2024-05-09T06:16:19.339 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T06:16:19.340 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535 --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535/orig.config.yaml 2024-05-09T06:16:19.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T06:16:19.480 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535 branch: squid description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7698535' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/reef ' name: teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2334 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 359/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 targets: smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOYFkKgAT/AIVxNH9mZa3ADyR3pC0nOjqV5uXAx2u+voylf2wL8cziVKm7ogjmYdUemwhflKsYmPWj6X8+SOM0U= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0dYtLFJMb1Cd3PqN/S8GwRyA6YC7HvN4IeszCSa/JDbc1gqrxc5QN+KNkbCHxxP5IfNYzSjDWGkFwmjsrjkrI= tasks: - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef roleless: true - print: '**** done end installing reef cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /home/teuthology/src/git.ceph.com_ceph_squid/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_21:24:04 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-05-09T06:16:19.480 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa; will attempt to use it 2024-05-09T06:16:19.480 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks 2024-05-09T06:16:19.481 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T06:16:19.482 INFO:teuthology.task.internal:Checking packages... 2024-05-09T06:16:19.504 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-09T06:16:19.504 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T06:16:19.504 INFO:teuthology.packaging:ref: None 2024-05-09T06:16:19.504 INFO:teuthology.packaging:tag: None 2024-05-09T06:16:19.504 INFO:teuthology.packaging:branch: squid 2024-05-09T06:16:19.504 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:16:19.504 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-09T06:16:20.000 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.gbf77c77e 2024-05-09T06:16:20.001 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T06:16:20.008 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T06:16:20.008 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T06:16:20.014 INFO:teuthology.task.internal:Saving configuration 2024-05-09T06:16:20.034 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T06:16:20.040 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T06:16:20.061 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:11:21.541231', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOYFkKgAT/AIVxNH9mZa3ADyR3pC0nOjqV5uXAx2u+voylf2wL8cziVKm7ogjmYdUemwhflKsYmPWj6X8+SOM0U='} 2024-05-09T06:16:20.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:11:21.540236', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0dYtLFJMb1Cd3PqN/S8GwRyA6YC7HvN4IeszCSa/JDbc1gqrxc5QN+KNkbCHxxP5IfNYzSjDWGkFwmjsrjkrI='} 2024-05-09T06:16:20.085 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T06:16:20.091 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-05-09T06:16:20.092 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-05-09T06:16:20.092 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T06:16:20.161 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7febb06f6160>, signals=[15]) 2024-05-09T06:16:20.162 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T06:16:20.167 INFO:teuthology.task.internal:Opening connections... 2024-05-09T06:16:20.168 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:16:20.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:16:20.244 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-05-09T06:16:20.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:16:20.318 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T06:16:20.325 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-05-09T06:16:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-05-09T06:16:20.343 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-05-09T06:16:20.401 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T06:16:20.402 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T06:16:20.403 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-05-09T06:16:20.442 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-05-09T06:16:20.460 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-05-09T06:16:20.460 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-05-09T06:16:20.517 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-05-09T06:16:20.517 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-05-09T06:16:20.517 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-05-09T06:16:20.517 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-05-09T06:16:20.517 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-05-09T06:16:20.517 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-05-09T06:16:20.518 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T06:16:20.518 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-05-09T06:16:20.518 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-05-09T06:16:20.518 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T06:16:20.518 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-05-09T06:16:20.518 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T06:16:20.518 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T06:16:20.518 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T06:16:20.518 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-05-09T06:16:20.541 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T06:16:20.550 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T06:16:20.555 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T06:16:20.555 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-05-09T06:16:20.558 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-05-09T06:16:20.573 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T06:16:20.579 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T06:16:20.580 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-05-09T06:16:20.615 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-05-09T06:16:20.714 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T06:16:20.768 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T06:16:20.782 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-09T06:16:20.782 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-09T06:16:20.782 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T06:16:20.783 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T06:16:20.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:16:20.800 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-05-09T06:16:20.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:16:20.801 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-09T06:16:20.857 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-437.el9.x86_64 2024-05-09T06:16:20.857 INFO:teuthology.task.kernel:Running kernel on smithi193: 5.14.0-437.el9.x86_64 2024-05-09T06:16:20.857 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2024-05-09T06:16:20.859 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-437.el9.x86_64 2024-05-09T06:16:20.859 INFO:teuthology.task.kernel:Running kernel on smithi063: 5.14.0-437.el9.x86_64 2024-05-09T06:16:20.859 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum install -y kernel 2024-05-09T06:16:23.215 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-09T06:16:24.530 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.1 MB 00:02 2024-05-09T06:16:30.930 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 3.6 MB/s | 19 MB 00:05 2024-05-09T06:16:33.740 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 2.0 MB/s | 19 MB 00:09 2024-05-09T06:16:36.731 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.2 MB 00:02 2024-05-09T06:16:38.156 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 92 kB/s | 16 kB 00:00 2024-05-09T06:16:39.973 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.2 MB 00:02 2024-05-09T06:16:41.832 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-09T06:16:44.044 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-09T06:16:48.277 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 2.1 MB/s | 21 MB 00:09 2024-05-09T06:16:49.191 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-09T06:16:51.181 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T06:16:51.181 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T06:16:51.272 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:16:51.280 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout:Install 4 Packages 2024-05-09T06:16:51.281 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:16:51.282 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 78 M 2024-05-09T06:16:51.282 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 123 M 2024-05-09T06:16:51.282 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:16:52.501 INFO:teuthology.orchestra.run.smithi193.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 452 kB/s | 484 kB 00:01 2024-05-09T06:16:53.418 INFO:teuthology.orchestra.run.smithi193.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 7.8 MB/s | 15 MB 00:01 2024-05-09T06:16:53.682 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-09T06:16:54.318 INFO:teuthology.orchestra.run.smithi193.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 12 MB/s | 34 MB 00:02 2024-05-09T06:16:55.193 INFO:teuthology.orchestra.run.smithi193.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 10 MB/s | 28 MB 00:02 2024-05-09T06:16:55.193 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:16:55.194 INFO:teuthology.orchestra.run.smithi193.stdout:Total 20 MB/s | 78 MB 00:03 2024-05-09T06:16:55.855 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T06:16:55.855 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T06:16:55.877 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:16:55.953 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:16:55.959 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:16:55.959 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-05-09T06:16:55.959 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:16:55.959 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:16:55.959 INFO:teuthology.orchestra.run.smithi063.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T06:16:55.960 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-09T06:16:55.960 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T06:16:55.960 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T06:16:55.960 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T06:16:55.960 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:16:55.960 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:16:55.960 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:16:55.960 INFO:teuthology.orchestra.run.smithi063.stdout:Install 4 Packages 2024-05-09T06:16:55.960 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:16:55.961 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 78 M 2024-05-09T06:16:55.961 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 123 M 2024-05-09T06:16:55.961 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:16:56.201 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:16:56.201 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:16:57.125 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:16:57.126 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:16:57.875 INFO:teuthology.orchestra.run.smithi063.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 301 kB/s | 484 kB 00:01 2024-05-09T06:16:59.534 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:17:00.142 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:17:00.226 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:17:02.906 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:17:03.069 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:17:07.759 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:17:07.812 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:12.558 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:13.090 INFO:teuthology.orchestra.run.smithi063.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 939 kB/s | 15 MB 00:16 2024-05-09T06:17:16.132 INFO:teuthology.orchestra.run.smithi063.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 1.5 MB/s | 28 MB 00:18 2024-05-09T06:17:26.964 INFO:teuthology.orchestra.run.smithi063.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 1.1 MB/s | 34 MB 00:30 2024-05-09T06:17:26.965 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:17:26.965 INFO:teuthology.orchestra.run.smithi063.stdout:Total 2.5 MB/s | 78 MB 00:31 2024-05-09T06:17:27.669 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:17:28.088 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:17:28.088 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:17:29.031 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:17:29.032 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:17:31.590 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:17:32.239 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:17:32.330 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:17:35.136 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:17:35.279 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:17:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:17:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:41.913 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:41.952 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:17:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:17:42.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:17:43.041 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:43.041 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:17:43.041 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:17:43.041 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T06:17:43.041 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T06:17:43.041 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T06:17:43.041 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T06:17:43.041 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:17:43.041 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:17:43.401 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:17:43.471 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:17:43.471 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:17:43.471 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T06:17:43.471 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T06:17:43.472 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T06:17:43.472 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-05-09T06:17:43.472 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi193.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T06:17:43.472 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:17:43.520 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:17:43.522 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:17:43.522 DEBUG:teuthology.orchestra.run.smithi193:> sudo rpm -qi grub2-tools 2024-05-09T06:17:43.561 INFO:teuthology.orchestra.run.smithi193.stdout:Name : grub2-tools 2024-05-09T06:17:43.561 INFO:teuthology.orchestra.run.smithi193.stdout:Epoch : 1 2024-05-09T06:17:43.561 INFO:teuthology.orchestra.run.smithi193.stdout:Version : 2.06 2024-05-09T06:17:43.561 INFO:teuthology.orchestra.run.smithi193.stdout:Release : 68.el9 2024-05-09T06:17:43.561 INFO:teuthology.orchestra.run.smithi193.stdout:Architecture: x86_64 2024-05-09T06:17:43.561 INFO:teuthology.orchestra.run.smithi193.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T06:17:43.561 INFO:teuthology.orchestra.run.smithi193.stdout:Group : Unspecified 2024-05-09T06:17:43.561 INFO:teuthology.orchestra.run.smithi193.stdout:Size : 8264254 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:License : GPLv3+ 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:Packager : builder@centos.org 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:Vendor : CentOS 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:Summary : Support tools for GRUB. 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:Description : 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T06:17:43.562 INFO:teuthology.orchestra.run.smithi193.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T06:17:43.563 INFO:teuthology.orchestra.run.smithi193.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T06:17:43.563 INFO:teuthology.orchestra.run.smithi193.stdout:hardware devices. 2024-05-09T06:17:43.563 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:17:43.563 INFO:teuthology.orchestra.run.smithi193.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T06:17:43.563 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T06:17:43.564 INFO:teuthology.task.kernel:Updating grub on smithi193 to boot 5.14.0-444.el9.x86_64 2024-05-09T06:17:43.564 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T06:17:44.169 INFO:teuthology.orchestra.run.smithi193.stderr:Generating grub configuration file ... 2024-05-09T06:17:45.502 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:17:45.567 INFO:teuthology.orchestra.run.smithi193.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T06:17:45.599 INFO:teuthology.orchestra.run.smithi193.stderr:done 2024-05-09T06:17:45.601 DEBUG:teuthology.orchestra.run.smithi193:> mktemp 2024-05-09T06:17:45.623 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.A3hOqmqsjN 2024-05-09T06:17:45.624 DEBUG:teuthology.orchestra.run.smithi193:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.A3hOqmqsjN 2024-05-09T06:17:45.700 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.A3hOqmqsjN 2024-05-09T06:17:45.835 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.A3hOqmqsjN is 6KB 2024-05-09T06:17:45.883 DEBUG:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.A3hOqmqsjN 2024-05-09T06:17:45.898 DEBUG:teuthology.orchestra.run.smithi193:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T06:17:45.959 INFO:teuthology.orchestra.run.smithi193.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T06:17:45.959 INFO:teuthology.orchestra.run.smithi193.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T06:17:45.959 INFO:teuthology.orchestra.run.smithi193.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T06:17:45.959 INFO:teuthology.orchestra.run.smithi193.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T06:17:45.961 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T06:17:46.123 DEBUG:teuthology.orchestra.run.smithi193:> sudo shutdown -r now 2024-05-09T06:18:16.125 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-05-09T06:18:16.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-09T06:18:16.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:18:16.371 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:18:16.411 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:18:16.763 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:18:16.763 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:18:16.763 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:18:16.763 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:18:17.612 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:18:17.612 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:18:17.612 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:18:17.612 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T06:18:17.613 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T06:18:17.613 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T06:18:17.613 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T06:18:17.613 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:18:17.613 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:18:18.002 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:18:18.074 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:18:18.075 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:18:18.075 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T06:18:18.075 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T06:18:18.075 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T06:18:18.075 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-05-09T06:18:18.076 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi063.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T06:18:18.076 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:18:18.126 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:18:18.127 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:18:18.127 DEBUG:teuthology.orchestra.run.smithi063:> sudo rpm -qi grub2-tools 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Name : grub2-tools 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Epoch : 1 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Version : 2.06 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Release : 68.el9 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: x86_64 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Group : Unspecified 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Size : 8264254 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:License : GPLv3+ 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T06:18:18.159 INFO:teuthology.orchestra.run.smithi063.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T06:18:18.160 INFO:teuthology.orchestra.run.smithi063.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T06:18:18.160 INFO:teuthology.orchestra.run.smithi063.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T06:18:18.160 INFO:teuthology.orchestra.run.smithi063.stdout:Packager : builder@centos.org 2024-05-09T06:18:18.160 INFO:teuthology.orchestra.run.smithi063.stdout:Vendor : CentOS 2024-05-09T06:18:18.160 INFO:teuthology.orchestra.run.smithi063.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T06:18:18.160 INFO:teuthology.orchestra.run.smithi063.stdout:Summary : Support tools for GRUB. 2024-05-09T06:18:18.160 INFO:teuthology.orchestra.run.smithi063.stdout:Description : 2024-05-09T06:18:18.161 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:18:18.161 INFO:teuthology.orchestra.run.smithi063.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T06:18:18.161 INFO:teuthology.orchestra.run.smithi063.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T06:18:18.161 INFO:teuthology.orchestra.run.smithi063.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T06:18:18.161 INFO:teuthology.orchestra.run.smithi063.stdout:hardware devices. 2024-05-09T06:18:18.161 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:18:18.161 INFO:teuthology.orchestra.run.smithi063.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T06:18:18.162 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T06:18:18.162 INFO:teuthology.task.kernel:Updating grub on smithi063 to boot 5.14.0-444.el9.x86_64 2024-05-09T06:18:18.162 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T06:18:18.847 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-05-09T06:18:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T06:18:20.362 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-05-09T06:18:20.364 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-09T06:18:20.383 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.FmBaRm8zla 2024-05-09T06:18:20.383 DEBUG:teuthology.orchestra.run.smithi063:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FmBaRm8zla 2024-05-09T06:18:20.450 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.FmBaRm8zla 2024-05-09T06:18:20.587 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.FmBaRm8zla is 6KB 2024-05-09T06:18:20.636 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.FmBaRm8zla 2024-05-09T06:18:20.651 DEBUG:teuthology.orchestra.run.smithi063:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T06:18:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T06:18:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T06:18:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T06:18:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T06:18:20.714 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T06:18:20.860 DEBUG:teuthology.orchestra.run.smithi063:> sudo shutdown -r now 2024-05-09T06:18:34.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-05-09T06:18:43.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-09T06:18:43.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:18:46.722 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-05-09T06:18:50.867 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-05-09T06:18:50.867 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T06:18:50.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:18:58.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-09T06:18:58.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:19:09.215 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-09T06:19:17.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-05-09T06:19:18.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T06:19:18.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:19:21.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-09T06:19:32.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-09T06:19:32.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:19:32.441 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-05-09T06:19:32.809 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-09T06:19:32.809 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-444.el9.x86_64"... 2024-05-09T06:19:32.810 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-05-09T06:19:32.824 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-444.el9.x86_64 2024-05-09T06:19:32.824 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T06:19:32.825 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T06:19:32.825 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T06:19:33.280 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T06:19:33.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:19:33.826 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-05-09T06:19:33.826 DEBUG:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T06:19:33.925 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2024-05-09T06:19:33.961 DEBUG:teuthology.parallel:result is None 2024-05-09T06:19:51.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-09T06:20:06.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T06:20:06.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:20:07.187 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-09T06:20:07.557 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T06:20:07.557 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-444.el9.x86_64"... 2024-05-09T06:20:07.558 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-09T06:20:07.574 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-444.el9.x86_64 2024-05-09T06:20:07.574 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T06:20:07.574 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T06:20:07.575 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T06:20:08.575 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-05-09T06:20:08.576 DEBUG:teuthology.orchestra.run.smithi063:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T06:20:08.651 INFO:teuthology.orchestra.run.smithi063.stdout:ttyS1 2024-05-09T06:20:08.680 DEBUG:teuthology.parallel:result is None 2024-05-09T06:20:08.681 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T06:20:08.689 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T06:20:08.689 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T06:20:08.691 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T06:20:08.710 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T06:20:08.717 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T06:20:08.725 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T06:20:08.725 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T06:20:08.744 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T06:20:08.809 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T06:20:08.816 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T06:20:08.816 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T06:20:08.819 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T06:20:08.845 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:20:08.854 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:20:08.857 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:20:08.869 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:20:08.871 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T06:20:08.879 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T06:20:08.879 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T06:20:08.896 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T06:20:08.940 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T06:20:08.950 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T06:20:08.950 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T06:20:08.961 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T06:20:08.997 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:20:09.060 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:20:09.109 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:20:09.185 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:20:09.227 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:20:09.227 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T06:20:09.290 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:20:09.332 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:20:09.385 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:20:09.469 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:20:09.511 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:20:09.511 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T06:20:09.579 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-05-09T06:20:09.583 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-05-09T06:20:09.651 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:20:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:20:10.159 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T06:20:10.168 INFO:teuthology.task.internal:Starting timer... 2024-05-09T06:20:10.168 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T06:20:10.183 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T06:20:10.191 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T06:20:10.192 DEBUG:teuthology.orchestra.run.smithi063:> sudo service auditd rotate 2024-05-09T06:20:10.288 INFO:teuthology.orchestra.run.smithi063.stdout:Rotating logs: 2024-05-09T06:20:10.291 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-05-09T06:20:10.337 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-05-09T06:20:10.339 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T06:20:10.339 DEBUG:teuthology.orchestra.run.smithi063:> /usr/sbin/getenforce 2024-05-09T06:20:10.373 INFO:teuthology.orchestra.run.smithi063.stdout:Permissive 2024-05-09T06:20:10.373 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-05-09T06:20:10.393 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-05-09T06:20:10.394 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi063.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-05-09T06:20:10.394 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:20:10.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:20:10.463 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:20:10.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:20:10.489 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T06:20:10.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo /usr/sbin/setenforce permissive 2024-05-09T06:20:10.535 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-05-09T06:20:10.561 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T06:20:10.572 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T06:20:10.591 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T06:20:10.592 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi063.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T06:24:22.057 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi063.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-05-09T06:24:22.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T06:24:22.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:24:22.141 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-09T06:24:22.177 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T06:24:22.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-09T06:24:22.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:24:22.257 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-05-09T06:24:22.338 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-09T06:24:22.338 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T06:24:22.349 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T06:24:22.349 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T06:24:22.350 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:24:22.352 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T06:24:22.352 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:24:22.381 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T06:24:22.395 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T06:24:22.407 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T06:24:22.421 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T06:24:22.422 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: ntpd: command not found 2024-05-09T06:24:22.432 INFO:teuthology.orchestra.run.smithi063.stdout:506 Cannot talk to daemon 2024-05-09T06:24:22.445 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-05-09T06:24:22.447 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T06:24:22.456 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-05-09T06:24:22.461 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T06:24:22.470 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T06:24:22.483 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T06:24:22.519 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:24:22.522 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:24:22.523 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-09T06:24:22.523 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:22.523 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:22.523 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:22.523 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:22.540 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:24:22.544 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:24:22.544 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-05-09T06:24:22.544 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:22.544 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:22.544 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:22.545 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:24:22.546 INFO:teuthology.run_tasks:Running task install... 2024-05-09T06:24:22.559 DEBUG:teuthology.task.install:project ceph 2024-05-09T06:24:22.560 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-09T06:24:22.560 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T06:24:22.560 INFO:teuthology.task.install:Using flavor: default 2024-05-09T06:24:22.567 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T06:24:22.568 INFO:teuthology.task.install:extra packages: [] 2024-05-09T06:24:22.568 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T06:24:22.569 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T06:24:22.569 INFO:teuthology.packaging:ref: None 2024-05-09T06:24:22.569 INFO:teuthology.packaging:tag: None 2024-05-09T06:24:22.569 INFO:teuthology.packaging:branch: reef 2024-05-09T06:24:22.569 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:24:22.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-05-09T06:24:22.573 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T06:24:22.573 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T06:24:22.573 INFO:teuthology.packaging:ref: None 2024-05-09T06:24:22.574 INFO:teuthology.packaging:tag: None 2024-05-09T06:24:22.574 INFO:teuthology.packaging:branch: reef 2024-05-09T06:24:22.574 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:24:22.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-05-09T06:24:22.813 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/flavors/default/ 2024-05-09T06:24:22.814 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1306.gbf64272e 2024-05-09T06:24:22.924 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/flavors/default/ 2024-05-09T06:24:22.924 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1306.gbf64272e 2024-05-09T06:24:23.001 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T06:24:23.001 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:24:23.002 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T06:24:23.031 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-09T06:24:23.031 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T06:24:23.031 INFO:teuthology.packaging:ref: None 2024-05-09T06:24:23.031 INFO:teuthology.packaging:tag: None 2024-05-09T06:24:23.031 INFO:teuthology.packaging:branch: reef 2024-05-09T06:24:23.031 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:24:23.032 DEBUG:teuthology.orchestra.run.smithi063:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T06:24:23.034 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T06:24:23.034 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:24:23.034 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T06:24:23.060 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-09T06:24:23.060 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T06:24:23.060 INFO:teuthology.packaging:ref: None 2024-05-09T06:24:23.060 INFO:teuthology.packaging:tag: None 2024-05-09T06:24:23.060 INFO:teuthology.packaging:branch: reef 2024-05-09T06:24:23.060 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:24:23.060 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T06:24:23.100 DEBUG:teuthology.orchestra.run.smithi063:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T06:24:23.131 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T06:24:23.189 DEBUG:teuthology.orchestra.run.smithi063:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T06:24:23.207 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T06:24:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:check_obsoletes = 1 2024-05-09T06:24:23.260 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum clean all 2024-05-09T06:24:23.277 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-05-09T06:24:23.279 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-05-09T06:24:23.504 INFO:teuthology.orchestra.run.smithi063.stdout:54 files removed 2024-05-09T06:24:23.512 INFO:teuthology.orchestra.run.smithi193.stdout:54 files removed 2024-05-09T06:24:23.531 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-radosgw 2024-05-09T06:24:23.539 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw 2024-05-09T06:24:24.227 INFO:teuthology.orchestra.run.smithi063.stdout:ceph packages for x86_64 195 kB/s | 77 kB 00:00 2024-05-09T06:24:24.228 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 185 kB/s | 77 kB 00:00 2024-05-09T06:24:24.653 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-05-09T06:24:24.677 INFO:teuthology.orchestra.run.smithi063.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-05-09T06:24:25.096 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-05-09T06:24:25.102 INFO:teuthology.orchestra.run.smithi063.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-09T06:24:25.569 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.1 MB 00:00 2024-05-09T06:24:26.700 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.1 MB 00:01 2024-05-09T06:24:27.362 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-09T06:24:31.505 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 19 MB 00:03 2024-05-09T06:24:32.347 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-05-09T06:24:34.006 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-05-09T06:24:34.966 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-05-09T06:24:36.747 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-05-09T06:24:39.156 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-09T06:24:40.283 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-05-09T06:24:40.302 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-05-09T06:24:42.793 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:24:42.801 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:24:42.801 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-09T06:24:42.801 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:24:42.801 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:24:42.801 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 7.7 M 2024-05-09T06:24:42.801 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrading: 2024-05-09T06:24:42.801 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 3.3 M 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 3.0 M 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 5.1 M 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 18 M 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 25 k 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 708 k 2024-05-09T06:24:42.802 INFO:teuthology.orchestra.run.smithi063.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 474 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 4.5 M 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 45 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 129 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 162 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 321 k 2024-05-09T06:24:42.803 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 298 k 2024-05-09T06:24:42.804 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 99 k 2024-05-09T06:24:42.804 INFO:teuthology.orchestra.run.smithi063.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T06:24:42.804 INFO:teuthology.orchestra.run.smithi063.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T06:24:42.804 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:24:42.804 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:24:42.804 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:24:42.804 INFO:teuthology.orchestra.run.smithi063.stdout:Install 28 Packages 2024-05-09T06:24:42.804 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrade 2 Packages 2024-05-09T06:24:42.805 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:24:42.805 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 53 M 2024-05-09T06:24:42.805 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:24:45.301 INFO:teuthology.orchestra.run.smithi063.stdout:(1/30): ceph-base-18.2.2-1306.gbf64272e.el9.x86 2.3 MB/s | 5.1 MB 00:02 2024-05-09T06:24:45.376 INFO:teuthology.orchestra.run.smithi063.stdout:(2/30): ceph-selinux-18.2.2-1306.gbf64272e.el9. 337 kB/s | 25 kB 00:00 2024-05-09T06:24:45.391 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-09T06:24:45.593 INFO:teuthology.orchestra.run.smithi063.stdout:(3/30): libcephfs2-18.2.2-1306.gbf64272e.el9.x8 3.2 MB/s | 708 kB 00:00 2024-05-09T06:24:45.809 INFO:teuthology.orchestra.run.smithi063.stdout:(4/30): libradosstriper1-18.2.2-1306.gbf64272e. 2.1 MB/s | 474 kB 00:00 2024-05-09T06:24:46.876 INFO:teuthology.orchestra.run.smithi063.stdout:(5/30): librgw2-18.2.2-1306.gbf64272e.el9.x86_6 4.2 MB/s | 4.5 MB 00:01 2024-05-09T06:24:46.952 INFO:teuthology.orchestra.run.smithi063.stdout:(6/30): python3-ceph-argparse-18.2.2-1306.gbf64 602 kB/s | 45 kB 00:00 2024-05-09T06:24:47.035 INFO:teuthology.orchestra.run.smithi063.stdout:(7/30): python3-ceph-common-18.2.2-1306.gbf6427 1.5 MB/s | 129 kB 00:00 2024-05-09T06:24:47.110 INFO:teuthology.orchestra.run.smithi063.stdout:(8/30): python3-cephfs-18.2.2-1306.gbf64272e.el 2.1 MB/s | 162 kB 00:00 2024-05-09T06:24:47.194 INFO:teuthology.orchestra.run.smithi063.stdout:(9/30): python3-rados-18.2.2-1306.gbf64272e.el9 3.8 MB/s | 321 kB 00:00 2024-05-09T06:24:47.286 INFO:teuthology.orchestra.run.smithi063.stdout:(10/30): python3-rbd-18.2.2-1306.gbf64272e.el9. 3.2 MB/s | 298 kB 00:00 2024-05-09T06:24:47.369 INFO:teuthology.orchestra.run.smithi063.stdout:(11/30): python3-rgw-18.2.2-1306.gbf64272e.el9. 1.2 MB/s | 99 kB 00:00 2024-05-09T06:24:47.748 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:24:47.751 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 7.7 M 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 3.3 M 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 3.0 M 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 5.1 M 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 18 M 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 25 k 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-09T06:24:47.752 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 708 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 474 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 4.5 M 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 45 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 129 k 2024-05-09T06:24:47.753 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 162 k 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 321 k 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 298 k 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 99 k 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout:Install 28 Packages 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 2 Packages 2024-05-09T06:24:47.754 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:24:47.755 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 53 M 2024-05-09T06:24:47.755 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:24:48.078 INFO:teuthology.orchestra.run.smithi063.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 151 kB/s | 107 kB 00:00 2024-05-09T06:24:48.836 INFO:teuthology.orchestra.run.smithi063.stdout:(13/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 211 kB/s | 160 kB 00:00 2024-05-09T06:24:49.478 INFO:teuthology.orchestra.run.smithi063.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-05-09T06:24:49.687 INFO:teuthology.orchestra.run.smithi063.stdout:(15/30): ceph-radosgw-18.2.2-1306.gbf64272e.el9 1.2 MB/s | 7.7 MB 00:06 2024-05-09T06:24:50.479 INFO:teuthology.orchestra.run.smithi063.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 662 kB/s | 662 kB 00:01 2024-05-09T06:24:50.546 INFO:teuthology.orchestra.run.smithi063.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 341 kB/s | 292 kB 00:00 2024-05-09T06:24:50.679 INFO:teuthology.orchestra.run.smithi063.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 833 kB/s | 111 kB 00:00 2024-05-09T06:24:50.771 INFO:teuthology.orchestra.run.smithi063.stdout:(19/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.3 MB/s | 309 kB 00:00 2024-05-09T06:24:51.234 INFO:teuthology.orchestra.run.smithi193.stdout:(1/30): ceph-radosgw-18.2.2-1306.gbf64272e.el9. 2.4 MB/s | 7.7 MB 00:03 2024-05-09T06:24:51.300 INFO:teuthology.orchestra.run.smithi193.stdout:(2/30): ceph-selinux-18.2.2-1306.gbf64272e.el9. 379 kB/s | 25 kB 00:00 2024-05-09T06:24:51.480 INFO:teuthology.orchestra.run.smithi063.stdout:(20/30): libarrow-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-05-09T06:24:51.530 INFO:teuthology.orchestra.run.smithi063.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 523 kB/s | 26 kB 00:00 2024-05-09T06:24:51.564 INFO:teuthology.orchestra.run.smithi063.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-05-09T06:24:51.601 INFO:teuthology.orchestra.run.smithi193.stdout:(3/30): libcephfs2-18.2.2-1306.gbf64272e.el9.x8 2.3 MB/s | 708 kB 00:00 2024-05-09T06:24:51.606 INFO:teuthology.orchestra.run.smithi063.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-09T06:24:51.789 INFO:teuthology.orchestra.run.smithi063.stdout:(24/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-05-09T06:24:51.840 INFO:teuthology.orchestra.run.smithi063.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-09T06:24:52.067 INFO:teuthology.orchestra.run.smithi193.stdout:(4/30): ceph-base-18.2.2-1306.gbf64272e.el9.x86 1.2 MB/s | 5.1 MB 00:04 2024-05-09T06:24:52.082 INFO:teuthology.orchestra.run.smithi063.stdout:(26/30): ceph-common-18.2.2-1306.gbf64272e.el9. 2.0 MB/s | 18 MB 00:09 2024-05-09T06:24:52.101 INFO:teuthology.orchestra.run.smithi193.stdout:(5/30): libradosstriper1-18.2.2-1306.gbf64272e. 948 kB/s | 474 kB 00:00 2024-05-09T06:24:52.185 INFO:teuthology.orchestra.run.smithi193.stdout:(6/30): python3-ceph-argparse-18.2.2-1306.gbf64 541 kB/s | 45 kB 00:00 2024-05-09T06:24:52.207 INFO:teuthology.orchestra.run.smithi063.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-05-09T06:24:52.268 INFO:teuthology.orchestra.run.smithi193.stdout:(7/30): python3-ceph-common-18.2.2-1306.gbf6427 1.5 MB/s | 129 kB 00:00 2024-05-09T06:24:52.352 INFO:teuthology.orchestra.run.smithi193.stdout:(8/30): python3-cephfs-18.2.2-1306.gbf64272e.el 1.9 MB/s | 162 kB 00:00 2024-05-09T06:24:52.527 INFO:teuthology.orchestra.run.smithi193.stdout:(9/30): python3-rados-18.2.2-1306.gbf64272e.el9 1.8 MB/s | 321 kB 00:00 2024-05-09T06:24:52.686 INFO:teuthology.orchestra.run.smithi193.stdout:(10/30): python3-rbd-18.2.2-1306.gbf64272e.el9. 1.8 MB/s | 298 kB 00:00 2024-05-09T06:24:52.770 INFO:teuthology.orchestra.run.smithi193.stdout:(11/30): python3-rgw-18.2.2-1306.gbf64272e.el9. 1.2 MB/s | 99 kB 00:00 2024-05-09T06:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout:(28/30): librados2-18.2.2-1306.gbf64272e.el9.x8 2.9 MB/s | 3.3 MB 00:01 2024-05-09T06:24:53.240 INFO:teuthology.orchestra.run.smithi063.stdout:(29/30): python3-prettytable-0.7.2-27.el9.noarc 15 kB/s | 42 kB 00:02 2024-05-09T06:24:54.612 INFO:teuthology.orchestra.run.smithi193.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 58 kB/s | 107 kB 00:01 2024-05-09T06:24:54.870 INFO:teuthology.orchestra.run.smithi193.stdout:(13/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 621 kB/s | 160 kB 00:00 2024-05-09T06:24:55.004 INFO:teuthology.orchestra.run.smithi193.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-05-09T06:24:55.288 INFO:teuthology.orchestra.run.smithi193.stdout:(15/30): ceph-common-18.2.2-1306.gbf64272e.el9. 2.5 MB/s | 18 MB 00:07 2024-05-09T06:24:55.573 INFO:teuthology.orchestra.run.smithi193.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-05-09T06:24:55.574 INFO:teuthology.orchestra.run.smithi063.stdout:(30/30): librbd1-18.2.2-1306.gbf64272e.el9.x86_ 935 kB/s | 3.0 MB 00:03 2024-05-09T06:24:55.574 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:24:55.574 INFO:teuthology.orchestra.run.smithi063.stdout:Total 4.2 MB/s | 53 MB 00:12 2024-05-09T06:24:55.579 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:24:55.620 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:24:55.620 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:24:55.888 INFO:teuthology.orchestra.run.smithi193.stdout:(17/30): librgw2-18.2.2-1306.gbf64272e.el9.x86_ 1.2 MB/s | 4.5 MB 00:03 2024-05-09T06:24:56.055 INFO:teuthology.orchestra.run.smithi193.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 381 kB/s | 292 kB 00:00 2024-05-09T06:24:56.209 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:24:56.210 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:24:56.405 INFO:teuthology.orchestra.run.smithi193.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 214 kB/s | 111 kB 00:00 2024-05-09T06:24:56.697 INFO:teuthology.orchestra.run.smithi193.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 481 kB/s | 309 kB 00:00 2024-05-09T06:24:56.822 INFO:teuthology.orchestra.run.smithi193.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-05-09T06:24:56.947 INFO:teuthology.orchestra.run.smithi193.stdout:(22/30): libarrow-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-05-09T06:24:56.972 INFO:teuthology.orchestra.run.smithi193.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 327 kB/s | 49 kB 00:00 2024-05-09T06:24:57.048 INFO:teuthology.orchestra.run.smithi193.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 673 kB/s | 67 kB 00:00 2024-05-09T06:24:57.098 INFO:teuthology.orchestra.run.smithi193.stdout:(25/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-05-09T06:24:57.190 INFO:teuthology.orchestra.run.smithi193.stdout:(26/30): python3-prettytable-0.7.2-27.el9.noarc 23 kB/s | 42 kB 00:01 2024-05-09T06:24:57.215 INFO:teuthology.orchestra.run.smithi193.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-05-09T06:24:57.249 INFO:teuthology.orchestra.run.smithi193.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-09T06:24:57.548 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:24:57.634 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-05-09T06:24:57.662 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-05-09T06:24:57.714 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-05-09T06:24:58.105 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-05-09T06:24:58.162 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 5/32 2024-05-09T06:24:58.238 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 5/32 2024-05-09T06:24:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rados-2:18.2.2-1306.gbf64272e.el9.x86_64 6/32 2024-05-09T06:24:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 7/32 2024-05-09T06:24:58.401 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 7/32 2024-05-09T06:24:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-05-09T06:24:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-argparse-2:18.2.2-1306.gbf64272e.el9. 9/32 2024-05-09T06:24:58.550 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cephfs-2:18.2.2-1306.gbf64272e.el9.x86_64 10/32 2024-05-09T06:24:58.574 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libradosstriper1-2:18.2.2-1306.gbf64272e.el9.x86_6 11/32 2024-05-09T06:24:58.640 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1306.gbf64272e.el9.x86_6 11/32 2024-05-09T06:24:58.755 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-05-09T06:24:58.780 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-05-09T06:24:58.844 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-05-09T06:24:58.878 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-05-09T06:24:58.916 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-05-09T06:24:58.942 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-05-09T06:24:58.975 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-05-09T06:24:59.174 INFO:teuthology.orchestra.run.smithi193.stdout:(29/30): librbd1-18.2.2-1306.gbf64272e.el9.x86_ 1.5 MB/s | 3.0 MB 00:01 2024-05-09T06:24:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-05-09T06:24:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 20/32 2024-05-09T06:24:59.422 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 20/32 2024-05-09T06:24:59.459 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rbd-2:18.2.2-1306.gbf64272e.el9.x86_64 21/32 2024-05-09T06:24:59.497 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-05-09T06:24:59.918 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-05-09T06:24:59.948 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librgw2-2:18.2.2-1306.gbf64272e.el9.x86_64 24/32 2024-05-09T06:25:00.020 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librgw2-2:18.2.2-1306.gbf64272e.el9.x86_64 24/32 2024-05-09T06:25:00.099 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rgw-2:18.2.2-1306.gbf64272e.el9.x86_64 25/32 2024-05-09T06:25:00.134 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-common-2:18.2.2-1306.gbf64272e.el9.x8 26/32 2024-05-09T06:25:00.557 INFO:teuthology.orchestra.run.smithi193.stdout:(30/30): librados2-18.2.2-1306.gbf64272e.el9.x8 1.0 MB/s | 3.3 MB 00:03 2024-05-09T06:25:00.563 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:00.563 INFO:teuthology.orchestra.run.smithi193.stdout:Total 4.1 MB/s | 53 MB 00:12 2024-05-09T06:25:00.633 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:25:00.670 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:25:00.670 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:25:01.235 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:25:01.235 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:25:02.544 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:25:02.546 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 27/32 2024-05-09T06:25:02.589 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 27/32 2024-05-09T06:25:02.625 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-05-09T06:25:02.662 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-05-09T06:25:02.722 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-05-09T06:25:03.103 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-05-09T06:25:03.153 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 5/32 2024-05-09T06:25:03.172 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 27/32 2024-05-09T06:25:03.228 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 5/32 2024-05-09T06:25:03.270 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 28/32 2024-05-09T06:25:03.321 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:18.2.2-1306.gbf64272e.el9.x86_64 6/32 2024-05-09T06:25:03.346 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 7/32 2024-05-09T06:25:03.367 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 28/32 2024-05-09T06:25:03.368 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T06:25:03.368 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T06:25:03.368 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:03.407 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 7/32 2024-05-09T06:25:03.410 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-selinux-2:18.2.2-1306.gbf64272e.el9.x86_64 29/32 2024-05-09T06:25:03.445 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-05-09T06:25:03.489 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:18.2.2-1306.gbf64272e.el9. 9/32 2024-05-09T06:25:03.555 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:18.2.2-1306.gbf64272e.el9.x86_64 10/32 2024-05-09T06:25:03.581 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:18.2.2-1306.gbf64272e.el9.x86_6 11/32 2024-05-09T06:25:03.630 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1306.gbf64272e.el9.x86_6 11/32 2024-05-09T06:25:03.736 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-05-09T06:25:03.763 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-05-09T06:25:03.805 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-05-09T06:25:03.837 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-05-09T06:25:03.864 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-05-09T06:25:03.900 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-05-09T06:25:03.932 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-05-09T06:25:04.230 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-05-09T06:25:04.270 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 20/32 2024-05-09T06:25:04.343 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 20/32 2024-05-09T06:25:04.375 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:18.2.2-1306.gbf64272e.el9.x86_64 21/32 2024-05-09T06:25:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-05-09T06:25:04.810 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-05-09T06:25:04.831 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:18.2.2-1306.gbf64272e.el9.x86_64 24/32 2024-05-09T06:25:04.886 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:18.2.2-1306.gbf64272e.el9.x86_64 24/32 2024-05-09T06:25:04.961 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:18.2.2-1306.gbf64272e.el9.x86_64 25/32 2024-05-09T06:25:04.992 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:18.2.2-1306.gbf64272e.el9.x8 26/32 2024-05-09T06:25:07.039 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 27/32 2024-05-09T06:25:07.072 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 27/32 2024-05-09T06:25:07.623 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 27/32 2024-05-09T06:25:07.659 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 28/32 2024-05-09T06:25:07.754 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 28/32 2024-05-09T06:25:07.755 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T06:25:07.755 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T06:25:07.755 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:07.785 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:18.2.2-1306.gbf64272e.el9.x86_64 29/32 2024-05-09T06:25:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1306.gbf64272e.el9.x86_64 29/32 2024-05-09T06:25:13.040 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /sys 2024-05-09T06:25:13.040 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /proc 2024-05-09T06:25:13.040 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /mnt 2024-05-09T06:25:13.040 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /var/tmp 2024-05-09T06:25:13.040 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /home 2024-05-09T06:25:13.040 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /root 2024-05-09T06:25:13.041 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /tmp 2024-05-09T06:25:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:13.811 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-radosgw-2:18.2.2-1306.gbf64272e.el9.x86_64 30/32 2024-05-09T06:25:13.838 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1306.gbf64272e.el9.x86_64 30/32 2024-05-09T06:25:13.838 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:13.838 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T06:25:13.838 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:25:13.838 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:25:13.838 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:13.840 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-09T06:25:13.931 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-09T06:25:13.931 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-09T06:25:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 1/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 2/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-radosgw-2:18.2.2-1306.gbf64272e.el9.x86_64 3/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-selinux-2:18.2.2-1306.gbf64272e.el9.x86_64 4/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 5/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libradosstriper1-2:18.2.2-1306.gbf64272e.el9.x86_6 6/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librgw2-2:18.2.2-1306.gbf64272e.el9.x86_64 7/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1306.gbf64272e.el9. 8/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-common-2:18.2.2-1306.gbf64272e.el9.x8 9/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cephfs-2:18.2.2-1306.gbf64272e.el9.x86_64 10/32 2024-05-09T06:25:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rados-2:18.2.2-1306.gbf64272e.el9.x86_64 11/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rbd-2:18.2.2-1306.gbf64272e.el9.x86_64 12/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rgw-2:18.2.2-1306.gbf64272e.el9.x86_64 13/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-05-09T06:25:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-05-09T06:25:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-05-09T06:25:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-05-09T06:25:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-05-09T06:25:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 29/32 2024-05-09T06:25:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-05-09T06:25:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 31/32 2024-05-09T06:25:15.847 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-05-09T06:25:15.847 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:15.847 INFO:teuthology.orchestra.run.smithi063.stdout:Upgraded: 2024-05-09T06:25:15.847 INFO:teuthology.orchestra.run.smithi063.stdout: librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.847 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T06:25:15.848 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T06:25:15.849 INFO:teuthology.orchestra.run.smithi063.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T06:25:15.850 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:15.850 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:25:16.218 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-test 2024-05-09T06:25:16.749 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1306.gbf64272e.el9.x86_64 29/32 2024-05-09T06:25:16.750 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-05-09T06:25:16.750 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-05-09T06:25:16.750 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-05-09T06:25:16.750 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-05-09T06:25:16.750 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-05-09T06:25:16.750 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-05-09T06:25:16.750 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-05-09T06:25:16.750 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:16.801 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:36 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:25:16.927 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:25:16.928 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-09T06:25:16.928 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:16.928 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:25:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 40 M 2024-05-09T06:25:16.928 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-09T06:25:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 166 k 2024-05-09T06:25:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T06:25:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T06:25:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:16.929 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:25:16.929 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:16.929 INFO:teuthology.orchestra.run.smithi063.stdout:Install 4 Packages 2024-05-09T06:25:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:16.930 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 40 M 2024-05-09T06:25:16.930 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 163 M 2024-05-09T06:25:16.930 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:25:17.152 INFO:teuthology.orchestra.run.smithi063.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-05-09T06:25:17.210 INFO:teuthology.orchestra.run.smithi063.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-09T06:25:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:(3/4): libcephsqlite-18.2.2-1306.gbf64272e.el9. 344 kB/s | 166 kB 00:00 2024-05-09T06:25:17.487 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:18.2.2-1306.gbf64272e.el9.x86_64 30/32 2024-05-09T06:25:17.512 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1306.gbf64272e.el9.x86_64 30/32 2024-05-09T06:25:17.512 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:17.512 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T06:25:17.512 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:25:17.512 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:25:17.512 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:17.514 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-09T06:25:17.598 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-09T06:25:17.598 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-09T06:25:18.619 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 1/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 2/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:18.2.2-1306.gbf64272e.el9.x86_64 3/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:18.2.2-1306.gbf64272e.el9.x86_64 4/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 5/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:18.2.2-1306.gbf64272e.el9.x86_6 6/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:18.2.2-1306.gbf64272e.el9.x86_64 7/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1306.gbf64272e.el9. 8/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:18.2.2-1306.gbf64272e.el9.x8 9/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:18.2.2-1306.gbf64272e.el9.x86_64 10/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:18.2.2-1306.gbf64272e.el9.x86_64 11/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:18.2.2-1306.gbf64272e.el9.x86_64 12/32 2024-05-09T06:25:18.620 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:18.2.2-1306.gbf64272e.el9.x86_64 13/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-05-09T06:25:18.621 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-05-09T06:25:18.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-05-09T06:25:18.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 29/32 2024-05-09T06:25:18.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-05-09T06:25:18.622 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 31/32 2024-05-09T06:25:19.414 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-05-09T06:25:19.414 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:19.414 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-05-09T06:25:19.414 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.414 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.414 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.415 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:19.416 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:25:19.752 INFO:teuthology.orchestra.run.smithi063.stdout:(4/4): ceph-test-18.2.2-1306.gbf64272e.el9.x86_ 14 MB/s | 40 MB 00:02 2024-05-09T06:25:19.754 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:19.755 INFO:teuthology.orchestra.run.smithi063.stdout:Total 14 MB/s | 40 MB 00:02 2024-05-09T06:25:19.763 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-test 2024-05-09T06:25:19.783 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:25:19.805 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:25:19.805 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:25:20.139 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:25:20.139 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:25:20.335 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:35 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:25:20.455 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 40 M 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 166 k 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:20.456 INFO:teuthology.orchestra.run.smithi193.stdout:Install 4 Packages 2024-05-09T06:25:20.457 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:20.457 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 40 M 2024-05-09T06:25:20.457 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 163 M 2024-05-09T06:25:20.457 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:25:20.611 INFO:teuthology.orchestra.run.smithi193.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-05-09T06:25:20.641 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:25:20.670 INFO:teuthology.orchestra.run.smithi193.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-09T06:25:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-09T06:25:20.736 INFO:teuthology.orchestra.run.smithi193.stdout:(3/4): libcephsqlite-18.2.2-1306.gbf64272e.el9. 691 kB/s | 166 kB 00:00 2024-05-09T06:25:20.777 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-09T06:25:20.804 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephsqlite-2:18.2.2-1306.gbf64272e.el9.x86_64 3/4 2024-05-09T06:25:23.020 INFO:teuthology.orchestra.run.smithi193.stdout:(4/4): ceph-test-18.2.2-1306.gbf64272e.el9.x86_ 16 MB/s | 40 MB 00:02 2024-05-09T06:25:23.022 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:23.023 INFO:teuthology.orchestra.run.smithi193.stdout:Total 16 MB/s | 40 MB 00:02 2024-05-09T06:25:23.049 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:25:23.069 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:25:23.069 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:25:23.382 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:25:23.382 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:25:23.866 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:25:23.938 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-09T06:25:23.977 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-09T06:25:24.005 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:18.2.2-1306.gbf64272e.el9.x86_64 3/4 2024-05-09T06:25:24.716 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1306.gbf64272e.el9.x86_64 3/4 2024-05-09T06:25:24.788 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-test-2:18.2.2-1306.gbf64272e.el9.x86_64 4/4 2024-05-09T06:25:26.742 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-test-2:18.2.2-1306.gbf64272e.el9.x86_64 4/4 2024-05-09T06:25:26.743 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-test-2:18.2.2-1306.gbf64272e.el9.x86_64 1/4 2024-05-09T06:25:26.743 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephsqlite-2:18.2.2-1306.gbf64272e.el9.x86_64 2/4 2024-05-09T06:25:26.743 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-09T06:25:26.981 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-09T06:25:26.981 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:26.981 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:25:26.981 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:26.981 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:26.981 INFO:teuthology.orchestra.run.smithi063.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T06:25:26.981 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T06:25:26.981 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:26.981 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:25:27.270 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph 2024-05-09T06:25:27.707 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1306.gbf64272e.el9.x86_64 3/4 2024-05-09T06:25:27.786 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:18.2.2-1306.gbf64272e.el9.x86_64 4/4 2024-05-09T06:25:27.852 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:47 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:25:27.981 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:25:27.984 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:27.984 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-09T06:25:27.984 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:27.984 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:25:27.984 INFO:teuthology.orchestra.run.smithi063.stdout: ceph x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 6.5 k 2024-05-09T06:25:27.984 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-09T06:25:27.984 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 2.1 M 2024-05-09T06:25:27.984 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 1.5 M 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 246 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 4.7 M 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 17 M 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T06:25:27.985 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T06:25:27.986 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T06:25:27.987 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T06:25:27.987 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T06:25:27.987 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:27.987 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:25:27.987 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:27.987 INFO:teuthology.orchestra.run.smithi063.stdout:Install 33 Packages 2024-05-09T06:25:27.987 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:27.988 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 29 M 2024-05-09T06:25:27.988 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 104 M 2024-05-09T06:25:27.988 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:25:28.820 INFO:teuthology.orchestra.run.smithi063.stdout:(1/33): ceph-18.2.2-1306.gbf64272e.el9.x86_64.r 20 kB/s | 6.5 kB 00:00 2024-05-09T06:25:29.355 INFO:teuthology.orchestra.run.smithi063.stdout:(2/33): ceph-mgr-18.2.2-1306.gbf64272e.el9.x86_ 1.7 MB/s | 1.5 MB 00:00 2024-05-09T06:25:29.655 INFO:teuthology.orchestra.run.smithi063.stdout:(3/33): ceph-mds-18.2.2-1306.gbf64272e.el9.x86_ 1.8 MB/s | 2.1 MB 00:01 2024-05-09T06:25:29.747 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-test-2:18.2.2-1306.gbf64272e.el9.x86_64 4/4 2024-05-09T06:25:29.747 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:18.2.2-1306.gbf64272e.el9.x86_64 1/4 2024-05-09T06:25:29.748 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:18.2.2-1306.gbf64272e.el9.x86_64 2/4 2024-05-09T06:25:29.748 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-09T06:25:29.780 INFO:teuthology.orchestra.run.smithi063.stdout:(4/33): ceph-mgr-modules-core-18.2.2-1306.gbf64 1.9 MB/s | 246 kB 00:00 2024-05-09T06:25:29.855 INFO:teuthology.orchestra.run.smithi063.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-05-09T06:25:29.964 INFO:teuthology.orchestra.run.smithi063.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 11 MB/s | 1.2 MB 00:00 2024-05-09T06:25:30.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-09T06:25:30.031 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:30.031 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:25:30.031 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:30.031 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:30.031 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T06:25:30.031 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T06:25:30.031 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:30.031 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:25:30.047 INFO:teuthology.orchestra.run.smithi063.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-05-09T06:25:30.264 INFO:teuthology.orchestra.run.smithi063.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 623 kB/s | 135 kB 00:00 2024-05-09T06:25:30.322 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph 2024-05-09T06:25:30.331 INFO:teuthology.orchestra.run.smithi063.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 1.8 MB/s | 125 kB 00:00 2024-05-09T06:25:30.398 INFO:teuthology.orchestra.run.smithi063.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-05-09T06:25:30.714 INFO:teuthology.orchestra.run.smithi063.stdout:(11/33): python3-mako-1.1.4-6.el9.noarch.rpm 544 kB/s | 172 kB 00:00 2024-05-09T06:25:30.873 INFO:teuthology.orchestra.run.smithi063.stdout:(12/33): ceph-osd-18.2.2-1306.gbf64272e.el9.x86 11 MB/s | 17 MB 00:01 2024-05-09T06:25:30.885 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:45 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:25:30.898 INFO:teuthology.orchestra.run.smithi063.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 189 kB/s | 35 kB 00:00 2024-05-09T06:25:30.982 INFO:teuthology.orchestra.run.smithi063.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-05-09T06:25:31.009 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:25:31.011 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:31.011 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-09T06:25:31.011 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:31.011 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:25:31.011 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 6.5 k 2024-05-09T06:25:31.011 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-09T06:25:31.011 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 2.1 M 2024-05-09T06:25:31.011 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 1.5 M 2024-05-09T06:25:31.011 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 246 k 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 4.7 M 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 17 M 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T06:25:31.012 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T06:25:31.013 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout:Install 33 Packages 2024-05-09T06:25:31.014 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:31.015 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 29 M 2024-05-09T06:25:31.015 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 104 M 2024-05-09T06:25:31.015 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:25:31.115 INFO:teuthology.orchestra.run.smithi063.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 211 kB/s | 51 kB 00:00 2024-05-09T06:25:31.199 INFO:teuthology.orchestra.run.smithi063.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 200 kB/s | 43 kB 00:00 2024-05-09T06:25:31.374 INFO:teuthology.orchestra.run.smithi063.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.0 MB/s | 358 kB 00:00 2024-05-09T06:25:31.433 INFO:teuthology.orchestra.run.smithi063.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 541 kB/s | 172 kB 00:00 2024-05-09T06:25:31.449 INFO:teuthology.orchestra.run.smithi063.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 142 kB/s | 11 kB 00:00 2024-05-09T06:25:31.500 INFO:teuthology.orchestra.run.smithi063.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 355 kB/s | 18 kB 00:00 2024-05-09T06:25:31.517 INFO:teuthology.orchestra.run.smithi063.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 347 kB/s | 23 kB 00:00 2024-05-09T06:25:31.558 INFO:teuthology.orchestra.run.smithi063.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 471 kB/s | 19 kB 00:00 2024-05-09T06:25:31.575 INFO:teuthology.orchestra.run.smithi063.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 338 kB/s | 20 kB 00:00 2024-05-09T06:25:31.601 INFO:teuthology.orchestra.run.smithi063.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-05-09T06:25:31.643 INFO:teuthology.orchestra.run.smithi063.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2024-05-09T06:25:31.701 INFO:teuthology.orchestra.run.smithi063.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-09T06:25:31.718 INFO:teuthology.orchestra.run.smithi063.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 219 kB/s | 16 kB 00:00 2024-05-09T06:25:31.729 INFO:teuthology.orchestra.run.smithi193.stdout:(1/33): ceph-18.2.2-1306.gbf64272e.el9.x86_64.r 39 kB/s | 6.5 kB 00:00 2024-05-09T06:25:31.769 INFO:teuthology.orchestra.run.smithi063.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-05-09T06:25:31.786 INFO:teuthology.orchestra.run.smithi063.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 534 kB/s | 36 kB 00:00 2024-05-09T06:25:31.869 INFO:teuthology.orchestra.run.smithi063.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-09T06:25:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-05-09T06:25:31.929 INFO:teuthology.orchestra.run.smithi193.stdout:(2/33): ceph-mds-18.2.2-1306.gbf64272e.el9.x86_ 5.7 MB/s | 2.1 MB 00:00 2024-05-09T06:25:32.028 INFO:teuthology.orchestra.run.smithi063.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.7 MB/s | 427 kB 00:00 2024-05-09T06:25:32.513 INFO:teuthology.orchestra.run.smithi193.stdout:(3/33): ceph-mon-18.2.2-1306.gbf64272e.el9.x86_ 6.0 MB/s | 4.7 MB 00:00 2024-05-09T06:25:32.588 INFO:teuthology.orchestra.run.smithi193.stdout:(4/33): ceph-mgr-modules-core-18.2.2-1306.gbf64 3.2 MB/s | 246 kB 00:00 2024-05-09T06:25:32.697 INFO:teuthology.orchestra.run.smithi193.stdout:(5/33): ceph-mgr-18.2.2-1306.gbf64272e.el9.x86_ 1.3 MB/s | 1.5 MB 00:01 2024-05-09T06:25:32.881 INFO:teuthology.orchestra.run.smithi193.stdout:(6/33): ceph-osd-18.2.2-1306.gbf64272e.el9.x86_ 17 MB/s | 17 MB 00:00 2024-05-09T06:25:32.905 INFO:teuthology.orchestra.run.smithi193.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 799 kB/s | 253 kB 00:00 2024-05-09T06:25:32.956 INFO:teuthology.orchestra.run.smithi193.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-05-09T06:25:33.006 INFO:teuthology.orchestra.run.smithi193.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-05-09T06:25:33.032 INFO:teuthology.orchestra.run.smithi193.stdout:(10/33): python3-ply-3.11-14.el9.noarch.rpm 708 kB/s | 106 kB 00:00 2024-05-09T06:25:33.056 INFO:teuthology.orchestra.run.smithi193.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-05-09T06:25:33.090 INFO:teuthology.orchestra.run.smithi193.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 3.1 MB/s | 1.2 MB 00:00 2024-05-09T06:25:33.199 INFO:teuthology.orchestra.run.smithi193.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 244 kB/s | 35 kB 00:00 2024-05-09T06:25:33.233 INFO:teuthology.orchestra.run.smithi193.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 358 kB/s | 51 kB 00:00 2024-05-09T06:25:33.275 INFO:teuthology.orchestra.run.smithi193.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 709 kB/s | 172 kB 00:00 2024-05-09T06:25:33.325 INFO:teuthology.orchestra.run.smithi193.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 331 kB/s | 42 kB 00:00 2024-05-09T06:25:33.500 INFO:teuthology.orchestra.run.smithi193.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 163 kB/s | 43 kB 00:00 2024-05-09T06:25:33.575 INFO:teuthology.orchestra.run.smithi193.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 625 kB/s | 172 kB 00:00 2024-05-09T06:25:33.578 INFO:teuthology.orchestra.run.smithi063.stdout:(33/33): ceph-mon-18.2.2-1306.gbf64272e.el9.x86 1.0 MB/s | 4.7 MB 00:04 2024-05-09T06:25:33.583 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:33.583 INFO:teuthology.orchestra.run.smithi063.stdout:Total 5.2 MB/s | 29 MB 00:05 2024-05-09T06:25:33.709 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:25:33.725 INFO:teuthology.orchestra.run.smithi193.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 118 kB/s | 18 kB 00:00 2024-05-09T06:25:33.750 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:25:33.751 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:25:33.777 INFO:teuthology.orchestra.run.smithi193.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 39 kB/s | 11 kB 00:00 2024-05-09T06:25:33.801 INFO:teuthology.orchestra.run.smithi193.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 308 kB/s | 23 kB 00:00 2024-05-09T06:25:33.834 INFO:teuthology.orchestra.run.smithi193.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 704 kB/s | 358 kB 00:00 2024-05-09T06:25:33.860 INFO:teuthology.orchestra.run.smithi193.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 235 kB/s | 19 kB 00:00 2024-05-09T06:25:33.893 INFO:teuthology.orchestra.run.smithi193.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-05-09T06:25:33.918 INFO:teuthology.orchestra.run.smithi193.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 553 kB/s | 46 kB 00:00 2024-05-09T06:25:33.944 INFO:teuthology.orchestra.run.smithi193.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 941 kB/s | 79 kB 00:00 2024-05-09T06:25:33.994 INFO:teuthology.orchestra.run.smithi193.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 218 kB/s | 16 kB 00:00 2024-05-09T06:25:34.011 INFO:teuthology.orchestra.run.smithi193.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-05-09T06:25:34.053 INFO:teuthology.orchestra.run.smithi193.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 612 kB/s | 36 kB 00:00 2024-05-09T06:25:34.078 INFO:teuthology.orchestra.run.smithi193.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-05-09T06:25:34.112 INFO:teuthology.orchestra.run.smithi193.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-09T06:25:34.137 INFO:teuthology.orchestra.run.smithi193.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 343 kB/s | 20 kB 00:00 2024-05-09T06:25:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:25:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:25:34.162 INFO:teuthology.orchestra.run.smithi193.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-05-09T06:25:34.166 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:34.167 INFO:teuthology.orchestra.run.smithi193.stdout:Total 9.2 MB/s | 29 MB 00:03 2024-05-09T06:25:34.289 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:25:34.326 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:25:34.326 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:25:34.697 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:25:34.697 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:25:35.304 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:25:35.364 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-09T06:25:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-09T06:25:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-09T06:25:35.526 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-09T06:25:35.603 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-09T06:25:35.649 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-09T06:25:35.692 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-09T06:25:35.721 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-09T06:25:35.756 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-09T06:25:35.789 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-09T06:25:35.830 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-09T06:25:35.850 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:25:35.874 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-09T06:25:35.898 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-09T06:25:35.908 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-09T06:25:35.938 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-09T06:25:35.955 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-09T06:25:35.972 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-09T06:25:36.040 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-09T06:25:36.062 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-09T06:25:36.150 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-09T06:25:36.200 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-09T06:25:36.200 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-09T06:25:36.235 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-09T06:25:36.264 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-09T06:25:36.290 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-09T06:25:36.299 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-09T06:25:36.332 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-09T06:25:36.349 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-09T06:25:36.372 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-09T06:25:36.387 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-09T06:25:36.407 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-09T06:25:36.442 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-09T06:25:36.446 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-09T06:25:36.488 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-09T06:25:36.509 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-09T06:25:36.561 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-09T06:25:36.699 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-09T06:25:36.706 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-09T06:25:36.778 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-09T06:25:36.816 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-09T06:25:36.833 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-09T06:25:36.872 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-09T06:25:36.897 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-09T06:25:36.920 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-09T06:25:36.976 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-09T06:25:37.119 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-09T06:25:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-09T06:25:37.174 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-09T06:25:37.208 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-09T06:25:37.278 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-09T06:25:37.297 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-09T06:25:37.366 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-09T06:25:37.406 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-09T06:25:37.522 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-09T06:25:37.579 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1306.gbf64272e.el9. 28/33 2024-05-09T06:25:37.616 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 29/33 2024-05-09T06:25:37.643 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 29/33 2024-05-09T06:25:37.643 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:37.643 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T06:25:37.643 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:25:37.644 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:25:37.644 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:37.682 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1306.gbf64272e.el9. 28/33 2024-05-09T06:25:37.718 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 29/33 2024-05-09T06:25:37.742 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 29/33 2024-05-09T06:25:37.743 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:37.743 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T06:25:37.743 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:25:37.743 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:25:37.743 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-osd-2:18.2.2-1306.gbf64272e.el9.x86_64 30/33 2024-05-09T06:25:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-osd-2:18.2.2-1306.gbf64272e.el9.x86_64 30/33 2024-05-09T06:25:39.182 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:39.183 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T06:25:39.183 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:25:39.183 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:25:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:18.2.2-1306.gbf64272e.el9.x86_64 30/33 2024-05-09T06:25:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:18.2.2-1306.gbf64272e.el9.x86_64 30/33 2024-05-09T06:25:39.225 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:39.225 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T06:25:39.225 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:25:39.225 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:25:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mon-2:18.2.2-1306.gbf64272e.el9.x86_64 31/33 2024-05-09T06:25:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mon-2:18.2.2-1306.gbf64272e.el9.x86_64 31/33 2024-05-09T06:25:39.677 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:39.677 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T06:25:39.677 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:25:39.677 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:25:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:39.678 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:18.2.2-1306.gbf64272e.el9.x86_64 31/33 2024-05-09T06:25:39.703 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:18.2.2-1306.gbf64272e.el9.x86_64 31/33 2024-05-09T06:25:39.703 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:39.703 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T06:25:39.703 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:25:39.703 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:25:39.703 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mds-2:18.2.2-1306.gbf64272e.el9.x86_64 32/33 2024-05-09T06:25:39.904 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:18.2.2-1306.gbf64272e.el9.x86_64 32/33 2024-05-09T06:25:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mds-2:18.2.2-1306.gbf64272e.el9.x86_64 32/33 2024-05-09T06:25:39.919 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:39.919 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T06:25:39.920 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:25:39.920 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:25:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:39.928 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:18.2.2-1306.gbf64272e.el9.x86_64 32/33 2024-05-09T06:25:39.928 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:39.928 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T06:25:39.928 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:25:39.928 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:25:39.928 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-2:18.2.2-1306.gbf64272e.el9.x86_64 33/33 2024-05-09T06:25:39.993 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:18.2.2-1306.gbf64272e.el9.x86_64 33/33 2024-05-09T06:25:41.663 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-2:18.2.2-1306.gbf64272e.el9.x86_64 33/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-2:18.2.2-1306.gbf64272e.el9.x86_64 1/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mds-2:18.2.2-1306.gbf64272e.el9.x86_64 2/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 3/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mon-2:18.2.2-1306.gbf64272e.el9.x86_64 4/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-osd-2:18.2.2-1306.gbf64272e.el9.x86_64 5/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1306.gbf64272e.el9. 6/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-09T06:25:41.664 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-09T06:25:41.665 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-09T06:25:41.666 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-09T06:25:41.666 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-09T06:25:41.666 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-09T06:25:41.666 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-09T06:25:41.666 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-09T06:25:41.667 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-09T06:25:41.667 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-09T06:25:41.845 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-2:18.2.2-1306.gbf64272e.el9.x86_64 33/33 2024-05-09T06:25:41.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:18.2.2-1306.gbf64272e.el9.x86_64 1/33 2024-05-09T06:25:41.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:18.2.2-1306.gbf64272e.el9.x86_64 2/33 2024-05-09T06:25:41.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 3/33 2024-05-09T06:25:41.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:18.2.2-1306.gbf64272e.el9.x86_64 4/33 2024-05-09T06:25:41.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:18.2.2-1306.gbf64272e.el9.x86_64 5/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1306.gbf64272e.el9. 6/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-09T06:25:41.846 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-09T06:25:41.847 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.639 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T06:25:42.640 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T06:25:42.641 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T06:25:42.641 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T06:25:42.641 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T06:25:42.641 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T06:25:42.641 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T06:25:42.641 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T06:25:42.641 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T06:25:42.641 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T06:25:42.641 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T06:25:42.642 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T06:25:42.642 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T06:25:42.642 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T06:25:42.642 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T06:25:42.642 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T06:25:42.642 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T06:25:42.642 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T06:25:42.642 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:42.642 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:25:42.873 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-09T06:25:42.873 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:42.873 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:25:42.873 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.873 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.873 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.873 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T06:25:42.874 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T06:25:42.875 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T06:25:42.876 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T06:25:42.876 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T06:25:42.876 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:42.876 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:25:42.979 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-base 2024-05-09T06:25:43.191 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-base 2024-05-09T06:25:43.571 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:03 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:25:43.655 INFO:teuthology.orchestra.run.smithi063.stdout:Package ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:25:43.698 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:25:43.699 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-09T06:25:43.700 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:25:43.759 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:58 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:25:43.777 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install cephadm 2024-05-09T06:25:43.840 INFO:teuthology.orchestra.run.smithi193.stdout:Package ceph-base-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:25:43.881 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:25:43.882 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-05-09T06:25:43.882 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:25:43.962 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install cephadm 2024-05-09T06:25:44.359 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:04 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:25:44.488 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:25:44.489 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:44.489 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-09T06:25:44.489 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:44.489 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:25:44.489 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 224 k 2024-05-09T06:25:44.489 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:44.490 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:25:44.490 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:44.490 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-05-09T06:25:44.490 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:44.490 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 224 k 2024-05-09T06:25:44.490 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 216 k 2024-05-09T06:25:44.490 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:25:44.526 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:59 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:25:44.650 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:25:44.650 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 224 k 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 224 k 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 216 k 2024-05-09T06:25:44.651 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:25:44.719 INFO:teuthology.orchestra.run.smithi063.stdout:cephadm-18.2.2-1306.gbf64272e.el9.noarch.rpm 982 kB/s | 224 kB 00:00 2024-05-09T06:25:44.719 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:44.719 INFO:teuthology.orchestra.run.smithi063.stdout:Total 975 kB/s | 224 kB 00:00 2024-05-09T06:25:44.720 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:25:44.725 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:25:44.725 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:25:44.728 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:25:44.728 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:25:44.800 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:25:44.920 INFO:teuthology.orchestra.run.smithi193.stdout:cephadm-18.2.2-1306.gbf64272e.el9.noarch.rpm 836 kB/s | 224 kB 00:00 2024-05-09T06:25:44.920 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:44.920 INFO:teuthology.orchestra.run.smithi193.stdout:Total 832 kB/s | 224 kB 00:00 2024-05-09T06:25:44.921 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:25:44.925 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:25:44.926 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:25:44.928 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:25:44.929 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:25:45.002 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:25:45.242 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/1 2024-05-09T06:25:45.333 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/1 2024-05-09T06:25:45.484 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/1 2024-05-09T06:25:45.574 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/1 2024-05-09T06:25:45.672 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/1 2024-05-09T06:25:45.902 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/1 2024-05-09T06:25:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/1 2024-05-09T06:25:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:45.903 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:25:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:45.903 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:45.903 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:25:46.132 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-immutable-object-cache 2024-05-09T06:25:46.165 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/1 2024-05-09T06:25:46.165 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:46.166 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:25:46.166 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:46.166 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:46.166 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:25:46.377 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-immutable-object-cache 2024-05-09T06:25:46.723 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:06 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:25:46.851 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:25:46.852 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:46.852 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-05-09T06:25:46.852 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:46.852 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:25:46.852 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 143 k 2024-05-09T06:25:46.853 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:46.853 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:25:46.853 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:46.853 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-05-09T06:25:46.853 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:46.853 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 143 k 2024-05-09T06:25:46.853 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 395 k 2024-05-09T06:25:46.853 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:25:46.948 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:01 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:25:47.073 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:25:47.074 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:47.074 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-05-09T06:25:47.074 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:47.074 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:25:47.075 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 143 k 2024-05-09T06:25:47.075 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:47.075 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:25:47.075 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:47.075 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-05-09T06:25:47.075 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:47.075 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 143 k 2024-05-09T06:25:47.075 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 395 k 2024-05-09T06:25:47.075 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:25:47.357 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-immutable-object-cache-18.2.2-1306.gbf6427 284 kB/s | 143 kB 00:00 2024-05-09T06:25:47.358 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:47.358 INFO:teuthology.orchestra.run.smithi063.stdout:Total 283 kB/s | 143 kB 00:00 2024-05-09T06:25:47.358 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:25:47.367 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:25:47.367 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:25:47.428 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:25:47.428 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:25:47.493 INFO:teuthology.orchestra.run.smithi193.stdout:ceph-immutable-object-cache-18.2.2-1306.gbf6427 343 kB/s | 143 kB 00:00 2024-05-09T06:25:47.493 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:47.493 INFO:teuthology.orchestra.run.smithi193.stdout:Total 341 kB/s | 143 kB 00:00 2024-05-09T06:25:47.494 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:25:47.501 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:25:47.501 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:25:47.558 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:25:47.558 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:25:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:25:47.658 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1306.gbf64272e. 1/1 2024-05-09T06:25:47.685 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1306.gbf64272e. 1/1 2024-05-09T06:25:47.686 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:47.686 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T06:25:47.686 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:47.696 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:25:47.744 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1306.gbf64272e. 1/1 2024-05-09T06:25:47.770 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1306.gbf64272e. 1/1 2024-05-09T06:25:47.770 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:25:47.770 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T06:25:47.770 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:48.624 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1306.gbf64272e. 1/1 2024-05-09T06:25:48.624 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:48.624 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:25:48.624 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:48.624 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:48.624 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:25:48.708 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1306.gbf64272e. 1/1 2024-05-09T06:25:48.708 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:48.708 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:25:48.709 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:25:48.709 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:48.709 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:25:48.853 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr 2024-05-09T06:25:48.939 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr 2024-05-09T06:25:49.445 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:09 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:25:49.523 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:04 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:25:49.529 INFO:teuthology.orchestra.run.smithi063.stdout:Package ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:25:49.574 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:25:49.575 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-09T06:25:49.575 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:25:49.605 INFO:teuthology.orchestra.run.smithi193.stdout:Package ceph-mgr-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:25:49.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-dashboard 2024-05-09T06:25:49.647 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:25:49.648 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-05-09T06:25:49.648 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:25:49.742 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-dashboard 2024-05-09T06:25:50.231 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:10 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:25:50.306 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:05 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:25:50.359 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:25:50.359 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:50.359 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-09T06:25:50.359 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:50.359 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:25:50.359 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 3.5 M 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 24 k 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 16 k 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout:Install 5 Packages 2024-05-09T06:25:50.360 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:50.361 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 3.8 M 2024-05-09T06:25:50.361 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 87 M 2024-05-09T06:25:50.361 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:25:50.430 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:25:50.431 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:50.431 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-09T06:25:50.431 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:50.431 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:25:50.431 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 3.5 M 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 24 k 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 16 k 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout:Install 5 Packages 2024-05-09T06:25:50.432 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:50.433 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 3.8 M 2024-05-09T06:25:50.433 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 87 M 2024-05-09T06:25:50.433 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:25:50.647 INFO:teuthology.orchestra.run.smithi063.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1306.gbf64 80 kB/s | 16 kB 00:00 2024-05-09T06:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout:(2/5): ceph-grafana-dashboards-18.2.2-1306.gbf6 82 kB/s | 24 kB 00:00 2024-05-09T06:25:50.939 INFO:teuthology.orchestra.run.smithi063.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 105 kB/s | 31 kB 00:00 2024-05-09T06:25:51.096 INFO:teuthology.orchestra.run.smithi193.stdout:(1/5): ceph-grafana-dashboards-18.2.2-1306.gbf6 79 kB/s | 24 kB 00:00 2024-05-09T06:25:51.146 INFO:teuthology.orchestra.run.smithi193.stdout:(2/5): ceph-prometheus-alerts-18.2.2-1306.gbf64 46 kB/s | 16 kB 00:00 2024-05-09T06:25:51.198 INFO:teuthology.orchestra.run.smithi063.stdout:(4/5): ceph-mgr-dashboard-18.2.2-1306.gbf64272e 4.7 MB/s | 3.5 MB 00:00 2024-05-09T06:25:51.222 INFO:teuthology.orchestra.run.smithi063.stdout:(5/5): python3-routes-2.5.1-5.el9.noarch.rpm 390 kB/s | 188 kB 00:00 2024-05-09T06:25:51.226 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:51.226 INFO:teuthology.orchestra.run.smithi063.stdout:Total 4.4 MB/s | 3.8 MB 00:00 2024-05-09T06:25:51.226 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:25:51.234 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:25:51.234 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:25:51.246 INFO:teuthology.orchestra.run.smithi193.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 205 kB/s | 31 kB 00:00 2024-05-09T06:25:51.300 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:25:51.300 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:25:51.363 INFO:teuthology.orchestra.run.smithi193.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 870 kB/s | 188 kB 00:00 2024-05-09T06:25:51.557 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:25:51.639 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-05-09T06:25:51.678 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-05-09T06:25:51.707 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1306.gbf64272e.el9.n 3/5 2024-05-09T06:25:51.772 INFO:teuthology.orchestra.run.smithi193.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1306.gbf64272e 3.6 MB/s | 3.5 MB 00:00 2024-05-09T06:25:51.774 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:51.775 INFO:teuthology.orchestra.run.smithi193.stdout:Total 2.8 MB/s | 3.8 MB 00:01 2024-05-09T06:25:51.775 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:25:51.782 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:25:51.782 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:25:51.843 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:25:51.844 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:25:52.095 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:25:52.178 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-05-09T06:25:52.219 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-05-09T06:25:52.247 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1306.gbf64272e.el9.n 3/5 2024-05-09T06:25:52.814 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1306.gbf64272e.el9. 4/5 2024-05-09T06:25:52.860 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1306.gbf64272e.el9.noarc 5/5 2024-05-09T06:25:53.298 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1306.gbf64272e.el9. 4/5 2024-05-09T06:25:53.342 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1306.gbf64272e.el9.noarc 5/5 2024-05-09T06:25:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1306.gbf64272e.el9.noarc 5/5 2024-05-09T06:25:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1306.gbf64272e.el9. 1/5 2024-05-09T06:25:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1306.gbf64272e.el9.noarc 2/5 2024-05-09T06:25:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1306.gbf64272e.el9.n 3/5 2024-05-09T06:25:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-05-09T06:25:54.142 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-05-09T06:25:54.142 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:54.142 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:25:54.142 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:54.142 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:54.143 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:54.143 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T06:25:54.143 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T06:25:54.143 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:54.143 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:25:54.352 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-09T06:25:54.468 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1306.gbf64272e.el9.noarc 5/5 2024-05-09T06:25:54.468 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1306.gbf64272e.el9. 1/5 2024-05-09T06:25:54.468 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1306.gbf64272e.el9.noarc 2/5 2024-05-09T06:25:54.468 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1306.gbf64272e.el9.n 3/5 2024-05-09T06:25:54.468 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:54.724 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:25:54.939 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:14 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:25:54.957 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-09T06:25:55.068 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 7.4 M 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T06:25:55.070 INFO:teuthology.orchestra.run.smithi063.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout:Install 12 Packages 2024-05-09T06:25:55.071 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:25:55.072 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 38 M 2024-05-09T06:25:55.072 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 209 M 2024-05-09T06:25:55.072 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:25:55.523 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:10 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:25:55.648 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:25:55.649 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-05-09T06:25:55.649 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-09T06:25:55.649 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-05-09T06:25:55.649 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:25:55.649 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 7.4 M 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T06:25:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:55.651 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:25:55.651 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-05-09T06:25:55.651 INFO:teuthology.orchestra.run.smithi193.stdout:Install 12 Packages 2024-05-09T06:25:55.651 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:25:55.651 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 38 M 2024-05-09T06:25:55.651 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 209 M 2024-05-09T06:25:55.651 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:25:55.797 INFO:teuthology.orchestra.run.smithi063.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 328 kB/s | 195 kB 00:00 2024-05-09T06:25:55.847 INFO:teuthology.orchestra.run.smithi063.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-05-09T06:25:56.122 INFO:teuthology.orchestra.run.smithi063.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-05-09T06:25:56.164 INFO:teuthology.orchestra.run.smithi063.stdout:(4/12): libgfortran-11.4.1-3.el9.x86_64.rpm 835 kB/s | 803 kB 00:00 2024-05-09T06:25:56.190 INFO:teuthology.orchestra.run.smithi063.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 272 kB/s | 18 kB 00:00 2024-05-09T06:25:56.223 INFO:teuthology.orchestra.run.smithi063.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 604 kB/s | 35 kB 00:00 2024-05-09T06:25:56.307 INFO:teuthology.orchestra.run.smithi063.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-09T06:25:56.487 INFO:teuthology.orchestra.run.smithi193.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 519 kB/s | 195 kB 00:00 2024-05-09T06:25:56.524 INFO:teuthology.orchestra.run.smithi063.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2024-05-09T06:25:56.607 INFO:teuthology.orchestra.run.smithi063.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-05-09T06:25:56.699 INFO:teuthology.orchestra.run.smithi063.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-09T06:25:56.722 INFO:teuthology.orchestra.run.smithi193.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-05-09T06:25:56.846 INFO:teuthology.orchestra.run.smithi193.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 91 kB/s | 33 kB 00:00 2024-05-09T06:25:56.971 INFO:teuthology.orchestra.run.smithi193.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 146 kB/s | 18 kB 00:00 2024-05-09T06:25:57.196 INFO:teuthology.orchestra.run.smithi193.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 156 kB/s | 35 kB 00:00 2024-05-09T06:25:57.499 INFO:teuthology.orchestra.run.smithi063.stdout:(11/12): ceph-mgr-diskprediction-local-18.2.2-1 3.2 MB/s | 7.4 MB 00:02 2024-05-09T06:25:58.063 INFO:teuthology.orchestra.run.smithi193.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.2 MB/s | 3.0 MB 00:01 2024-05-09T06:25:58.213 INFO:teuthology.orchestra.run.smithi193.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-05-09T06:25:58.805 INFO:teuthology.orchestra.run.smithi193.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 2.9 MB/s | 4.7 MB 00:01 2024-05-09T06:25:58.922 INFO:teuthology.orchestra.run.smithi193.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-05-09T06:25:58.966 INFO:teuthology.orchestra.run.smithi063.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.9 MB/s | 16 MB 00:02 2024-05-09T06:25:58.969 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:58.970 INFO:teuthology.orchestra.run.smithi063.stdout:Total 9.8 MB/s | 38 MB 00:03 2024-05-09T06:25:59.339 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:25:59.367 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:25:59.367 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:25:59.908 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:25:59.908 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:26:00.214 INFO:teuthology.orchestra.run.smithi193.stdout:(10/12): ceph-mgr-diskprediction-local-18.2.2-1 1.8 MB/s | 7.4 MB 00:04 2024-05-09T06:26:00.762 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:26:00.821 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-09T06:26:00.885 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-09T06:26:00.986 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-09T06:26:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-09T06:26:01.211 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-09T06:26:01.241 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-09T06:26:01.320 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-09T06:26:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-09T06:26:02.151 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-09T06:26:03.239 INFO:teuthology.orchestra.run.smithi193.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.0 MB/s | 5.1 MB 00:05 2024-05-09T06:26:03.503 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-09T06:26:04.588 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-09T06:26:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1306.gbf642 12/12 2024-05-09T06:26:07.065 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1306.gbf642 12/12 2024-05-09T06:26:07.065 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1306.gbf642 1/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-09T06:26:07.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-09T06:26:07.449 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-09T06:26:07.449 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:07.449 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:26:07.449 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:26:07.449 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T06:26:07.449 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T06:26:07.449 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T06:26:07.449 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T06:26:07.450 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T06:26:07.450 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T06:26:07.450 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T06:26:07.450 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T06:26:07.450 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T06:26:07.450 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T06:26:07.450 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T06:26:07.450 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:07.450 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:07.803 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-rook 2024-05-09T06:26:08.390 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:28 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:08.516 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:08.517 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:08.517 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-09T06:26:08.517 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:08.517 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:26:08.517 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 49 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T06:26:08.518 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T06:26:08.519 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:26:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Install 13 Packages 2024-05-09T06:26:08.519 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 2.2 M 2024-05-09T06:26:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 26 M 2024-05-09T06:26:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:26:08.699 INFO:teuthology.orchestra.run.smithi063.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 279 kB/s | 26 kB 00:00 2024-05-09T06:26:08.716 INFO:teuthology.orchestra.run.smithi063.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 177 kB/s | 19 kB 00:00 2024-05-09T06:26:08.825 INFO:teuthology.orchestra.run.smithi063.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-05-09T06:26:08.850 INFO:teuthology.orchestra.run.smithi063.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-09T06:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout:(5/13): ceph-mgr-rook-18.2.2-1306.gbf64272e.el9 173 kB/s | 49 kB 00:00 2024-05-09T06:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 640 kB/s | 54 kB 00:00 2024-05-09T06:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.0 MB/s | 279 kB 00:00 2024-05-09T06:26:09.193 INFO:teuthology.orchestra.run.smithi063.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 58 kB/s | 15 kB 00:00 2024-05-09T06:26:09.210 INFO:teuthology.orchestra.run.smithi063.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 101 kB/s | 32 kB 00:00 2024-05-09T06:26:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 417 kB/s | 59 kB 00:00 2024-05-09T06:26:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 493 kB/s | 206 kB 00:00 2024-05-09T06:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 825 kB/s | 90 kB 00:00 2024-05-09T06:26:09.494 INFO:teuthology.orchestra.run.smithi063.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.4 MB/s | 1.0 MB 00:00 2024-05-09T06:26:09.497 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:09.497 INFO:teuthology.orchestra.run.smithi063.stdout:Total 2.2 MB/s | 2.2 MB 00:00 2024-05-09T06:26:09.565 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:26:09.581 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:26:09.581 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:26:09.678 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:26:09.678 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:26:10.204 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:26:10.391 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-09T06:26:10.447 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-09T06:26:10.504 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-09T06:26:10.539 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-09T06:26:10.582 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-09T06:26:10.679 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-09T06:26:10.798 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-09T06:26:10.849 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-09T06:26:11.146 INFO:teuthology.orchestra.run.smithi193.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.3 MB/s | 16 MB 00:12 2024-05-09T06:26:11.148 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:11.148 INFO:teuthology.orchestra.run.smithi193.stdout:Total 2.5 MB/s | 38 MB 00:15 2024-05-09T06:26:11.506 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:26:11.533 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:26:11.533 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:26:11.607 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-09T06:26:11.654 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-09T06:26:11.688 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-09T06:26:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-09T06:26:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-rook-2:18.2.2-1306.gbf64272e.el9.noarch 13/13 2024-05-09T06:26:12.046 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:26:12.046 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:26:12.560 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1306.gbf64272e.el9.noarch 13/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1306.gbf64272e.el9.noarch 1/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-09T06:26:12.561 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-09T06:26:12.562 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-09T06:26:12.873 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-09T06:26:12.873 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:12.873 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T06:26:12.874 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:12.875 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:12.894 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:26:12.968 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-09T06:26:13.018 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-09T06:26:13.093 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-cephadm 2024-05-09T06:26:13.121 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-09T06:26:13.162 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-09T06:26:13.339 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-09T06:26:13.369 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-09T06:26:13.447 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-09T06:26:13.685 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:33 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:13.814 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:13.815 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:13.815 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-09T06:26:13.815 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:13.815 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:26:13.815 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 139 k 2024-05-09T06:26:13.815 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-09T06:26:13.815 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T06:26:13.815 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T06:26:13.815 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T06:26:13.816 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T06:26:13.816 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T06:26:13.816 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:13.816 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:26:13.816 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:13.816 INFO:teuthology.orchestra.run.smithi063.stdout:Install 6 Packages 2024-05-09T06:26:13.816 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:13.816 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 7.0 M 2024-05-09T06:26:13.817 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 33 M 2024-05-09T06:26:13.817 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:26:14.109 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-09T06:26:14.234 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-09T06:26:14.380 INFO:teuthology.orchestra.run.smithi063.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-05-09T06:26:14.405 INFO:teuthology.orchestra.run.smithi063.stdout:(2/6): ceph-mgr-cephadm-18.2.2-1306.gbf64272e.e 543 kB/s | 139 kB 00:00 2024-05-09T06:26:14.522 INFO:teuthology.orchestra.run.smithi063.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 495 kB/s | 58 kB 00:00 2024-05-09T06:26:14.572 INFO:teuthology.orchestra.run.smithi063.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 1.0 MB/s | 51 kB 00:00 2024-05-09T06:26:14.614 INFO:teuthology.orchestra.run.smithi063.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.3 MB/s | 548 kB 00:00 2024-05-09T06:26:15.231 INFO:teuthology.orchestra.run.smithi063.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-05-09T06:26:15.232 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:15.233 INFO:teuthology.orchestra.run.smithi063.stdout:Total 4.9 MB/s | 7.0 MB 00:01 2024-05-09T06:26:15.306 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:26:15.318 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:26:15.318 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:26:15.422 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:26:15.422 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:26:15.479 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-09T06:26:15.735 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:26:15.915 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-09T06:26:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-09T06:26:16.515 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-09T06:26:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-09T06:26:16.546 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1306.gbf642 12/12 2024-05-09T06:26:16.624 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-09T06:26:16.692 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-09T06:26:16.720 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 6/6 2024-05-09T06:26:17.426 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 6/6 2024-05-09T06:26:17.427 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/6 2024-05-09T06:26:17.427 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-09T06:26:17.427 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-09T06:26:17.427 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-09T06:26:17.427 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:17.699 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:17.950 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-fuse 2024-05-09T06:26:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:38 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:18.670 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:18.671 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:18.671 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-09T06:26:18.671 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:18.671 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:26:18.671 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 848 k 2024-05-09T06:26:18.671 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-09T06:26:18.671 INFO:teuthology.orchestra.run.smithi063.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T06:26:18.671 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:18.671 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:26:18.672 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:18.672 INFO:teuthology.orchestra.run.smithi063.stdout:Install 2 Packages 2024-05-09T06:26:18.672 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:18.672 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 928 k 2024-05-09T06:26:18.672 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 2.7 M 2024-05-09T06:26:18.672 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:26:18.998 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1306.gbf642 12/12 2024-05-09T06:26:18.998 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1306.gbf642 1/12 2024-05-09T06:26:18.998 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-09T06:26:18.998 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-09T06:26:18.999 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-09T06:26:18.999 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-09T06:26:18.999 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-09T06:26:18.999 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-09T06:26:18.999 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-09T06:26:18.999 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-09T06:26:18.999 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-09T06:26:18.999 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-09T06:26:19.025 INFO:teuthology.orchestra.run.smithi063.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 80 kB 00:00 2024-05-09T06:26:19.401 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-09T06:26:19.401 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:19.401 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:26:19.401 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:26:19.401 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T06:26:19.402 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:19.403 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:19.408 INFO:teuthology.orchestra.run.smithi063.stdout:(2/2): ceph-fuse-18.2.2-1306.gbf64272e.el9.x86_ 1.8 MB/s | 848 kB 00:00 2024-05-09T06:26:19.409 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:19.410 INFO:teuthology.orchestra.run.smithi063.stdout:Total 1.2 MB/s | 928 kB 00:00 2024-05-09T06:26:19.422 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:26:19.446 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:26:19.446 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:26:19.528 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:26:19.528 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:26:19.717 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:26:19.757 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-rook 2024-05-09T06:26:19.861 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-09T06:26:19.935 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 2/2 2024-05-09T06:26:20.330 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:35 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:20.452 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:20.453 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:20.453 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-09T06:26:20.453 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:20.453 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:26:20.453 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 49 k 2024-05-09T06:26:20.453 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-09T06:26:20.453 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T06:26:20.453 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:20.454 INFO:teuthology.orchestra.run.smithi193.stdout:Install 13 Packages 2024-05-09T06:26:20.455 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:20.455 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 2.2 M 2024-05-09T06:26:20.455 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 26 M 2024-05-09T06:26:20.455 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:26:20.623 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 2/2 2024-05-09T06:26:20.623 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 1/2 2024-05-09T06:26:20.804 INFO:teuthology.orchestra.run.smithi193.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 151 kB/s | 19 kB 00:00 2024-05-09T06:26:20.829 INFO:teuthology.orchestra.run.smithi193.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 169 kB/s | 26 kB 00:00 2024-05-09T06:26:20.958 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-09T06:26:20.958 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:20.958 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:26:20.958 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-09T06:26:20.958 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:20.958 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:20.963 INFO:teuthology.orchestra.run.smithi193.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-09T06:26:20.997 INFO:teuthology.orchestra.run.smithi193.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.1 MB/s | 222 kB 00:00 2024-05-09T06:26:21.022 INFO:teuthology.orchestra.run.smithi193.stdout:(5/13): ceph-mgr-rook-18.2.2-1306.gbf64272e.el9 143 kB/s | 49 kB 00:00 2024-05-09T06:26:21.047 INFO:teuthology.orchestra.run.smithi193.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2024-05-09T06:26:21.106 INFO:teuthology.orchestra.run.smithi193.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 386 kB/s | 32 kB 00:00 2024-05-09T06:26:21.131 INFO:teuthology.orchestra.run.smithi193.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 179 kB/s | 15 kB 00:00 2024-05-09T06:26:21.181 INFO:teuthology.orchestra.run.smithi193.stdout:(9/13): python3-google-auth-2.29.0-1.el9.noarch 2.7 MB/s | 206 kB 00:00 2024-05-09T06:26:21.223 INFO:teuthology.orchestra.run.smithi193.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-05-09T06:26:21.249 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librados-devel 2024-05-09T06:26:21.274 INFO:teuthology.orchestra.run.smithi193.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-05-09T06:26:21.407 INFO:teuthology.orchestra.run.smithi193.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 3.7 MB/s | 1.0 MB 00:00 2024-05-09T06:26:21.841 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:41 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:21.969 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 127 k 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:21.970 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 127 k 2024-05-09T06:26:21.971 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 456 k 2024-05-09T06:26:21.971 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:26:22.189 INFO:teuthology.orchestra.run.smithi063.stdout:librados-devel-18.2.2-1306.gbf64272e.el9.x86_64 581 kB/s | 127 kB 00:00 2024-05-09T06:26:22.190 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:22.190 INFO:teuthology.orchestra.run.smithi063.stdout:Total 577 kB/s | 127 kB 00:00 2024-05-09T06:26:22.190 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:26:22.196 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:26:22.196 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:26:22.263 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:26:22.263 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:26:22.416 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:26:22.523 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librados-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:22.889 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:23.032 INFO:teuthology.orchestra.run.smithi193.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 26 kB/s | 54 kB 00:02 2024-05-09T06:26:23.033 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:23.033 INFO:teuthology.orchestra.run.smithi193.stdout:Total 874 kB/s | 2.2 MB 00:02 2024-05-09T06:26:23.105 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:26:23.120 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:26:23.120 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:23.206 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:26:23.208 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:26:23.439 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install libcephfs2 2024-05-09T06:26:23.712 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:26:23.881 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-09T06:26:23.933 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-09T06:26:23.988 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-09T06:26:24.026 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-09T06:26:24.040 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:44 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:24.067 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-09T06:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout:Package libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:24.161 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-09T06:26:24.168 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:24.169 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-09T06:26:24.169 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:24.242 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install libcephfs-devel 2024-05-09T06:26:24.270 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-09T06:26:24.328 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-09T06:26:24.831 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:44 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 31 k 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 31 k 2024-05-09T06:26:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 139 k 2024-05-09T06:26:24.962 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:26:25.050 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-09T06:26:25.099 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-09T06:26:25.133 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-09T06:26:25.181 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-09T06:26:25.202 INFO:teuthology.orchestra.run.smithi063.stdout:libcephfs-devel-18.2.2-1306.gbf64272e.el9.x86_6 129 kB/s | 31 kB 00:00 2024-05-09T06:26:25.203 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:25.203 INFO:teuthology.orchestra.run.smithi063.stdout:Total 128 kB/s | 31 kB 00:00 2024-05-09T06:26:25.203 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:26:25.203 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:18.2.2-1306.gbf64272e.el9.noarch 13/13 2024-05-09T06:26:25.207 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:26:25.207 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:26:25.230 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:26:25.230 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:26:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:26:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:25.817 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:25.996 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1306.gbf64272e.el9.noarch 13/13 2024-05-09T06:26:25.996 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1306.gbf64272e.el9.noarch 1/13 2024-05-09T06:26:25.996 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-09T06:26:25.996 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-09T06:26:25.996 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-09T06:26:25.996 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-09T06:26:25.997 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-09T06:26:25.997 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-09T06:26:25.997 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-09T06:26:25.997 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-09T06:26:25.997 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-09T06:26:25.997 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-09T06:26:25.997 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-09T06:26:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:26.055 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:26:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:26.055 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:26.251 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librados2 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T06:26:26.336 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T06:26:26.337 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T06:26:26.337 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T06:26:26.337 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T06:26:26.337 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T06:26:26.337 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T06:26:26.337 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:26.337 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:26.605 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-cephadm 2024-05-09T06:26:26.845 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:46 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:26.929 INFO:teuthology.orchestra.run.smithi063.stdout:Package librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:26.973 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:26.974 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-09T06:26:26.974 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:27.047 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librbd1 2024-05-09T06:26:27.175 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:42 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:27.296 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:27.297 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:27.297 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-09T06:26:27.297 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:27.297 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:26:27.297 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1306.gbf64272e.el9 ceph-noarch 139 k 2024-05-09T06:26:27.297 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-09T06:26:27.297 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T06:26:27.298 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T06:26:27.298 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T06:26:27.298 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T06:26:27.298 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T06:26:27.298 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:27.298 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:26:27.298 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:27.299 INFO:teuthology.orchestra.run.smithi193.stdout:Install 6 Packages 2024-05-09T06:26:27.299 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:27.300 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 7.0 M 2024-05-09T06:26:27.300 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 33 M 2024-05-09T06:26:27.300 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:26:27.636 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:47 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:27.719 INFO:teuthology.orchestra.run.smithi063.stdout:Package librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:27.725 INFO:teuthology.orchestra.run.smithi193.stdout:(1/6): ceph-mgr-cephadm-18.2.2-1306.gbf64272e.e 601 kB/s | 139 kB 00:00 2024-05-09T06:26:27.762 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:27.763 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-09T06:26:27.764 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:27.830 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rados 2024-05-09T06:26:28.334 INFO:teuthology.orchestra.run.smithi193.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 295 kB/s | 248 kB 00:00 2024-05-09T06:26:28.421 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:48 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:28.506 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rados-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:28.550 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:28.552 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-09T06:26:28.552 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:28.622 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rgw 2024-05-09T06:26:28.726 INFO:teuthology.orchestra.run.smithi193.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-05-09T06:26:28.868 INFO:teuthology.orchestra.run.smithi193.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 108 kB/s | 58 kB 00:00 2024-05-09T06:26:28.993 INFO:teuthology.orchestra.run.smithi193.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 450 kB/s | 548 kB 00:01 2024-05-09T06:26:29.214 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:49 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:29.252 INFO:teuthology.orchestra.run.smithi193.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 97 kB/s | 51 kB 00:00 2024-05-09T06:26:29.255 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:29.255 INFO:teuthology.orchestra.run.smithi193.stdout:Total 3.6 MB/s | 7.0 MB 00:01 2024-05-09T06:26:29.298 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rgw-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:29.326 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:26:29.337 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:26:29.337 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:26:29.342 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:29.343 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-09T06:26:29.343 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:29.414 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-cephfs 2024-05-09T06:26:29.434 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:26:29.436 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:26:29.739 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:26:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-09T06:26:29.945 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-09T06:26:30.001 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:50 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:30.083 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-cephfs-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:30.126 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:30.127 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-09T06:26:30.127 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:30.196 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rbd 2024-05-09T06:26:30.479 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-09T06:26:30.565 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-09T06:26:30.626 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-09T06:26:30.658 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 6/6 2024-05-09T06:26:30.789 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:50 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:30.873 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rbd-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:30.916 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:30.917 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-09T06:26:30.917 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:30.987 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-fuse 2024-05-09T06:26:31.425 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 6/6 2024-05-09T06:26:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 1/6 2024-05-09T06:26:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-09T06:26:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-09T06:26:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-09T06:26:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-09T06:26:31.579 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:51 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:31.684 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-09T06:26:31.684 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:31.684 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:26:31.684 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:18.2.2-1306.gbf64272e.el9.noarch 2024-05-09T06:26:31.685 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T06:26:31.685 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T06:26:31.685 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T06:26:31.685 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T06:26:31.685 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T06:26:31.685 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:31.685 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:31.707 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:31.708 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:31.708 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-05-09T06:26:31.708 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:31.708 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:26:31.708 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 87 k 2024-05-09T06:26:31.708 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:31.708 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:26:31.709 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:31.709 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-05-09T06:26:31.709 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:31.709 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 87 k 2024-05-09T06:26:31.709 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 227 k 2024-05-09T06:26:31.709 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:26:31.967 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-fuse 2024-05-09T06:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-fuse-18.2.2-1306.gbf64272e.el9.x86_64.rpm 269 kB/s | 87 kB 00:00 2024-05-09T06:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout:Total 268 kB/s | 87 kB 00:00 2024-05-09T06:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:26:32.107 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:26:32.107 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:26:32.254 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:26:32.364 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:32.536 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:47 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:32.660 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:32.660 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:32.660 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-09T06:26:32.660 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:32.661 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:26:32.661 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 848 k 2024-05-09T06:26:32.661 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-09T06:26:32.661 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T06:26:32.661 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:32.661 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:26:32.661 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:32.661 INFO:teuthology.orchestra.run.smithi193.stdout:Install 2 Packages 2024-05-09T06:26:32.661 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:32.662 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 928 k 2024-05-09T06:26:32.662 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 2.7 M 2024-05-09T06:26:32.662 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:26:32.736 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:32.996 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:26:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:32.996 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:33.225 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-mirror 2024-05-09T06:26:33.246 INFO:teuthology.orchestra.run.smithi193.stdout:(1/2): ceph-fuse-18.2.2-1306.gbf64272e.el9.x86_ 2.7 MB/s | 848 kB 00:00 2024-05-09T06:26:33.263 INFO:teuthology.orchestra.run.smithi193.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 245 kB/s | 80 kB 00:00 2024-05-09T06:26:33.263 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:33.263 INFO:teuthology.orchestra.run.smithi193.stdout:Total 1.5 MB/s | 928 kB 00:00 2024-05-09T06:26:33.275 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:26:33.297 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:26:33.297 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:26:33.378 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:26:33.378 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:26:33.566 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:26:33.714 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-09T06:26:33.777 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 2/2 2024-05-09T06:26:33.817 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:53 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:33.947 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 3.0 M 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:33.948 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 3.0 M 2024-05-09T06:26:33.949 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 12 M 2024-05-09T06:26:33.949 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:26:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 2/2 2024-05-09T06:26:34.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 1/2 2024-05-09T06:26:34.568 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-mirror-18.2.2-1306.gbf64272e.el9.x86_64.rpm 4.8 MB/s | 3.0 MB 00:00 2024-05-09T06:26:34.568 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:34.569 INFO:teuthology.orchestra.run.smithi063.stdout:Total 4.8 MB/s | 3.0 MB 00:00 2024-05-09T06:26:34.569 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:26:34.579 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:26:34.579 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:26:34.664 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:26:34.664 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:26:34.730 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-09T06:26:34.730 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:34.730 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:26:34.730 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-09T06:26:34.730 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:34.730 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:35.051 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librados-devel 2024-05-09T06:26:35.116 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:26:35.177 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-mirror-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:35.204 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:35.204 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:26:35.204 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T06:26:35.205 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:26:35.205 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:26:35.205 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:35.631 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:50 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:35.753 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:35.754 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:35.755 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-05-09T06:26:35.755 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:35.755 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:26:35.755 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 127 k 2024-05-09T06:26:35.755 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:35.755 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:26:35.755 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:35.756 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-05-09T06:26:35.756 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:35.756 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 127 k 2024-05-09T06:26:35.756 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 456 k 2024-05-09T06:26:35.756 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:26:36.068 INFO:teuthology.orchestra.run.smithi193.stdout:librados-devel-18.2.2-1306.gbf64272e.el9.x86_64 406 kB/s | 127 kB 00:00 2024-05-09T06:26:36.068 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:36.068 INFO:teuthology.orchestra.run.smithi193.stdout:Total 404 kB/s | 127 kB 00:00 2024-05-09T06:26:36.069 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:26:36.074 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:26:36.074 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:26:36.137 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:26:36.137 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:26:36.254 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-mirror-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:36.254 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:36.255 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:26:36.255 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:36.255 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:36.255 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:36.283 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:26:36.372 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:36.489 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-nbd 2024-05-09T06:26:36.796 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:37.067 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:37.068 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:26:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:37.068 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:37.068 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:37.104 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:57 ago on Thu 09 May 2024 06:24:40 AM UTC. 2024-05-09T06:26:37.231 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-09T06:26:37.232 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:37.232 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-05-09T06:26:37.232 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:37.232 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-09T06:26:37.232 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 172 k 2024-05-09T06:26:37.232 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:37.232 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-09T06:26:37.233 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-09T06:26:37.233 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-05-09T06:26:37.233 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:37.233 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 172 k 2024-05-09T06:26:37.233 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 494 k 2024-05-09T06:26:37.233 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-09T06:26:37.337 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs2 2024-05-09T06:26:37.588 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-nbd-18.2.2-1306.gbf64272e.el9.x86_64.rpm 484 kB/s | 172 kB 00:00 2024-05-09T06:26:37.589 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:37.589 INFO:teuthology.orchestra.run.smithi063.stdout:Total 483 kB/s | 172 kB 00:00 2024-05-09T06:26:37.589 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-09T06:26:37.598 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-09T06:26:37.598 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-09T06:26:37.669 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-09T06:26:37.670 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-09T06:26:37.832 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-09T06:26:37.919 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:52 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:37.930 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-nbd-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:38.001 INFO:teuthology.orchestra.run.smithi193.stdout:Package libcephfs2-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:38.043 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:38.044 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-05-09T06:26:38.044 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:38.141 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs-devel 2024-05-09T06:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:38.587 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-nbd-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:38.587 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:38.587 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-09T06:26:38.587 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:38.587 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:26:38.587 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-09T06:26:38.706 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:53 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:38.827 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:38.827 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:38.827 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-05-09T06:26:38.828 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:38.828 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:26:38.828 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 31 k 2024-05-09T06:26:38.828 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:38.828 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:26:38.828 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:38.828 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-05-09T06:26:38.828 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:38.829 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 31 k 2024-05-09T06:26:38.829 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 139 k 2024-05-09T06:26:38.829 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:26:38.863 DEBUG:teuthology.parallel:result is None 2024-05-09T06:26:39.063 INFO:teuthology.orchestra.run.smithi193.stdout:libcephfs-devel-18.2.2-1306.gbf64272e.el9.x86_6 132 kB/s | 31 kB 00:00 2024-05-09T06:26:39.064 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:39.064 INFO:teuthology.orchestra.run.smithi193.stdout:Total 132 kB/s | 31 kB 00:00 2024-05-09T06:26:39.064 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:26:39.067 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:26:39.067 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:26:39.084 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:26:39.085 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:26:39.164 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:26:39.268 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:39.604 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:39.840 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:39.840 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:39.840 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:26:39.841 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:39.841 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:39.841 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:40.053 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librados2 2024-05-09T06:26:40.618 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:55 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:40.699 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:40.741 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:40.742 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-05-09T06:26:40.742 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:40.828 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librbd1 2024-05-09T06:26:41.397 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:56 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:41.478 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:41.520 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:41.521 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-05-09T06:26:41.522 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:41.603 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rados 2024-05-09T06:26:42.167 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:57 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:42.247 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rados-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:42.288 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:42.289 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-05-09T06:26:42.289 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:42.369 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rgw 2024-05-09T06:26:42.939 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:57 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:43.021 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rgw-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:43.063 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:43.064 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-05-09T06:26:43.064 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:43.143 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-cephfs 2024-05-09T06:26:43.712 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:58 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:43.792 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-cephfs-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:43.833 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:43.834 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-05-09T06:26:43.834 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:43.919 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rbd 2024-05-09T06:26:44.483 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:59 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:44.564 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rbd-2:18.2.2-1306.gbf64272e.el9.x86_64 is already installed. 2024-05-09T06:26:44.605 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:44.606 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-05-09T06:26:44.606 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:44.698 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-fuse 2024-05-09T06:26:45.268 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:00 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:45.392 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:45.393 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:45.393 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-05-09T06:26:45.393 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:45.393 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:26:45.393 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 87 k 2024-05-09T06:26:45.393 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:45.393 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:26:45.394 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:45.394 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-05-09T06:26:45.394 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:45.394 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 87 k 2024-05-09T06:26:45.394 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 227 k 2024-05-09T06:26:45.394 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:26:45.599 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-fuse-18.2.2-1306.gbf64272e.el9.x86_64.rpm 421 kB/s | 87 kB 00:00 2024-05-09T06:26:45.600 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:45.600 INFO:teuthology.orchestra.run.smithi193.stdout:Total 418 kB/s | 87 kB 00:00 2024-05-09T06:26:45.600 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:26:45.607 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:26:45.607 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:26:45.671 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:26:45.671 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:26:45.817 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:26:45.904 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:46.281 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:46.544 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:46.544 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:46.544 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:26:46.544 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:46.544 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:46.544 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:46.784 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-mirror 2024-05-09T06:26:47.386 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:02 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:47.519 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 3.0 M 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:47.520 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 3.0 M 2024-05-09T06:26:47.521 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 12 M 2024-05-09T06:26:47.521 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:26:52.661 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-mirror-18.2.2-1306.gbf64272e.el9.x86_64.rpm 595 kB/s | 3.0 MB 00:05 2024-05-09T06:26:52.662 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:52.663 INFO:teuthology.orchestra.run.smithi193.stdout:Total 595 kB/s | 3.0 MB 00:05 2024-05-09T06:26:52.663 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:26:52.674 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:26:52.674 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:26:52.759 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:26:52.759 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:26:53.179 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:26:53.254 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:53.280 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:53.280 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:26:53.280 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T06:26:53.280 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:26:53.280 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:26:53.281 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:54.397 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:54.398 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:54.398 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:26:54.398 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:54.398 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:54.398 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:54.676 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-nbd 2024-05-09T06:26:55.260 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:10 ago on Thu 09 May 2024 06:24:45 AM UTC. 2024-05-09T06:26:55.385 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-09T06:26:55.386 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:55.386 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-05-09T06:26:55.386 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:55.386 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-09T06:26:55.386 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:18.2.2-1306.gbf64272e.el9 ceph 172 k 2024-05-09T06:26:55.386 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:55.386 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-09T06:26:55.386 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-05-09T06:26:55.386 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-05-09T06:26:55.387 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:55.387 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 172 k 2024-05-09T06:26:55.387 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 494 k 2024-05-09T06:26:55.387 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-09T06:26:56.088 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-nbd-18.2.2-1306.gbf64272e.el9.x86_64.rpm 245 kB/s | 172 kB 00:00 2024-05-09T06:26:56.089 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:56.089 INFO:teuthology.orchestra.run.smithi193.stdout:Total 245 kB/s | 172 kB 00:00 2024-05-09T06:26:56.089 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-09T06:26:56.098 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-09T06:26:56.098 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-09T06:26:56.170 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-09T06:26:56.171 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-09T06:26:56.339 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-09T06:26:56.437 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:56.836 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:57.096 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:18.2.2-1306.gbf64272e.el9.x86_64 1/1 2024-05-09T06:26:57.096 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:57.096 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-09T06:26:57.096 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:18.2.2-1306.gbf64272e.el9.x86_64 2024-05-09T06:26:57.096 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:26:57.096 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-09T06:26:57.348 DEBUG:teuthology.parallel:result is None 2024-05-09T06:26:57.349 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T06:26:57.349 INFO:teuthology.packaging:ref: None 2024-05-09T06:26:57.349 INFO:teuthology.packaging:tag: None 2024-05-09T06:26:57.349 INFO:teuthology.packaging:branch: reef 2024-05-09T06:26:57.349 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:26:57.349 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-05-09T06:26:57.694 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T06:26:57.723 INFO:teuthology.orchestra.run.smithi063.stdout:18.2.2-1306.gbf64272e.el9 2024-05-09T06:26:57.723 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1306.gbf64272e.el9 2024-05-09T06:26:57.723 INFO:teuthology.task.install:The correct ceph version 18.2.2-1306.gbf64272e is installed. 2024-05-09T06:26:57.725 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T06:26:57.725 INFO:teuthology.packaging:ref: None 2024-05-09T06:26:57.725 INFO:teuthology.packaging:tag: None 2024-05-09T06:26:57.726 INFO:teuthology.packaging:branch: reef 2024-05-09T06:26:57.726 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:26:57.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-05-09T06:26:57.947 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T06:26:57.968 INFO:teuthology.orchestra.run.smithi193.stdout:18.2.2-1306.gbf64272e.el9 2024-05-09T06:26:57.969 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1306.gbf64272e.el9 2024-05-09T06:26:57.969 INFO:teuthology.task.install:The correct ceph version 18.2.2-1306.gbf64272e is installed. 2024-05-09T06:26:57.970 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T06:26:57.971 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:26:57.971 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T06:26:58.003 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:26:58.003 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T06:26:58.057 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T06:26:58.058 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:26:58.058 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T06:26:58.088 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T06:26:58.157 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:26:58.158 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T06:26:58.182 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T06:26:58.246 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T06:26:58.246 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:26:58.246 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T06:26:58.274 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T06:26:58.340 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:26:58.341 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T06:26:58.365 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T06:26:58.430 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T06:26:58.430 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:26:58.430 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T06:26:58.457 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T06:26:58.524 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:26:58.524 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T06:26:58.549 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T06:26:58.614 INFO:teuthology.run_tasks:Running task print... 2024-05-09T06:26:58.626 INFO:teuthology.task.print:**** done install task... 2024-05-09T06:26:58.626 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T06:26:58.725 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T06:26:58.726 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-05-09T06:26:58.726 INFO:tasks.cephadm:Cluster fsid is 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:26:58.726 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T06:26:58.726 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-05-09T06:26:58.726 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi063': '172.21.15.63', 'mon.smithi193': '172.21.15.193'} 2024-05-09T06:26:58.727 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T06:26:58.727 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname $(hostname -s) 2024-05-09T06:26:58.754 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-05-09T06:26:58.779 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-05-09T06:26:58.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:26:59.025 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55097/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2515-g96805dcf', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2515.g96805dcf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-07 19:00:25.721799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-09T06:26:59.252 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 bf64272e3d7edf5c7f8706082e690401a7a11750 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-05-09T06:26:59.254 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/x86_64/flavors/default/cephadm 2024-05-09T06:26:59.254 DEBUG:teuthology.orchestra.run.smithi063:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T06:26:59.886 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r--. 1 ubuntu ubuntu 215333 May 9 06:26 /home/ubuntu/cephtest/cephadm 2024-05-09T06:26:59.886 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/bf64272e3d7edf5c7f8706082e690401a7a11750/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T06:27:00.816 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 215333 May 9 06:27 /home/ubuntu/cephtest/cephadm 2024-05-09T06:27:00.817 DEBUG:teuthology.orchestra.run.smithi063:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T06:27:00.837 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T06:27:00.867 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-05-09T06:27:00.867 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-05-09T06:27:00.880 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-05-09T06:27:02.374 INFO:teuthology.orchestra.run.smithi063.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-05-09T06:27:02.651 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-05-09T06:27:37.421 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:27:37.422 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)", 2024-05-09T06:27:37.422 INFO:teuthology.orchestra.run.smithi063.stdout: "image_id": "196422f1b64117f056e4ad8be5d26eeefdfc0c98363316c4529effef2079a699", 2024-05-09T06:27:37.422 INFO:teuthology.orchestra.run.smithi063.stdout: "repo_digests": [ 2024-05-09T06:27:37.422 INFO:teuthology.orchestra.run.smithi063.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f" 2024-05-09T06:27:37.422 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-09T06:27:37.422 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:27:41.466 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-09T06:27:41.467 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)", 2024-05-09T06:27:41.467 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "196422f1b64117f056e4ad8be5d26eeefdfc0c98363316c4529effef2079a699", 2024-05-09T06:27:41.467 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-05-09T06:27:41.467 INFO:teuthology.orchestra.run.smithi193.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f" 2024-05-09T06:27:41.467 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-05-09T06:27:41.467 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-09T06:27:41.486 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph 2024-05-09T06:27:41.520 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-05-09T06:27:41.547 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /etc/ceph 2024-05-09T06:27:41.591 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-05-09T06:27:41.615 INFO:tasks.cephadm:Writing seed config... 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T06:27:41.616 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T06:27:41.617 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T06:27:41.618 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T06:27:41.619 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:27:41.619 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T06:27:41.650 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2386be1c-0dcd-11ef-bc97-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T06:27:41.650 DEBUG:teuthology.orchestra.run.smithi063:mon.smithi063> sudo journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063.service 2024-05-09T06:27:41.693 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T06:27:41.693 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.63 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:27:41.863 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:27:41.863 INFO:teuthology.orchestra.run.smithi063.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', '2386be1c-0dcd-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.63', '--skip-admin-label'] 2024-05-09T06:27:41.891 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T06:27:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T06:27:41.892 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying podman|docker is present... 2024-05-09T06:27:41.917 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T06:27:41.917 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying lvm2 is present... 2024-05-09T06:27:41.917 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying time synchronization is in place... 2024-05-09T06:27:41.925 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T06:27:41.925 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T06:27:41.931 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T06:27:41.932 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-09T06:27:41.938 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-05-09T06:27:41.945 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-05-09T06:27:41.945 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-05-09T06:27:41.945 INFO:teuthology.orchestra.run.smithi063.stdout:Repeating the final host check... 2024-05-09T06:27:41.969 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T06:27:41.970 INFO:teuthology.orchestra.run.smithi063.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T06:27:41.970 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl is present 2024-05-09T06:27:41.970 INFO:teuthology.orchestra.run.smithi063.stdout:lvcreate is present 2024-05-09T06:27:41.976 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T06:27:41.976 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T06:27:41.982 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T06:27:41.982 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-09T06:27:41.989 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-05-09T06:27:41.996 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-05-09T06:27:41.996 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-05-09T06:27:41.996 INFO:teuthology.orchestra.run.smithi063.stdout:Host looks OK 2024-05-09T06:27:41.996 INFO:teuthology.orchestra.run.smithi063.stdout:Cluster fsid: 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:27:41.996 INFO:teuthology.orchestra.run.smithi063.stdout:Acquiring lock 140180973159424 on /run/cephadm/2386be1c-0dcd-11ef-bc97-c7b262605968.lock 2024-05-09T06:27:41.996 INFO:teuthology.orchestra.run.smithi063.stdout:Lock 140180973159424 acquired on /run/cephadm/2386be1c-0dcd-11ef-bc97-c7b262605968.lock 2024-05-09T06:27:41.997 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 3300 ... 2024-05-09T06:27:41.997 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 6789 ... 2024-05-09T06:27:41.997 INFO:teuthology.orchestra.run.smithi063.stdout:Base mon IP(s) is [172.21.15.63:3300, 172.21.15.63:6789], mon addrv is [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-05-09T06:27:42.000 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.63 metric 100 2024-05-09T06:27:42.000 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.63 metric 100 2024-05-09T06:27:42.003 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T06:27:42.003 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-09T06:27:42.003 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1673sec hoplimit 64 pref medium 2024-05-09T06:27:42.005 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T06:27:42.005 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T06:27:42.006 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T06:27:42.006 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-09T06:27:42.006 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-09T06:27:42.006 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T06:27:42.006 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-05-09T06:27:42.006 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-05-09T06:27:42.006 INFO:teuthology.orchestra.run.smithi063.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T06:27:42.007 INFO:teuthology.orchestra.run.smithi063.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T06:27:42.008 INFO:teuthology.orchestra.run.smithi063.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-05-09T06:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 196422f1b64117f056e4ad8be5d26eeefdfc0c98363316c4529effef2079a699 2024-05-09T06:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-05-09T06:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T06:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:38b279daa26f8c11f5739a1a92ca1eec669d760ec0d0eeb5cbce2b1587315830 2024-05-09T06:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-09T06:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying config sha256:196422f1b64117f056e4ad8be5d26eeefdfc0c98363316c4529effef2079a699 2024-05-09T06:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T06:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout:ceph: stdout ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable) 2024-05-09T06:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph version: ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable) 2024-05-09T06:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T06:27:45.092 INFO:teuthology.orchestra.run.smithi063.stdout:stat: stdout 167 167 2024-05-09T06:27:45.092 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial keys... 2024-05-09T06:27:45.930 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQDhbDxmH8ChGxAAOoIaVOdap0sDmoykCWCPMw== 2024-05-09T06:27:46.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQDibDxmN0zfDxAAaSuQwF0vxdjLfqaCKDUWlQ== 2024-05-09T06:27:47.824 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQDjbDxmCUG9DhAAO9Y25KJUfmE4kweiICPnOw== 2024-05-09T06:27:47.825 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial monmap... 2024-05-09T06:27:48.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T06:27:48.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-09T06:27:48.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:27:48.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T06:27:48.723 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool for smithi063 [v2:172.21.15.63:3300,v1:172.21.15.63:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T06:27:48.723 INFO:teuthology.orchestra.run.smithi063.stdout:setting min_mon_release = pacific 2024-05-09T06:27:48.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: set fsid to 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:27:48.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T06:27:48.723 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:27:48.723 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mon... 2024-05-09T06:27:49.586 INFO:teuthology.orchestra.run.smithi063.stdout:create mon.smithi063 on 2024-05-09T06:27:49.802 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T06:27:49.944 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T06:27:50.112 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968.target. 2024-05-09T06:27:50.112 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968.target. 2024-05-09T06:27:50.323 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063 2024-05-09T06:27:50.323 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063.service: Unit ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063.service not loaded. 2024-05-09T06:27:50.492 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968.target.wants/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063.service → /etc/systemd/system/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@.service. 2024-05-09T06:27:51.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:50 smithi063 podman[26776]: 2024-05-09 06:27:50.741376046 +0000 UTC m=+0.018774497 image pull 196422f1b64117f056e4ad8be5d26eeefdfc0c98363316c4529effef2079a699 quay.ceph.io/ceph-ci/ceph:reef 2024-05-09T06:27:51.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:51 smithi063 podman[26776]: 2024-05-09 06:27:51.543026704 +0000 UTC m=+0.820425185 container create 2f450eebe644263243629c082f1520c49cc271acd0863a798052e61253adbf39 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=reef-1758e8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T06:27:53.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:53 smithi063 podman[26776]: 2024-05-09 06:27:53.337224767 +0000 UTC m=+2.614623202 container init 2f450eebe644263243629c082f1520c49cc271acd0863a798052e61253adbf39 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-1758e8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T06:27:53.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:53 smithi063 podman[26776]: 2024-05-09 06:27:53.341332742 +0000 UTC m=+2.618731178 container start 2f450eebe644263243629c082f1520c49cc271acd0863a798052e61253adbf39 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=reef-1758e8e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430) 2024-05-09T06:27:53.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:53 smithi063 ceph-mon[26796]: mkfs 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:27:53.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:53 smithi063 ceph-mon[26796]: mon.smithi063 is new leader, mons smithi063 in quorum (ranks 0) 2024-05-09T06:27:54.378 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T06:27:54.378 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-09T06:27:54.387 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T06:27:54.387 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-09T06:27:54.387 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-09T06:27:54.387 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T06:27:54.388 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon to start... 2024-05-09T06:27:54.388 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon... 2024-05-09T06:27:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:54 smithi063 bash[26776]: 2f450eebe644263243629c082f1520c49cc271acd0863a798052e61253adbf39 2024-05-09T06:27:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:54 smithi063 systemd[1]: Started Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:27:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:54 smithi063 ceph-mon[26796]: mon.smithi063 is new leader, mons smithi063 in quorum (ranks 0) 2024-05-09T06:27:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:54 smithi063 ceph-mon[26796]: monmap e1: 1 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-09T06:27:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:54 smithi063 ceph-mon[26796]: fsmap 2024-05-09T06:27:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:54 smithi063 ceph-mon[26796]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T06:27:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:54 smithi063 ceph-mon[26796]: mgrmap e1: no daemons active 2024-05-09T06:27:57.520 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:27:57 smithi063 ceph-mon[26796]: from='client.? 172.21.15.63:0/1574407769' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T06:28:03.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout id: 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout services: 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi063 (age 3s) 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout data: 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T06:28:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.264+0000 7ff79bace700 1 Processor -- start 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.264+0000 7ff79bace700 1 -- start start 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.264+0000 7ff79bace700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794105920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.264+0000 7ff79bace700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff794105ef0 con 0x7ff794105500 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.265+0000 7ff79986a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.265+0000 7ff79986a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794105920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58208/0 (socket says 172.21.15.63:58208) 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.265+0000 7ff79986a700 1 -- 172.21.15.63:0/959583465 learned_addr learned my addr 172.21.15.63:0/959583465 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.265+0000 7ff79986a700 1 -- 172.21.15.63:0/959583465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794106700 con 0x7ff794105500 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff79986a700 1 --2- 172.21.15.63:0/959583465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794105920 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff784009e80 tx=0x7ff78400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf808a54c219ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff798868700 1 -- 172.21.15.63:0/959583465 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff78400e070 con 0x7ff794105500 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff798868700 1 -- 172.21.15.63:0/959583465 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff784003a20 con 0x7ff794105500 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff798868700 1 -- 172.21.15.63:0/959583465 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff784003bd0 con 0x7ff794105500 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff79bace700 1 -- 172.21.15.63:0/959583465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 msgr2=0x7ff794105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:03.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff79bace700 1 --2- 172.21.15.63:0/959583465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794105920 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff784009e80 tx=0x7ff78400b330 comp rx=0 tx=0).stop 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff79bace700 1 -- 172.21.15.63:0/959583465 shutdown_connections 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff79bace700 1 --2- 172.21.15.63:0/959583465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794105920 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff79bace700 1 -- 172.21.15.63:0/959583465 >> 172.21.15.63:0/959583465 conn(0x7ff794100b20 msgr2=0x7ff794102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff79bace700 1 -- 172.21.15.63:0/959583465 shutdown_connections 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.266+0000 7ff79bace700 1 -- 172.21.15.63:0/959583465 wait complete. 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.267+0000 7ff79bace700 1 Processor -- start 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.267+0000 7ff79bace700 1 -- start start 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.267+0000 7ff79bace700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.267+0000 7ff79bace700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79419a090 con 0x7ff794105500 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.267+0000 7ff79986a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.267+0000 7ff79986a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58212/0 (socket says 172.21.15.63:58212) 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.267+0000 7ff79986a700 1 -- 172.21.15.63:0/1574407769 learned_addr learned my addr 172.21.15.63:0/1574407769 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.268+0000 7ff79986a700 1 -- 172.21.15.63:0/1574407769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff784009b30 con 0x7ff794105500 2024-05-09T06:28:03.100 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.268+0000 7ff79986a700 1 --2- 172.21.15.63:0/1574407769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794199b50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff784004000 tx=0x7ff784004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.268+0000 7ff792ffd700 1 -- 172.21.15.63:0/1574407769 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff784004240 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.268+0000 7ff792ffd700 1 -- 172.21.15.63:0/1574407769 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7840043e0 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.268+0000 7ff79bace700 1 -- 172.21.15.63:0/1574407769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79419a290 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.268+0000 7ff792ffd700 1 -- 172.21.15.63:0/1574407769 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7840046e0 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.268+0000 7ff79bace700 1 -- 172.21.15.63:0/1574407769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79419a730 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.268+0000 7ff792ffd700 1 -- 172.21.15.63:0/1574407769 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff784019050 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.268+0000 7ff792ffd700 1 -- 172.21.15.63:0/1574407769 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff784025790 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.269+0000 7ff79bace700 1 -- 172.21.15.63:0/1574407769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff794193860 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.272+0000 7ff792ffd700 1 -- 172.21.15.63:0/1574407769 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff78402a060 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.345+0000 7ff79bace700 1 -- 172.21.15.63:0/1574407769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff794106a70 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.346+0000 7ff792ffd700 1 -- 172.21.15.63:0/1574407769 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+318 (secure 0 0 0) 0x7ff784020030 con 0x7ff794105500 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.346+0000 7ff79bace700 1 -- 172.21.15.63:0/1574407769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 msgr2=0x7ff794199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.346+0000 7ff79bace700 1 --2- 172.21.15.63:0/1574407769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794199b50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff784004000 tx=0x7ff784004ad0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.346+0000 7ff79bace700 1 -- 172.21.15.63:0/1574407769 shutdown_connections 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.346+0000 7ff79bace700 1 --2- 172.21.15.63:0/1574407769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff794105500 0x7ff794199b50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:03.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.346+0000 7ff79bace700 1 -- 172.21.15.63:0/1574407769 >> 172.21.15.63:0/1574407769 conn(0x7ff794100b20 msgr2=0x7ff7941909a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:03.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.347+0000 7ff79bace700 1 -- 172.21.15.63:0/1574407769 shutdown_connections 2024-05-09T06:28:03.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:27:57.347+0000 7ff79bace700 1 -- 172.21.15.63:0/1574407769 wait complete. 2024-05-09T06:28:03.102 INFO:teuthology.orchestra.run.smithi063.stdout:mon is available 2024-05-09T06:28:03.102 INFO:teuthology.orchestra.run.smithi063.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T06:28:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:03 smithi063 ceph-mon[26796]: from='client.? 172.21.15.63:0/2141902259' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T06:28:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:03 smithi063 ceph-mon[26796]: from='client.? 172.21.15.63:0/2141902259' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T06:28:04.188 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:04.195 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T06:28:04.195 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:28:04.195 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-05-09T06:28:04.195 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T06:28:04.195 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T06:28:04.195 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T06:28:04.195 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T06:28:04.195 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.558+0000 7f8f55c56700 1 Processor -- start 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.558+0000 7f8f55c56700 1 -- start start 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.558+0000 7f8f55c56700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50109540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.558+0000 7f8f55c56700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50079dd0 con 0x7f8f50107150 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.558+0000 7f8f4f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50109540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.196 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.558+0000 7f8f4f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50109540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58228/0 (socket says 172.21.15.63:58228) 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.558+0000 7f8f4f7fe700 1 -- 172.21.15.63:0/283797385 learned_addr learned my addr 172.21.15.63:0/283797385 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.559+0000 7f8f4f7fe700 1 -- 172.21.15.63:0/283797385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f50079f10 con 0x7f8f50107150 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.559+0000 7f8f4f7fe700 1 --2- 172.21.15.63:0/283797385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50109540 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f400098a0 tx=0x7f8f40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e9e5716408d64c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.559+0000 7f8f4e7fc700 1 -- 172.21.15.63:0/283797385 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f40004180 con 0x7f8f50107150 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.559+0000 7f8f4e7fc700 1 -- 172.21.15.63:0/283797385 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400042e0 con 0x7f8f50107150 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.559+0000 7f8f4e7fc700 1 -- 172.21.15.63:0/283797385 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f400036a0 con 0x7f8f50107150 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.560+0000 7f8f55c56700 1 -- 172.21.15.63:0/283797385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 msgr2=0x7f8f50109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.560+0000 7f8f55c56700 1 --2- 172.21.15.63:0/283797385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50109540 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f400098a0 tx=0x7f8f40009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.560+0000 7f8f55c56700 1 -- 172.21.15.63:0/283797385 shutdown_connections 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.560+0000 7f8f55c56700 1 --2- 172.21.15.63:0/283797385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50109540 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.560+0000 7f8f55c56700 1 -- 172.21.15.63:0/283797385 >> 172.21.15.63:0/283797385 conn(0x7f8f50100b00 msgr2=0x7f8f50102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.560+0000 7f8f55c56700 1 -- 172.21.15.63:0/283797385 shutdown_connections 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.560+0000 7f8f55c56700 1 -- 172.21.15.63:0/283797385 wait complete. 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.560+0000 7f8f55c56700 1 Processor -- start 2024-05-09T06:28:04.197 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.561+0000 7f8f55c56700 1 -- start start 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.561+0000 7f8f55c56700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.561+0000 7f8f55c56700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5019a000 con 0x7f8f50107150 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.561+0000 7f8f4f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.561+0000 7f8f4f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58244/0 (socket says 172.21.15.63:58244) 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.561+0000 7f8f4f7fe700 1 -- 172.21.15.63:0/2141902259 learned_addr learned my addr 172.21.15.63:0/2141902259 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.561+0000 7f8f4f7fe700 1 -- 172.21.15.63:0/2141902259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40009550 con 0x7f8f50107150 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.561+0000 7f8f4f7fe700 1 --2- 172.21.15.63:0/2141902259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50199ac0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8f50079210 tx=0x7f8f40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.561+0000 7f8f4cff9700 1 -- 172.21.15.63:0/2141902259 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f40003ce0 con 0x7f8f50107150 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.562+0000 7f8f4cff9700 1 -- 172.21.15.63:0/2141902259 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f40020070 con 0x7f8f50107150 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.562+0000 7f8f55c56700 1 -- 172.21.15.63:0/2141902259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5019a200 con 0x7f8f50107150 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.562+0000 7f8f4cff9700 1 -- 172.21.15.63:0/2141902259 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f40011630 con 0x7f8f50107150 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.562+0000 7f8f55c56700 1 -- 172.21.15.63:0/2141902259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5019a6a0 con 0x7f8f50107150 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.562+0000 7f8f4cff9700 1 -- 172.21.15.63:0/2141902259 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f8f40022030 con 0x7f8f50107150 2024-05-09T06:28:04.198 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.562+0000 7f8f4cff9700 1 -- 172.21.15.63:0/2141902259 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8f40025d00 con 0x7f8f50107150 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.563+0000 7f8f55c56700 1 -- 172.21.15.63:0/2141902259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f50193920 con 0x7f8f50107150 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.565+0000 7f8f4cff9700 1 -- 172.21.15.63:0/2141902259 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f8f40025480 con 0x7f8f50107150 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.636+0000 7f8f55c56700 1 -- 172.21.15.63:0/2141902259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8f50109c60 con 0x7f8f50107150 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.639+0000 7f8f4cff9700 1 -- 172.21.15.63:0/2141902259 <== mon.0 v2:172.21.15.63:3300/0 7 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f8f40011c20 con 0x7f8f50107150 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.639+0000 7f8f4cff9700 1 -- 172.21.15.63:0/2141902259 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f8f4001c500 con 0x7f8f50107150 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.640+0000 7f8f55c56700 1 -- 172.21.15.63:0/2141902259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 msgr2=0x7f8f50199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.640+0000 7f8f55c56700 1 --2- 172.21.15.63:0/2141902259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50199ac0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8f50079210 tx=0x7f8f40004750 comp rx=0 tx=0).stop 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.640+0000 7f8f55c56700 1 -- 172.21.15.63:0/2141902259 shutdown_connections 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.640+0000 7f8f55c56700 1 --2- 172.21.15.63:0/2141902259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f50107150 0x7f8f50199ac0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.640+0000 7f8f55c56700 1 -- 172.21.15.63:0/2141902259 >> 172.21.15.63:0/2141902259 conn(0x7f8f50100b00 msgr2=0x7f8f50100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.640+0000 7f8f55c56700 1 -- 172.21.15.63:0/2141902259 shutdown_connections 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:03.640+0000 7f8f55c56700 1 -- 172.21.15.63:0/2141902259 wait complete. 2024-05-09T06:28:04.199 INFO:teuthology.orchestra.run.smithi063.stdout:Generating new minimal ceph.conf... 2024-05-09T06:28:04.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:04 smithi063 ceph-mon[26796]: from='client.? 172.21.15.63:0/3063427142' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:28:05.266 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.653+0000 7f1ab619c700 1 Processor -- start 2024-05-09T06:28:05.266 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.653+0000 7f1ab619c700 1 -- start start 2024-05-09T06:28:05.266 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.653+0000 7f1ab619c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab0106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.266 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.653+0000 7f1ab619c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab0107030 con 0x7f1ab0106640 2024-05-09T06:28:05.266 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.653+0000 7f1aaf7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab0106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.266 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.653+0000 7f1aaf7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab0106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58258/0 (socket says 172.21.15.63:58258) 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.653+0000 7f1aaf7fe700 1 -- 172.21.15.63:0/3086412143 learned_addr learned my addr 172.21.15.63:0/3086412143 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.654+0000 7f1aaf7fe700 1 -- 172.21.15.63:0/3086412143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab01077f0 con 0x7f1ab0106640 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.654+0000 7f1aaf7fe700 1 --2- 172.21.15.63:0/3086412143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab0106a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009e80 tx=0x7f1aa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f94af4196aa85a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.654+0000 7f1aae7fc700 1 -- 172.21.15.63:0/3086412143 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1aa000e070 con 0x7f1ab0106640 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.654+0000 7f1aae7fc700 1 -- 172.21.15.63:0/3086412143 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1aa000b8c0 con 0x7f1ab0106640 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.654+0000 7f1aae7fc700 1 -- 172.21.15.63:0/3086412143 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1aa0004440 con 0x7f1ab0106640 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.655+0000 7f1ab619c700 1 -- 172.21.15.63:0/3086412143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 msgr2=0x7f1ab0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.655+0000 7f1ab619c700 1 --2- 172.21.15.63:0/3086412143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab0106a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009e80 tx=0x7f1aa000b330 comp rx=0 tx=0).stop 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.655+0000 7f1ab619c700 1 -- 172.21.15.63:0/3086412143 shutdown_connections 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.655+0000 7f1ab619c700 1 --2- 172.21.15.63:0/3086412143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab0106a60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.267 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.655+0000 7f1ab619c700 1 -- 172.21.15.63:0/3086412143 >> 172.21.15.63:0/3086412143 conn(0x7f1ab0101c40 msgr2=0x7f1ab0104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.655+0000 7f1ab619c700 1 -- 172.21.15.63:0/3086412143 shutdown_connections 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.655+0000 7f1ab619c700 1 -- 172.21.15.63:0/3086412143 wait complete. 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.655+0000 7f1ab619c700 1 Processor -- start 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.656+0000 7f1ab619c700 1 -- start start 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.656+0000 7f1ab619c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab007bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.656+0000 7f1ab619c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab007c040 con 0x7f1ab0106640 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.656+0000 7f1aaf7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab007bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.656+0000 7f1aaf7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab007bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58260/0 (socket says 172.21.15.63:58260) 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.656+0000 7f1aaf7fe700 1 -- 172.21.15.63:0/3063427142 learned_addr learned my addr 172.21.15.63:0/3063427142 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.656+0000 7f1aaf7fe700 1 -- 172.21.15.63:0/3063427142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa0009b30 con 0x7f1ab0106640 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.656+0000 7f1aaf7fe700 1 --2- 172.21.15.63:0/3063427142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab007bb00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00099e0 tx=0x7f1aa0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.656+0000 7f1aacff9700 1 -- 172.21.15.63:0/3063427142 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1aa000e040 con 0x7f1ab0106640 2024-05-09T06:28:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.657+0000 7f1aacff9700 1 -- 172.21.15.63:0/3063427142 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1aa000bc40 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.657+0000 7f1ab619c700 1 -- 172.21.15.63:0/3063427142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab007a280 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.657+0000 7f1aacff9700 1 -- 172.21.15.63:0/3063427142 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1aa00125e0 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.657+0000 7f1ab619c700 1 -- 172.21.15.63:0/3063427142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab007a780 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.657+0000 7f1aacff9700 1 -- 172.21.15.63:0/3063427142 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1aa0019040 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.657+0000 7f1aacff9700 1 -- 172.21.15.63:0/3063427142 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1aa0024bb0 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.658+0000 7f1ab619c700 1 -- 172.21.15.63:0/3063427142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab0062520 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.660+0000 7f1aacff9700 1 -- 172.21.15.63:0/3063427142 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f1aa001f020 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.730+0000 7f1ab619c700 1 -- 172.21.15.63:0/3063427142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1ab010ab00 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.731+0000 7f1aacff9700 1 -- 172.21.15.63:0/3063427142 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f1aa0033090 con 0x7f1ab0106640 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.731+0000 7f1ab619c700 1 -- 172.21.15.63:0/3063427142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 msgr2=0x7f1ab007bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.731+0000 7f1ab619c700 1 --2- 172.21.15.63:0/3063427142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab007bb00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00099e0 tx=0x7f1aa0003d00 comp rx=0 tx=0).stop 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.731+0000 7f1ab619c700 1 -- 172.21.15.63:0/3063427142 shutdown_connections 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.731+0000 7f1ab619c700 1 --2- 172.21.15.63:0/3063427142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab0106640 0x7f1ab007bb00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.731+0000 7f1ab619c700 1 -- 172.21.15.63:0/3063427142 >> 172.21.15.63:0/3063427142 conn(0x7f1ab0101c40 msgr2=0x7f1ab01027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.732+0000 7f1ab619c700 1 -- 172.21.15.63:0/3063427142 shutdown_connections 2024-05-09T06:28:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:04.732+0000 7f1ab619c700 1 -- 172.21.15.63:0/3063427142 wait complete. 2024-05-09T06:28:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:Restarting the monitor... 2024-05-09T06:28:05.546 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:05 smithi063 systemd[1]: Stopping Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:28:05.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:05 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063[26792]: 2024-05-09T06:28:05.564+0000 7fc67dfc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi063 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:28:05.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:05 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063[26792]: 2024-05-09T06:28:05.564+0000 7fc67dfc0700 -1 mon.smithi063@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T06:28:05.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:05 smithi063 podman[27040]: 2024-05-09 06:28:05.718255557 +0000 UTC m=+0.269447491 container died 2f450eebe644263243629c082f1520c49cc271acd0863a798052e61253adbf39 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=reef-1758e8e, maintainer=Guillaume Abrioux ) 2024-05-09T06:28:06.221 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 podman[27040]: 2024-05-09 06:28:06.039159851 +0000 UTC m=+0.590351786 container cleanup 2f450eebe644263243629c082f1520c49cc271acd0863a798052e61253adbf39 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-1758e8e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T06:28:06.221 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 bash[27040]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063 2024-05-09T06:28:06.473 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 podman[27054]: 2024-05-09 06:28:06.240304647 +0000 UTC m=+0.522696257 container remove 2f450eebe644263243629c082f1520c49cc271acd0863a798052e61253adbf39 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=reef-1758e8e, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:28:06.473 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063.service: Deactivated successfully. 2024-05-09T06:28:06.473 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 systemd[1]: Stopped Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:28:06.473 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 systemd[1]: Starting Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:28:06.765 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 podman[27152]: 2024-05-09 06:28:06.68268964 +0000 UTC m=+0.114689969 container create 7d88727e4bbbd85e71c5790a652c29c6566266eae7cb5bedc66f7483ccdb0c5f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-1758e8e, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:28:06.766 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 podman[27152]: 2024-05-09 06:28:06.586749332 +0000 UTC m=+0.018749672 image pull 196422f1b64117f056e4ad8be5d26eeefdfc0c98363316c4529effef2079a699 quay.ceph.io/ceph-ci/ceph:reef 2024-05-09T06:28:06.921 INFO:teuthology.orchestra.run.smithi063.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T06:28:07.169 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 podman[27152]: 2024-05-09 06:28:06.876609587 +0000 UTC m=+0.308609915 container init 7d88727e4bbbd85e71c5790a652c29c6566266eae7cb5bedc66f7483ccdb0c5f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-1758e8e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:28:07.169 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 podman[27152]: 2024-05-09 06:28:06.879940682 +0000 UTC m=+0.311941012 container start 7d88727e4bbbd85e71c5790a652c29c6566266eae7cb5bedc66f7483ccdb0c5f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-1758e8e) 2024-05-09T06:28:07.169 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T06:28:07.169 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable), process ceph-mon, pid 2 2024-05-09T06:28:07.169 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: pidfile_write: ignore empty --pid-file 2024-05-09T06:28:07.169 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: load: jerasure load: lrc 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: RocksDB version: 7.9.2 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Git sha 0 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Compile date 2024-05-04 20:27:46 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: DB SUMMARY 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: DB Session ID: D8T0Z444FBNKFB4OQYLZ 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: CURRENT file: CURRENT 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi063/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi063/store.db: 000009.log size: 98784 ; 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.error_if_exists: 0 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.create_if_missing: 0 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.paranoid_checks: 1 2024-05-09T06:28:07.170 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T06:28:07.171 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T06:28:07.171 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T06:28:07.171 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.env: 0x560f19c5e820 2024-05-09T06:28:07.171 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T06:28:07.171 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.info_log: 0x560f1caff3c0 2024-05-09T06:28:07.171 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T06:28:07.171 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.statistics: (nil) 2024-05-09T06:28:07.171 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.use_fsync: 0 2024-05-09T06:28:07.172 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_log_file_size: 0 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.allow_fallocate: 1 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.use_direct_reads: 0 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.db_log_dir: 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.wal_dir: 2024-05-09T06:28:07.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.write_buffer_manager: 0x560f1bd74140 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T06:28:07.174 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.unordered_write: 0 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.row_cache: None 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.wal_filter: None 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.two_write_queues: 0 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.wal_compression: 0 2024-05-09T06:28:07.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.atomic_flush: 0 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.log_readahead_size: 0 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_background_jobs: 2 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_background_compactions: -1 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_subcompactions: 1 2024-05-09T06:28:07.176 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_open_files: -1 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_background_flushes: -1 2024-05-09T06:28:07.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Compression algorithms supported: 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: kZSTD supported: 0 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: kXpressCompression supported: 0 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: kZlibCompression supported: 1 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: kSnappyCompression supported: 1 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: kLZ4Compression supported: 1 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: kBZip2Compression supported: 0 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000010 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.merge_operator: 2024-05-09T06:28:07.178 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_filter: None 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560f1caff4e0) 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks: 1 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_top_level_index_and_filter: 1 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_type: 0 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_index_type: 0 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_shortening: 1 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T06:28:07.179 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: checksum: 4 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: no_block_cache: 0 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache: 0x560f1be171f0 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_name: BinnedLRUCache 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_options: 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: capacity : 536870912 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_shard_bits : 4 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: strict_capacity_limit : 0 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: high_pri_pool_ratio: 0.000 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_compressed: (nil) 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: persistent_cache: (nil) 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size: 4096 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size_deviation: 10 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_restart_interval: 16 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_block_restart_interval: 1 2024-05-09T06:28:07.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: metadata_block_size: 4096 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: partition_filters: 0 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: use_delta_encoding: 1 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: filter_policy: bloomfilter 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: whole_key_filtering: 1 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: verify_compression: 0 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: read_amp_bytes_per_bit: 0 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: format_version: 5 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_index_compression: 1 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_align: 0 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_auto_readahead_size: 262144 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: prepopulate_block_cache: 0 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: initial_auto_readahead_size: 8192 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T06:28:07.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression: NoCompression 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.num_levels: 7 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T06:28:07.182 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T06:28:07.183 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T06:28:07.184 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.table_properties_collectors: 2024-05-09T06:28:07.185 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.inplace_update_support: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.bloom_locality: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.max_successive_merges: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.ttl: 2592000 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T06:28:07.186 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.enable_blob_files: false 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.min_blob_size: 0 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3aeef634-646c-4050-9255-05b80ab7fcac 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236086909491, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T06:28:07.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236086910578, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93913, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 91947, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13606, "raw_average_key_size": 49, "raw_value_size": 84485, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715236086, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3aeef634-646c-4050-9255-05b80ab7fcac", "db_session_id": "D8T0Z444FBNKFB4OQYLZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236086910662, "job": 1, "event": "recovery_finished"} 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi063/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560f1beb4000 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: DB pointer 0x560f1bea2000 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** DB Stats ** 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T06:28:07.188 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L0 2/0 93.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Sum 2/0 93.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 133.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:28:07.189 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative compaction: 0.00 GB write, 28.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval compaction: 0.00 GB write, 28.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Block cache BinnedLRUCache@0x560f1be171f0#2 capacity: 512.00 MB usage: 1.33 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: starting mon.smithi063 rank 0 at public addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] at bind addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi063 fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mon.smithi063@-1(???) e1 preinit fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:28:07.190 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mon.smithi063@-1(???).mds e1 new map 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mon.smithi063@-1(???).mds e1 print_map 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: e1 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: legacy client fscid: -1 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: No filesystems configured 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mon.smithi063@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mon.smithi063@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mon.smithi063@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mon.smithi063@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mon.smithi063@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 bash[27152]: 7d88727e4bbbd85e71c5790a652c29c6566266eae7cb5bedc66f7483ccdb0c5f 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 systemd[1]: Started Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mon.smithi063 is new leader, mons smithi063 in quorum (ranks 0) 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: monmap e1: 1 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-09T06:28:07.191 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: fsmap 2024-05-09T06:28:07.192 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T06:28:07.192 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:06 smithi063 ceph-mon[27171]: mgrmap e1: no daemons active 2024-05-09T06:28:08.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.402+0000 7f029f5c1700 1 Processor -- start 2024-05-09T06:28:08.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.402+0000 7f029f5c1700 1 -- start start 2024-05-09T06:28:08.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.402+0000 7f029f5c1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f0298106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.402+0000 7f029f5c1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0298107020 con 0x7f0298106630 2024-05-09T06:28:08.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.403+0000 7f029d35d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f0298106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.403+0000 7f029d35d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f0298106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42360/0 (socket says 172.21.15.63:42360) 2024-05-09T06:28:08.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.403+0000 7f029d35d700 1 -- 172.21.15.63:0/876334301 learned_addr learned my addr 172.21.15.63:0/876334301 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.403+0000 7f029d35d700 1 -- 172.21.15.63:0/876334301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02981077e0 con 0x7f0298106630 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.403+0000 7f029d35d700 1 --2- 172.21.15.63:0/876334301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f0298106a50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f02940098a0 tx=0x7f0294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d995ee2b1e3a8ac1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f028ffff700 1 -- 172.21.15.63:0/876334301 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0294004180 con 0x7f0298106630 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f028ffff700 1 -- 172.21.15.63:0/876334301 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f02940042e0 con 0x7f0298106630 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f028ffff700 1 -- 172.21.15.63:0/876334301 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0294011a70 con 0x7f0298106630 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f029f5c1700 1 -- 172.21.15.63:0/876334301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 msgr2=0x7f0298106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f029f5c1700 1 --2- 172.21.15.63:0/876334301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f0298106a50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f02940098a0 tx=0x7f0294009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f029f5c1700 1 -- 172.21.15.63:0/876334301 shutdown_connections 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f029f5c1700 1 --2- 172.21.15.63:0/876334301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f0298106a50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f029f5c1700 1 -- 172.21.15.63:0/876334301 >> 172.21.15.63:0/876334301 conn(0x7f0298101c30 msgr2=0x7f0298104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f029f5c1700 1 -- 172.21.15.63:0/876334301 shutdown_connections 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.404+0000 7f029f5c1700 1 -- 172.21.15.63:0/876334301 wait complete. 2024-05-09T06:28:08.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.405+0000 7f029f5c1700 1 Processor -- start 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.405+0000 7f029f5c1700 1 -- start start 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.405+0000 7f029f5c1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f029807bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.405+0000 7f029f5c1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029807c040 con 0x7f0298106630 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.405+0000 7f029d35d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f029807bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.405+0000 7f029d35d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f029807bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42370/0 (socket says 172.21.15.63:42370) 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.405+0000 7f029d35d700 1 -- 172.21.15.63:0/3805439236 learned_addr learned my addr 172.21.15.63:0/3805439236 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.406+0000 7f029d35d700 1 -- 172.21.15.63:0/3805439236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294009550 con 0x7f0298106630 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.406+0000 7f029d35d700 1 --2- 172.21.15.63:0/3805439236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f029807bb00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0294003df0 tx=0x7f02940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.406+0000 7f028e7fc700 1 -- 172.21.15.63:0/3805439236 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0294003a00 con 0x7f0298106630 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.406+0000 7f028e7fc700 1 -- 172.21.15.63:0/3805439236 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0294003b60 con 0x7f0298106630 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.406+0000 7f028e7fc700 1 -- 172.21.15.63:0/3805439236 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0294023770 con 0x7f0298106630 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.406+0000 7f029f5c1700 1 -- 172.21.15.63:0/3805439236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029807a280 con 0x7f0298106630 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.406+0000 7f029f5c1700 1 -- 172.21.15.63:0/3805439236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029807a720 con 0x7f0298106630 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.406+0000 7f028e7fc700 1 -- 172.21.15.63:0/3805439236 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0294023c20 con 0x7f0298106630 2024-05-09T06:28:08.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.406+0000 7f028e7fc700 1 -- 172.21.15.63:0/3805439236 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0294019710 con 0x7f0298106630 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.407+0000 7f029f5c1700 1 -- 172.21.15.63:0/3805439236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0298062520 con 0x7f0298106630 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.409+0000 7f028e7fc700 1 -- 172.21.15.63:0/3805439236 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f029401bbe0 con 0x7f0298106630 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.479+0000 7f029f5c1700 1 -- 172.21.15.63:0/3805439236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f029807a960 con 0x7f0298106630 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.482+0000 7f028e7fc700 1 -- 172.21.15.63:0/3805439236 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f02940198d0 con 0x7f0298106630 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.482+0000 7f029f5c1700 1 -- 172.21.15.63:0/3805439236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 msgr2=0x7f029807bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.482+0000 7f029f5c1700 1 --2- 172.21.15.63:0/3805439236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f029807bb00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0294003df0 tx=0x7f02940047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.482+0000 7f029f5c1700 1 -- 172.21.15.63:0/3805439236 shutdown_connections 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.482+0000 7f029f5c1700 1 --2- 172.21.15.63:0/3805439236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0298106630 0x7f029807bb00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.482+0000 7f029f5c1700 1 -- 172.21.15.63:0/3805439236 >> 172.21.15.63:0/3805439236 conn(0x7f0298101c30 msgr2=0x7f0298102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.482+0000 7f029f5c1700 1 -- 172.21.15.63:0/3805439236 shutdown_connections 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:07.483+0000 7f029f5c1700 1 -- 172.21.15.63:0/3805439236 wait complete. 2024-05-09T06:28:08.040 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T06:28:08.055 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:28:08.055 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mgr... 2024-05-09T06:28:08.056 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T06:28:08.056 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T06:28:08.056 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying port 0.0.0.0:8443 ... 2024-05-09T06:28:08.309 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mgr.smithi063.egfguy 2024-05-09T06:28:08.309 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mgr.smithi063.egfguy.service: Unit ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mgr.smithi063.egfguy.service not loaded. 2024-05-09T06:28:08.509 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968.target.wants/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mgr.smithi063.egfguy.service → /etc/systemd/system/ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@.service. 2024-05-09T06:28:08.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:08 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3805439236' entity='client.admin' 2024-05-09T06:28:10.130 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T06:28:10.130 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-09T06:28:10.138 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T06:28:10.138 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-09T06:28:10.138 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-09T06:28:10.138 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T06:28:10.147 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T06:28:10.147 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-09T06:28:10.155 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T06:28:10.155 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-09T06:28:10.155 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-09T06:28:10.155 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-05-09T06:28:10.155 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr to start... 2024-05-09T06:28:10.155 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr... 2024-05-09T06:28:10.980 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:10 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3012416721' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T06:28:11.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:11.236 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-09T06:28:11.236 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "2386be1c-0dcd-11ef-bc97-c7b262605968", 2024-05-09T06:28:11.236 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T06:28:11.236 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T06:28:11.236 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T06:28:11.236 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T06:28:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:11.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T06:28:11.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T06:28:11.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T06:28:11.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T06:28:11.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:11.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T06:28:11.247 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T06:27:53.399254+0000", 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.608+0000 7f2049606700 1 Processor -- start 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.608+0000 7f2049606700 1 -- start start 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.609+0000 7f2049606700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f2044109460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.609+0000 7f2049606700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2044079e90 con 0x7f2044107070 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.609+0000 7f2042ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f2044109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.609+0000 7f2042ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f2044109460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42398/0 (socket says 172.21.15.63:42398) 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.609+0000 7f2042ffd700 1 -- 172.21.15.63:0/516943021 learned_addr learned my addr 172.21.15.63:0/516943021 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.609+0000 7f2042ffd700 1 -- 172.21.15.63:0/516943021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20441099a0 con 0x7f2044107070 2024-05-09T06:28:11.248 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.609+0000 7f2042ffd700 1 --2- 172.21.15.63:0/516943021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f2044109460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f20340098a0 tx=0x7f2034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392919f90e2caae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.610+0000 7f2041ffb700 1 -- 172.21.15.63:0/516943021 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2034004180 con 0x7f2044107070 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.610+0000 7f2041ffb700 1 -- 172.21.15.63:0/516943021 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f20340042e0 con 0x7f2044107070 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.610+0000 7f2041ffb700 1 -- 172.21.15.63:0/516943021 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2034011a70 con 0x7f2044107070 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.610+0000 7f2049606700 1 -- 172.21.15.63:0/516943021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 msgr2=0x7f2044109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.610+0000 7f2049606700 1 --2- 172.21.15.63:0/516943021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f2044109460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f20340098a0 tx=0x7f2034009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.610+0000 7f2049606700 1 -- 172.21.15.63:0/516943021 shutdown_connections 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.610+0000 7f2049606700 1 --2- 172.21.15.63:0/516943021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f2044109460 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.611+0000 7f2049606700 1 -- 172.21.15.63:0/516943021 >> 172.21.15.63:0/516943021 conn(0x7f2044100b20 msgr2=0x7f2044102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.611+0000 7f2049606700 1 -- 172.21.15.63:0/516943021 shutdown_connections 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.611+0000 7f2049606700 1 -- 172.21.15.63:0/516943021 wait complete. 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.611+0000 7f2049606700 1 Processor -- start 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.612+0000 7f2049606700 1 -- start start 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.612+0000 7f2049606700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f204419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.612+0000 7f2049606700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204419a870 con 0x7f2044107070 2024-05-09T06:28:11.249 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.612+0000 7f2042ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f204419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.612+0000 7f2042ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f204419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42414/0 (socket says 172.21.15.63:42414) 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.612+0000 7f2042ffd700 1 -- 172.21.15.63:0/3012416721 learned_addr learned my addr 172.21.15.63:0/3012416721 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.612+0000 7f2042ffd700 1 -- 172.21.15.63:0/3012416721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2034009550 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.612+0000 7f2042ffd700 1 --2- 172.21.15.63:0/3012416721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f204419a330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2034011a20 tx=0x7f20340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.613+0000 7f202bfff700 1 -- 172.21.15.63:0/3012416721 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2034003b10 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.613+0000 7f202bfff700 1 -- 172.21.15.63:0/3012416721 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f203401b8c0 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.613+0000 7f2049606700 1 -- 172.21.15.63:0/3012416721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204419aa70 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.613+0000 7f202bfff700 1 -- 172.21.15.63:0/3012416721 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20340237b0 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.613+0000 7f2049606700 1 -- 172.21.15.63:0/3012416721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204419af10 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.613+0000 7f202bfff700 1 -- 172.21.15.63:0/3012416721 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f2034023ce0 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.613+0000 7f202bfff700 1 -- 172.21.15.63:0/3012416721 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2034019710 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.614+0000 7f2049606700 1 -- 172.21.15.63:0/3012416721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204410a720 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.616+0000 7f202bfff700 1 -- 172.21.15.63:0/3012416721 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f203401ba30 con 0x7f2044107070 2024-05-09T06:28:11.250 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.689+0000 7f2049606700 1 -- 172.21.15.63:0/3012416721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f204410a910 con 0x7f2044107070 2024-05-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.690+0000 7f202bfff700 1 -- 172.21.15.63:0/3012416721 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f203401bbd0 con 0x7f2044107070 2024-05-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.691+0000 7f2049606700 1 -- 172.21.15.63:0/3012416721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 msgr2=0x7f204419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.691+0000 7f2049606700 1 --2- 172.21.15.63:0/3012416721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f204419a330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2034011a20 tx=0x7f20340047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.691+0000 7f2049606700 1 -- 172.21.15.63:0/3012416721 shutdown_connections 2024-05-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.691+0000 7f2049606700 1 --2- 172.21.15.63:0/3012416721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2044107070 0x7f204419a330 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.691+0000 7f2049606700 1 -- 172.21.15.63:0/3012416721 >> 172.21.15.63:0/3012416721 conn(0x7f2044100b20 msgr2=0x7f2044101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.691+0000 7f2049606700 1 -- 172.21.15.63:0/3012416721 shutdown_connections 2024-05-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:10.691+0000 7f2049606700 1 -- 172.21.15.63:0/3012416721 wait complete. 2024-05-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (1/15)... 2024-05-09T06:28:14.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:13 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2496526687' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T06:28:14.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:14.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "2386be1c-0dcd-11ef-bc97-c7b262605968", 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:14.392 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:14.393 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:14.394 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T06:27:53.399254+0000", 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:14.395 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.695+0000 7fc80ec20700 1 Processor -- start 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.695+0000 7fc80ec20700 1 -- start start 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.695+0000 7fc80ec20700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc808106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.695+0000 7fc80ec20700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc808106fd0 con 0x7fc8081065e0 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.695+0000 7fc80c9bc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc808106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.695+0000 7fc80c9bc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc808106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42426/0 (socket says 172.21.15.63:42426) 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.695+0000 7fc80c9bc700 1 -- 172.21.15.63:0/727938026 learned_addr learned my addr 172.21.15.63:0/727938026 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.696+0000 7fc80c9bc700 1 -- 172.21.15.63:0/727938026 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808107790 con 0x7fc8081065e0 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.696+0000 7fc80c9bc700 1 --2- 172.21.15.63:0/727938026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc808106a00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098a0 tx=0x7fc7fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e774da53b5f6d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.696+0000 7fc8077fe700 1 -- 172.21.15.63:0/727938026 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7fc004180 con 0x7fc8081065e0 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.696+0000 7fc8077fe700 1 -- 172.21.15.63:0/727938026 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc7fc0042e0 con 0x7fc8081065e0 2024-05-09T06:28:14.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.696+0000 7fc8077fe700 1 -- 172.21.15.63:0/727938026 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7fc011a70 con 0x7fc8081065e0 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.697+0000 7fc80ec20700 1 -- 172.21.15.63:0/727938026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 msgr2=0x7fc808106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.697+0000 7fc80ec20700 1 --2- 172.21.15.63:0/727938026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc808106a00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098a0 tx=0x7fc7fc009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.697+0000 7fc80ec20700 1 -- 172.21.15.63:0/727938026 shutdown_connections 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.697+0000 7fc80ec20700 1 --2- 172.21.15.63:0/727938026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc808106a00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.697+0000 7fc80ec20700 1 -- 172.21.15.63:0/727938026 >> 172.21.15.63:0/727938026 conn(0x7fc808101c00 msgr2=0x7fc808104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.697+0000 7fc80ec20700 1 -- 172.21.15.63:0/727938026 shutdown_connections 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.697+0000 7fc80ec20700 1 -- 172.21.15.63:0/727938026 wait complete. 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.697+0000 7fc80ec20700 1 Processor -- start 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.697+0000 7fc80ec20700 1 -- start start 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc80ec20700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc80819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc80ec20700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80819a890 con 0x7fc8081065e0 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc80c9bc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc80819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc80c9bc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc80819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42438/0 (socket says 172.21.15.63:42438) 2024-05-09T06:28:14.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc80c9bc700 1 -- 172.21.15.63:0/2496526687 learned_addr learned my addr 172.21.15.63:0/2496526687 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc80c9bc700 1 -- 172.21.15.63:0/2496526687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc009550 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc80c9bc700 1 --2- 172.21.15.63:0/2496526687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc80819a350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc011710 tx=0x7fc7fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc805ffb700 1 -- 172.21.15.63:0/2496526687 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7fc003b10 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc805ffb700 1 -- 172.21.15.63:0/2496526687 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc7fc01b8c0 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.698+0000 7fc805ffb700 1 -- 172.21.15.63:0/2496526687 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7fc019770 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.699+0000 7fc80ec20700 1 -- 172.21.15.63:0/2496526687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80819aa90 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.699+0000 7fc80ec20700 1 -- 172.21.15.63:0/2496526687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80819af30 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.699+0000 7fc805ffb700 1 -- 172.21.15.63:0/2496526687 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fc7fc019ca0 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.699+0000 7fc805ffb700 1 -- 172.21.15.63:0/2496526687 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc7fc01a710 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.700+0000 7fc80ec20700 1 -- 172.21.15.63:0/2496526687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc808062520 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.704+0000 7fc805ffb700 1 -- 172.21.15.63:0/2496526687 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fc7fc029060 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.778+0000 7fc80ec20700 1 -- 172.21.15.63:0/2496526687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc80810a960 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.779+0000 7fc805ffb700 1 -- 172.21.15.63:0/2496526687 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fc7fc01f070 con 0x7fc8081065e0 2024-05-09T06:28:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.780+0000 7fc80ec20700 1 -- 172.21.15.63:0/2496526687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 msgr2=0x7fc80819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:14.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.780+0000 7fc80ec20700 1 --2- 172.21.15.63:0/2496526687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc80819a350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc011710 tx=0x7fc7fc0047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.780+0000 7fc80ec20700 1 -- 172.21.15.63:0/2496526687 shutdown_connections 2024-05-09T06:28:14.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.780+0000 7fc80ec20700 1 --2- 172.21.15.63:0/2496526687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8081065e0 0x7fc80819a350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:14.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.780+0000 7fc80ec20700 1 -- 172.21.15.63:0/2496526687 >> 172.21.15.63:0/2496526687 conn(0x7fc808101c00 msgr2=0x7fc808102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:14.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.780+0000 7fc80ec20700 1 -- 172.21.15.63:0/2496526687 shutdown_connections 2024-05-09T06:28:14.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:13.780+0000 7fc80ec20700 1 -- 172.21.15.63:0/2496526687 wait complete. 2024-05-09T06:28:14.399 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (2/15)... 2024-05-09T06:28:17.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:16 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/797015675' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T06:28:17.436 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:17.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-09T06:28:17.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "2386be1c-0dcd-11ef-bc97-c7b262605968", 2024-05-09T06:28:17.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T06:28:17.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-09T06:28:17.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:17.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:17.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:17.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T06:27:53.399254+0000", 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.835+0000 7f9664ce4700 1 Processor -- start 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.835+0000 7f9664ce4700 1 -- start start 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.836+0000 7f9664ce4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f9660109490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.836+0000 7f9664ce4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660074900 con 0x7f96601070a0 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.836+0000 7f965e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f9660109490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.836+0000 7f965e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f9660109490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38110/0 (socket says 172.21.15.63:38110) 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.836+0000 7f965e59c700 1 -- 172.21.15.63:0/4115182297 learned_addr learned my addr 172.21.15.63:0/4115182297 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.836+0000 7f965e59c700 1 -- 172.21.15.63:0/4115182297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96601099d0 con 0x7f96601070a0 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.836+0000 7f965e59c700 1 --2- 172.21.15.63:0/4115182297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f9660109490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96500098a0 tx=0x7f9650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2294820ffdb76a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.837+0000 7f965d59a700 1 -- 172.21.15.63:0/4115182297 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9650004180 con 0x7f96601070a0 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.837+0000 7f965d59a700 1 -- 172.21.15.63:0/4115182297 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f96500042e0 con 0x7f96601070a0 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.837+0000 7f965d59a700 1 -- 172.21.15.63:0/4115182297 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9650011a70 con 0x7f96601070a0 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.837+0000 7f9664ce4700 1 -- 172.21.15.63:0/4115182297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 msgr2=0x7f9660109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.837+0000 7f9664ce4700 1 --2- 172.21.15.63:0/4115182297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f9660109490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96500098a0 tx=0x7f9650009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.837+0000 7f9664ce4700 1 -- 172.21.15.63:0/4115182297 shutdown_connections 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.837+0000 7f9664ce4700 1 --2- 172.21.15.63:0/4115182297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f9660109490 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.837+0000 7f9664ce4700 1 -- 172.21.15.63:0/4115182297 >> 172.21.15.63:0/4115182297 conn(0x7f9660100a90 msgr2=0x7f9660102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.837+0000 7f9664ce4700 1 -- 172.21.15.63:0/4115182297 shutdown_connections 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.838+0000 7f9664ce4700 1 -- 172.21.15.63:0/4115182297 wait complete. 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.838+0000 7f9664ce4700 1 Processor -- start 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.839+0000 7f9664ce4700 1 -- start start 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.839+0000 7f9664ce4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f966019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.839+0000 7f9664ce4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019a880 con 0x7f96601070a0 2024-05-09T06:28:17.455 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.839+0000 7f965e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f966019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.839+0000 7f965e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f966019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38126/0 (socket says 172.21.15.63:38126) 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.839+0000 7f965e59c700 1 -- 172.21.15.63:0/797015675 learned_addr learned my addr 172.21.15.63:0/797015675 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.839+0000 7f965e59c700 1 -- 172.21.15.63:0/797015675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650009550 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.839+0000 7f965e59c700 1 --2- 172.21.15.63:0/797015675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f966019a340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9650011a20 tx=0x7f96500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.840+0000 7f964f7fe700 1 -- 172.21.15.63:0/797015675 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9650003b10 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.840+0000 7f964f7fe700 1 -- 172.21.15.63:0/797015675 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f965001b8c0 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.840+0000 7f964f7fe700 1 -- 172.21.15.63:0/797015675 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96500237b0 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.840+0000 7f9664ce4700 1 -- 172.21.15.63:0/797015675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966019aa80 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.840+0000 7f9664ce4700 1 -- 172.21.15.63:0/797015675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966019af20 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.841+0000 7f964f7fe700 1 -- 172.21.15.63:0/797015675 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9650023d30 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.841+0000 7f964f7fe700 1 -- 172.21.15.63:0/797015675 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9650019710 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.841+0000 7f9664ce4700 1 -- 172.21.15.63:0/797015675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966010a750 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.843+0000 7f964f7fe700 1 -- 172.21.15.63:0/797015675 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f965001ba30 con 0x7f96601070a0 2024-05-09T06:28:17.456 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.918+0000 7f9664ce4700 1 -- 172.21.15.63:0/797015675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f966010a940 con 0x7f96601070a0 2024-05-09T06:28:17.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.918+0000 7f964f7fe700 1 -- 172.21.15.63:0/797015675 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f965001bbd0 con 0x7f96601070a0 2024-05-09T06:28:17.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.920+0000 7f9664ce4700 1 -- 172.21.15.63:0/797015675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 msgr2=0x7f966019a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:17.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.920+0000 7f9664ce4700 1 --2- 172.21.15.63:0/797015675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f966019a340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9650011a20 tx=0x7f96500047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.920+0000 7f9664ce4700 1 -- 172.21.15.63:0/797015675 shutdown_connections 2024-05-09T06:28:17.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.920+0000 7f9664ce4700 1 --2- 172.21.15.63:0/797015675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96601070a0 0x7f966019a340 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:17.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.920+0000 7f9664ce4700 1 -- 172.21.15.63:0/797015675 >> 172.21.15.63:0/797015675 conn(0x7f9660100a90 msgr2=0x7f9660101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:17.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.920+0000 7f9664ce4700 1 -- 172.21.15.63:0/797015675 shutdown_connections 2024-05-09T06:28:17.457 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:16.920+0000 7f9664ce4700 1 -- 172.21.15.63:0/797015675 wait complete. 2024-05-09T06:28:17.457 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (3/15)... 2024-05-09T06:28:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: Activating manager daemon smithi063.egfguy 2024-05-09T06:28:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: mgrmap e2: smithi063.egfguy(active, starting, since 0.00320316s) 2024-05-09T06:28:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:28:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:28:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:28:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:28:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:28:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: Manager daemon smithi063.egfguy is now available 2024-05-09T06:28:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:28:18.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/trash_purge_schedule"}]: dispatch 2024-05-09T06:28:18.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:18.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:18.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:17 smithi063 ceph-mon[27171]: from='mgr.14100 172.21.15.63:0/3458720987' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:19.568 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:19 smithi063 ceph-mon[27171]: mgrmap e3: smithi063.egfguy(active, since 1.0074s) 2024-05-09T06:28:20.415 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:20 smithi063 ceph-mon[27171]: mgrmap e4: smithi063.egfguy(active, since 2s) 2024-05-09T06:28:20.415 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:20 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/933999004' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T06:28:20.678 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:20.678 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-09T06:28:20.678 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "2386be1c-0dcd-11ef-bc97-c7b262605968", 2024-05-09T06:28:20.678 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T06:28:20.678 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T06:28:20.678 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T06:28:20.678 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T06:28:20.678 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:20.679 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T06:28:20.679 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T06:28:20.690 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T06:28:20.691 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T06:27:53.399254+0000", 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.847+0000 7f420f1c4700 1 Processor -- start 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.847+0000 7f420f1c4700 1 -- start start 2024-05-09T06:28:20.692 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.847+0000 7f420f1c4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f4208106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.847+0000 7f420f1c4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4208107030 con 0x7f4208106640 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.847+0000 7f420cf60700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f4208106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.847+0000 7f420cf60700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f4208106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38190/0 (socket says 172.21.15.63:38190) 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.847+0000 7f420cf60700 1 -- 172.21.15.63:0/327326624 learned_addr learned my addr 172.21.15.63:0/327326624 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.848+0000 7f420cf60700 1 -- 172.21.15.63:0/327326624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42081077f0 con 0x7f4208106640 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.848+0000 7f420cf60700 1 --2- 172.21.15.63:0/327326624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f4208106a60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41f00098a0 tx=0x7f41f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957de1e420fd5cca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.848+0000 7f42077fe700 1 -- 172.21.15.63:0/327326624 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41f0004180 con 0x7f4208106640 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.848+0000 7f42077fe700 1 -- 172.21.15.63:0/327326624 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f41f00042e0 con 0x7f4208106640 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.848+0000 7f42077fe700 1 -- 172.21.15.63:0/327326624 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41f0011a70 con 0x7f4208106640 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.849+0000 7f420f1c4700 1 -- 172.21.15.63:0/327326624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 msgr2=0x7f4208106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.849+0000 7f420f1c4700 1 --2- 172.21.15.63:0/327326624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f4208106a60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41f00098a0 tx=0x7f41f0009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.849+0000 7f420f1c4700 1 -- 172.21.15.63:0/327326624 shutdown_connections 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.849+0000 7f420f1c4700 1 --2- 172.21.15.63:0/327326624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f4208106a60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.693 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.849+0000 7f420f1c4700 1 -- 172.21.15.63:0/327326624 >> 172.21.15.63:0/327326624 conn(0x7f4208101c40 msgr2=0x7f4208104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.849+0000 7f420f1c4700 1 -- 172.21.15.63:0/327326624 shutdown_connections 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.849+0000 7f420f1c4700 1 -- 172.21.15.63:0/327326624 wait complete. 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.849+0000 7f420f1c4700 1 Processor -- start 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.849+0000 7f420f1c4700 1 -- start start 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f420f1c4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f420819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f420f1c4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420819a900 con 0x7f4208106640 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f420cf60700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f420819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f420cf60700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f420819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38206/0 (socket says 172.21.15.63:38206) 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f420cf60700 1 -- 172.21.15.63:0/933999004 learned_addr learned my addr 172.21.15.63:0/933999004 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f420cf60700 1 -- 172.21.15.63:0/933999004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0009550 con 0x7f4208106640 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f420cf60700 1 --2- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f420819a3c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006af0 tx=0x7f41f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f4205ffb700 1 -- 172.21.15.63:0/933999004 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41f0003a00 con 0x7f4208106640 2024-05-09T06:28:20.694 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f4205ffb700 1 -- 172.21.15.63:0/933999004 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f41f0003b60 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420819ab00 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.850+0000 7f4205ffb700 1 -- 172.21.15.63:0/933999004 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41f0023750 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.851+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420819afa0 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.852+0000 7f4205ffb700 1 -- 172.21.15.63:0/933999004 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) v1 ==== 45545+0+0 (secure 0 0 0) 0x7f41f0023990 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.852+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420810aa20 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.852+0000 7f4205ffb700 1 --2- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f41f40386d0 0x7f41f403ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.852+0000 7f4205ffb700 1 -- 172.21.15.63:0/933999004 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f41f004c950 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.852+0000 7f4207fff700 1 --2- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f41f40386d0 0x7f41f403ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.852+0000 7f4207fff700 1 --2- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f41f40386d0 0x7f41f403ab90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f41f800ad30 tx=0x7f41f80093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:19.857+0000 7f4205ffb700 1 -- 172.21.15.63:0/933999004 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f41f002bd20 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.141+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f420819d890 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.141+0000 7f4205ffb700 1 -- 172.21.15.63:0/933999004 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f41f002bb40 con 0x7f4208106640 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f41f40386d0 msgr2=0x7f41f403ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 --2- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f41f40386d0 0x7f41f403ab90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f41f800ad30 tx=0x7f41f80093f0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.696 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 msgr2=0x7f420819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:20.696 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 --2- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f420819a3c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006af0 tx=0x7f41f00047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.696 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 shutdown_connections 2024-05-09T06:28:20.696 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 --2- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f41f40386d0 0x7f41f403ab90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.696 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 --2- 172.21.15.63:0/933999004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4208106640 0x7f420819a3c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:20.696 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 >> 172.21.15.63:0/933999004 conn(0x7f4208101c40 msgr2=0x7f4208102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:20.696 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 shutdown_connections 2024-05-09T06:28:20.696 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:20.144+0000 7f420f1c4700 1 -- 172.21.15.63:0/933999004 wait complete. 2024-05-09T06:28:20.696 INFO:teuthology.orchestra.run.smithi063.stdout:mgr is available 2024-05-09T06:28:21.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:21 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/702780213' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T06:28:21.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:21.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T06:28:21.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:28:21.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T06:28:21.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T06:28:21.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T06:28:21.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T06:28:21.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.145+0000 7f13b6ad8700 1 Processor -- start 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.145+0000 7f13b6ad8700 1 -- start start 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.145+0000 7f13b6ad8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b01052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.145+0000 7f13b6ad8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0079e30 con 0x7f13b0104ed0 2024-05-09T06:28:21.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.145+0000 7f13b4874700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b01052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.145+0000 7f13b4874700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b01052f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38214/0 (socket says 172.21.15.63:38214) 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.145+0000 7f13b4874700 1 -- 172.21.15.63:0/74903706 learned_addr learned my addr 172.21.15.63:0/74903706 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.145+0000 7f13b4874700 1 -- 172.21.15.63:0/74903706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b0105830 con 0x7f13b0104ed0 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.146+0000 7f13b4874700 1 --2- 172.21.15.63:0/74903706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b01052f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f13a40098a0 tx=0x7f13a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d328a536564e0e57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.146+0000 7f13af7fe700 1 -- 172.21.15.63:0/74903706 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13a4004180 con 0x7f13b0104ed0 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.146+0000 7f13af7fe700 1 -- 172.21.15.63:0/74903706 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f13a40042e0 con 0x7f13b0104ed0 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.146+0000 7f13af7fe700 1 -- 172.21.15.63:0/74903706 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13a4011a70 con 0x7f13b0104ed0 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.146+0000 7f13b6ad8700 1 -- 172.21.15.63:0/74903706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 msgr2=0x7f13b01052f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.146+0000 7f13b6ad8700 1 --2- 172.21.15.63:0/74903706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b01052f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f13a40098a0 tx=0x7f13a4009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.146+0000 7f13b6ad8700 1 -- 172.21.15.63:0/74903706 shutdown_connections 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.146+0000 7f13b6ad8700 1 --2- 172.21.15.63:0/74903706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b01052f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.146+0000 7f13b6ad8700 1 -- 172.21.15.63:0/74903706 >> 172.21.15.63:0/74903706 conn(0x7f13b0100b10 msgr2=0x7f13b0102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.147+0000 7f13b6ad8700 1 -- 172.21.15.63:0/74903706 shutdown_connections 2024-05-09T06:28:21.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.147+0000 7f13b6ad8700 1 -- 172.21.15.63:0/74903706 wait complete. 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.147+0000 7f13b6ad8700 1 Processor -- start 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.147+0000 7f13b6ad8700 1 -- start start 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.147+0000 7f13b6ad8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.147+0000 7f13b6ad8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b019ca80 con 0x7f13b0104ed0 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.147+0000 7f13b4874700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.148+0000 7f13b4874700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38222/0 (socket says 172.21.15.63:38222) 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.148+0000 7f13b4874700 1 -- 172.21.15.63:0/702780213 learned_addr learned my addr 172.21.15.63:0/702780213 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.148+0000 7f13b4874700 1 -- 172.21.15.63:0/702780213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4009550 con 0x7f13b0104ed0 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.148+0000 7f13b4874700 1 --2- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b019c540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13a4011a20 tx=0x7f13a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.148+0000 7f13adffb700 1 -- 172.21.15.63:0/702780213 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13a4003b10 con 0x7f13b0104ed0 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.148+0000 7f13adffb700 1 -- 172.21.15.63:0/702780213 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f13a401b8c0 con 0x7f13b0104ed0 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.148+0000 7f13adffb700 1 -- 172.21.15.63:0/702780213 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13a40237b0 con 0x7f13b0104ed0 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.148+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b019cc80 con 0x7f13b0104ed0 2024-05-09T06:28:21.937 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.148+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b019d120 con 0x7f13b0104ed0 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.149+0000 7f13adffb700 1 -- 172.21.15.63:0/702780213 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) v1 ==== 45545+0+0 (secure 0 0 0) 0x7f13a401ba30 con 0x7f13b0104ed0 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.149+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b0196250 con 0x7f13b0104ed0 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.149+0000 7f13adffb700 1 --2- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f13a00386e0 0x7f13a003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.149+0000 7f13adffb700 1 -- 172.21.15.63:0/702780213 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f13a404c990 con 0x7f13b0104ed0 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.149+0000 7f13affff700 1 --2- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f13a00386e0 0x7f13a003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.150+0000 7f13affff700 1 --2- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f13a00386e0 0x7f13a003aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f139c006fd0 tx=0x7f139c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.154+0000 7f13adffb700 1 -- 172.21.15.63:0/702780213 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f13a402b940 con 0x7f13b0104ed0 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.355+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f13b0106650 con 0x7f13b0104ed0 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.356+0000 7f13adffb700 1 -- 172.21.15.63:0/702780213 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f13a400bc90 con 0x7f13b0104ed0 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.358+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f13a00386e0 msgr2=0x7f13a003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.358+0000 7f13b6ad8700 1 --2- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f13a00386e0 0x7f13a003aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f139c006fd0 tx=0x7f139c006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.359+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 msgr2=0x7f13b019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:21.938 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.359+0000 7f13b6ad8700 1 --2- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b019c540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13a4011a20 tx=0x7f13a40047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.939 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.359+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 shutdown_connections 2024-05-09T06:28:21.939 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.359+0000 7f13b6ad8700 1 --2- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f13a00386e0 0x7f13a003aba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.939 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.359+0000 7f13b6ad8700 1 --2- 172.21.15.63:0/702780213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b0104ed0 0x7f13b019c540 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:21.939 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.359+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 >> 172.21.15.63:0/702780213 conn(0x7f13b0100b10 msgr2=0x7f13b0193390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:21.939 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.359+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 shutdown_connections 2024-05-09T06:28:21.939 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:21.359+0000 7f13b6ad8700 1 -- 172.21.15.63:0/702780213 wait complete. 2024-05-09T06:28:21.939 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling cephadm module... 2024-05-09T06:28:22.952 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:22 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1120648450' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T06:28:23.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.371+0000 7f1776a96700 1 Processor -- start 2024-05-09T06:28:23.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.371+0000 7f1776a96700 1 -- start start 2024-05-09T06:28:23.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.371+0000 7f1776a96700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.371+0000 7f1776a96700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177007a8b0 con 0x7f177007be30 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.372+0000 7f1774832700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.372+0000 7f1774832700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38224/0 (socket says 172.21.15.63:38224) 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.372+0000 7f1774832700 1 -- 172.21.15.63:0/2658804538 learned_addr learned my addr 172.21.15.63:0/2658804538 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.372+0000 7f1774832700 1 -- 172.21.15.63:0/2658804538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177007a9f0 con 0x7f177007be30 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.372+0000 7f1774832700 1 --2- 172.21.15.63:0/2658804538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177007a2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1760009e80 tx=0x7f176000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c432b27aa7ffa0a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.372+0000 7f176f7fe700 1 -- 172.21.15.63:0/2658804538 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f176000e070 con 0x7f177007be30 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.372+0000 7f176f7fe700 1 -- 172.21.15.63:0/2658804538 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f176000b8c0 con 0x7f177007be30 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.372+0000 7f176f7fe700 1 -- 172.21.15.63:0/2658804538 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1760004440 con 0x7f177007be30 2024-05-09T06:28:23.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.373+0000 7f1776a96700 1 -- 172.21.15.63:0/2658804538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 msgr2=0x7f177007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.373+0000 7f1776a96700 1 --2- 172.21.15.63:0/2658804538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177007a2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1760009e80 tx=0x7f176000b330 comp rx=0 tx=0).stop 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.373+0000 7f1776a96700 1 -- 172.21.15.63:0/2658804538 shutdown_connections 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.373+0000 7f1776a96700 1 --2- 172.21.15.63:0/2658804538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177007a2e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.373+0000 7f1776a96700 1 -- 172.21.15.63:0/2658804538 >> 172.21.15.63:0/2658804538 conn(0x7f1770101c00 msgr2=0x7f1770104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.373+0000 7f1776a96700 1 -- 172.21.15.63:0/2658804538 shutdown_connections 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.373+0000 7f1776a96700 1 -- 172.21.15.63:0/2658804538 wait complete. 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.373+0000 7f1776a96700 1 Processor -- start 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.373+0000 7f1776a96700 1 -- start start 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f1776a96700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f1776a96700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177019c9f0 con 0x7f177007be30 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f1774832700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f1774832700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38230/0 (socket says 172.21.15.63:38230) 2024-05-09T06:28:23.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f1774832700 1 -- 172.21.15.63:0/1120648450 learned_addr learned my addr 172.21.15.63:0/1120648450 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f1774832700 1 -- 172.21.15.63:0/1120648450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1760009b30 con 0x7f177007be30 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f1774832700 1 --2- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177019c4b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1760003c50 tx=0x7f1760003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f176dffb700 1 -- 172.21.15.63:0/1120648450 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f176000e040 con 0x7f177007be30 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177019cbf0 con 0x7f177007be30 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f176dffb700 1 -- 172.21.15.63:0/1120648450 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f176000bc40 con 0x7f177007be30 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.374+0000 7f176dffb700 1 -- 172.21.15.63:0/1120648450 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17600125e0 con 0x7f177007be30 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.375+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177019d0f0 con 0x7f177007be30 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.375+0000 7f176dffb700 1 -- 172.21.15.63:0/1120648450 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) v1 ==== 45545+0+0 (secure 0 0 0) 0x7f1760019040 con 0x7f177007be30 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.376+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177010c8e0 con 0x7f177007be30 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.376+0000 7f176dffb700 1 --2- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f175c0386b0 0x7f175c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.376+0000 7f176dffb700 1 -- 172.21.15.63:0/1120648450 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f176004bd10 con 0x7f177007be30 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.376+0000 7f176ffff700 1 --2- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f175c0386b0 0x7f175c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:23.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.376+0000 7f176ffff700 1 --2- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f175c0386b0 0x7f175c03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1758006fd0 tx=0x7f1758006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.381+0000 7f176dffb700 1 -- 172.21.15.63:0/1120648450 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1760051020 con 0x7f177007be30 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.639+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f177010cb50 con 0x7f177007be30 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.692+0000 7f176dffb700 1 -- 172.21.15.63:0/1120648450 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 5) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f176000f250 con 0x7f177007be30 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.693+0000 7f176dffb700 1 -- 172.21.15.63:0/1120648450 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f176004ba70 con 0x7f177007be30 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f175c0386b0 msgr2=0x7f175c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 --2- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f175c0386b0 0x7f175c03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1758006fd0 tx=0x7f1758006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 msgr2=0x7f177019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 --2- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177019c4b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1760003c50 tx=0x7f1760003d30 comp rx=0 tx=0).stop 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 shutdown_connections 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 --2- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f175c0386b0 0x7f175c03ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 --2- 172.21.15.63:0/1120648450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f177007be30 0x7f177019c4b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:23.308 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 >> 172.21.15.63:0/1120648450 conn(0x7f1770101c00 msgr2=0x7f1770102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:23.309 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 shutdown_connections 2024-05-09T06:28:23.309 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:22.696+0000 7f1776a96700 1 -- 172.21.15.63:0/1120648450 wait complete. 2024-05-09T06:28:23.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:23 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1120648450' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T06:28:23.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:23 smithi063 ceph-mon[27171]: mgrmap e5: smithi063.egfguy(active, since 5s) 2024-05-09T06:28:24.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-09T06:28:24.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "smithi063.egfguy", 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.777+0000 7f1d6999f700 1 Processor -- start 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.777+0000 7f1d6999f700 1 -- start start 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.777+0000 7f1d6999f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d64106a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.777+0000 7f1d6999f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64107060 con 0x7f1d64106670 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.777+0000 7f1d62ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d64106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.777+0000 7f1d62ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d64106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38262/0 (socket says 172.21.15.63:38262) 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.777+0000 7f1d62ffd700 1 -- 172.21.15.63:0/412133867 learned_addr learned my addr 172.21.15.63:0/412133867 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.777+0000 7f1d62ffd700 1 -- 172.21.15.63:0/412133867 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64107820 con 0x7f1d64106670 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.778+0000 7f1d62ffd700 1 --2- 172.21.15.63:0/412133867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d64106a90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1d540098a0 tx=0x7f1d54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb4301554b4be1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.778+0000 7f1d61ffb700 1 -- 172.21.15.63:0/412133867 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d54004180 con 0x7f1d64106670 2024-05-09T06:28:24.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.778+0000 7f1d61ffb700 1 -- 172.21.15.63:0/412133867 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1d540042e0 con 0x7f1d64106670 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.778+0000 7f1d61ffb700 1 -- 172.21.15.63:0/412133867 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d54011a70 con 0x7f1d64106670 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.778+0000 7f1d6999f700 1 -- 172.21.15.63:0/412133867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 msgr2=0x7f1d64106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.778+0000 7f1d6999f700 1 --2- 172.21.15.63:0/412133867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d64106a90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1d540098a0 tx=0x7f1d54009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.778+0000 7f1d6999f700 1 -- 172.21.15.63:0/412133867 shutdown_connections 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.778+0000 7f1d6999f700 1 --2- 172.21.15.63:0/412133867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d64106a90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.778+0000 7f1d6999f700 1 -- 172.21.15.63:0/412133867 >> 172.21.15.63:0/412133867 conn(0x7f1d64101c70 msgr2=0x7f1d64104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.779+0000 7f1d6999f700 1 -- 172.21.15.63:0/412133867 shutdown_connections 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.779+0000 7f1d6999f700 1 -- 172.21.15.63:0/412133867 wait complete. 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.779+0000 7f1d6999f700 1 Processor -- start 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.779+0000 7f1d6999f700 1 -- start start 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.779+0000 7f1d6999f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d6419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.779+0000 7f1d6999f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419a940 con 0x7f1d64106670 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d62ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d6419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d62ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d6419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38278/0 (socket says 172.21.15.63:38278) 2024-05-09T06:28:24.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d62ffd700 1 -- 172.21.15.63:0/2365986229 learned_addr learned my addr 172.21.15.63:0/2365986229 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d62ffd700 1 -- 172.21.15.63:0/2365986229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54009550 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d62ffd700 1 --2- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d6419a400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d54003df0 tx=0x7f1d540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d6899d700 1 -- 172.21.15.63:0/2365986229 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d54003a00 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d6899d700 1 -- 172.21.15.63:0/2365986229 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1d54003b60 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6419ab40 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d6899d700 1 -- 172.21.15.63:0/2365986229 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d54023770 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.780+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6419afe0 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.781+0000 7f1d6899d700 1 -- 172.21.15.63:0/2365986229 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f1d5402c3f0 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.782+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6410aa50 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.782+0000 7f1d6899d700 1 --2- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f1d4c0386e0 0x7f1d4c03aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.782+0000 7f1d6899d700 1 -- 172.21.15.63:0/2365986229 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1d5404c9c0 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.782+0000 7f1d627fc700 1 -- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f1d4c0386e0 msgr2=0x7f1d4c03aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1739893919 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.782+0000 7f1d627fc700 1 --2- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f1d4c0386e0 0x7f1d4c03aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.786+0000 7f1d6899d700 1 -- 172.21.15.63:0/2365986229 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1d54018710 con 0x7f1d64106670 2024-05-09T06:28:24.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.983+0000 7f1d627fc700 1 -- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f1d4c0386e0 msgr2=0x7f1d4c03aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1739893919 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:23.983+0000 7f1d627fc700 1 --2- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f1d4c0386e0 0x7f1d4c03aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.091+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1d6407a8b0 con 0x7f1d64106670 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.092+0000 7f1d6899d700 1 -- 172.21.15.63:0/2365986229 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f1d54018350 con 0x7f1d64106670 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f1d4c0386e0 msgr2=0x7f1d4c03aba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 --2- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f1d4c0386e0 0x7f1d4c03aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 msgr2=0x7f1d6419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 --2- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d6419a400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d54003df0 tx=0x7f1d540047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 shutdown_connections 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 --2- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f1d4c0386e0 0x7f1d4c03aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 --2- 172.21.15.63:0/2365986229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d64106670 0x7f1d6419a400 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 >> 172.21.15.63:0/2365986229 conn(0x7f1d64101c70 msgr2=0x7f1d64102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 shutdown_connections 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.094+0000 7f1d6999f700 1 -- 172.21.15.63:0/2365986229 wait complete. 2024-05-09T06:28:24.587 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-05-09T06:28:24.588 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 5... 2024-05-09T06:28:24.955 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:24 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2365986229' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T06:28:30.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: Active manager daemon smithi063.egfguy restarted 2024-05-09T06:28:30.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: Activating manager daemon smithi063.egfguy 2024-05-09T06:28:30.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T06:28:30.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: mgrmap e6: smithi063.egfguy(active, starting, since 0.00299357s) 2024-05-09T06:28:30.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:28:30.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: Manager daemon smithi063.egfguy is now available 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:28:30.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:29 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/trash_purge_schedule"}]: dispatch 2024-05-09T06:28:31.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:30 smithi063 ceph-mon[27171]: Found migration_current of "None". Setting to last migration. 2024-05-09T06:28:31.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:30 smithi063 ceph-mon[27171]: mgrmap e7: smithi063.egfguy(active, since 1.00535s) 2024-05-09T06:28:31.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-09T06:28:31.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T06:28:31.449 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.990+0000 7f906a1cb700 1 Processor -- start 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.990+0000 7f906a1cb700 1 -- start start 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.990+0000 7f906a1cb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f9064105930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.990+0000 7f906a1cb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9064105f00 con 0x7f9064105510 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.991+0000 7f90637fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f9064105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.991+0000 7f90637fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f9064105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42644/0 (socket says 172.21.15.63:42644) 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.991+0000 7f90637fe700 1 -- 172.21.15.63:0/2755355098 learned_addr learned my addr 172.21.15.63:0/2755355098 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.991+0000 7f90637fe700 1 -- 172.21.15.63:0/2755355098 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9064106710 con 0x7f9064105510 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.991+0000 7f90637fe700 1 --2- 172.21.15.63:0/2755355098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f9064105930 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f90580098a0 tx=0x7f9058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55a868b1f07c293 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.991+0000 7f90627fc700 1 -- 172.21.15.63:0/2755355098 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9058004180 con 0x7f9064105510 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.991+0000 7f90627fc700 1 -- 172.21.15.63:0/2755355098 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f90580042e0 con 0x7f9064105510 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.991+0000 7f90627fc700 1 -- 172.21.15.63:0/2755355098 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9058011a70 con 0x7f9064105510 2024-05-09T06:28:31.450 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.992+0000 7f906a1cb700 1 -- 172.21.15.63:0/2755355098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 msgr2=0x7f9064105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.992+0000 7f906a1cb700 1 --2- 172.21.15.63:0/2755355098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f9064105930 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f90580098a0 tx=0x7f9058009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.992+0000 7f906a1cb700 1 -- 172.21.15.63:0/2755355098 shutdown_connections 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.992+0000 7f906a1cb700 1 --2- 172.21.15.63:0/2755355098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f9064105930 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.992+0000 7f906a1cb700 1 -- 172.21.15.63:0/2755355098 >> 172.21.15.63:0/2755355098 conn(0x7f9064100b10 msgr2=0x7f9064102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.992+0000 7f906a1cb700 1 -- 172.21.15.63:0/2755355098 shutdown_connections 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.992+0000 7f906a1cb700 1 -- 172.21.15.63:0/2755355098 wait complete. 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.993+0000 7f906a1cb700 1 Processor -- start 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.993+0000 7f906a1cb700 1 -- start start 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.993+0000 7f906a1cb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f906419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.993+0000 7f906a1cb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906419a8e0 con 0x7f9064105510 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.993+0000 7f90637fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f906419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.993+0000 7f90637fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f906419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42650/0 (socket says 172.21.15.63:42650) 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.993+0000 7f90637fe700 1 -- 172.21.15.63:0/3563723843 learned_addr learned my addr 172.21.15.63:0/3563723843 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.994+0000 7f90637fe700 1 -- 172.21.15.63:0/3563723843 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058009550 con 0x7f9064105510 2024-05-09T06:28:31.451 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.994+0000 7f90637fe700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f906419a3a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9058011710 tx=0x7f90580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.994+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9058003b10 con 0x7f9064105510 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.994+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f905801b8c0 con 0x7f9064105510 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.994+0000 7f906a1cb700 1 -- 172.21.15.63:0/3563723843 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906419aae0 con 0x7f9064105510 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.994+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9058019710 con 0x7f9064105510 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.994+0000 7f906a1cb700 1 -- 172.21.15.63:0/3563723843 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906419af80 con 0x7f9064105510 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.995+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f9058019940 con 0x7f9064105510 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.995+0000 7f9060ff9700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 0x7f904c03abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.995+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 --> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f904c03b300 con 0x7f904c038730 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.995+0000 7f9062ffd700 1 -- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 msgr2=0x7f904c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1739893919 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.995+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f905804cd60 con 0x7f9064105510 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:24.995+0000 7f9062ffd700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 0x7f904c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:25.196+0000 7f9062ffd700 1 -- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 msgr2=0x7f904c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1739893919 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:25.196+0000 7f9062ffd700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 0x7f904c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T06:28:31.452 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:25.597+0000 7f9062ffd700 1 -- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 msgr2=0x7f904c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1739893919 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:25.597+0000 7f9062ffd700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 0x7f904c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:26.398+0000 7f9062ffd700 1 -- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 msgr2=0x7f904c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1739893919 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:26.398+0000 7f9062ffd700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 0x7f904c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:28.000+0000 7f9062ffd700 1 -- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 msgr2=0x7f904c03abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1739893919 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:28.000+0000 7f9062ffd700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 0x7f904c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:29.801+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 6) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f905801a940 con 0x7f9064105510 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:29.801+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 msgr2=0x7f904c03abf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:29.801+0000 7f9060ff9700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 0x7f904c03abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.802+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 7) v1 ==== 45450+0+0 (secure 0 0 0) 0x7f905804e590 con 0x7f9064105510 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.802+0000 7f9060ff9700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f904c03bfc0 0x7f904c03e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.803+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f904c03b300 con 0x7f904c03bfc0 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.803+0000 7f9062ffd700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f904c03bfc0 0x7f904c03e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.804+0000 7f9062ffd700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f904c03bfc0 0x7f904c03e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9054003a10 tx=0x7f90540092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.804+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f904c03b300 con 0x7f904c03bfc0 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f906a1cb700 1 -- 172.21.15.63:0/3563723843 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f9064062520 con 0x7f904c03bfc0 2024-05-09T06:28:31.453 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f9060ff9700 1 -- 172.21.15.63:0/3563723843 <== mgr.14120 v2:172.21.15.63:6800/3629307253 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9064062520 con 0x7f904c03bfc0 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f906a1cb700 1 -- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f904c03bfc0 msgr2=0x7f904c03e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f906a1cb700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f904c03bfc0 0x7f904c03e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9054003a10 tx=0x7f90540092b0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f906a1cb700 1 -- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 msgr2=0x7f906419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f906a1cb700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f906419a3a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9058011710 tx=0x7f90580047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f906a1cb700 1 -- 172.21.15.63:0/3563723843 shutdown_connections 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f906a1cb700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9064105510 0x7f906419a3a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f906a1cb700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f904c03bfc0 0x7f904c03e3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.811+0000 7f906a1cb700 1 --2- 172.21.15.63:0/3563723843 >> [v2:172.21.15.63:6800/1739893919,v1:172.21.15.63:6801/1739893919] conn(0x7f904c038730 0x7f904c03abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.812+0000 7f906a1cb700 1 -- 172.21.15.63:0/3563723843 >> 172.21.15.63:0/3563723843 conn(0x7f9064100b10 msgr2=0x7f9064101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.812+0000 7f906a1cb700 1 -- 172.21.15.63:0/3563723843 shutdown_connections 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:30.812+0000 7f906a1cb700 1 -- 172.21.15.63:0/3563723843 wait complete. 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 5 is available 2024-05-09T06:28:31.454 INFO:teuthology.orchestra.run.smithi063.stdout:Setting orchestrator backend to cephadm... 2024-05-09T06:28:31.950 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:31 smithi063 ceph-mon[27171]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T06:28:31.950 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:31 smithi063 ceph-mon[27171]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T06:28:31.950 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:31 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:31.950 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:31 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:31.951 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:31 smithi063 ceph-mon[27171]: [09/May/2024:06:28:31] ENGINE Bus STARTING 2024-05-09T06:28:31.951 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:31 smithi063 ceph-mon[27171]: [09/May/2024:06:28:31] ENGINE Serving on http://172.21.15.63:8765 2024-05-09T06:28:31.951 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:31 smithi063 ceph-mon[27171]: [09/May/2024:06:28:31] ENGINE Serving on https://172.21.15.63:7150 2024-05-09T06:28:31.951 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:31 smithi063 ceph-mon[27171]: [09/May/2024:06:28:31] ENGINE Bus STARTED 2024-05-09T06:28:31.951 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:31 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:28:32.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.945+0000 7fcc174a6700 1 Processor -- start 2024-05-09T06:28:32.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.945+0000 7fcc174a6700 1 -- start start 2024-05-09T06:28:32.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc174a6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc10106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc174a6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc10106ff0 con 0x7fcc10106600 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc15242700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc10106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc15242700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc10106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42740/0 (socket says 172.21.15.63:42740) 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc15242700 1 -- 172.21.15.63:0/1237953981 learned_addr learned my addr 172.21.15.63:0/1237953981 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc15242700 1 -- 172.21.15.63:0/1237953981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc101077b0 con 0x7fcc10106600 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc15242700 1 --2- 172.21.15.63:0/1237953981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc10106a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0098a0 tx=0x7fcc0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd8ed55d628513d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc07fff700 1 -- 172.21.15.63:0/1237953981 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc0c004180 con 0x7fcc10106600 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc07fff700 1 -- 172.21.15.63:0/1237953981 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcc0c0042e0 con 0x7fcc10106600 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.946+0000 7fcc07fff700 1 -- 172.21.15.63:0/1237953981 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc0c011a70 con 0x7fcc10106600 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.947+0000 7fcc174a6700 1 -- 172.21.15.63:0/1237953981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 msgr2=0x7fcc10106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.947+0000 7fcc174a6700 1 --2- 172.21.15.63:0/1237953981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc10106a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0098a0 tx=0x7fcc0c009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.947+0000 7fcc174a6700 1 -- 172.21.15.63:0/1237953981 shutdown_connections 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.947+0000 7fcc174a6700 1 --2- 172.21.15.63:0/1237953981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc10106a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.947+0000 7fcc174a6700 1 -- 172.21.15.63:0/1237953981 >> 172.21.15.63:0/1237953981 conn(0x7fcc10101c60 msgr2=0x7fcc10104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:32.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.947+0000 7fcc174a6700 1 -- 172.21.15.63:0/1237953981 shutdown_connections 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.947+0000 7fcc174a6700 1 -- 172.21.15.63:0/1237953981 wait complete. 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.948+0000 7fcc174a6700 1 Processor -- start 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.948+0000 7fcc174a6700 1 -- start start 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.948+0000 7fcc174a6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc1007bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.948+0000 7fcc174a6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1007c040 con 0x7fcc10106600 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.948+0000 7fcc15242700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc1007bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.948+0000 7fcc15242700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc1007bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42754/0 (socket says 172.21.15.63:42754) 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.948+0000 7fcc15242700 1 -- 172.21.15.63:0/878993142 learned_addr learned my addr 172.21.15.63:0/878993142 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.948+0000 7fcc15242700 1 -- 172.21.15.63:0/878993142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c009550 con 0x7fcc10106600 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.948+0000 7fcc15242700 1 --2- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc1007bb00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c011710 tx=0x7fcc0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.949+0000 7fcc067fc700 1 -- 172.21.15.63:0/878993142 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc0c003b10 con 0x7fcc10106600 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.949+0000 7fcc067fc700 1 -- 172.21.15.63:0/878993142 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcc0c01b8c0 con 0x7fcc10106600 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.949+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1007a280 con 0x7fcc10106600 2024-05-09T06:28:32.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.949+0000 7fcc067fc700 1 -- 172.21.15.63:0/878993142 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc0c019770 con 0x7fcc10106600 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.949+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1007a720 con 0x7fcc10106600 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.950+0000 7fcc067fc700 1 -- 172.21.15.63:0/878993142 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 7) v1 ==== 45450+0+0 (secure 0 0 0) 0x7fcc0c01b440 con 0x7fcc10106600 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.950+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc10062520 con 0x7fcc10106600 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.950+0000 7fcc067fc700 1 --2- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fcbfc038650 0x7fcbfc03ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.950+0000 7fcc067fc700 1 -- 172.21.15.63:0/878993142 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc0c04c6e0 con 0x7fcc10106600 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.950+0000 7fcc14a41700 1 --2- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fcbfc038650 0x7fcbfc03ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.950+0000 7fcc14a41700 1 --2- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fcbfc038650 0x7fcbfc03ab10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc00006fd0 tx=0x7fcc00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.955+0000 7fcc067fc700 1 -- 172.21.15.63:0/878993142 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcc0c01a3d0 con 0x7fcc10106600 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:31.968+0000 7fcc067fc700 1 -- 172.21.15.63:0/878993142 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fcc0c02cc50 con 0x7fcc10106600 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.179+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc1007ad40 con 0x7fcbfc038650 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.189+0000 7fcc067fc700 1 -- 172.21.15.63:0/878993142 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcc1007ad40 con 0x7fcbfc038650 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fcbfc038650 msgr2=0x7fcbfc03ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.793 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 --2- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fcbfc038650 0x7fcbfc03ab10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc00006fd0 tx=0x7fcc00006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:32.794 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 msgr2=0x7fcc1007bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:32.794 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 --2- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc1007bb00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c011710 tx=0x7fcc0c0047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.794 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 shutdown_connections 2024-05-09T06:28:32.794 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 --2- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fcbfc038650 0x7fcbfc03ab10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.794 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 --2- 172.21.15.63:0/878993142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc10106600 0x7fcc1007bb00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:32.794 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 >> 172.21.15.63:0/878993142 conn(0x7fcc10101c60 msgr2=0x7fcc10102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:32.794 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 shutdown_connections 2024-05-09T06:28:32.794 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:32.192+0000 7fcc174a6700 1 -- 172.21.15.63:0/878993142 wait complete. 2024-05-09T06:28:33.048 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:32 smithi063 ceph-mon[27171]: mgrmap e8: smithi063.egfguy(active, since 2s) 2024-05-09T06:28:33.049 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:32 smithi063 ceph-mon[27171]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:33.049 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:32 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:33.049 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:32 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:28:34.064 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:33 smithi063 ceph-mon[27171]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.255+0000 7f51862c4700 1 Processor -- start 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.255+0000 7f51862c4700 1 -- start start 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.255+0000 7f51862c4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f5180106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.255+0000 7f51862c4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180107030 con 0x7f5180106640 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.255+0000 7f517ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f5180106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.255+0000 7f517ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f5180106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42758/0 (socket says 172.21.15.63:42758) 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.255+0000 7f517ffff700 1 -- 172.21.15.63:0/3857910932 learned_addr learned my addr 172.21.15.63:0/3857910932 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.256+0000 7f517ffff700 1 -- 172.21.15.63:0/3857910932 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51801077f0 con 0x7f5180106640 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.256+0000 7f517ffff700 1 --2- 172.21.15.63:0/3857910932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f5180106a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51700098a0 tx=0x7f5170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f09c14896bbb299 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:34.073 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.256+0000 7f517effd700 1 -- 172.21.15.63:0/3857910932 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5170004180 con 0x7f5180106640 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.256+0000 7f517effd700 1 -- 172.21.15.63:0/3857910932 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f51700042e0 con 0x7f5180106640 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.256+0000 7f517effd700 1 -- 172.21.15.63:0/3857910932 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5170011a70 con 0x7f5180106640 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.256+0000 7f51862c4700 1 -- 172.21.15.63:0/3857910932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 msgr2=0x7f5180106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.256+0000 7f51862c4700 1 --2- 172.21.15.63:0/3857910932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f5180106a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51700098a0 tx=0x7f5170009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.257+0000 7f51862c4700 1 -- 172.21.15.63:0/3857910932 shutdown_connections 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.257+0000 7f51862c4700 1 --2- 172.21.15.63:0/3857910932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f5180106a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.257+0000 7f51862c4700 1 -- 172.21.15.63:0/3857910932 >> 172.21.15.63:0/3857910932 conn(0x7f5180101c40 msgr2=0x7f5180104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.257+0000 7f51862c4700 1 -- 172.21.15.63:0/3857910932 shutdown_connections 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.257+0000 7f51862c4700 1 -- 172.21.15.63:0/3857910932 wait complete. 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.257+0000 7f51862c4700 1 Processor -- start 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.257+0000 7f51862c4700 1 -- start start 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f51862c4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f518007bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:34.074 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f51862c4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518007c040 con 0x7f5180106640 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f517ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f518007bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f517ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f518007bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42760/0 (socket says 172.21.15.63:42760) 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f517ffff700 1 -- 172.21.15.63:0/347267707 learned_addr learned my addr 172.21.15.63:0/347267707 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f517ffff700 1 -- 172.21.15.63:0/347267707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5170009550 con 0x7f5180106640 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f517ffff700 1 --2- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f518007bb00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5170006af0 tx=0x7f51700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f517d7fa700 1 -- 172.21.15.63:0/347267707 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5170003a00 con 0x7f5180106640 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f517d7fa700 1 -- 172.21.15.63:0/347267707 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5170003b60 con 0x7f5180106640 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518007a280 con 0x7f5180106640 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f517d7fa700 1 -- 172.21.15.63:0/347267707 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5170023750 con 0x7f5180106640 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.258+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518007a720 con 0x7f5180106640 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.259+0000 7f517d7fa700 1 -- 172.21.15.63:0/347267707 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f5170023990 con 0x7f5180106640 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.259+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5180062520 con 0x7f5180106640 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.259+0000 7f517d7fa700 1 --2- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f5168038720 0x7f516803abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:34.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.260+0000 7f517d7fa700 1 -- 172.21.15.63:0/347267707 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f517004c9d0 con 0x7f5180106640 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.260+0000 7f517f7fe700 1 --2- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f5168038720 0x7f516803abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.260+0000 7f517f7fe700 1 --2- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f5168038720 0x7f516803abe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5174006fd0 tx=0x7f5174006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.264+0000 7f517d7fa700 1 -- 172.21.15.63:0/347267707 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f517002b760 con 0x7f5180106640 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.470+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f518007ad50 con 0x7f5168038720 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.471+0000 7f517d7fa700 1 -- 172.21.15.63:0/347267707 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f518007ad50 con 0x7f5168038720 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f5168038720 msgr2=0x7f516803abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 --2- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f5168038720 0x7f516803abe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5174006fd0 tx=0x7f5174006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 msgr2=0x7f518007bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 --2- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f518007bb00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5170006af0 tx=0x7f51700047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 shutdown_connections 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 --2- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f5168038720 0x7f516803abe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 --2- 172.21.15.63:0/347267707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5180106640 0x7f518007bb00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 >> 172.21.15.63:0/347267707 conn(0x7f5180101c40 msgr2=0x7f5180102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 shutdown_connections 2024-05-09T06:28:34.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:33.473+0000 7f51862c4700 1 -- 172.21.15.63:0/347267707 wait complete. 2024-05-09T06:28:34.077 INFO:teuthology.orchestra.run.smithi063.stdout:Generating ssh key... 2024-05-09T06:28:35.320 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:34 smithi063 ceph-mon[27171]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:35.320 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:34 smithi063 ceph-mon[27171]: Generating ssh key... 2024-05-09T06:28:35.612 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.484+0000 7fca96dd9700 1 Processor -- start 2024-05-09T06:28:35.612 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.484+0000 7fca96dd9700 1 -- start start 2024-05-09T06:28:35.612 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.484+0000 7fca96dd9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:35.612 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.484+0000 7fca96dd9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90079e50 con 0x7fca901070c0 2024-05-09T06:28:35.612 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.484+0000 7fca94b75700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:35.612 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.484+0000 7fca94b75700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42776/0 (socket says 172.21.15.63:42776) 2024-05-09T06:28:35.612 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.484+0000 7fca94b75700 1 -- 172.21.15.63:0/1005316143 learned_addr learned my addr 172.21.15.63:0/1005316143 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.485+0000 7fca94b75700 1 -- 172.21.15.63:0/1005316143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca901099f0 con 0x7fca901070c0 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.485+0000 7fca94b75700 1 --2- 172.21.15.63:0/1005316143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901094b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca840098a0 tx=0x7fca84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68b3106fe558975 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.485+0000 7fca8f7fe700 1 -- 172.21.15.63:0/1005316143 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca84004180 con 0x7fca901070c0 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.485+0000 7fca8f7fe700 1 -- 172.21.15.63:0/1005316143 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fca840042e0 con 0x7fca901070c0 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.485+0000 7fca8f7fe700 1 -- 172.21.15.63:0/1005316143 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca84011a70 con 0x7fca901070c0 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.485+0000 7fca96dd9700 1 -- 172.21.15.63:0/1005316143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 msgr2=0x7fca901094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.485+0000 7fca96dd9700 1 --2- 172.21.15.63:0/1005316143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901094b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca840098a0 tx=0x7fca84009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.486+0000 7fca96dd9700 1 -- 172.21.15.63:0/1005316143 shutdown_connections 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.486+0000 7fca96dd9700 1 --2- 172.21.15.63:0/1005316143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901094b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.486+0000 7fca96dd9700 1 -- 172.21.15.63:0/1005316143 >> 172.21.15.63:0/1005316143 conn(0x7fca90100b10 msgr2=0x7fca90102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.486+0000 7fca96dd9700 1 -- 172.21.15.63:0/1005316143 shutdown_connections 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.486+0000 7fca96dd9700 1 -- 172.21.15.63:0/1005316143 wait complete. 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.486+0000 7fca96dd9700 1 Processor -- start 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.486+0000 7fca96dd9700 1 -- start start 2024-05-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.486+0000 7fca96dd9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901068f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.486+0000 7fca96dd9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90106e30 con 0x7fca901070c0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca94b75700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca94b75700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42782/0 (socket says 172.21.15.63:42782) 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca94b75700 1 -- 172.21.15.63:0/3448324369 learned_addr learned my addr 172.21.15.63:0/3448324369 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca94b75700 1 -- 172.21.15.63:0/3448324369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca84009550 con 0x7fca901070c0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca94b75700 1 --2- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901068f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca84011710 tx=0x7fca840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca8dffb700 1 -- 172.21.15.63:0/3448324369 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca84003b10 con 0x7fca901070c0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca8dffb700 1 -- 172.21.15.63:0/3448324369 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fca8401b8c0 con 0x7fca901070c0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca90105070 con 0x7fca901070c0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca8dffb700 1 -- 172.21.15.63:0/3448324369 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca840197e0 con 0x7fca901070c0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.487+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90105510 con 0x7fca901070c0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.488+0000 7fca8dffb700 1 -- 172.21.15.63:0/3448324369 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fca840199d0 con 0x7fca901070c0 2024-05-09T06:28:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.489+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca90062520 con 0x7fca901070c0 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.489+0000 7fca8dffb700 1 --2- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fca80038770 0x7fca8003ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.489+0000 7fca8dffb700 1 -- 172.21.15.63:0/3448324369 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fca8404ca90 con 0x7fca901070c0 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.489+0000 7fca8ffff700 1 --2- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fca80038770 0x7fca8003ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.489+0000 7fca8ffff700 1 --2- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fca80038770 0x7fca8003ac30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca7c006fd0 tx=0x7fca7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.494+0000 7fca8dffb700 1 -- 172.21.15.63:0/3448324369 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fca84018db0 con 0x7fca901070c0 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:34.698+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fca90105bc0 con 0x7fca80038770 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.031+0000 7fca8dffb700 1 -- 172.21.15.63:0/3448324369 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fca90105bc0 con 0x7fca80038770 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fca80038770 msgr2=0x7fca8003ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 --2- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fca80038770 0x7fca8003ac30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca7c006fd0 tx=0x7fca7c006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 msgr2=0x7fca901068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 --2- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901068f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca84011710 tx=0x7fca840047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 shutdown_connections 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 --2- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fca80038770 0x7fca8003ac30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 --2- 172.21.15.63:0/3448324369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca901070c0 0x7fca901068f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:35.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 >> 172.21.15.63:0/3448324369 conn(0x7fca90100b10 msgr2=0x7fca901016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:35.616 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 shutdown_connections 2024-05-09T06:28:35.616 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:35.033+0000 7fca96dd9700 1 -- 172.21.15.63:0/3448324369 wait complete. 2024-05-09T06:28:36.236 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:36 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:36.236 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:36 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:36.910 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtDQyYtQyu5JkYanv7gXevOt+OzkE3sIQF1sd51ZtIoqOi8pDbjIqeOL8B2b+W4wbrKH93YyfM410N5RQM6QKIwCzSDBHDVlNLrDk4MDvtanumxRWk3LPHUZGSm/H9IRRdcq7R+bfQEw8bI7hC3v6yyG6sVrMnZBtdM/Rhl7xa/Yl25QFPJHYlB/N6DT4N2rDfXNs0nw8z1qWL3gqjkDqR3HiNqh7QLq3qY8TNspL8N1EXsiUUMpz4RlPdVFBkOzNCJ3jRcmsyknsQvAOlEtyZ1hztitLLEARRsSW9ZnPR7hmzUSXGEOcKO2ni91Q378lL8hTOBinxNIMB2lR+udS/uiOF7J3HZqFf3A8GQgawgc1DvNtubK4CbSve/k22DOT/Kf/2iiQhvs0R+FQx264PjnWkQTlFlnOdtz9CX/IQqJbnKnrPwTzgv1nKaqDovd4+qLeDS6sjnZI45spFggener6cCv5pYyUx5CHy6vRX+KaT7pqqadlp32q45E0m53k= ceph-2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:28:36.910 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.040+0000 7fb41ad27700 1 Processor -- start 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.040+0000 7fb41ad27700 1 -- start start 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.040+0000 7fb41ad27700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb414106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.040+0000 7fb41ad27700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb414107030 con 0x7fb414106640 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.040+0000 7fb418ac3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb414106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.040+0000 7fb418ac3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb414106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33340/0 (socket says 172.21.15.63:33340) 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.040+0000 7fb418ac3700 1 -- 172.21.15.63:0/2531593982 learned_addr learned my addr 172.21.15.63:0/2531593982 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.040+0000 7fb418ac3700 1 -- 172.21.15.63:0/2531593982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4141077f0 con 0x7fb414106640 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb418ac3700 1 --2- 172.21.15.63:0/2531593982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb414106a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4080098a0 tx=0x7fb408009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a5582dc5edd2ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb4137fe700 1 -- 172.21.15.63:0/2531593982 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb408004180 con 0x7fb414106640 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb4137fe700 1 -- 172.21.15.63:0/2531593982 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb4080042e0 con 0x7fb414106640 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb4137fe700 1 -- 172.21.15.63:0/2531593982 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb408011a70 con 0x7fb414106640 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb41ad27700 1 -- 172.21.15.63:0/2531593982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 msgr2=0x7fb414106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb41ad27700 1 --2- 172.21.15.63:0/2531593982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb414106a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4080098a0 tx=0x7fb408009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.911 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb41ad27700 1 -- 172.21.15.63:0/2531593982 shutdown_connections 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb41ad27700 1 --2- 172.21.15.63:0/2531593982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb414106a60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb41ad27700 1 -- 172.21.15.63:0/2531593982 >> 172.21.15.63:0/2531593982 conn(0x7fb414101c40 msgr2=0x7fb414104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb41ad27700 1 -- 172.21.15.63:0/2531593982 shutdown_connections 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.041+0000 7fb41ad27700 1 -- 172.21.15.63:0/2531593982 wait complete. 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.042+0000 7fb41ad27700 1 Processor -- start 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.042+0000 7fb41ad27700 1 -- start start 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.042+0000 7fb41ad27700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb41419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.042+0000 7fb41ad27700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41419a900 con 0x7fb414106640 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.042+0000 7fb418ac3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb41419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.042+0000 7fb418ac3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb41419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33354/0 (socket says 172.21.15.63:33354) 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.042+0000 7fb418ac3700 1 -- 172.21.15.63:0/4133705243 learned_addr learned my addr 172.21.15.63:0/4133705243 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.043+0000 7fb418ac3700 1 -- 172.21.15.63:0/4133705243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb408009550 con 0x7fb414106640 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.043+0000 7fb418ac3700 1 --2- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb41419a3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb408003df0 tx=0x7fb4080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:36.912 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.043+0000 7fb411ffb700 1 -- 172.21.15.63:0/4133705243 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb408003a00 con 0x7fb414106640 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.043+0000 7fb411ffb700 1 -- 172.21.15.63:0/4133705243 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb408003b60 con 0x7fb414106640 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.043+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41419ab00 con 0x7fb414106640 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.043+0000 7fb411ffb700 1 -- 172.21.15.63:0/4133705243 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb408023770 con 0x7fb414106640 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.043+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41419afa0 con 0x7fb414106640 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.044+0000 7fb411ffb700 1 -- 172.21.15.63:0/4133705243 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fb40802c3f0 con 0x7fb414106640 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.044+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41410aa20 con 0x7fb414106640 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.044+0000 7fb411ffb700 1 --2- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fb4040386e0 0x7fb40403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.044+0000 7fb411ffb700 1 -- 172.21.15.63:0/4133705243 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb40804cb90 con 0x7fb414106640 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.044+0000 7fb413fff700 1 --2- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fb4040386e0 0x7fb40403aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.045+0000 7fb413fff700 1 --2- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fb4040386e0 0x7fb40403aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb400006fd0 tx=0x7fb400006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.049+0000 7fb411ffb700 1 -- 172.21.15.63:0/4133705243 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb408018350 con 0x7fb414106640 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.254+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb414102f90 con 0x7fb4040386e0 2024-05-09T06:28:36.913 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.255+0000 7fb411ffb700 1 -- 172.21.15.63:0/4133705243 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fb414102f90 con 0x7fb4040386e0 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fb4040386e0 msgr2=0x7fb40403aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 --2- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fb4040386e0 0x7fb40403aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb400006fd0 tx=0x7fb400006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 msgr2=0x7fb41419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 --2- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb41419a3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb408003df0 tx=0x7fb4080047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 shutdown_connections 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 --2- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fb4040386e0 0x7fb40403aba0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 --2- 172.21.15.63:0/4133705243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb414106640 0x7fb41419a3c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 >> 172.21.15.63:0/4133705243 conn(0x7fb414101c40 msgr2=0x7fb414102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 shutdown_connections 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:36.258+0000 7fb41ad27700 1 -- 172.21.15.63:0/4133705243 wait complete. 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T06:28:36.914 INFO:teuthology.orchestra.run.smithi063.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T06:28:36.915 INFO:teuthology.orchestra.run.smithi063.stdout:Adding host smithi063... 2024-05-09T06:28:37.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:37 smithi063 ceph-mon[27171]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:38.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:38 smithi063 ceph-mon[27171]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:39.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:39 smithi063 ceph-mon[27171]: Deploying cephadm binary to smithi063 2024-05-09T06:28:40.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Added host 'smithi063' with addr '172.21.15.63' 2024-05-09T06:28:40.760 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.385+0000 7f69087cf700 1 Processor -- start 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.385+0000 7f69087cf700 1 -- start start 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.385+0000 7f69087cf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f6900104d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.385+0000 7f69087cf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6900105310 con 0x7f6900104920 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.386+0000 7f690656b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f6900104d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.386+0000 7f690656b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f6900104d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33358/0 (socket says 172.21.15.63:33358) 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.386+0000 7f690656b700 1 -- 172.21.15.63:0/2615650792 learned_addr learned my addr 172.21.15.63:0/2615650792 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.386+0000 7f690656b700 1 -- 172.21.15.63:0/2615650792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69001078b0 con 0x7f6900104920 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.386+0000 7f690656b700 1 --2- 172.21.15.63:0/2615650792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f6900104d40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0098a0 tx=0x7f68fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a689e8ceb60baf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.386+0000 7f6905569700 1 -- 172.21.15.63:0/2615650792 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68fc004180 con 0x7f6900104920 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.386+0000 7f6905569700 1 -- 172.21.15.63:0/2615650792 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f68fc0042e0 con 0x7f6900104920 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.386+0000 7f6905569700 1 -- 172.21.15.63:0/2615650792 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68fc011a70 con 0x7f6900104920 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.387+0000 7f69087cf700 1 -- 172.21.15.63:0/2615650792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 msgr2=0x7f6900104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.761 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.387+0000 7f69087cf700 1 --2- 172.21.15.63:0/2615650792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f6900104d40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0098a0 tx=0x7f68fc009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.387+0000 7f69087cf700 1 -- 172.21.15.63:0/2615650792 shutdown_connections 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.387+0000 7f69087cf700 1 --2- 172.21.15.63:0/2615650792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f6900104d40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.387+0000 7f69087cf700 1 -- 172.21.15.63:0/2615650792 >> 172.21.15.63:0/2615650792 conn(0x7f6900100290 msgr2=0x7f69001026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.387+0000 7f69087cf700 1 -- 172.21.15.63:0/2615650792 shutdown_connections 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.387+0000 7f69087cf700 1 -- 172.21.15.63:0/2615650792 wait complete. 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.388+0000 7f69087cf700 1 Processor -- start 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.388+0000 7f69087cf700 1 -- start start 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.388+0000 7f69087cf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f690019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.388+0000 7f69087cf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690019a8f0 con 0x7f6900104920 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.388+0000 7f690656b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f690019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.388+0000 7f690656b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f690019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33368/0 (socket says 172.21.15.63:33368) 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.388+0000 7f690656b700 1 -- 172.21.15.63:0/4043283540 learned_addr learned my addr 172.21.15.63:0/4043283540 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.388+0000 7f690656b700 1 -- 172.21.15.63:0/4043283540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc009550 con 0x7f6900104920 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.388+0000 7f690656b700 1 --2- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f690019a3b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f68fc011710 tx=0x7f68fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:40.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.389+0000 7f68f77fe700 1 -- 172.21.15.63:0/4043283540 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68fc003b10 con 0x7f6900104920 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.389+0000 7f68f77fe700 1 -- 172.21.15.63:0/4043283540 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f68fc01b8c0 con 0x7f6900104920 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.389+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690019aaf0 con 0x7f6900104920 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.389+0000 7f68f77fe700 1 -- 172.21.15.63:0/4043283540 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68fc019770 con 0x7f6900104920 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.389+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690019af90 con 0x7f6900104920 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.390+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690010aae0 con 0x7f6900104920 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.390+0000 7f68f77fe700 1 -- 172.21.15.63:0/4043283540 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f68fc01ba30 con 0x7f6900104920 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.390+0000 7f68f77fe700 1 --2- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f68ec038770 0x7f68ec03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.390+0000 7f68f77fe700 1 -- 172.21.15.63:0/4043283540 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f68fc04c8b0 con 0x7f6900104920 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.390+0000 7f6905d6a700 1 --2- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f68ec038770 0x7f68ec03ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.391+0000 7f6905d6a700 1 --2- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f68ec038770 0x7f68ec03ac30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68f0006fd0 tx=0x7f68f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.395+0000 7f68f77fe700 1 -- 172.21.15.63:0/4043283540 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f68fc02cce0 con 0x7f6900104920 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:37.605+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}) v1 -- 0x7f690019d830 con 0x7f68ec038770 2024-05-09T06:28:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.159+0000 7f68f77fe700 1 -- 172.21.15.63:0/4043283540 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f690019d830 con 0x7f68ec038770 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f68ec038770 msgr2=0x7f68ec03ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 --2- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f68ec038770 0x7f68ec03ac30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68f0006fd0 tx=0x7f68f0006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 msgr2=0x7f690019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 --2- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f690019a3b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f68fc011710 tx=0x7f68fc0047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 shutdown_connections 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 --2- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f68ec038770 0x7f68ec03ac30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 --2- 172.21.15.63:0/4043283540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900104920 0x7f690019a3b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 >> 172.21.15.63:0/4043283540 conn(0x7f6900100290 msgr2=0x7f6900100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 shutdown_connections 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:40.162+0000 7f69087cf700 1 -- 172.21.15.63:0/4043283540 wait complete. 2024-05-09T06:28:40.764 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying mon service with default placement... 2024-05-09T06:28:41.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:41 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:41.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:41 smithi063 ceph-mon[27171]: Added host smithi063 2024-05-09T06:28:41.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:41 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:28:42.153 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T06:28:42.153 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.164+0000 7fc9711bb700 1 Processor -- start 2024-05-09T06:28:42.153 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.164+0000 7fc9711bb700 1 -- start start 2024-05-09T06:28:42.153 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.164+0000 7fc9711bb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:42.153 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.164+0000 7fc9711bb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c107030 con 0x7fc96c106640 2024-05-09T06:28:42.153 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.164+0000 7fc96ad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:42.153 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.164+0000 7fc96ad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33370/0 (socket says 172.21.15.63:33370) 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.164+0000 7fc96ad9d700 1 -- 172.21.15.63:0/482454188 learned_addr learned my addr 172.21.15.63:0/482454188 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.164+0000 7fc96ad9d700 1 -- 172.21.15.63:0/482454188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c1077f0 con 0x7fc96c106640 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.165+0000 7fc96ad9d700 1 --2- 172.21.15.63:0/482454188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c106a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0098a0 tx=0x7fc95c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d111e07697d208 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.165+0000 7fc969d9b700 1 -- 172.21.15.63:0/482454188 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc95c004180 con 0x7fc96c106640 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.165+0000 7fc969d9b700 1 -- 172.21.15.63:0/482454188 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc95c0042e0 con 0x7fc96c106640 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.165+0000 7fc969d9b700 1 -- 172.21.15.63:0/482454188 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc95c011a70 con 0x7fc96c106640 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.165+0000 7fc9711bb700 1 -- 172.21.15.63:0/482454188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 msgr2=0x7fc96c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.165+0000 7fc9711bb700 1 --2- 172.21.15.63:0/482454188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c106a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0098a0 tx=0x7fc95c009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.166+0000 7fc9711bb700 1 -- 172.21.15.63:0/482454188 shutdown_connections 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.166+0000 7fc9711bb700 1 --2- 172.21.15.63:0/482454188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c106a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.166+0000 7fc9711bb700 1 -- 172.21.15.63:0/482454188 >> 172.21.15.63:0/482454188 conn(0x7fc96c101c40 msgr2=0x7fc96c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.166+0000 7fc9711bb700 1 -- 172.21.15.63:0/482454188 shutdown_connections 2024-05-09T06:28:42.154 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.166+0000 7fc9711bb700 1 -- 172.21.15.63:0/482454188 wait complete. 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.166+0000 7fc9711bb700 1 Processor -- start 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.166+0000 7fc9711bb700 1 -- start start 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc9711bb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c07bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc9711bb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c07c040 con 0x7fc96c106640 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc96ad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c07bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc96ad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c07bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33382/0 (socket says 172.21.15.63:33382) 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc96ad9d700 1 -- 172.21.15.63:0/741341307 learned_addr learned my addr 172.21.15.63:0/741341307 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc96ad9d700 1 -- 172.21.15.63:0/741341307 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c009550 con 0x7fc96c106640 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc96ad9d700 1 --2- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c07bb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006af0 tx=0x7fc95c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc953fff700 1 -- 172.21.15.63:0/741341307 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc95c003a00 con 0x7fc96c106640 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc953fff700 1 -- 172.21.15.63:0/741341307 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc95c003b60 con 0x7fc96c106640 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc953fff700 1 -- 172.21.15.63:0/741341307 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc95c023750 con 0x7fc96c106640 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.167+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96c07a280 con 0x7fc96c106640 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.168+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c07a720 con 0x7fc96c106640 2024-05-09T06:28:42.155 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.168+0000 7fc953fff700 1 -- 172.21.15.63:0/741341307 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fc95c023940 con 0x7fc96c106640 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.168+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96c062520 con 0x7fc96c106640 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.169+0000 7fc953fff700 1 --2- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc954038770 0x7fc95403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.169+0000 7fc953fff700 1 -- 172.21.15.63:0/741341307 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc95c04c980 con 0x7fc96c106640 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.169+0000 7fc96a59c700 1 --2- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc954038770 0x7fc95403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.169+0000 7fc96a59c700 1 --2- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc954038770 0x7fc95403ac30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc960006fd0 tx=0x7fc960006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.173+0000 7fc953fff700 1 -- 172.21.15.63:0/741341307 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc95c02bd20 con 0x7fc96c106640 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.407+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fc96c07ad60 con 0x7fc954038770 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.414+0000 7fc953fff700 1 -- 172.21.15.63:0/741341307 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc96c07ad60 con 0x7fc954038770 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc954038770 msgr2=0x7fc95403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 --2- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc954038770 0x7fc95403ac30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc960006fd0 tx=0x7fc960006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 msgr2=0x7fc96c07bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 --2- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c07bb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006af0 tx=0x7fc95c0047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 shutdown_connections 2024-05-09T06:28:42.156 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 --2- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc954038770 0x7fc95403ac30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.157 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 --2- 172.21.15.63:0/741341307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc96c106640 0x7fc96c07bb00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:42.157 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 >> 172.21.15.63:0/741341307 conn(0x7fc96c101c40 msgr2=0x7fc96c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:42.157 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 shutdown_connections 2024-05-09T06:28:42.157 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:41.416+0000 7fc9711bb700 1 -- 172.21.15.63:0/741341307 wait complete. 2024-05-09T06:28:42.157 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying mgr service with default placement... 2024-05-09T06:28:42.572 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:42 smithi063 ceph-mon[27171]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:42.573 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:42 smithi063 ceph-mon[27171]: Saving service mon spec with placement count:5 2024-05-09T06:28:42.573 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:42 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.623+0000 7f639f4e2700 1 Processor -- start 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.624+0000 7f639f4e2700 1 -- start start 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.624+0000 7f639f4e2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f6398106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.624+0000 7f639f4e2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398107000 con 0x7f6398106610 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.624+0000 7f639d27e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f6398106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.624+0000 7f639d27e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f6398106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33384/0 (socket says 172.21.15.63:33384) 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.624+0000 7f639d27e700 1 -- 172.21.15.63:0/642737343 learned_addr learned my addr 172.21.15.63:0/642737343 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.624+0000 7f639d27e700 1 -- 172.21.15.63:0/642737343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63981077c0 con 0x7f6398106610 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f639d27e700 1 --2- 172.21.15.63:0/642737343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f6398106a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63940098a0 tx=0x7f6394009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb1d4840b29c8e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f638bfff700 1 -- 172.21.15.63:0/642737343 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6394004180 con 0x7f6398106610 2024-05-09T06:28:43.536 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f638bfff700 1 -- 172.21.15.63:0/642737343 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f63940042e0 con 0x7f6398106610 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f638bfff700 1 -- 172.21.15.63:0/642737343 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6394011a70 con 0x7f6398106610 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f639f4e2700 1 -- 172.21.15.63:0/642737343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 msgr2=0x7f6398106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f639f4e2700 1 --2- 172.21.15.63:0/642737343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f6398106a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63940098a0 tx=0x7f6394009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f639f4e2700 1 -- 172.21.15.63:0/642737343 shutdown_connections 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f639f4e2700 1 --2- 172.21.15.63:0/642737343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f6398106a30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f639f4e2700 1 -- 172.21.15.63:0/642737343 >> 172.21.15.63:0/642737343 conn(0x7f6398101c10 msgr2=0x7f6398104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.625+0000 7f639f4e2700 1 -- 172.21.15.63:0/642737343 shutdown_connections 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.626+0000 7f639f4e2700 1 -- 172.21.15.63:0/642737343 wait complete. 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.626+0000 7f639f4e2700 1 Processor -- start 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.626+0000 7f639f4e2700 1 -- start start 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.626+0000 7f639f4e2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f639819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.626+0000 7f639f4e2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639819a8e0 con 0x7f6398106610 2024-05-09T06:28:43.537 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.626+0000 7f639d27e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f639819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.626+0000 7f639d27e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f639819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33390/0 (socket says 172.21.15.63:33390) 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.626+0000 7f639d27e700 1 -- 172.21.15.63:0/2885888782 learned_addr learned my addr 172.21.15.63:0/2885888782 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.627+0000 7f639d27e700 1 -- 172.21.15.63:0/2885888782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394009550 con 0x7f6398106610 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.627+0000 7f639d27e700 1 --2- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f639819a3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6394006af0 tx=0x7f63940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.627+0000 7f638a7fc700 1 -- 172.21.15.63:0/2885888782 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6394003a00 con 0x7f6398106610 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.627+0000 7f638a7fc700 1 -- 172.21.15.63:0/2885888782 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6394003b60 con 0x7f6398106610 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.627+0000 7f638a7fc700 1 -- 172.21.15.63:0/2885888782 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6394023750 con 0x7f6398106610 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.627+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639819aae0 con 0x7f6398106610 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.627+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639819af80 con 0x7f6398106610 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.628+0000 7f638a7fc700 1 -- 172.21.15.63:0/2885888782 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f6394023940 con 0x7f6398106610 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.628+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639810a9f0 con 0x7f6398106610 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.628+0000 7f638a7fc700 1 --2- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f6384038770 0x7f638403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:43.538 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.628+0000 7f638a7fc700 1 -- 172.21.15.63:0/2885888782 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f639404c980 con 0x7f6398106610 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.628+0000 7f639ca7d700 1 --2- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f6384038770 0x7f638403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.629+0000 7f639ca7d700 1 --2- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f6384038770 0x7f638403ac30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.633+0000 7f638a7fc700 1 -- 172.21.15.63:0/2885888782 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f639402b760 con 0x7f6398106610 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.852+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f6398102fc0 con 0x7f6384038770 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.860+0000 7f638a7fc700 1 -- 172.21.15.63:0/2885888782 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6398102fc0 con 0x7f6384038770 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.862+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f6384038770 msgr2=0x7f638403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.862+0000 7f639f4e2700 1 --2- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f6384038770 0x7f638403ac30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.862+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 msgr2=0x7f639819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.862+0000 7f639f4e2700 1 --2- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f639819a3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6394006af0 tx=0x7f63940047c0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.862+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 shutdown_connections 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.862+0000 7f639f4e2700 1 --2- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f6384038770 0x7f638403ac30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.862+0000 7f639f4e2700 1 --2- 172.21.15.63:0/2885888782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6398106610 0x7f639819a3a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.862+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 >> 172.21.15.63:0/2885888782 conn(0x7f6398101c10 msgr2=0x7f63981028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:43.539 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.863+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 shutdown_connections 2024-05-09T06:28:43.540 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:42.863+0000 7f639f4e2700 1 -- 172.21.15.63:0/2885888782 wait complete. 2024-05-09T06:28:43.540 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying crash service with default placement... 2024-05-09T06:28:43.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:43 smithi063 ceph-mon[27171]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:43.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:43 smithi063 ceph-mon[27171]: Saving service mgr spec with placement count:2 2024-05-09T06:28:43.923 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:43 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:45.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-05-09T06:28:45.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.433+0000 7fc3bd6e4700 1 Processor -- start 2024-05-09T06:28:45.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.433+0000 7fc3bd6e4700 1 -- start start 2024-05-09T06:28:45.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.433+0000 7fc3bd6e4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b8105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:45.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.433+0000 7fc3bd6e4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8105f20 con 0x7fc3b8105530 2024-05-09T06:28:45.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.433+0000 7fc3b6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b8105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.433+0000 7fc3b6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b8105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33400/0 (socket says 172.21.15.63:33400) 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.433+0000 7fc3b6ffd700 1 -- 172.21.15.63:0/3730313438 learned_addr learned my addr 172.21.15.63:0/3730313438 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.434+0000 7fc3b6ffd700 1 -- 172.21.15.63:0/3730313438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b8106730 con 0x7fc3b8105530 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.434+0000 7fc3b6ffd700 1 --2- 172.21.15.63:0/3730313438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b8105950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80098a0 tx=0x7fc3a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c755ae767716e52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.434+0000 7fc3b5ffb700 1 -- 172.21.15.63:0/3730313438 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3a8004180 con 0x7fc3b8105530 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.434+0000 7fc3b5ffb700 1 -- 172.21.15.63:0/3730313438 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3a80042e0 con 0x7fc3b8105530 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.434+0000 7fc3b5ffb700 1 -- 172.21.15.63:0/3730313438 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3a8011b10 con 0x7fc3b8105530 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.435+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/3730313438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 msgr2=0x7fc3b8105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.435+0000 7fc3bd6e4700 1 --2- 172.21.15.63:0/3730313438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b8105950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80098a0 tx=0x7fc3a8009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.435+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/3730313438 shutdown_connections 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.435+0000 7fc3bd6e4700 1 --2- 172.21.15.63:0/3730313438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b8105950 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.435+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/3730313438 >> 172.21.15.63:0/3730313438 conn(0x7fc3b8100a90 msgr2=0x7fc3b8102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.435+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/3730313438 shutdown_connections 2024-05-09T06:28:45.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.435+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/3730313438 wait complete. 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.435+0000 7fc3bd6e4700 1 Processor -- start 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.436+0000 7fc3bd6e4700 1 -- start start 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.436+0000 7fc3bd6e4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.436+0000 7fc3bd6e4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b819a940 con 0x7fc3b8105530 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.436+0000 7fc3b6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.436+0000 7fc3b6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33408/0 (socket says 172.21.15.63:33408) 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.436+0000 7fc3b6ffd700 1 -- 172.21.15.63:0/2022148217 learned_addr learned my addr 172.21.15.63:0/2022148217 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.436+0000 7fc3b6ffd700 1 -- 172.21.15.63:0/2022148217 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a8009550 con 0x7fc3b8105530 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.436+0000 7fc3b6ffd700 1 --2- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b819a400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80117b0 tx=0x7fc3a8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.436+0000 7fc39ffff700 1 -- 172.21.15.63:0/2022148217 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3a80038b0 con 0x7fc3b8105530 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.437+0000 7fc39ffff700 1 -- 172.21.15.63:0/2022148217 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3a8003a10 con 0x7fc3b8105530 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.437+0000 7fc39ffff700 1 -- 172.21.15.63:0/2022148217 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3a80246e0 con 0x7fc3b8105530 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.437+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b819ab40 con 0x7fc3b8105530 2024-05-09T06:28:45.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.437+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b819afe0 con 0x7fc3b8105530 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.438+0000 7fc39ffff700 1 -- 172.21.15.63:0/2022148217 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fc3a802d3f0 con 0x7fc3b8105530 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.438+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b8194570 con 0x7fc3b8105530 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.438+0000 7fc39ffff700 1 --2- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc3a0038770 0x7fc3a003ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.438+0000 7fc39ffff700 1 -- 172.21.15.63:0/2022148217 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc3a804c7e0 con 0x7fc3b8105530 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.438+0000 7fc3b67fc700 1 --2- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc3a0038770 0x7fc3a003ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.438+0000 7fc3b67fc700 1 --2- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc3a0038770 0x7fc3a003ac30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac006fd0 tx=0x7fc3ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.444+0000 7fc39ffff700 1 -- 172.21.15.63:0/2022148217 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc3a8010930 con 0x7fc3b8105530 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.672+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3b8061330 con 0x7fc3a0038770 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.680+0000 7fc39ffff700 1 -- 172.21.15.63:0/2022148217 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fc3b8061330 con 0x7fc3a0038770 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc3a0038770 msgr2=0x7fc3a003ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 --2- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc3a0038770 0x7fc3a003ac30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac006fd0 tx=0x7fc3ac006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 msgr2=0x7fc3b819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 --2- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b819a400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80117b0 tx=0x7fc3a8011890 comp rx=0 tx=0).stop 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 shutdown_connections 2024-05-09T06:28:45.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 --2- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fc3a0038770 0x7fc3a003ac30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:45.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 --2- 172.21.15.63:0/2022148217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b8105530 0x7fc3b819a400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:45.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 >> 172.21.15.63:0/2022148217 conn(0x7fc3b8100a90 msgr2=0x7fc3b8101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:45.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 shutdown_connections 2024-05-09T06:28:45.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:44.682+0000 7fc3bd6e4700 1 -- 172.21.15.63:0/2022148217 wait complete. 2024-05-09T06:28:45.216 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying ceph-exporter service with default placement... 2024-05-09T06:28:45.330 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:45 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:45.331 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:45 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:45.331 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:45 smithi063 ceph-mon[27171]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:45.331 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:45 smithi063 ceph-mon[27171]: Saving service crash spec with placement * 2024-05-09T06:28:45.331 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:45 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:46.579 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.630+0000 7fd261ba9700 1 Processor -- start 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.630+0000 7fd261ba9700 1 -- start start 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.631+0000 7fd261ba9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.631+0000 7fd261ba9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c106fe0 con 0x7fd25c1065f0 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.631+0000 7fd25b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.631+0000 7fd25b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50492/0 (socket says 172.21.15.63:50492) 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.631+0000 7fd25b7fe700 1 -- 172.21.15.63:0/4127606488 learned_addr learned my addr 172.21.15.63:0/4127606488 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.631+0000 7fd25b7fe700 1 -- 172.21.15.63:0/4127606488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c1077a0 con 0x7fd25c1065f0 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.631+0000 7fd25b7fe700 1 --2- 172.21.15.63:0/4127606488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c106a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0098a0 tx=0x7fd24c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a15adda029b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.631+0000 7fd25a7fc700 1 -- 172.21.15.63:0/4127606488 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd24c004180 con 0x7fd25c1065f0 2024-05-09T06:28:46.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.632+0000 7fd25a7fc700 1 -- 172.21.15.63:0/4127606488 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd24c0042e0 con 0x7fd25c1065f0 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.632+0000 7fd25a7fc700 1 -- 172.21.15.63:0/4127606488 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd24c011b10 con 0x7fd25c1065f0 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.632+0000 7fd261ba9700 1 -- 172.21.15.63:0/4127606488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 msgr2=0x7fd25c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.632+0000 7fd261ba9700 1 --2- 172.21.15.63:0/4127606488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c106a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0098a0 tx=0x7fd24c009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.632+0000 7fd261ba9700 1 -- 172.21.15.63:0/4127606488 shutdown_connections 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.632+0000 7fd261ba9700 1 --2- 172.21.15.63:0/4127606488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c106a10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.632+0000 7fd261ba9700 1 -- 172.21.15.63:0/4127606488 >> 172.21.15.63:0/4127606488 conn(0x7fd25c101c30 msgr2=0x7fd25c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.632+0000 7fd261ba9700 1 -- 172.21.15.63:0/4127606488 shutdown_connections 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.632+0000 7fd261ba9700 1 -- 172.21.15.63:0/4127606488 wait complete. 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.633+0000 7fd261ba9700 1 Processor -- start 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.633+0000 7fd261ba9700 1 -- start start 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.633+0000 7fd261ba9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.633+0000 7fd261ba9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c19a8b0 con 0x7fd25c1065f0 2024-05-09T06:28:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd25b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd25b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50500/0 (socket says 172.21.15.63:50500) 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd25b7fe700 1 -- 172.21.15.63:0/4279362790 learned_addr learned my addr 172.21.15.63:0/4279362790 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd25b7fe700 1 -- 172.21.15.63:0/4279362790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c009550 con 0x7fd25c1065f0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd25b7fe700 1 --2- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c19a370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009fb0 tx=0x7fd24c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd258ff9700 1 -- 172.21.15.63:0/4279362790 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd24c0038b0 con 0x7fd25c1065f0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c19aab0 con 0x7fd25c1065f0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd258ff9700 1 -- 172.21.15.63:0/4279362790 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd24c003a10 con 0x7fd25c1065f0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd258ff9700 1 -- 172.21.15.63:0/4279362790 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd24c019770 con 0x7fd25c1065f0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.634+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c19af50 con 0x7fd25c1065f0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.635+0000 7fd258ff9700 1 -- 172.21.15.63:0/4279362790 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7fd24c0198d0 con 0x7fd25c1065f0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.635+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c10a990 con 0x7fd25c1065f0 2024-05-09T06:28:46.582 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.636+0000 7fd258ff9700 1 --2- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fd244038730 0x7fd24403abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.636+0000 7fd258ff9700 1 -- 172.21.15.63:0/4279362790 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd24c04c780 con 0x7fd25c1065f0 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.636+0000 7fd25affd700 1 --2- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fd244038730 0x7fd24403abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.636+0000 7fd25affd700 1 --2- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fd244038730 0x7fd24403abf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd250006fd0 tx=0x7fd250006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.641+0000 7fd258ff9700 1 -- 172.21.15.63:0/4279362790 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd24c022d70 con 0x7fd25c1065f0 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.879+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fd25c19d840 con 0x7fd244038730 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.883+0000 7fd258ff9700 1 -- 172.21.15.63:0/4279362790 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fd25c19d840 con 0x7fd244038730 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fd244038730 msgr2=0x7fd24403abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 --2- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fd244038730 0x7fd24403abf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd250006fd0 tx=0x7fd250006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 msgr2=0x7fd25c19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 --2- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c19a370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009fb0 tx=0x7fd24c011890 comp rx=0 tx=0).stop 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 shutdown_connections 2024-05-09T06:28:46.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 --2- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7fd244038730 0x7fd24403abf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 --2- 172.21.15.63:0/4279362790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25c1065f0 0x7fd25c19a370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:46.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 >> 172.21.15.63:0/4279362790 conn(0x7fd25c101c30 msgr2=0x7fd25c102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:46.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 shutdown_connections 2024-05-09T06:28:46.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:45.886+0000 7fd261ba9700 1 -- 172.21.15.63:0/4279362790 wait complete. 2024-05-09T06:28:46.584 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying prometheus service with default placement... 2024-05-09T06:28:46.949 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:46 smithi063 ceph-mon[27171]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:46.949 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:46 smithi063 ceph-mon[27171]: Saving service ceph-exporter spec with placement * 2024-05-09T06:28:46.950 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:46 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:47.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-05-09T06:28:47.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:46.998+0000 7f6614752700 1 Processor -- start 2024-05-09T06:28:47.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:46.998+0000 7f6614752700 1 -- start start 2024-05-09T06:28:47.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:46.999+0000 7f6614752700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.762 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:46.999+0000 7f6614752700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c107010 con 0x7f660c106620 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:46.999+0000 7f66124ee700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:46.999+0000 7f66124ee700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50504/0 (socket says 172.21.15.63:50504) 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:46.999+0000 7f66124ee700 1 -- 172.21.15.63:0/1998548514 learned_addr learned my addr 172.21.15.63:0/1998548514 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:46.999+0000 7f66124ee700 1 -- 172.21.15.63:0/1998548514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c1077d0 con 0x7f660c106620 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:46.999+0000 7f66124ee700 1 --2- 172.21.15.63:0/1998548514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c106a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f66080098a0 tx=0x7f6608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69ca275506b8ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f66114ec700 1 -- 172.21.15.63:0/1998548514 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6608004180 con 0x7f660c106620 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f66114ec700 1 -- 172.21.15.63:0/1998548514 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66080042e0 con 0x7f660c106620 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f66114ec700 1 -- 172.21.15.63:0/1998548514 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6608011b10 con 0x7f660c106620 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f6614752700 1 -- 172.21.15.63:0/1998548514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 msgr2=0x7f660c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f6614752700 1 --2- 172.21.15.63:0/1998548514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c106a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f66080098a0 tx=0x7f6608009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f6614752700 1 -- 172.21.15.63:0/1998548514 shutdown_connections 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f6614752700 1 --2- 172.21.15.63:0/1998548514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c106a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f6614752700 1 -- 172.21.15.63:0/1998548514 >> 172.21.15.63:0/1998548514 conn(0x7f660c101c60 msgr2=0x7f660c104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:47.763 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f6614752700 1 -- 172.21.15.63:0/1998548514 shutdown_connections 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.000+0000 7f6614752700 1 -- 172.21.15.63:0/1998548514 wait complete. 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.001+0000 7f6614752700 1 Processor -- start 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.001+0000 7f6614752700 1 -- start start 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.001+0000 7f6614752700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c07bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.001+0000 7f6614752700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c07c040 con 0x7f660c106620 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.001+0000 7f66124ee700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c07bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.001+0000 7f66124ee700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c07bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50512/0 (socket says 172.21.15.63:50512) 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.001+0000 7f66124ee700 1 -- 172.21.15.63:0/1093702782 learned_addr learned my addr 172.21.15.63:0/1093702782 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.002+0000 7f66124ee700 1 -- 172.21.15.63:0/1093702782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6608009550 con 0x7f660c106620 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.002+0000 7f66124ee700 1 --2- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c07bb00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f66080117b0 tx=0x7f6608011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.002+0000 7f66037fe700 1 -- 172.21.15.63:0/1093702782 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66080038b0 con 0x7f660c106620 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.002+0000 7f66037fe700 1 -- 172.21.15.63:0/1093702782 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6608003a10 con 0x7f660c106620 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.002+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c07a280 con 0x7f660c106620 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.002+0000 7f66037fe700 1 -- 172.21.15.63:0/1093702782 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66080246e0 con 0x7f660c106620 2024-05-09T06:28:47.764 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.002+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c07a720 con 0x7f660c106620 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.003+0000 7f66037fe700 1 -- 172.21.15.63:0/1093702782 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f660801a430 con 0x7f660c106620 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.003+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c062520 con 0x7f660c106620 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.004+0000 7f66037fe700 1 --2- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f65f8038770 0x7f65f803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.004+0000 7f66037fe700 1 -- 172.21.15.63:0/1093702782 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f660804c7f0 con 0x7f660c106620 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.004+0000 7f6611ced700 1 --2- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f65f8038770 0x7f65f803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.004+0000 7f6611ced700 1 --2- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f65f8038770 0x7f65f803ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65fc006fd0 tx=0x7f65fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.009+0000 7f66037fe700 1 -- 172.21.15.63:0/1093702782 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6608018b40 con 0x7f660c106620 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.229+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f660c07ae80 con 0x7f65f8038770 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.235+0000 7f66037fe700 1 -- 172.21.15.63:0/1093702782 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f660c07ae80 con 0x7f65f8038770 2024-05-09T06:28:47.765 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f65f8038770 msgr2=0x7f65f803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 --2- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f65f8038770 0x7f65f803ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65fc006fd0 tx=0x7f65fc006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 msgr2=0x7f660c07bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 --2- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c07bb00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f66080117b0 tx=0x7f6608011890 comp rx=0 tx=0).stop 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 shutdown_connections 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 --2- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f65f8038770 0x7f65f803ac30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 --2- 172.21.15.63:0/1093702782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f660c106620 0x7f660c07bb00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 >> 172.21.15.63:0/1093702782 conn(0x7f660c101c60 msgr2=0x7f660c1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 shutdown_connections 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:47.238+0000 7f6614752700 1 -- 172.21.15.63:0/1093702782 wait complete. 2024-05-09T06:28:47.766 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying grafana service with default placement... 2024-05-09T06:28:48.475 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:48 smithi063 ceph-mon[27171]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:48.475 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:48 smithi063 ceph-mon[27171]: Saving service prometheus spec with placement count:1 2024-05-09T06:28:48.475 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:48 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:49.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-05-09T06:28:49.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.255+0000 7f3b83d30700 1 Processor -- start 2024-05-09T06:28:49.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.255+0000 7f3b83d30700 1 -- start start 2024-05-09T06:28:49.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b83d30700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c109490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b83d30700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c074900 con 0x7f3b7c1070a0 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b81acc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c109490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b81acc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c109490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50526/0 (socket says 172.21.15.63:50526) 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b81acc700 1 -- 172.21.15.63:0/1078659905 learned_addr learned my addr 172.21.15.63:0/1078659905 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b81acc700 1 -- 172.21.15.63:0/1078659905 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c1099d0 con 0x7f3b7c1070a0 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b81acc700 1 --2- 172.21.15.63:0/1078659905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c109490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098a0 tx=0x7f3b78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedd08ef4779f4b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b80aca700 1 -- 172.21.15.63:0/1078659905 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b78004180 con 0x7f3b7c1070a0 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b80aca700 1 -- 172.21.15.63:0/1078659905 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b780042e0 con 0x7f3b7c1070a0 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.256+0000 7f3b80aca700 1 -- 172.21.15.63:0/1078659905 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b78011b10 con 0x7f3b7c1070a0 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.257+0000 7f3b83d30700 1 -- 172.21.15.63:0/1078659905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 msgr2=0x7f3b7c109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.257+0000 7f3b83d30700 1 --2- 172.21.15.63:0/1078659905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c109490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098a0 tx=0x7f3b78009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.257+0000 7f3b83d30700 1 -- 172.21.15.63:0/1078659905 shutdown_connections 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.257+0000 7f3b83d30700 1 --2- 172.21.15.63:0/1078659905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c109490 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.257+0000 7f3b83d30700 1 -- 172.21.15.63:0/1078659905 >> 172.21.15.63:0/1078659905 conn(0x7f3b7c100a90 msgr2=0x7f3b7c102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:49.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.257+0000 7f3b83d30700 1 -- 172.21.15.63:0/1078659905 shutdown_connections 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.257+0000 7f3b83d30700 1 -- 172.21.15.63:0/1078659905 wait complete. 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.258+0000 7f3b83d30700 1 Processor -- start 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.258+0000 7f3b83d30700 1 -- start start 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.258+0000 7f3b83d30700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c1068d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.258+0000 7f3b83d30700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c106e10 con 0x7f3b7c1070a0 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.258+0000 7f3b81acc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c1068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.258+0000 7f3b81acc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c1068d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50528/0 (socket says 172.21.15.63:50528) 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.258+0000 7f3b81acc700 1 -- 172.21.15.63:0/1170761459 learned_addr learned my addr 172.21.15.63:0/1170761459 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.259+0000 7f3b81acc700 1 -- 172.21.15.63:0/1170761459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b78009550 con 0x7f3b7c1070a0 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.259+0000 7f3b81acc700 1 --2- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c1068d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b780117b0 tx=0x7f3b78011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.259+0000 7f3b72ffd700 1 -- 172.21.15.63:0/1170761459 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b780038b0 con 0x7f3b7c1070a0 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.259+0000 7f3b72ffd700 1 -- 172.21.15.63:0/1170761459 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b78003a10 con 0x7f3b7c1070a0 2024-05-09T06:28:49.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.259+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c105050 con 0x7f3b7c1070a0 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.259+0000 7f3b72ffd700 1 -- 172.21.15.63:0/1170761459 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b78024780 con 0x7f3b7c1070a0 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.259+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c1054f0 con 0x7f3b7c1070a0 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.260+0000 7f3b72ffd700 1 -- 172.21.15.63:0/1170761459 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f3b780248e0 con 0x7f3b7c1070a0 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.260+0000 7f3b72ffd700 1 --2- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f3b68038770 0x7f3b6803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.260+0000 7f3b72ffd700 1 -- 172.21.15.63:0/1170761459 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3b7804c750 con 0x7f3b7c1070a0 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.260+0000 7f3b812cb700 1 --2- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f3b68038770 0x7f3b6803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.260+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c062520 con 0x7f3b7c1070a0 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.261+0000 7f3b812cb700 1 --2- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f3b68038770 0x7f3b6803ac30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c006fd0 tx=0x7f3b6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.267+0000 7f3b72ffd700 1 -- 172.21.15.63:0/1170761459 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3b78024b90 con 0x7f3b7c1070a0 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.493+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b7c105a10 con 0x7f3b68038770 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.499+0000 7f3b72ffd700 1 -- 172.21.15.63:0/1170761459 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f3b7c105a10 con 0x7f3b68038770 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.501+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f3b68038770 msgr2=0x7f3b6803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.501+0000 7f3b83d30700 1 --2- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f3b68038770 0x7f3b6803ac30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c006fd0 tx=0x7f3b6c006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:49.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.501+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 msgr2=0x7f3b7c1068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:49.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.501+0000 7f3b83d30700 1 --2- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c1068d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b780117b0 tx=0x7f3b78011890 comp rx=0 tx=0).stop 2024-05-09T06:28:49.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.502+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 shutdown_connections 2024-05-09T06:28:49.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.502+0000 7f3b83d30700 1 --2- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f3b68038770 0x7f3b6803ac30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.502+0000 7f3b83d30700 1 --2- 172.21.15.63:0/1170761459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b7c1070a0 0x7f3b7c1068d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:49.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.502+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 >> 172.21.15.63:0/1170761459 conn(0x7f3b7c100a90 msgr2=0x7f3b7c101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:49.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.502+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 shutdown_connections 2024-05-09T06:28:49.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:48.502+0000 7f3b83d30700 1 -- 172.21.15.63:0/1170761459 wait complete. 2024-05-09T06:28:49.106 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying node-exporter service with default placement... 2024-05-09T06:28:49.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:49 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:49.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:49 smithi063 ceph-mon[27171]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:49.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:49 smithi063 ceph-mon[27171]: Saving service grafana spec with placement count:1 2024-05-09T06:28:49.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:49 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:50.417 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-05-09T06:28:50.417 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.602+0000 7f96ef3e6700 1 Processor -- start 2024-05-09T06:28:50.417 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.603+0000 7f96ef3e6700 1 -- start start 2024-05-09T06:28:50.417 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.603+0000 7f96ef3e6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e8106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.603+0000 7f96ef3e6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8107010 con 0x7f96e8106620 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.603+0000 7f96ed182700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e8106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.603+0000 7f96ed182700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e8106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50532/0 (socket says 172.21.15.63:50532) 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.603+0000 7f96ed182700 1 -- 172.21.15.63:0/1220041716 learned_addr learned my addr 172.21.15.63:0/1220041716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.604+0000 7f96ed182700 1 -- 172.21.15.63:0/1220041716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e81077d0 con 0x7f96e8106620 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.604+0000 7f96ed182700 1 --2- 172.21.15.63:0/1220041716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e8106a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009e80 tx=0x7f96d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1f34f0f91df558 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.604+0000 7f96e7fff700 1 -- 172.21.15.63:0/1220041716 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96d800e070 con 0x7f96e8106620 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.604+0000 7f96e7fff700 1 -- 172.21.15.63:0/1220041716 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96d800b8c0 con 0x7f96e8106620 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.604+0000 7f96e7fff700 1 -- 172.21.15.63:0/1220041716 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96d80044d0 con 0x7f96e8106620 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.605+0000 7f96ef3e6700 1 -- 172.21.15.63:0/1220041716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 msgr2=0x7f96e8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.605+0000 7f96ef3e6700 1 --2- 172.21.15.63:0/1220041716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e8106a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009e80 tx=0x7f96d800b330 comp rx=0 tx=0).stop 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.605+0000 7f96ef3e6700 1 -- 172.21.15.63:0/1220041716 shutdown_connections 2024-05-09T06:28:50.418 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.605+0000 7f96ef3e6700 1 --2- 172.21.15.63:0/1220041716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e8106a40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.605+0000 7f96ef3e6700 1 -- 172.21.15.63:0/1220041716 >> 172.21.15.63:0/1220041716 conn(0x7f96e8101c60 msgr2=0x7f96e8104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.605+0000 7f96ef3e6700 1 -- 172.21.15.63:0/1220041716 shutdown_connections 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.605+0000 7f96ef3e6700 1 -- 172.21.15.63:0/1220041716 wait complete. 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.605+0000 7f96ef3e6700 1 Processor -- start 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96ef3e6700 1 -- start start 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96ef3e6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96ef3e6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819a860 con 0x7f96e8106620 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96ed182700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96ed182700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50544/0 (socket says 172.21.15.63:50544) 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96ed182700 1 -- 172.21.15.63:0/2954785656 learned_addr learned my addr 172.21.15.63:0/2954785656 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96ed182700 1 -- 172.21.15.63:0/2954785656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8009b30 con 0x7f96e8106620 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96ed182700 1 --2- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e819a320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f96d80072a0 tx=0x7f96d800bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96e67fc700 1 -- 172.21.15.63:0/2954785656 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96d800e040 con 0x7f96e8106620 2024-05-09T06:28:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96e67fc700 1 -- 172.21.15.63:0/2954785656 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96d8003dc0 con 0x7f96e8106620 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e819aa60 con 0x7f96e8106620 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.606+0000 7f96e67fc700 1 -- 172.21.15.63:0/2954785656 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96d8012600 con 0x7f96e8106620 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.607+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e819af60 con 0x7f96e8106620 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.607+0000 7f96e67fc700 1 -- 172.21.15.63:0/2954785656 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f96d8019040 con 0x7f96e8106620 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.608+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e810aa00 con 0x7f96e8106620 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.608+0000 7f96e67fc700 1 --2- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f96d0038710 0x7f96d003abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.608+0000 7f96e67fc700 1 -- 172.21.15.63:0/2954785656 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f96d804c190 con 0x7f96e8106620 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.608+0000 7f96ec981700 1 --2- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f96d0038710 0x7f96d003abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.608+0000 7f96ec981700 1 --2- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f96d0038710 0x7f96d003abd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006fd0 tx=0x7f96dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.613+0000 7f96e67fc700 1 -- 172.21.15.63:0/2954785656 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f96d801b430 con 0x7f96e8106620 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.835+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f96e819d790 con 0x7f96d0038710 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.841+0000 7f96e67fc700 1 -- 172.21.15.63:0/2954785656 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f96e819d790 con 0x7f96d0038710 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.843+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f96d0038710 msgr2=0x7f96d003abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.843+0000 7f96ef3e6700 1 --2- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f96d0038710 0x7f96d003abd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006fd0 tx=0x7f96dc006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.843+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 msgr2=0x7f96e819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.843+0000 7f96ef3e6700 1 --2- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e819a320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f96d80072a0 tx=0x7f96d800bf70 comp rx=0 tx=0).stop 2024-05-09T06:28:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.843+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 shutdown_connections 2024-05-09T06:28:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.843+0000 7f96ef3e6700 1 --2- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f96d0038710 0x7f96d003abd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.843+0000 7f96ef3e6700 1 --2- 172.21.15.63:0/2954785656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e8106620 0x7f96e819a320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.843+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 >> 172.21.15.63:0/2954785656 conn(0x7f96e8101c60 msgr2=0x7f96e81027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.843+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 shutdown_connections 2024-05-09T06:28:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:49.844+0000 7f96ef3e6700 1 -- 172.21.15.63:0/2954785656 wait complete. 2024-05-09T06:28:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying alertmanager service with default placement... 2024-05-09T06:28:51.088 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:50 smithi063 ceph-mon[27171]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:28:51.088 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:50 smithi063 ceph-mon[27171]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:51.089 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:50 smithi063 ceph-mon[27171]: Saving service node-exporter spec with placement * 2024-05-09T06:28:51.089 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:50 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:51.617 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-05-09T06:28:51.618 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.878+0000 7f5115810700 1 Processor -- start 2024-05-09T06:28:51.618 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.878+0000 7f5115810700 1 -- start start 2024-05-09T06:28:51.618 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f5115810700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f5110109450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:51.618 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f5115810700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5110079e90 con 0x7f5110107060 2024-05-09T06:28:51.618 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f510effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f5110109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:51.618 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f510effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f5110109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50554/0 (socket says 172.21.15.63:50554) 2024-05-09T06:28:51.618 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f510effd700 1 -- 172.21.15.63:0/1386180969 learned_addr learned my addr 172.21.15.63:0/1386180969 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:51.619 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f510effd700 1 -- 172.21.15.63:0/1386180969 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5110109990 con 0x7f5110107060 2024-05-09T06:28:51.619 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f510effd700 1 --2- 172.21.15.63:0/1386180969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f5110109450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5100009e80 tx=0x7f510000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7771ab25b171adc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:51.619 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f510dffb700 1 -- 172.21.15.63:0/1386180969 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f510000e070 con 0x7f5110107060 2024-05-09T06:28:51.619 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f510dffb700 1 -- 172.21.15.63:0/1386180969 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f510000b8c0 con 0x7f5110107060 2024-05-09T06:28:51.619 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.879+0000 7f510dffb700 1 -- 172.21.15.63:0/1386180969 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51000044d0 con 0x7f5110107060 2024-05-09T06:28:51.619 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.880+0000 7f5115810700 1 -- 172.21.15.63:0/1386180969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 msgr2=0x7f5110109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:51.619 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.880+0000 7f5115810700 1 --2- 172.21.15.63:0/1386180969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f5110109450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5100009e80 tx=0x7f510000b330 comp rx=0 tx=0).stop 2024-05-09T06:28:51.619 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.880+0000 7f5115810700 1 -- 172.21.15.63:0/1386180969 shutdown_connections 2024-05-09T06:28:51.619 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.880+0000 7f5115810700 1 --2- 172.21.15.63:0/1386180969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f5110109450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:51.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.880+0000 7f5115810700 1 -- 172.21.15.63:0/1386180969 >> 172.21.15.63:0/1386180969 conn(0x7f5110100ad0 msgr2=0x7f5110102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:51.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.880+0000 7f5115810700 1 -- 172.21.15.63:0/1386180969 shutdown_connections 2024-05-09T06:28:51.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.880+0000 7f5115810700 1 -- 172.21.15.63:0/1386180969 wait complete. 2024-05-09T06:28:51.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.881+0000 7f5115810700 1 Processor -- start 2024-05-09T06:28:51.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.881+0000 7f5115810700 1 -- start start 2024-05-09T06:28:51.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.881+0000 7f5115810700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f511019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:51.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.881+0000 7f5115810700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511019a840 con 0x7f5110107060 2024-05-09T06:28:51.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.881+0000 7f510effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f511019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:51.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.881+0000 7f510effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f511019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50570/0 (socket says 172.21.15.63:50570) 2024-05-09T06:28:51.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.881+0000 7f510effd700 1 -- 172.21.15.63:0/934267915 learned_addr learned my addr 172.21.15.63:0/934267915 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:51.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.881+0000 7f510effd700 1 -- 172.21.15.63:0/934267915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100009b30 con 0x7f5110107060 2024-05-09T06:28:51.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.882+0000 7f510effd700 1 --2- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f511019a300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f51000072a0 tx=0x7f510000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:51.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.882+0000 7f511480e700 1 -- 172.21.15.63:0/934267915 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f510000e040 con 0x7f5110107060 2024-05-09T06:28:51.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.882+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511019aa40 con 0x7f5110107060 2024-05-09T06:28:51.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.882+0000 7f511480e700 1 -- 172.21.15.63:0/934267915 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5100003dc0 con 0x7f5110107060 2024-05-09T06:28:51.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.882+0000 7f511480e700 1 -- 172.21.15.63:0/934267915 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5100012620 con 0x7f5110107060 2024-05-09T06:28:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.882+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511019af40 con 0x7f5110107060 2024-05-09T06:28:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.883+0000 7f511480e700 1 -- 172.21.15.63:0/934267915 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f5100019040 con 0x7f5110107060 2024-05-09T06:28:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.883+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511010a780 con 0x7f5110107060 2024-05-09T06:28:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.883+0000 7f511480e700 1 --2- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f50fc0386c0 0x7f50fc03ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.883+0000 7f511480e700 1 -- 172.21.15.63:0/934267915 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f510004c490 con 0x7f5110107060 2024-05-09T06:28:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.883+0000 7f510e7fc700 1 --2- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f50fc0386c0 0x7f50fc03ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.884+0000 7f510e7fc700 1 --2- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f50fc0386c0 0x7f50fc03ab80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006fd0 tx=0x7f50f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:50.888+0000 7f511480e700 1 -- 172.21.15.63:0/934267915 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5100011390 con 0x7f5110107060 2024-05-09T06:28:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.106+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f5110101d40 con 0x7f50fc0386c0 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.112+0000 7f511480e700 1 -- 172.21.15.63:0/934267915 <== mgr.14120 v2:172.21.15.63:6800/3629307253 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f5110101d40 con 0x7f50fc0386c0 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f50fc0386c0 msgr2=0x7f50fc03ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 --2- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f50fc0386c0 0x7f50fc03ab80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006fd0 tx=0x7f50f8006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 msgr2=0x7f511019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 --2- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f511019a300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f51000072a0 tx=0x7f510000bf70 comp rx=0 tx=0).stop 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 shutdown_connections 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 --2- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f50fc0386c0 0x7f50fc03ab80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 --2- 172.21.15.63:0/934267915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5110107060 0x7f511019a300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 >> 172.21.15.63:0/934267915 conn(0x7f5110100ad0 msgr2=0x7f5110101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 shutdown_connections 2024-05-09T06:28:51.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:51.114+0000 7f5115810700 1 -- 172.21.15.63:0/934267915 wait complete. 2024-05-09T06:28:52.299 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:52 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:52.300 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:52 smithi063 ceph-mon[27171]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:28:52.300 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:52 smithi063 ceph-mon[27171]: Saving service alertmanager spec with placement count:1 2024-05-09T06:28:52.300 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:52 smithi063 ceph-mon[27171]: from='mgr.14120 172.21.15.63:0/3935343242' entity='mgr.smithi063.egfguy' 2024-05-09T06:28:53.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:53 smithi063 ceph-mon[27171]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:28:53.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:53 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/912121721' entity='client.admin' 2024-05-09T06:28:53.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.105+0000 7f7cdbc17700 1 Processor -- start 2024-05-09T06:28:53.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.106+0000 7f7cdbc17700 1 -- start start 2024-05-09T06:28:53.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.106+0000 7f7cdbc17700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd41094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.106+0000 7f7cdbc17700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4079e50 con 0x7f7cd41070c0 2024-05-09T06:28:53.620 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.106+0000 7f7cd99b3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd41094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.106+0000 7f7cd99b3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd41094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50574/0 (socket says 172.21.15.63:50574) 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.106+0000 7f7cd99b3700 1 -- 172.21.15.63:0/3682312858 learned_addr learned my addr 172.21.15.63:0/3682312858 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.106+0000 7f7cd99b3700 1 -- 172.21.15.63:0/3682312858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd41099f0 con 0x7f7cd41070c0 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.107+0000 7f7cd99b3700 1 --2- 172.21.15.63:0/3682312858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd41094b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00098a0 tx=0x7f7cd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dc68a2dd45214f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.107+0000 7f7cd89b1700 1 -- 172.21.15.63:0/3682312858 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cd0004180 con 0x7f7cd41070c0 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.107+0000 7f7cd89b1700 1 -- 172.21.15.63:0/3682312858 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cd00042e0 con 0x7f7cd41070c0 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.107+0000 7f7cd89b1700 1 -- 172.21.15.63:0/3682312858 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cd0011b10 con 0x7f7cd41070c0 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.108+0000 7f7cdbc17700 1 -- 172.21.15.63:0/3682312858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 msgr2=0x7f7cd41094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.108+0000 7f7cdbc17700 1 --2- 172.21.15.63:0/3682312858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd41094b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00098a0 tx=0x7f7cd0009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.108+0000 7f7cdbc17700 1 -- 172.21.15.63:0/3682312858 shutdown_connections 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.108+0000 7f7cdbc17700 1 --2- 172.21.15.63:0/3682312858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd41094b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.108+0000 7f7cdbc17700 1 -- 172.21.15.63:0/3682312858 >> 172.21.15.63:0/3682312858 conn(0x7f7cd4100b10 msgr2=0x7f7cd4102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.108+0000 7f7cdbc17700 1 -- 172.21.15.63:0/3682312858 shutdown_connections 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.108+0000 7f7cdbc17700 1 -- 172.21.15.63:0/3682312858 wait complete. 2024-05-09T06:28:53.621 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.108+0000 7f7cdbc17700 1 Processor -- start 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cdbc17700 1 -- start start 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cdbc17700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cdbc17700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd419a910 con 0x7f7cd41070c0 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cd99b3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cd99b3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50588/0 (socket says 172.21.15.63:50588) 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cd99b3700 1 -- 172.21.15.63:0/912121721 learned_addr learned my addr 172.21.15.63:0/912121721 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cd99b3700 1 -- 172.21.15.63:0/912121721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd0009550 con 0x7f7cd41070c0 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cd99b3700 1 --2- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd419a3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00117b0 tx=0x7f7cd0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7ccaffd700 1 -- 172.21.15.63:0/912121721 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cd00038b0 con 0x7f7cd41070c0 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd419ab10 con 0x7f7cd41070c0 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7ccaffd700 1 -- 172.21.15.63:0/912121721 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cd0003a10 con 0x7f7cd41070c0 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7ccaffd700 1 -- 172.21.15.63:0/912121721 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cd0024780 con 0x7f7cd41070c0 2024-05-09T06:28:53.622 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.109+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd419afb0 con 0x7f7cd41070c0 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.110+0000 7f7ccaffd700 1 -- 172.21.15.63:0/912121721 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f7cd00248e0 con 0x7f7cd41070c0 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.111+0000 7f7ccaffd700 1 --2- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f7cc0038770 0x7f7cc003ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.110+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd410a780 con 0x7f7cd41070c0 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.111+0000 7f7ccaffd700 1 -- 172.21.15.63:0/912121721 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7cd004c8b0 con 0x7f7cd41070c0 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.111+0000 7f7cd91b2700 1 --2- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f7cc0038770 0x7f7cc003ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.111+0000 7f7cd91b2700 1 --2- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f7cc0038770 0x7f7cc003ac30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006fd0 tx=0x7f7cc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.116+0000 7f7ccaffd700 1 -- 172.21.15.63:0/912121721 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7cd0018db0 con 0x7f7cd41070c0 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.325+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f7cd410a9c0 con 0x7f7cd41070c0 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.329+0000 7f7ccaffd700 1 -- 172.21.15.63:0/912121721 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f7cd000f2e0 con 0x7f7cd41070c0 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f7cc0038770 msgr2=0x7f7cc003ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 --2- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f7cc0038770 0x7f7cc003ac30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006fd0 tx=0x7f7cc4006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 msgr2=0x7f7cd419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 --2- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd419a3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00117b0 tx=0x7f7cd0011890 comp rx=0 tx=0).stop 2024-05-09T06:28:53.623 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 shutdown_connections 2024-05-09T06:28:53.624 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 --2- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f7cc0038770 0x7f7cc003ac30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.624 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 --2- 172.21.15.63:0/912121721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cd41070c0 0x7f7cd419a3d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:53.624 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 >> 172.21.15.63:0/912121721 conn(0x7f7cd4100b10 msgr2=0x7f7cd41016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:53.624 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 shutdown_connections 2024-05-09T06:28:53.624 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:52.332+0000 7f7cdbc17700 1 -- 172.21.15.63:0/912121721 wait complete. 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.051+0000 7f02f6152700 1 Processor -- start 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.051+0000 7f02f6152700 1 -- start start 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.051+0000 7f02f6152700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f0104d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.051+0000 7f02f6152700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f0105330 con 0x7f02f0104940 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.051+0000 7f02ef7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f0104d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.051+0000 7f02ef7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f0104d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50598/0 (socket says 172.21.15.63:50598) 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.051+0000 7f02ef7fe700 1 -- 172.21.15.63:0/2679149209 learned_addr learned my addr 172.21.15.63:0/2679149209 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.051+0000 7f02ef7fe700 1 -- 172.21.15.63:0/2679149209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f01078e0 con 0x7f02f0104940 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.052+0000 7f02ef7fe700 1 --2- 172.21.15.63:0/2679149209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f0104d60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098a0 tx=0x7f02e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3644d6e0ed4f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.052+0000 7f02ee7fc700 1 -- 172.21.15.63:0/2679149209 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02e0004180 con 0x7f02f0104940 2024-05-09T06:28:54.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.052+0000 7f02ee7fc700 1 -- 172.21.15.63:0/2679149209 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02e00042e0 con 0x7f02f0104940 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.052+0000 7f02ee7fc700 1 -- 172.21.15.63:0/2679149209 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02e0011b10 con 0x7f02f0104940 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.053+0000 7f02f6152700 1 -- 172.21.15.63:0/2679149209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 msgr2=0x7f02f0104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.053+0000 7f02f6152700 1 --2- 172.21.15.63:0/2679149209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f0104d60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098a0 tx=0x7f02e0009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.053+0000 7f02f6152700 1 -- 172.21.15.63:0/2679149209 shutdown_connections 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.053+0000 7f02f6152700 1 --2- 172.21.15.63:0/2679149209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f0104d60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.053+0000 7f02f6152700 1 -- 172.21.15.63:0/2679149209 >> 172.21.15.63:0/2679149209 conn(0x7f02f01001e0 msgr2=0x7f02f0102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.053+0000 7f02f6152700 1 -- 172.21.15.63:0/2679149209 shutdown_connections 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.053+0000 7f02f6152700 1 -- 172.21.15.63:0/2679149209 wait complete. 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.053+0000 7f02f6152700 1 Processor -- start 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.054+0000 7f02f6152700 1 -- start start 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.054+0000 7f02f6152700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.054+0000 7f02f6152700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f019a8e0 con 0x7f02f0104940 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.054+0000 7f02ef7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.054+0000 7f02ef7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50600/0 (socket says 172.21.15.63:50600) 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.054+0000 7f02ef7fe700 1 -- 172.21.15.63:0/786528781 learned_addr learned my addr 172.21.15.63:0/786528781 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.054+0000 7f02ef7fe700 1 -- 172.21.15.63:0/786528781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e0009550 con 0x7f02f0104940 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.054+0000 7f02ef7fe700 1 --2- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f019a3a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02e00117b0 tx=0x7f02e0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.054+0000 7f02ecff9700 1 -- 172.21.15.63:0/786528781 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02e00038b0 con 0x7f02f0104940 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.055+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f019aae0 con 0x7f02f0104940 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.055+0000 7f02ecff9700 1 -- 172.21.15.63:0/786528781 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02e0003a10 con 0x7f02f0104940 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.055+0000 7f02ecff9700 1 -- 172.21.15.63:0/786528781 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02e0024780 con 0x7f02f0104940 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.055+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f019af80 con 0x7f02f0104940 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.055+0000 7f02ecff9700 1 -- 172.21.15.63:0/786528781 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f02e00248e0 con 0x7f02f0104940 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.056+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f010ab10 con 0x7f02f0104940 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.056+0000 7f02ecff9700 1 --2- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f02d8038720 0x7f02d803abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.056+0000 7f02ecff9700 1 -- 172.21.15.63:0/786528781 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f02e004c7b0 con 0x7f02f0104940 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.056+0000 7f02eeffd700 1 --2- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f02d8038720 0x7f02d803abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:54.846 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.056+0000 7f02eeffd700 1 --2- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f02d8038720 0x7f02d803abe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f02e4006fd0 tx=0x7f02e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.061+0000 7f02ecff9700 1 -- 172.21.15.63:0/786528781 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f02e0024b90 con 0x7f02f0104940 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.263+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f02f019d810 con 0x7f02f0104940 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.266+0000 7f02ecff9700 1 -- 172.21.15.63:0/786528781 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f02e0018350 con 0x7f02f0104940 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.268+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f02d8038720 msgr2=0x7f02d803abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.268+0000 7f02f6152700 1 --2- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f02d8038720 0x7f02d803abe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f02e4006fd0 tx=0x7f02e4006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.268+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 msgr2=0x7f02f019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.268+0000 7f02f6152700 1 --2- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f019a3a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02e00117b0 tx=0x7f02e0011890 comp rx=0 tx=0).stop 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.268+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 shutdown_connections 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.268+0000 7f02f6152700 1 --2- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f02d8038720 0x7f02d803abe0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.268+0000 7f02f6152700 1 --2- 172.21.15.63:0/786528781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02f0104940 0x7f02f019a3a0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02e00117b0 tx=0x7f02e0011890 comp rx=0 tx=0).stop 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.268+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 >> 172.21.15.63:0/786528781 conn(0x7f02f01001e0 msgr2=0x7f02f0100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.269+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 shutdown_connections 2024-05-09T06:28:54.848 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:54.269+0000 7f02f6152700 1 -- 172.21.15.63:0/786528781 wait complete. 2024-05-09T06:28:54.848 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling the dashboard module... 2024-05-09T06:28:55.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:55 smithi063 ceph-mon[27171]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:28:55.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:55 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/786528781' entity='client.admin' 2024-05-09T06:28:56.576 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:56 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1675708292' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T06:28:56.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.321+0000 7f0df2a69700 1 Processor -- start 2024-05-09T06:28:56.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.321+0000 7f0df2a69700 1 -- start start 2024-05-09T06:28:56.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0df2a69700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec105910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:56.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0df2a69700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec105ee0 con 0x7f0dec1054f0 2024-05-09T06:28:56.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0df0805700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec105910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:56.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0df0805700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec105910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50864/0 (socket says 172.21.15.63:50864) 2024-05-09T06:28:56.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0df0805700 1 -- 172.21.15.63:0/1435343109 learned_addr learned my addr 172.21.15.63:0/1435343109 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:56.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0df0805700 1 -- 172.21.15.63:0/1435343109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec1066f0 con 0x7f0dec1054f0 2024-05-09T06:28:56.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0df0805700 1 --2- 172.21.15.63:0/1435343109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec105910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098a0 tx=0x7f0ddc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63811a6cd3d8d72c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:56.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0de37fe700 1 -- 172.21.15.63:0/1435343109 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ddc004180 con 0x7f0dec1054f0 2024-05-09T06:28:56.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0de37fe700 1 -- 172.21.15.63:0/1435343109 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ddc0042e0 con 0x7f0dec1054f0 2024-05-09T06:28:56.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.322+0000 7f0de37fe700 1 -- 172.21.15.63:0/1435343109 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ddc011b10 con 0x7f0dec1054f0 2024-05-09T06:28:56.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.323+0000 7f0df2a69700 1 -- 172.21.15.63:0/1435343109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 msgr2=0x7f0dec105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:56.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.323+0000 7f0df2a69700 1 --2- 172.21.15.63:0/1435343109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec105910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098a0 tx=0x7f0ddc009bb0 comp rx=0 tx=0).stop 2024-05-09T06:28:56.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.323+0000 7f0df2a69700 1 -- 172.21.15.63:0/1435343109 shutdown_connections 2024-05-09T06:28:56.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.323+0000 7f0df2a69700 1 --2- 172.21.15.63:0/1435343109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec105910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:56.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.323+0000 7f0df2a69700 1 -- 172.21.15.63:0/1435343109 >> 172.21.15.63:0/1435343109 conn(0x7f0dec100ad0 msgr2=0x7f0dec102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:56.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.323+0000 7f0df2a69700 1 -- 172.21.15.63:0/1435343109 shutdown_connections 2024-05-09T06:28:56.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.323+0000 7f0df2a69700 1 -- 172.21.15.63:0/1435343109 wait complete. 2024-05-09T06:28:56.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0df2a69700 1 Processor -- start 2024-05-09T06:28:56.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0df2a69700 1 -- start start 2024-05-09T06:28:56.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0df2a69700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:56.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0df2a69700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec19a900 con 0x7f0dec1054f0 2024-05-09T06:28:56.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0df0805700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:56.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0df0805700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec19a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50870/0 (socket says 172.21.15.63:50870) 2024-05-09T06:28:56.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0df0805700 1 -- 172.21.15.63:0/1675708292 learned_addr learned my addr 172.21.15.63:0/1675708292 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:56.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0df0805700 1 -- 172.21.15.63:0/1675708292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc009550 con 0x7f0dec1054f0 2024-05-09T06:28:56.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0df0805700 1 --2- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec19a3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0117b0 tx=0x7f0ddc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:56.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.324+0000 7f0de1ffb700 1 -- 172.21.15.63:0/1675708292 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ddc0038b0 con 0x7f0dec1054f0 2024-05-09T06:28:56.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.325+0000 7f0de1ffb700 1 -- 172.21.15.63:0/1675708292 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ddc003a10 con 0x7f0dec1054f0 2024-05-09T06:28:56.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.325+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec19ab00 con 0x7f0dec1054f0 2024-05-09T06:28:56.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.325+0000 7f0de1ffb700 1 -- 172.21.15.63:0/1675708292 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ddc0246e0 con 0x7f0dec1054f0 2024-05-09T06:28:56.877 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.325+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec19afa0 con 0x7f0dec1054f0 2024-05-09T06:28:56.878 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.326+0000 7f0de1ffb700 1 -- 172.21.15.63:0/1675708292 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 45556+0+0 (secure 0 0 0) 0x7f0ddc02d3f0 con 0x7f0dec1054f0 2024-05-09T06:28:56.878 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.326+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec194530 con 0x7f0dec1054f0 2024-05-09T06:28:56.878 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.326+0000 7f0de1ffb700 1 --2- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0dd4038770 0x7f0dd403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:56.878 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.326+0000 7f0de1ffb700 1 -- 172.21.15.63:0/1675708292 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0ddc04c9d0 con 0x7f0dec1054f0 2024-05-09T06:28:56.878 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.326+0000 7f0de3fff700 1 --2- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0dd4038770 0x7f0dd403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:56.878 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.327+0000 7f0de3fff700 1 --2- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0dd4038770 0x7f0dd403ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0de8006fd0 tx=0x7f0de8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:56.879 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.331+0000 7f0de1ffb700 1 -- 172.21.15.63:0/1675708292 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0ddc018cd0 con 0x7f0dec1054f0 2024-05-09T06:28:56.879 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:55.593+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f0dec109920 con 0x7f0dec1054f0 2024-05-09T06:28:56.879 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.270+0000 7f0de1ffb700 1 -- 172.21.15.63:0/1675708292 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 9) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f0ddc01ab80 con 0x7f0dec1054f0 2024-05-09T06:28:56.879 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.271+0000 7f0de1ffb700 1 -- 172.21.15.63:0/1675708292 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f0ddc02d6a0 con 0x7f0dec1054f0 2024-05-09T06:28:56.879 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0dd4038770 msgr2=0x7f0dd403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:56.880 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 --2- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0dd4038770 0x7f0dd403ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0de8006fd0 tx=0x7f0de8006e40 comp rx=0 tx=0).stop 2024-05-09T06:28:56.880 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 msgr2=0x7f0dec19a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:56.880 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 --2- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec19a3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0117b0 tx=0x7f0ddc011890 comp rx=0 tx=0).stop 2024-05-09T06:28:56.880 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 shutdown_connections 2024-05-09T06:28:56.880 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 --2- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0dd4038770 0x7f0dd403ac30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:56.881 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 --2- 172.21.15.63:0/1675708292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0dec1054f0 0x7f0dec19a3c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:56.881 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 >> 172.21.15.63:0/1675708292 conn(0x7f0dec100ad0 msgr2=0x7f0dec101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:56.881 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 shutdown_connections 2024-05-09T06:28:56.881 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:56.274+0000 7f0df2a69700 1 -- 172.21.15.63:0/1675708292 wait complete. 2024-05-09T06:28:57.516 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:57 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1675708292' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T06:28:57.517 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:57 smithi063 ceph-mon[27171]: mgrmap e9: smithi063.egfguy(active, since 26s) 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "smithi063.egfguy", 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.328+0000 7f79a452c700 1 Processor -- start 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.328+0000 7f79a452c700 1 -- start start 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.328+0000 7f79a452c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c1094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.328+0000 7f79a452c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c0748c0 con 0x7f799c1070e0 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.329+0000 7f79a22c8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c1094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.175 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.329+0000 7f79a22c8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c1094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50890/0 (socket says 172.21.15.63:50890) 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.329+0000 7f79a22c8700 1 -- 172.21.15.63:0/3137385075 learned_addr learned my addr 172.21.15.63:0/3137385075 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.329+0000 7f79a22c8700 1 -- 172.21.15.63:0/3137385075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c109a10 con 0x7f799c1070e0 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.329+0000 7f79a22c8700 1 --2- 172.21.15.63:0/3137385075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c1094d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7998009e80 tx=0x7f799800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a15548f1485d75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.329+0000 7f79a12c6700 1 -- 172.21.15.63:0/3137385075 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f799800e070 con 0x7f799c1070e0 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.329+0000 7f79a12c6700 1 -- 172.21.15.63:0/3137385075 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f799800b8c0 con 0x7f799c1070e0 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.329+0000 7f79a12c6700 1 -- 172.21.15.63:0/3137385075 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79980044d0 con 0x7f799c1070e0 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.330+0000 7f79a452c700 1 -- 172.21.15.63:0/3137385075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 msgr2=0x7f799c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.330+0000 7f79a452c700 1 --2- 172.21.15.63:0/3137385075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c1094d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7998009e80 tx=0x7f799800b330 comp rx=0 tx=0).stop 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.330+0000 7f79a452c700 1 -- 172.21.15.63:0/3137385075 shutdown_connections 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.330+0000 7f79a452c700 1 --2- 172.21.15.63:0/3137385075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c1094d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.330+0000 7f79a452c700 1 -- 172.21.15.63:0/3137385075 >> 172.21.15.63:0/3137385075 conn(0x7f799c100a90 msgr2=0x7f799c102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.330+0000 7f79a452c700 1 -- 172.21.15.63:0/3137385075 shutdown_connections 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.330+0000 7f79a452c700 1 -- 172.21.15.63:0/3137385075 wait complete. 2024-05-09T06:28:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.331+0000 7f79a452c700 1 Processor -- start 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.331+0000 7f79a452c700 1 -- start start 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.331+0000 7f79a452c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.331+0000 7f79a452c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c19a890 con 0x7f799c1070e0 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.331+0000 7f79a22c8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.331+0000 7f79a22c8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50902/0 (socket says 172.21.15.63:50902) 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.331+0000 7f79a22c8700 1 -- 172.21.15.63:0/286914438 learned_addr learned my addr 172.21.15.63:0/286914438 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.332+0000 7f79a22c8700 1 -- 172.21.15.63:0/286914438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998009b30 con 0x7f799c1070e0 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.332+0000 7f79a22c8700 1 --2- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c19a350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79980072a0 tx=0x7f799800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.332+0000 7f798f7fe700 1 -- 172.21.15.63:0/286914438 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f799800e040 con 0x7f799c1070e0 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.332+0000 7f798f7fe700 1 -- 172.21.15.63:0/286914438 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7998003dc0 con 0x7f799c1070e0 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.332+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c19aa90 con 0x7f799c1070e0 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.332+0000 7f798f7fe700 1 -- 172.21.15.63:0/286914438 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7998012750 con 0x7f799c1070e0 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.332+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c19af90 con 0x7f799c1070e0 2024-05-09T06:28:58.177 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.333+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c10a7a0 con 0x7f799c1070e0 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.333+0000 7f798f7fe700 1 -- 172.21.15.63:0/286914438 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f7998019040 con 0x7f799c1070e0 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.334+0000 7f798f7fe700 1 --2- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f79880387c0 0x7f798803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.334+0000 7f798f7fe700 1 -- 172.21.15.63:0/286914438 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f799804c110 con 0x7f799c1070e0 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.334+0000 7f79a1ac7700 1 -- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f79880387c0 msgr2=0x7f798803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3629307253 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.334+0000 7f79a1ac7700 1 --2- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f79880387c0 0x7f798803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.339+0000 7f798f7fe700 1 -- 172.21.15.63:0/286914438 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f799801a570 con 0x7f799c1070e0 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.535+0000 7f79a1ac7700 1 -- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f79880387c0 msgr2=0x7f798803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3629307253 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.535+0000 7f79a1ac7700 1 --2- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f79880387c0 0x7f798803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.613+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f799c19dba0 con 0x7f799c1070e0 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.615+0000 7f798f7fe700 1 -- 172.21.15.63:0/286914438 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f799801f020 con 0x7f799c1070e0 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.620+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f79880387c0 msgr2=0x7f798803ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.620+0000 7f79a452c700 1 --2- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f79880387c0 0x7f798803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.178 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.620+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 msgr2=0x7f799c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:28:58.179 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.620+0000 7f79a452c700 1 --2- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c19a350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79980072a0 tx=0x7f799800be00 comp rx=0 tx=0).stop 2024-05-09T06:28:58.179 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.620+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 shutdown_connections 2024-05-09T06:28:58.179 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.620+0000 7f79a452c700 1 --2- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f79880387c0 0x7f798803ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:28:58.179 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.620+0000 7f79a452c700 1 --2- 172.21.15.63:0/286914438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f799c1070e0 0x7f799c19a350 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79980072a0 tx=0x7f799800be00 comp rx=0 tx=0).stop 2024-05-09T06:28:58.179 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.620+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 >> 172.21.15.63:0/286914438 conn(0x7f799c100a90 msgr2=0x7f799c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:28:58.179 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.621+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 shutdown_connections 2024-05-09T06:28:58.179 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:57.622+0000 7f79a452c700 1 -- 172.21.15.63:0/286914438 wait complete. 2024-05-09T06:28:58.179 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-05-09T06:28:58.179 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 9... 2024-05-09T06:28:58.294 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:28:58 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/286914438' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: Active manager daemon smithi063.egfguy restarted 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: Activating manager daemon smithi063.egfguy 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: mgrmap e10: smithi063.egfguy(active, starting, since 0.00565816s) 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: Manager daemon smithi063.egfguy is now available 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:29:03.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:03 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:29:04.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:04 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/trash_purge_schedule"}]: dispatch 2024-05-09T06:29:04.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:04 smithi063 ceph-mon[27171]: [09/May/2024:06:29:04] ENGINE Bus STARTING 2024-05-09T06:29:04.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:04 smithi063 ceph-mon[27171]: mgrmap e11: smithi063.egfguy(active, since 1.00542s) 2024-05-09T06:29:05.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-09T06:29:05.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T06:29:05.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T06:29:05.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.675+0000 7f0da4db2700 1 Processor -- start 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.675+0000 7f0da4db2700 1 -- start start 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.676+0000 7f0da4db2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da0109510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.676+0000 7f0da4db2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da0074880 con 0x7f0da0107120 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.676+0000 7f0d9e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da0109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.676+0000 7f0d9e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da0109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50918/0 (socket says 172.21.15.63:50918) 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.676+0000 7f0d9e59c700 1 -- 172.21.15.63:0/3787348860 learned_addr learned my addr 172.21.15.63:0/3787348860 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.676+0000 7f0d9e59c700 1 -- 172.21.15.63:0/3787348860 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0109a50 con 0x7f0da0107120 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.676+0000 7f0d9e59c700 1 --2- 172.21.15.63:0/3787348860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da0109510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d900098a0 tx=0x7f0d90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6bf14affb0e346 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.676+0000 7f0d9d59a700 1 -- 172.21.15.63:0/3787348860 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d90004180 con 0x7f0da0107120 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.677+0000 7f0d9d59a700 1 -- 172.21.15.63:0/3787348860 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d900042e0 con 0x7f0da0107120 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.677+0000 7f0d9d59a700 1 -- 172.21.15.63:0/3787348860 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d90011b10 con 0x7f0da0107120 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.677+0000 7f0da4db2700 1 -- 172.21.15.63:0/3787348860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 msgr2=0x7f0da0109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.677+0000 7f0da4db2700 1 --2- 172.21.15.63:0/3787348860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da0109510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d900098a0 tx=0x7f0d90009bb0 comp rx=0 tx=0).stop 2024-05-09T06:29:05.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.677+0000 7f0da4db2700 1 -- 172.21.15.63:0/3787348860 shutdown_connections 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.677+0000 7f0da4db2700 1 --2- 172.21.15.63:0/3787348860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da0109510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.677+0000 7f0da4db2700 1 -- 172.21.15.63:0/3787348860 >> 172.21.15.63:0/3787348860 conn(0x7f0da0100a90 msgr2=0x7f0da0102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.677+0000 7f0da4db2700 1 -- 172.21.15.63:0/3787348860 shutdown_connections 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.677+0000 7f0da4db2700 1 -- 172.21.15.63:0/3787348860 wait complete. 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.678+0000 7f0da4db2700 1 Processor -- start 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.678+0000 7f0da4db2700 1 -- start start 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.678+0000 7f0da4db2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.678+0000 7f0da4db2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da019a8c0 con 0x7f0da0107120 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.678+0000 7f0d9e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.679+0000 7f0d9e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50922/0 (socket says 172.21.15.63:50922) 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.679+0000 7f0d9e59c700 1 -- 172.21.15.63:0/132553491 learned_addr learned my addr 172.21.15.63:0/132553491 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.679+0000 7f0d9e59c700 1 -- 172.21.15.63:0/132553491 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d90009550 con 0x7f0da0107120 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.679+0000 7f0d9e59c700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da019a380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d900117b0 tx=0x7f0d90011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:05.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.679+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d900038b0 con 0x7f0da0107120 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.679+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d90003a10 con 0x7f0da0107120 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.679+0000 7f0da4db2700 1 -- 172.21.15.63:0/132553491 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da019aac0 con 0x7f0da0107120 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.679+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d90024780 con 0x7f0da0107120 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.679+0000 7f0da4db2700 1 -- 172.21.15.63:0/132553491 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da019af60 con 0x7f0da0107120 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.680+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f0d900248e0 con 0x7f0da0107120 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.680+0000 7f0d8f7fe700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 0x7f0d8803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.680+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 --> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d8803b390 con 0x7f0d880387c0 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.681+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0d9004c810 con 0x7f0da0107120 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.681+0000 7f0d9dd9b700 1 -- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 msgr2=0x7f0d8803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3629307253 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.681+0000 7f0d9dd9b700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 0x7f0d8803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.882+0000 7f0d9dd9b700 1 -- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 msgr2=0x7f0d8803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3629307253 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:58.882+0000 7f0d9dd9b700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 0x7f0d8803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T06:29:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:59.283+0000 7f0d9dd9b700 1 -- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 msgr2=0x7f0d8803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3629307253 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:28:59.283+0000 7f0d9dd9b700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 0x7f0d8803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:00.084+0000 7f0d9dd9b700 1 -- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 msgr2=0x7f0d8803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3629307253 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:00.084+0000 7f0d9dd9b700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 0x7f0d8803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:01.685+0000 7f0d9dd9b700 1 -- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 msgr2=0x7f0d8803ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3629307253 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:01.685+0000 7f0d9dd9b700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 0x7f0d8803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:03.471+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 10) v1 ==== 45336+0+0 (secure 0 0 0) 0x7f0d9002d9b0 con 0x7f0da0107120 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:03.471+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 msgr2=0x7f0d8803ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:03.471+0000 7f0d8f7fe700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 0x7f0d8803ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.470+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 11) v1 ==== 45463+0+0 (secure 0 0 0) 0x7f0d90001ba0 con 0x7f0da0107120 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.471+0000 7f0d8f7fe700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0d8803bfc0 0x7f0d8803e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.471+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 --> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d8803b390 con 0x7f0d8803bfc0 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.471+0000 7f0d9dd9b700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0d8803bfc0 0x7f0d8803e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.472+0000 7f0d9dd9b700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0d8803bfc0 0x7f0d8803e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0d94003a10 tx=0x7f0d94009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:05.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.472+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 <== mgr.14164 v2:172.21.15.63:6800/3098785268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f0d8803b390 con 0x7f0d8803bfc0 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 -- 172.21.15.63:0/132553491 --> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0da010a790 con 0x7f0d8803bfc0 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0d8f7fe700 1 -- 172.21.15.63:0/132553491 <== mgr.14164 v2:172.21.15.63:6800/3098785268 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f0da010a790 con 0x7f0d8803bfc0 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 -- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0d8803bfc0 msgr2=0x7f0d8803e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0d8803bfc0 0x7f0d8803e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0d94003a10 tx=0x7f0d94009270 comp rx=0 tx=0).stop 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 -- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 msgr2=0x7f0da019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da019a380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d900117b0 tx=0x7f0d90011890 comp rx=0 tx=0).stop 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 -- 172.21.15.63:0/132553491 shutdown_connections 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0d8803bfc0 0x7f0d8803e3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0da0107120 0x7f0da019a380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 --2- 172.21.15.63:0/132553491 >> [v2:172.21.15.63:6800/3629307253,v1:172.21.15.63:6801/3629307253] conn(0x7f0d880387c0 0x7f0d8803ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 -- 172.21.15.63:0/132553491 >> 172.21.15.63:0/132553491 conn(0x7f0da0100a90 msgr2=0x7f0da0101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 -- 172.21.15.63:0/132553491 shutdown_connections 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:04.479+0000 7f0da4db2700 1 -- 172.21.15.63:0/132553491 wait complete. 2024-05-09T06:29:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 9 is available 2024-05-09T06:29:05.142 INFO:teuthology.orchestra.run.smithi063.stdout:Generating a dashboard self-signed certificate... 2024-05-09T06:29:06.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:05 smithi063 ceph-mon[27171]: [09/May/2024:06:29:04] ENGINE Serving on https://172.21.15.63:7150 2024-05-09T06:29:06.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:05 smithi063 ceph-mon[27171]: [09/May/2024:06:29:04] ENGINE Serving on http://172.21.15.63:8765 2024-05-09T06:29:06.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:05 smithi063 ceph-mon[27171]: [09/May/2024:06:29:04] ENGINE Bus STARTED 2024-05-09T06:29:06.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:05 smithi063 ceph-mon[27171]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T06:29:06.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:05 smithi063 ceph-mon[27171]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T06:29:06.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:05 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:06.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:05 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.587+0000 7f25b7797700 1 Processor -- start 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.588+0000 7f25b7797700 1 -- start start 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.588+0000 7f25b7797700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b0104d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.588+0000 7f25b7797700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b01052d0 con 0x7f25b01048e0 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.588+0000 7f25b5533700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b0104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.588+0000 7f25b5533700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b0104d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35372/0 (socket says 172.21.15.63:35372) 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.588+0000 7f25b5533700 1 -- 172.21.15.63:0/4260349053 learned_addr learned my addr 172.21.15.63:0/4260349053 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.588+0000 7f25b5533700 1 -- 172.21.15.63:0/4260349053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0107870 con 0x7f25b01048e0 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.589+0000 7f25b5533700 1 --2- 172.21.15.63:0/4260349053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b0104d00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0098a0 tx=0x7f25ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb6a01a16c07a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:06.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.589+0000 7f25a7fff700 1 -- 172.21.15.63:0/4260349053 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ac004180 con 0x7f25b01048e0 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.589+0000 7f25a7fff700 1 -- 172.21.15.63:0/4260349053 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25ac0042e0 con 0x7f25b01048e0 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.589+0000 7f25a7fff700 1 -- 172.21.15.63:0/4260349053 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ac011b10 con 0x7f25b01048e0 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.589+0000 7f25b7797700 1 -- 172.21.15.63:0/4260349053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 msgr2=0x7f25b0104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.589+0000 7f25b7797700 1 --2- 172.21.15.63:0/4260349053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b0104d00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0098a0 tx=0x7f25ac009bb0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.590+0000 7f25b7797700 1 -- 172.21.15.63:0/4260349053 shutdown_connections 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.590+0000 7f25b7797700 1 --2- 172.21.15.63:0/4260349053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b0104d00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.590+0000 7f25b7797700 1 -- 172.21.15.63:0/4260349053 >> 172.21.15.63:0/4260349053 conn(0x7f25b01001f0 msgr2=0x7f25b0102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.590+0000 7f25b7797700 1 -- 172.21.15.63:0/4260349053 shutdown_connections 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.590+0000 7f25b7797700 1 -- 172.21.15.63:0/4260349053 wait complete. 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.590+0000 7f25b7797700 1 Processor -- start 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.590+0000 7f25b7797700 1 -- start start 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25b7797700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25b7797700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b019a940 con 0x7f25b01048e0 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25b5533700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25b5533700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35388/0 (socket says 172.21.15.63:35388) 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25b5533700 1 -- 172.21.15.63:0/3363243809 learned_addr learned my addr 172.21.15.63:0/3363243809 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25b5533700 1 -- 172.21.15.63:0/3363243809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac009550 con 0x7f25b01048e0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25b5533700 1 --2- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b019a400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0117b0 tx=0x7f25ac011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25a67fc700 1 -- 172.21.15.63:0/3363243809 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ac0038b0 con 0x7f25b01048e0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25a67fc700 1 -- 172.21.15.63:0/3363243809 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25ac003a10 con 0x7f25b01048e0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b019ab40 con 0x7f25b01048e0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25a67fc700 1 -- 172.21.15.63:0/3363243809 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ac0246e0 con 0x7f25b01048e0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.591+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b019afe0 con 0x7f25b01048e0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.592+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b010aa60 con 0x7f25b01048e0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.593+0000 7f25a67fc700 1 -- 172.21.15.63:0/3363243809 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 11) v1 ==== 45463+0+0 (secure 0 0 0) 0x7f25ac02d3f0 con 0x7f25b01048e0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.593+0000 7f25a67fc700 1 --2- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f259c038650 0x7f259c03ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.593+0000 7f25a67fc700 1 -- 172.21.15.63:0/3363243809 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f25ac04c7b0 con 0x7f25b01048e0 2024-05-09T06:29:06.502 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.593+0000 7f25b4d32700 1 --2- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f259c038650 0x7f259c03ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.593+0000 7f25b4d32700 1 --2- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f259c038650 0x7f259c03ab10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006fd0 tx=0x7f25a0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.598+0000 7f25a67fc700 1 -- 172.21.15.63:0/3363243809 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f25ac01ab80 con 0x7f25b01048e0 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.809+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 --> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f25b019d8d0 con 0x7f259c038650 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.822+0000 7f25a67fc700 1 -- 172.21.15.63:0/3363243809 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 12) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f25ac04ed00 con 0x7f25b01048e0 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.899+0000 7f25a67fc700 1 -- 172.21.15.63:0/3363243809 <== mgr.14164 v2:172.21.15.63:6800/3098785268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f25b019d8d0 con 0x7f259c038650 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.901+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f259c038650 msgr2=0x7f259c03ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.901+0000 7f25b7797700 1 --2- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f259c038650 0x7f259c03ab10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006fd0 tx=0x7f25a0006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.901+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 msgr2=0x7f25b019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.901+0000 7f25b7797700 1 --2- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b019a400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0117b0 tx=0x7f25ac011890 comp rx=0 tx=0).stop 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.901+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 shutdown_connections 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.901+0000 7f25b7797700 1 --2- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f259c038650 0x7f259c03ab10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.901+0000 7f25b7797700 1 --2- 172.21.15.63:0/3363243809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25b01048e0 0x7f25b019a400 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.901+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 >> 172.21.15.63:0/3363243809 conn(0x7f25b01001f0 msgr2=0x7f25b0100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.902+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 shutdown_connections 2024-05-09T06:29:06.503 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:05.902+0000 7f25b7797700 1 -- 172.21.15.63:0/3363243809 wait complete. 2024-05-09T06:29:06.504 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial admin user... 2024-05-09T06:29:06.934 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:06 smithi063 ceph-mon[27171]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:29:06.935 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:06 smithi063 ceph-mon[27171]: mgrmap e12: smithi063.egfguy(active, since 2s) 2024-05-09T06:29:06.935 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:06 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:06.935 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:06 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:08.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$.PbXUfjYocsbkYOANPOEiuA5gCQ1zZxbNqKlRzcwS/K9Bk4ULDPZe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715236147, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T06:29:08.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.148+0000 7fa0d23e4700 1 Processor -- start 2024-05-09T06:29:08.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.148+0000 7fa0d23e4700 1 -- start start 2024-05-09T06:29:08.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.148+0000 7fa0d23e4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:08.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.148+0000 7fa0d23e4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc105f20 con 0x7fa0cc105530 2024-05-09T06:29:08.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.148+0000 7fa0cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:08.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.148+0000 7fa0cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35402/0 (socket says 172.21.15.63:35402) 2024-05-09T06:29:08.498 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.148+0000 7fa0cbfff700 1 -- 172.21.15.63:0/1966478591 learned_addr learned my addr 172.21.15.63:0/1966478591 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.148+0000 7fa0cbfff700 1 -- 172.21.15.63:0/1966478591 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc106730 con 0x7fa0cc105530 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.149+0000 7fa0cbfff700 1 --2- 172.21.15.63:0/1966478591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc105950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0098a0 tx=0x7fa0bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fb84cc1ba96f2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.149+0000 7fa0cb7fe700 1 -- 172.21.15.63:0/1966478591 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0bc004180 con 0x7fa0cc105530 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.149+0000 7fa0cb7fe700 1 -- 172.21.15.63:0/1966478591 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0bc0042e0 con 0x7fa0cc105530 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.149+0000 7fa0cb7fe700 1 -- 172.21.15.63:0/1966478591 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0bc011b10 con 0x7fa0cc105530 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.150+0000 7fa0d23e4700 1 -- 172.21.15.63:0/1966478591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 msgr2=0x7fa0cc105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.150+0000 7fa0d23e4700 1 --2- 172.21.15.63:0/1966478591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc105950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0098a0 tx=0x7fa0bc009bb0 comp rx=0 tx=0).stop 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.150+0000 7fa0d23e4700 1 -- 172.21.15.63:0/1966478591 shutdown_connections 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.150+0000 7fa0d23e4700 1 --2- 172.21.15.63:0/1966478591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc105950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.150+0000 7fa0d23e4700 1 -- 172.21.15.63:0/1966478591 >> 172.21.15.63:0/1966478591 conn(0x7fa0cc100a90 msgr2=0x7fa0cc102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.150+0000 7fa0d23e4700 1 -- 172.21.15.63:0/1966478591 shutdown_connections 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.150+0000 7fa0d23e4700 1 -- 172.21.15.63:0/1966478591 wait complete. 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.150+0000 7fa0d23e4700 1 Processor -- start 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.150+0000 7fa0d23e4700 1 -- start start 2024-05-09T06:29:08.499 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0d23e4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0d23e4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc199820 con 0x7fa0cc105530 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35414/0 (socket says 172.21.15.63:35414) 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0cbfff700 1 -- 172.21.15.63:0/2213049391 learned_addr learned my addr 172.21.15.63:0/2213049391 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0cbfff700 1 -- 172.21.15.63:0/2213049391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009550 con 0x7fa0cc105530 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0cbfff700 1 --2- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc1992e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0117b0 tx=0x7fa0bc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0c9ffb700 1 -- 172.21.15.63:0/2213049391 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0bc0038b0 con 0x7fa0cc105530 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc199a20 con 0x7fa0cc105530 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.151+0000 7fa0c9ffb700 1 -- 172.21.15.63:0/2213049391 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0bc003a10 con 0x7fa0cc105530 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.152+0000 7fa0c9ffb700 1 -- 172.21.15.63:0/2213049391 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0bc0246e0 con 0x7fa0cc105530 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.152+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc199ec0 con 0x7fa0cc105530 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.152+0000 7fa0c9ffb700 1 -- 172.21.15.63:0/2213049391 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fa0bc02d3f0 con 0x7fa0cc105530 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.153+0000 7fa0c9ffb700 1 --2- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa0b8038770 0x7fa0b803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.153+0000 7fa0c9ffb700 1 -- 172.21.15.63:0/2213049391 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa0bc04d960 con 0x7fa0cc105530 2024-05-09T06:29:08.500 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.153+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc062520 con 0x7fa0cc105530 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.153+0000 7fa0c3fff700 1 --2- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa0b8038770 0x7fa0b803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.153+0000 7fa0c3fff700 1 --2- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa0b8038770 0x7fa0b803ac30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.158+0000 7fa0c9ffb700 1 -- 172.21.15.63:0/2213049391 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa0bc02d740 con 0x7fa0cc105530 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.378+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 --> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa0cc062340 con 0x7fa0b8038770 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.625+0000 7fa0c9ffb700 1 -- 172.21.15.63:0/2213049391 <== mgr.14164 v2:172.21.15.63:6800/3098785268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fa0cc062340 con 0x7fa0b8038770 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa0b8038770 msgr2=0x7fa0b803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 --2- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa0b8038770 0x7fa0b803ac30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 msgr2=0x7fa0cc1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 --2- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc1992e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0117b0 tx=0x7fa0bc011890 comp rx=0 tx=0).stop 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 shutdown_connections 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 --2- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa0b8038770 0x7fa0b803ac30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 --2- 172.21.15.63:0/2213049391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0cc105530 0x7fa0cc1992e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 >> 172.21.15.63:0/2213049391 conn(0x7fa0cc100a90 msgr2=0x7fa0cc1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 shutdown_connections 2024-05-09T06:29:08.501 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:07.627+0000 7fa0d23e4700 1 -- 172.21.15.63:0/2213049391 wait complete. 2024-05-09T06:29:08.502 INFO:teuthology.orchestra.run.smithi063.stdout:Fetching dashboard port number... 2024-05-09T06:29:08.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:08 smithi063 ceph-mon[27171]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:29:08.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:08 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:09.789 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:09 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/630376657' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T06:29:09.797 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T06:29:09.797 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.980+0000 7fb6fba85700 1 Processor -- start 2024-05-09T06:29:09.797 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.980+0000 7fb6fba85700 1 -- start start 2024-05-09T06:29:09.797 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.980+0000 7fb6fba85700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f4105330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.980+0000 7fb6fba85700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4079e10 con 0x7fb6f4104f10 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.980+0000 7fb6f9821700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f4105330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.981+0000 7fb6f9821700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f4105330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35428/0 (socket says 172.21.15.63:35428) 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.981+0000 7fb6f9821700 1 -- 172.21.15.63:0/3175317759 learned_addr learned my addr 172.21.15.63:0/3175317759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.981+0000 7fb6f9821700 1 -- 172.21.15.63:0/3175317759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f4105870 con 0x7fb6f4104f10 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.981+0000 7fb6f9821700 1 --2- 172.21.15.63:0/3175317759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f4105330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00098a0 tx=0x7fb6f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159e9dbfea933154 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.981+0000 7fb6f881f700 1 -- 172.21.15.63:0/3175317759 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f0004180 con 0x7fb6f4104f10 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.981+0000 7fb6f881f700 1 -- 172.21.15.63:0/3175317759 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6f00042e0 con 0x7fb6f4104f10 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.981+0000 7fb6f881f700 1 -- 172.21.15.63:0/3175317759 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f0011b10 con 0x7fb6f4104f10 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.982+0000 7fb6fba85700 1 -- 172.21.15.63:0/3175317759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 msgr2=0x7fb6f4105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.982+0000 7fb6fba85700 1 --2- 172.21.15.63:0/3175317759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f4105330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00098a0 tx=0x7fb6f0009bb0 comp rx=0 tx=0).stop 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.982+0000 7fb6fba85700 1 -- 172.21.15.63:0/3175317759 shutdown_connections 2024-05-09T06:29:09.798 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.982+0000 7fb6fba85700 1 --2- 172.21.15.63:0/3175317759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f4105330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.982+0000 7fb6fba85700 1 -- 172.21.15.63:0/3175317759 >> 172.21.15.63:0/3175317759 conn(0x7fb6f4100ad0 msgr2=0x7fb6f4102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.982+0000 7fb6fba85700 1 -- 172.21.15.63:0/3175317759 shutdown_connections 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.982+0000 7fb6fba85700 1 -- 172.21.15.63:0/3175317759 wait complete. 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.982+0000 7fb6fba85700 1 Processor -- start 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.982+0000 7fb6fba85700 1 -- start start 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6fba85700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6fba85700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419cac0 con 0x7fb6f4104f10 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6f9821700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6f9821700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35430/0 (socket says 172.21.15.63:35430) 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6f9821700 1 -- 172.21.15.63:0/630376657 learned_addr learned my addr 172.21.15.63:0/630376657 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6f9821700 1 -- 172.21.15.63:0/630376657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f0009550 con 0x7fb6f4104f10 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6f9821700 1 --2- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f419c580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00117b0 tx=0x7fb6f0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6eaffd700 1 -- 172.21.15.63:0/630376657 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f00038b0 con 0x7fb6f4104f10 2024-05-09T06:29:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6eaffd700 1 -- 172.21.15.63:0/630376657 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6f0003a10 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6eaffd700 1 -- 172.21.15.63:0/630376657 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f0024780 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f419ccc0 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.983+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f419d160 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.984+0000 7fb6eaffd700 1 -- 172.21.15.63:0/630376657 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fb6f001ac10 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.984+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f4196230 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.985+0000 7fb6eaffd700 1 --2- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fb6e00387c0 0x7fb6e003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.985+0000 7fb6eaffd700 1 -- 172.21.15.63:0/630376657 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb6f004c900 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.985+0000 7fb6f9020700 1 --2- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fb6e00387c0 0x7fb6e003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.985+0000 7fb6f9020700 1 --2- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fb6e00387c0 0x7fb6e003ac80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006fd0 tx=0x7fb6e4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:08.989+0000 7fb6eaffd700 1 -- 172.21.15.63:0/630376657 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb6f002cb00 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.192+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb6f402cf40 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.192+0000 7fb6eaffd700 1 -- 172.21.15.63:0/630376657 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fb6f002cb00 con 0x7fb6f4104f10 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.194+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fb6e00387c0 msgr2=0x7fb6e003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:09.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.194+0000 7fb6fba85700 1 --2- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fb6e00387c0 0x7fb6e003ac80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006fd0 tx=0x7fb6e4006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:09.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.194+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 msgr2=0x7fb6f419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:09.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.194+0000 7fb6fba85700 1 --2- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f419c580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00117b0 tx=0x7fb6f0011890 comp rx=0 tx=0).stop 2024-05-09T06:29:09.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.195+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 shutdown_connections 2024-05-09T06:29:09.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.195+0000 7fb6fba85700 1 --2- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fb6e00387c0 0x7fb6e003ac80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:09.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.195+0000 7fb6fba85700 1 --2- 172.21.15.63:0/630376657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6f4104f10 0x7fb6f419c580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:09.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.195+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 >> 172.21.15.63:0/630376657 conn(0x7fb6f4100ad0 msgr2=0x7fb6f4193370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:09.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.195+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 shutdown_connections 2024-05-09T06:29:09.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:09.195+0000 7fb6fba85700 1 -- 172.21.15.63:0/630376657 wait complete. 2024-05-09T06:29:09.805 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T06:29:09.805 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-09T06:29:09.813 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T06:29:09.813 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-09T06:29:09.813 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-09T06:29:09.813 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T06:29:09.814 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph Dashboard is now available at: 2024-05-09T06:29:09.814 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:09.814 INFO:teuthology.orchestra.run.smithi063.stdout: URL: https://smithi063.front.sepia.ceph.com:8443/ 2024-05-09T06:29:09.814 INFO:teuthology.orchestra.run.smithi063.stdout: User: admin 2024-05-09T06:29:09.815 INFO:teuthology.orchestra.run.smithi063.stdout: Password: zmekv67zgc 2024-05-09T06:29:09.815 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:09.815 INFO:teuthology.orchestra.run.smithi063.stdout:Saving cluster configuration to /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config directory 2024-05-09T06:29:09.815 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling autotune for osd_memory_target 2024-05-09T06:29:10.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:10 smithi063 ceph-mon[27171]: mgrmap e13: smithi063.egfguy(active, since 6s) 2024-05-09T06:29:11.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.248+0000 7f3893424700 1 Processor -- start 2024-05-09T06:29:11.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.248+0000 7f3893424700 1 -- start start 2024-05-09T06:29:11.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.248+0000 7f3893424700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:11.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.248+0000 7f3893424700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c107020 con 0x7f388c106630 2024-05-09T06:29:11.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.248+0000 7f38911c0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:11.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.248+0000 7f38911c0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35434/0 (socket says 172.21.15.63:35434) 2024-05-09T06:29:11.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.248+0000 7f38911c0700 1 -- 172.21.15.63:0/1779310722 learned_addr learned my addr 172.21.15.63:0/1779310722 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:11.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.249+0000 7f38911c0700 1 -- 172.21.15.63:0/1779310722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c1077e0 con 0x7f388c106630 2024-05-09T06:29:11.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.249+0000 7f38911c0700 1 --2- 172.21.15.63:0/1779310722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c106a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f38880098a0 tx=0x7f3888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfe869bb67759f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.249+0000 7f3883fff700 1 -- 172.21.15.63:0/1779310722 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3888004180 con 0x7f388c106630 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.249+0000 7f3883fff700 1 -- 172.21.15.63:0/1779310722 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38880042e0 con 0x7f388c106630 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.249+0000 7f3883fff700 1 -- 172.21.15.63:0/1779310722 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3888011b10 con 0x7f388c106630 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.249+0000 7f3893424700 1 -- 172.21.15.63:0/1779310722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 msgr2=0x7f388c106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.249+0000 7f3893424700 1 --2- 172.21.15.63:0/1779310722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c106a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f38880098a0 tx=0x7f3888009bb0 comp rx=0 tx=0).stop 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.250+0000 7f3893424700 1 -- 172.21.15.63:0/1779310722 shutdown_connections 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.250+0000 7f3893424700 1 --2- 172.21.15.63:0/1779310722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c106a50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.250+0000 7f3893424700 1 -- 172.21.15.63:0/1779310722 >> 172.21.15.63:0/1779310722 conn(0x7f388c101c30 msgr2=0x7f388c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.250+0000 7f3893424700 1 -- 172.21.15.63:0/1779310722 shutdown_connections 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.250+0000 7f3893424700 1 -- 172.21.15.63:0/1779310722 wait complete. 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.250+0000 7f3893424700 1 Processor -- start 2024-05-09T06:29:11.037 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.250+0000 7f3893424700 1 -- start start 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.250+0000 7f3893424700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.250+0000 7f3893424700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c19a960 con 0x7f388c106630 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f38911c0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f38911c0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35440/0 (socket says 172.21.15.63:35440) 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f38911c0700 1 -- 172.21.15.63:0/3969097968 learned_addr learned my addr 172.21.15.63:0/3969097968 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f38911c0700 1 -- 172.21.15.63:0/3969097968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888009550 con 0x7f388c106630 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f38911c0700 1 --2- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c19a420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c00 tx=0x7f3888011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f38827fc700 1 -- 172.21.15.63:0/3969097968 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38880038b0 con 0x7f388c106630 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c19ab60 con 0x7f388c106630 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f38827fc700 1 -- 172.21.15.63:0/3969097968 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3888003a10 con 0x7f388c106630 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f38827fc700 1 -- 172.21.15.63:0/3969097968 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3888024780 con 0x7f388c106630 2024-05-09T06:29:11.038 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.251+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c19b000 con 0x7f388c106630 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.252+0000 7f38827fc700 1 -- 172.21.15.63:0/3969097968 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f388801ac10 con 0x7f388c106630 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.252+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c10aa10 con 0x7f388c106630 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.252+0000 7f38827fc700 1 --2- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f3878038810 0x7f387803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.252+0000 7f38827fc700 1 -- 172.21.15.63:0/3969097968 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f388804c7d0 con 0x7f388c106630 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.252+0000 7f38909bf700 1 --2- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f3878038810 0x7f387803acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.253+0000 7f38909bf700 1 --2- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f3878038810 0x7f387803acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f387c006fd0 tx=0x7f387c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.257+0000 7f38827fc700 1 -- 172.21.15.63:0/3969097968 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f388801a430 con 0x7f388c106630 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.460+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f388c19d890 con 0x7f388c106630 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.460+0000 7f38827fc700 1 -- 172.21.15.63:0/3969097968 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f3888018b40 con 0x7f388c106630 2024-05-09T06:29:11.039 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f3878038810 msgr2=0x7f387803acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:11.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 --2- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f3878038810 0x7f387803acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f387c006fd0 tx=0x7f387c006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:11.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 msgr2=0x7f388c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:11.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 --2- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c19a420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c00 tx=0x7f3888011890 comp rx=0 tx=0).stop 2024-05-09T06:29:11.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 shutdown_connections 2024-05-09T06:29:11.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 --2- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f3878038810 0x7f387803acd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:11.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 --2- 172.21.15.63:0/3969097968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f388c106630 0x7f388c19a420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:11.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 >> 172.21.15.63:0/3969097968 conn(0x7f388c101c30 msgr2=0x7f388c102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:11.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 shutdown_connections 2024-05-09T06:29:11.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:10.462+0000 7f3893424700 1 -- 172.21.15.63:0/3969097968 wait complete. 2024-05-09T06:29:12.400 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.481+0000 7fad39ddb700 1 Processor -- start 2024-05-09T06:29:12.400 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.481+0000 7fad39ddb700 1 -- start start 2024-05-09T06:29:12.400 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.481+0000 7fad39ddb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad341094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.481+0000 7fad39ddb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad34079e10 con 0x7fad34107100 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.481+0000 7fad337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad341094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.481+0000 7fad337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad341094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35446/0 (socket says 172.21.15.63:35446) 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.481+0000 7fad337fe700 1 -- 172.21.15.63:0/2217231216 learned_addr learned my addr 172.21.15.63:0/2217231216 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.481+0000 7fad337fe700 1 -- 172.21.15.63:0/2217231216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad34109a30 con 0x7fad34107100 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.481+0000 7fad337fe700 1 --2- 172.21.15.63:0/2217231216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad341094f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad24009e80 tx=0x7fad2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b51a8b54f1d353 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad327fc700 1 -- 172.21.15.63:0/2217231216 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad2400e070 con 0x7fad34107100 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad327fc700 1 -- 172.21.15.63:0/2217231216 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad2400b8c0 con 0x7fad34107100 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad327fc700 1 -- 172.21.15.63:0/2217231216 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad240044d0 con 0x7fad34107100 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad39ddb700 1 -- 172.21.15.63:0/2217231216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 msgr2=0x7fad341094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad39ddb700 1 --2- 172.21.15.63:0/2217231216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad341094f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad24009e80 tx=0x7fad2400b330 comp rx=0 tx=0).stop 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad39ddb700 1 -- 172.21.15.63:0/2217231216 shutdown_connections 2024-05-09T06:29:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad39ddb700 1 --2- 172.21.15.63:0/2217231216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad341094f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad39ddb700 1 -- 172.21.15.63:0/2217231216 >> 172.21.15.63:0/2217231216 conn(0x7fad34100ad0 msgr2=0x7fad34102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad39ddb700 1 -- 172.21.15.63:0/2217231216 shutdown_connections 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.482+0000 7fad39ddb700 1 -- 172.21.15.63:0/2217231216 wait complete. 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.483+0000 7fad39ddb700 1 Processor -- start 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.483+0000 7fad39ddb700 1 -- start start 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.483+0000 7fad39ddb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad3419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.483+0000 7fad39ddb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419a860 con 0x7fad34107100 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.483+0000 7fad337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad3419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.483+0000 7fad337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad3419a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35462/0 (socket says 172.21.15.63:35462) 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.483+0000 7fad337fe700 1 -- 172.21.15.63:0/171260416 learned_addr learned my addr 172.21.15.63:0/171260416 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.484+0000 7fad337fe700 1 -- 172.21.15.63:0/171260416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad24009b30 con 0x7fad34107100 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.484+0000 7fad337fe700 1 --2- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad3419a320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad240072a0 tx=0x7fad2400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.484+0000 7fad30ff9700 1 -- 172.21.15.63:0/171260416 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad2400e040 con 0x7fad34107100 2024-05-09T06:29:12.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.484+0000 7fad30ff9700 1 -- 172.21.15.63:0/171260416 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad24003dc0 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.484+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3419aa60 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.484+0000 7fad30ff9700 1 -- 172.21.15.63:0/171260416 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad24012750 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.484+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3419af60 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.485+0000 7fad30ff9700 1 -- 172.21.15.63:0/171260416 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fad24019040 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.485+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3410a770 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.485+0000 7fad30ff9700 1 --2- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fad1c038860 0x7fad1c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.485+0000 7fad30ff9700 1 -- 172.21.15.63:0/171260416 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fad2404c3c0 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.485+0000 7fad32ffd700 1 --2- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fad1c038860 0x7fad1c03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.486+0000 7fad32ffd700 1 --2- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fad1c038860 0x7fad1c03ad20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fad28006fd0 tx=0x7fad28006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.490+0000 7fad30ff9700 1 -- 172.21.15.63:0/171260416 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fad2402cb40 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.803+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fad3410a9e0 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.805+0000 7fad30ff9700 1 -- 172.21.15.63:0/171260416 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7fad2401f070 con 0x7fad34107100 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.807+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fad1c038860 msgr2=0x7fad1c03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:12.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.807+0000 7fad39ddb700 1 --2- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fad1c038860 0x7fad1c03ad20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fad28006fd0 tx=0x7fad28006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.807+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 msgr2=0x7fad3419a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.807+0000 7fad39ddb700 1 --2- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad3419a320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad240072a0 tx=0x7fad2400be00 comp rx=0 tx=0).stop 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.807+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 shutdown_connections 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.807+0000 7fad39ddb700 1 --2- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fad1c038860 0x7fad1c03ad20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.807+0000 7fad39ddb700 1 --2- 172.21.15.63:0/171260416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad34107100 0x7fad3419a320 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.807+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 >> 172.21.15.63:0/171260416 conn(0x7fad34100ad0 msgr2=0x7fad34101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.808+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 shutdown_connections 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-09T06:29:11.808+0000 7fad39ddb700 1 -- 172.21.15.63:0/171260416 wait complete. 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:12.404 INFO:teuthology.orchestra.run.smithi063.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout: ceph telemetry on 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout:For more information see: 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:12.405 INFO:teuthology.orchestra.run.smithi063.stdout:Bootstrap complete. 2024-05-09T06:29:12.432 INFO:tasks.cephadm:Fetching config... 2024-05-09T06:29:12.432 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:29:12.432 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T06:29:12.449 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T06:29:12.449 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:29:12.449 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T06:29:12.506 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T06:29:12.506 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:29:12.506 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/keyring of=/dev/stdout 2024-05-09T06:29:12.572 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T06:29:12.572 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:29:12.573 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T06:29:12.628 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T06:29:12.629 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtDQyYtQyu5JkYanv7gXevOt+OzkE3sIQF1sd51ZtIoqOi8pDbjIqeOL8B2b+W4wbrKH93YyfM410N5RQM6QKIwCzSDBHDVlNLrDk4MDvtanumxRWk3LPHUZGSm/H9IRRdcq7R+bfQEw8bI7hC3v6yyG6sVrMnZBtdM/Rhl7xa/Yl25QFPJHYlB/N6DT4N2rDfXNs0nw8z1qWL3gqjkDqR3HiNqh7QLq3qY8TNspL8N1EXsiUUMpz4RlPdVFBkOzNCJ3jRcmsyknsQvAOlEtyZ1hztitLLEARRsSW9ZnPR7hmzUSXGEOcKO2ni91Q378lL8hTOBinxNIMB2lR+udS/uiOF7J3HZqFf3A8GQgawgc1DvNtubK4CbSve/k22DOT/Kf/2iiQhvs0R+FQx264PjnWkQTlFlnOdtz9CX/IQqJbnKnrPwTzgv1nKaqDovd4+qLeDS6sjnZI45spFggener6cCv5pYyUx5CHy6vRX+KaT7pqqadlp32q45E0m53k= ceph-2386be1c-0dcd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T06:29:12.720 INFO:teuthology.orchestra.run.smithi063.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtDQyYtQyu5JkYanv7gXevOt+OzkE3sIQF1sd51ZtIoqOi8pDbjIqeOL8B2b+W4wbrKH93YyfM410N5RQM6QKIwCzSDBHDVlNLrDk4MDvtanumxRWk3LPHUZGSm/H9IRRdcq7R+bfQEw8bI7hC3v6yyG6sVrMnZBtdM/Rhl7xa/Yl25QFPJHYlB/N6DT4N2rDfXNs0nw8z1qWL3gqjkDqR3HiNqh7QLq3qY8TNspL8N1EXsiUUMpz4RlPdVFBkOzNCJ3jRcmsyknsQvAOlEtyZ1hztitLLEARRsSW9ZnPR7hmzUSXGEOcKO2ni91Q378lL8hTOBinxNIMB2lR+udS/uiOF7J3HZqFf3A8GQgawgc1DvNtubK4CbSve/k22DOT/Kf/2iiQhvs0R+FQx264PjnWkQTlFlnOdtz9CX/IQqJbnKnrPwTzgv1nKaqDovd4+qLeDS6sjnZI45spFggener6cCv5pYyUx5CHy6vRX+KaT7pqqadlp32q45E0m53k= ceph-2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:29:12.734 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtDQyYtQyu5JkYanv7gXevOt+OzkE3sIQF1sd51ZtIoqOi8pDbjIqeOL8B2b+W4wbrKH93YyfM410N5RQM6QKIwCzSDBHDVlNLrDk4MDvtanumxRWk3LPHUZGSm/H9IRRdcq7R+bfQEw8bI7hC3v6yyG6sVrMnZBtdM/Rhl7xa/Yl25QFPJHYlB/N6DT4N2rDfXNs0nw8z1qWL3gqjkDqR3HiNqh7QLq3qY8TNspL8N1EXsiUUMpz4RlPdVFBkOzNCJ3jRcmsyknsQvAOlEtyZ1hztitLLEARRsSW9ZnPR7hmzUSXGEOcKO2ni91Q378lL8hTOBinxNIMB2lR+udS/uiOF7J3HZqFf3A8GQgawgc1DvNtubK4CbSve/k22DOT/Kf/2iiQhvs0R+FQx264PjnWkQTlFlnOdtz9CX/IQqJbnKnrPwTzgv1nKaqDovd4+qLeDS6sjnZI45spFggener6cCv5pYyUx5CHy6vRX+KaT7pqqadlp32q45E0m53k= ceph-2386be1c-0dcd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T06:29:12.775 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtDQyYtQyu5JkYanv7gXevOt+OzkE3sIQF1sd51ZtIoqOi8pDbjIqeOL8B2b+W4wbrKH93YyfM410N5RQM6QKIwCzSDBHDVlNLrDk4MDvtanumxRWk3LPHUZGSm/H9IRRdcq7R+bfQEw8bI7hC3v6yyG6sVrMnZBtdM/Rhl7xa/Yl25QFPJHYlB/N6DT4N2rDfXNs0nw8z1qWL3gqjkDqR3HiNqh7QLq3qY8TNspL8N1EXsiUUMpz4RlPdVFBkOzNCJ3jRcmsyknsQvAOlEtyZ1hztitLLEARRsSW9ZnPR7hmzUSXGEOcKO2ni91Q378lL8hTOBinxNIMB2lR+udS/uiOF7J3HZqFf3A8GQgawgc1DvNtubK4CbSve/k22DOT/Kf/2iiQhvs0R+FQx264PjnWkQTlFlnOdtz9CX/IQqJbnKnrPwTzgv1nKaqDovd4+qLeDS6sjnZI45spFggener6cCv5pYyUx5CHy6vRX+KaT7pqqadlp32q45E0m53k= ceph-2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:29:12.787 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T06:29:12.994 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:29:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:12 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/171260416' entity='client.admin' 2024-05-09T06:29:15.530 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:15 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:15.531 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:15 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:15.531 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:15 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:29:15.531 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:15 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:15.531 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:15 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:29:15.531 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:15 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-09T06:29:15.531 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:15 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:29:15.531 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:15 smithi063 ceph-mon[27171]: Deploying daemon ceph-exporter.smithi063 on smithi063 2024-05-09T06:29:15.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.578+0000 7fc00cb44700 1 -- 172.21.15.63:0/259534168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 msgr2=0x7fc008100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.578+0000 7fc00cb44700 1 --2- 172.21.15.63:0/259534168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 0x7fc008100ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009910 tx=0x7fbff8009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.579+0000 7fc00cb44700 1 -- 172.21.15.63:0/259534168 shutdown_connections 2024-05-09T06:29:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.579+0000 7fc00cb44700 1 --2- 172.21.15.63:0/259534168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 0x7fc008100ca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.579+0000 7fc00cb44700 1 -- 172.21.15.63:0/259534168 >> 172.21.15.63:0/259534168 conn(0x7fc0080fbe20 msgr2=0x7fc0080fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.579+0000 7fc00cb44700 1 -- 172.21.15.63:0/259534168 shutdown_connections 2024-05-09T06:29:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.579+0000 7fc00cb44700 1 -- 172.21.15.63:0/259534168 wait complete. 2024-05-09T06:29:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.579+0000 7fc00cb44700 1 Processor -- start 2024-05-09T06:29:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.579+0000 7fc00cb44700 1 -- start start 2024-05-09T06:29:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fc00cb44700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 0x7fc008195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fc00cb44700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc008196260 con 0x7fc008100880 2024-05-09T06:29:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fc00659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 0x7fc008195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fc00659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 0x7fc008195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47298/0 (socket says 172.21.15.63:47298) 2024-05-09T06:29:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fc00659c700 1 -- 172.21.15.63:0/1738153282 learned_addr learned my addr 172.21.15.63:0/1738153282 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fc00659c700 1 -- 172.21.15.63:0/1738153282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff80095f0 con 0x7fc008100880 2024-05-09T06:29:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fc00659c700 1 --2- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 0x7fc008195d20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbff8006010 tx=0x7fbff80050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fbff77fe700 1 -- 172.21.15.63:0/1738153282 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbff801d070 con 0x7fc008100880 2024-05-09T06:29:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fbff77fe700 1 -- 172.21.15.63:0/1738153282 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbff800be20 con 0x7fc008100880 2024-05-09T06:29:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.580+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc008196460 con 0x7fc008100880 2024-05-09T06:29:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.581+0000 7fbff77fe700 1 -- 172.21.15.63:0/1738153282 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbff8022de0 con 0x7fc008100880 2024-05-09T06:29:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.581+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc008196900 con 0x7fc008100880 2024-05-09T06:29:15.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.582+0000 7fbff77fe700 1 -- 172.21.15.63:0/1738153282 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fbff800f4b0 con 0x7fc008100880 2024-05-09T06:29:15.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.582+0000 7fbff77fe700 1 --2- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbff00387b0 0x7fbff003ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:15.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.582+0000 7fbff77fe700 1 -- 172.21.15.63:0/1738153282 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbff804c880 con 0x7fc008100880 2024-05-09T06:29:15.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.582+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00818f6b0 con 0x7fc008100880 2024-05-09T06:29:15.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.582+0000 7fc005d9b700 1 --2- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbff00387b0 0x7fbff003ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:15.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.582+0000 7fc005d9b700 1 --2- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbff00387b0 0x7fbff003ac70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbffc006fd0 tx=0x7fbffc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:15.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.587+0000 7fbff77fe700 1 -- 172.21.15.63:0/1738153282 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbff8027020 con 0x7fc008100880 2024-05-09T06:29:15.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.844+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fc008062520 con 0x7fc008100880 2024-05-09T06:29:15.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.848+0000 7fbff77fe700 1 -- 172.21.15.63:0/1738153282 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbff800fe60 con 0x7fc008100880 2024-05-09T06:29:15.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbff00387b0 msgr2=0x7fbff003ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 --2- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbff00387b0 0x7fbff003ac70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbffc006fd0 tx=0x7fbffc006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:15.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 msgr2=0x7fc008195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:15.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 --2- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 0x7fc008195d20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbff8006010 tx=0x7fbff80050b0 comp rx=0 tx=0).stop 2024-05-09T06:29:15.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 shutdown_connections 2024-05-09T06:29:15.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 --2- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbff00387b0 0x7fbff003ac70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 --2- 172.21.15.63:0/1738153282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc008100880 0x7fc008195d20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 >> 172.21.15.63:0/1738153282 conn(0x7fc0080fbe20 msgr2=0x7fc0080fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 shutdown_connections 2024-05-09T06:29:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:15.850+0000 7fc00cb44700 1 -- 172.21.15.63:0/1738153282 wait complete. 2024-05-09T06:29:16.655 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T06:29:16.656 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T06:29:16.929 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:16 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1738153282' entity='client.admin' 2024-05-09T06:29:17.008 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:29:19.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.409+0000 7f540fb7a700 1 -- 172.21.15.63:0/989942365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 msgr2=0x7f54080ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.409+0000 7f540fb7a700 1 --2- 172.21.15.63:0/989942365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 0x7f54080ff530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5404009910 tx=0x7f5404009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.409+0000 7f540fb7a700 1 -- 172.21.15.63:0/989942365 shutdown_connections 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.409+0000 7f540fb7a700 1 --2- 172.21.15.63:0/989942365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 0x7f54080ff530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.409+0000 7f540fb7a700 1 -- 172.21.15.63:0/989942365 >> 172.21.15.63:0/989942365 conn(0x7f54080fad50 msgr2=0x7f54080fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.409+0000 7f540fb7a700 1 -- 172.21.15.63:0/989942365 shutdown_connections 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.409+0000 7f540fb7a700 1 -- 172.21.15.63:0/989942365 wait complete. 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.409+0000 7f540fb7a700 1 Processor -- start 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.410+0000 7f540fb7a700 1 -- start start 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.410+0000 7f540fb7a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 0x7f5408197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.410+0000 7f540fb7a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5408198440 con 0x7f54080ff110 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.410+0000 7f540d916700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 0x7f5408197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.410+0000 7f540d916700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 0x7f5408197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47306/0 (socket says 172.21.15.63:47306) 2024-05-09T06:29:19.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.410+0000 7f540d916700 1 -- 172.21.15.63:0/2992566185 learned_addr learned my addr 172.21.15.63:0/2992566185 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:19.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.410+0000 7f540d916700 1 -- 172.21.15.63:0/2992566185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54040095f0 con 0x7f54080ff110 2024-05-09T06:29:19.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.410+0000 7f540d916700 1 --2- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 0x7f5408197f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5404009ed0 tx=0x7f5404004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.410+0000 7f53feffd700 1 -- 172.21.15.63:0/2992566185 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f540401d070 con 0x7f54080ff110 2024-05-09T06:29:19.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.411+0000 7f53feffd700 1 -- 172.21.15.63:0/2992566185 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f540400be20 con 0x7f54080ff110 2024-05-09T06:29:19.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.411+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5408198640 con 0x7f54080ff110 2024-05-09T06:29:19.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.411+0000 7f53feffd700 1 -- 172.21.15.63:0/2992566185 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f540400f460 con 0x7f54080ff110 2024-05-09T06:29:19.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.411+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5408198ae0 con 0x7f54080ff110 2024-05-09T06:29:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.411+0000 7f53feffd700 1 -- 172.21.15.63:0/2992566185 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f540400f5c0 con 0x7f54080ff110 2024-05-09T06:29:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.412+0000 7f53feffd700 1 --2- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f53f4038820 0x7f53f403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.412+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54081918b0 con 0x7f54080ff110 2024-05-09T06:29:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.412+0000 7f53feffd700 1 -- 172.21.15.63:0/2992566185 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f540404c8a0 con 0x7f54080ff110 2024-05-09T06:29:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.412+0000 7f540d115700 1 --2- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f53f4038820 0x7f53f403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:19.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.412+0000 7f540d115700 1 --2- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f53f4038820 0x7f53f403ace0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006fd0 tx=0x7f53f8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:19.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.417+0000 7f53feffd700 1 -- 172.21.15.63:0/2992566185 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5404027070 con 0x7f54080ff110 2024-05-09T06:29:19.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.629+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 --> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f540802cdb0 con 0x7f53f4038820 2024-05-09T06:29:19.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.632+0000 7f53feffd700 1 -- 172.21.15.63:0/2992566185 <== mgr.14164 v2:172.21.15.63:6800/3098785268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f540802cdb0 con 0x7f53f4038820 2024-05-09T06:29:19.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.634+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f53f4038820 msgr2=0x7f53f403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.634+0000 7f540fb7a700 1 --2- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f53f4038820 0x7f53f403ace0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006fd0 tx=0x7f53f8006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:19.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.634+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 msgr2=0x7f5408197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:19.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.635+0000 7f540fb7a700 1 --2- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 0x7f5408197f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5404009ed0 tx=0x7f5404004690 comp rx=0 tx=0).stop 2024-05-09T06:29:19.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.635+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 shutdown_connections 2024-05-09T06:29:19.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.635+0000 7f540fb7a700 1 --2- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f53f4038820 0x7f53f403ace0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.635+0000 7f540fb7a700 1 --2- 172.21.15.63:0/2992566185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54080ff110 0x7f5408197f00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:19.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.635+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 >> 172.21.15.63:0/2992566185 conn(0x7f54080fad50 msgr2=0x7f54080fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:19.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.635+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 shutdown_connections 2024-05-09T06:29:19.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:19.635+0000 7f540fb7a700 1 -- 172.21.15.63:0/2992566185 wait complete. 2024-05-09T06:29:20.224 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-05-09T06:29:20.224 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:29:20.224 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-05-09T06:29:20.245 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:29:20.245 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:29:20.303 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-05-09T06:29:20.304 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch host add smithi193 2024-05-09T06:29:20.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:29:20.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:20.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:20.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:20.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:20.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:20.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:29:20.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-09T06:29:20.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:29:20.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:20 smithi063 ceph-mon[27171]: Deploying daemon crash.smithi063 on smithi063 2024-05-09T06:29:21.310 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:29:23.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.314+0000 7f0541c2f700 1 -- 172.21.15.63:0/1246075166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 msgr2=0x7f053c0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:23.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.314+0000 7f0541c2f700 1 --2- 172.21.15.63:0/1246075166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 0x7f053c0ffba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f052c009910 tx=0x7f052c009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:23.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.314+0000 7f0541c2f700 1 -- 172.21.15.63:0/1246075166 shutdown_connections 2024-05-09T06:29:23.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.314+0000 7f0541c2f700 1 --2- 172.21.15.63:0/1246075166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 0x7f053c0ffba0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:23.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.314+0000 7f0541c2f700 1 -- 172.21.15.63:0/1246075166 >> 172.21.15.63:0/1246075166 conn(0x7f053c0fad20 msgr2=0x7f053c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:23.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.314+0000 7f0541c2f700 1 -- 172.21.15.63:0/1246075166 shutdown_connections 2024-05-09T06:29:23.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.314+0000 7f0541c2f700 1 -- 172.21.15.63:0/1246075166 wait complete. 2024-05-09T06:29:23.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.314+0000 7f0541c2f700 1 Processor -- start 2024-05-09T06:29:23.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.315+0000 7f0541c2f700 1 -- start start 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.315+0000 7f0541c2f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 0x7f053c194c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.315+0000 7f0541c2f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c195160 con 0x7f053c0ff780 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.315+0000 7f053b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 0x7f053c194c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.315+0000 7f053b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 0x7f053c194c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47338/0 (socket says 172.21.15.63:47338) 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.315+0000 7f053b7fe700 1 -- 172.21.15.63:0/2089503680 learned_addr learned my addr 172.21.15.63:0/2089503680 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.315+0000 7f053b7fe700 1 -- 172.21.15.63:0/2089503680 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c0095f0 con 0x7f053c0ff780 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.315+0000 7f053b7fe700 1 --2- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 0x7f053c194c20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f052c006010 tx=0x7f052c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.316+0000 7f0538ff9700 1 -- 172.21.15.63:0/2089503680 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f052c01d070 con 0x7f053c0ff780 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.316+0000 7f0538ff9700 1 -- 172.21.15.63:0/2089503680 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f052c00be20 con 0x7f053c0ff780 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.316+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c195360 con 0x7f053c0ff780 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.316+0000 7f0538ff9700 1 -- 172.21.15.63:0/2089503680 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f052c022de0 con 0x7f053c0ff780 2024-05-09T06:29:23.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.316+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c195800 con 0x7f053c0ff780 2024-05-09T06:29:23.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.317+0000 7f0538ff9700 1 -- 172.21.15.63:0/2089503680 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f052c00f460 con 0x7f053c0ff780 2024-05-09T06:29:23.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.317+0000 7f0538ff9700 1 --2- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0524038820 0x7f052403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:23.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.317+0000 7f0538ff9700 1 -- 172.21.15.63:0/2089503680 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f052c04c870 con 0x7f053c0ff780 2024-05-09T06:29:23.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.317+0000 7f053affd700 1 --2- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0524038820 0x7f052403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:23.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.317+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c18e5b0 con 0x7f053c0ff780 2024-05-09T06:29:23.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.318+0000 7f053affd700 1 --2- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0524038820 0x7f052403ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0530006fd0 tx=0x7f0530006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:23.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.322+0000 7f0538ff9700 1 -- 172.21.15.63:0/2089503680 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f052c027030 con 0x7f053c0ff780 2024-05-09T06:29:23.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:23.540+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 --> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) v1 -- 0x7f053c1058b0 con 0x7f0524038820 2024-05-09T06:29:24.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:24 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:24.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:24 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:24.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:24 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:24.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:24 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:24.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:24 smithi063 ceph-mon[27171]: Deploying daemon node-exporter.smithi063 on smithi063 2024-05-09T06:29:24.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:24 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:25.386 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:25 smithi063 ceph-mon[27171]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:25.386 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:25 smithi063 ceph-mon[27171]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:29:25.386 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:25 smithi063 ceph-mon[27171]: Deploying cephadm binary to smithi193 2024-05-09T06:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-05-09T06:29:25.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.016+0000 7f0538ff9700 1 -- 172.21.15.63:0/2089503680 <== mgr.14164 v2:172.21.15.63:6800/3098785268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f053c1058b0 con 0x7f0524038820 2024-05-09T06:29:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.018+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0524038820 msgr2=0x7f052403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.018+0000 7f0541c2f700 1 --2- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0524038820 0x7f052403ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0530006fd0 tx=0x7f0530006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.019+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 msgr2=0x7f053c194c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.019+0000 7f0541c2f700 1 --2- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 0x7f053c194c20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f052c006010 tx=0x7f052c0050b0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.019+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 shutdown_connections 2024-05-09T06:29:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.019+0000 7f0541c2f700 1 --2- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0524038820 0x7f052403ace0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.019+0000 7f0541c2f700 1 --2- 172.21.15.63:0/2089503680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f053c0ff780 0x7f053c194c20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.019+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 >> 172.21.15.63:0/2089503680 conn(0x7f053c0fad20 msgr2=0x7f053c0fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.019+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 shutdown_connections 2024-05-09T06:29:26.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:26.019+0000 7f0541c2f700 1 -- 172.21.15.63:0/2089503680 wait complete. 2024-05-09T06:29:27.080 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T06:29:27.318 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:27 smithi063 ceph-mon[27171]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:27.319 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:27 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:27.319 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:27 smithi063 ceph-mon[27171]: Added host smithi193 2024-05-09T06:29:27.566 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:29:28.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:28 smithi063 ceph-mon[27171]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:28.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:28 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:28.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:28 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:28.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:28 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:28.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:28 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:28.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:28 smithi063 ceph-mon[27171]: Deploying daemon alertmanager.smithi063 on smithi063 2024-05-09T06:29:28.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:28 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:28.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.001+0000 7f99d278f700 1 -- 172.21.15.63:0/4265817276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 msgr2=0x7f99cc103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.001+0000 7f99d278f700 1 --2- 172.21.15.63:0/4265817276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 0x7f99cc103720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009910 tx=0x7f99bc009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.001+0000 7f99d278f700 1 -- 172.21.15.63:0/4265817276 shutdown_connections 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.001+0000 7f99d278f700 1 --2- 172.21.15.63:0/4265817276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 0x7f99cc103720 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.001+0000 7f99d278f700 1 -- 172.21.15.63:0/4265817276 >> 172.21.15.63:0/4265817276 conn(0x7f99cc0fad40 msgr2=0x7f99cc0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.001+0000 7f99d278f700 1 -- 172.21.15.63:0/4265817276 shutdown_connections 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.001+0000 7f99d278f700 1 -- 172.21.15.63:0/4265817276 wait complete. 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.001+0000 7f99d278f700 1 Processor -- start 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.001+0000 7f99d278f700 1 -- start start 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99d278f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 0x7f99cc100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99d278f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc1010a0 con 0x7f99cc101330 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 0x7f99cc100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 0x7f99cc100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60908/0 (socket says 172.21.15.63:60908) 2024-05-09T06:29:28.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99cbfff700 1 -- 172.21.15.63:0/2078510190 learned_addr learned my addr 172.21.15.63:0/2078510190 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99cbfff700 1 -- 172.21.15.63:0/2078510190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc0095f0 con 0x7f99cc101330 2024-05-09T06:29:28.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99cbfff700 1 --2- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 0x7f99cc100b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009ed0 tx=0x7f99bc004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99c97fa700 1 -- 172.21.15.63:0/2078510190 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99bc01d070 con 0x7f99cc101330 2024-05-09T06:29:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99c97fa700 1 -- 172.21.15.63:0/2078510190 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99bc00be20 con 0x7f99cc101330 2024-05-09T06:29:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc0ff2e0 con 0x7f99cc101330 2024-05-09T06:29:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.002+0000 7f99c97fa700 1 -- 172.21.15.63:0/2078510190 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99bc00f460 con 0x7f99cc101330 2024-05-09T06:29:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.003+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc0ff780 con 0x7f99cc101330 2024-05-09T06:29:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.003+0000 7f99c97fa700 1 -- 172.21.15.63:0/2078510190 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f99bc00f650 con 0x7f99cc101330 2024-05-09T06:29:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.003+0000 7f99c97fa700 1 --2- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f99b40387d0 0x7f99b403ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:28.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.004+0000 7f99c97fa700 1 -- 172.21.15.63:0/2078510190 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f99bc022470 con 0x7f99cc101330 2024-05-09T06:29:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.004+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc107000 con 0x7f99cc101330 2024-05-09T06:29:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.004+0000 7f99cb7fe700 1 --2- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f99b40387d0 0x7f99b403ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:28.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.004+0000 7f99cb7fe700 1 --2- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f99b40387d0 0x7f99b403ac90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f99c0006fd0 tx=0x7f99c0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:28.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.009+0000 7f99c97fa700 1 -- 172.21.15.63:0/2078510190 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f99bc027070 con 0x7f99cc101330 2024-05-09T06:29:29.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.218+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 --> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99cc0ffa80 con 0x7f99b40387d0 2024-05-09T06:29:29.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.219+0000 7f99c97fa700 1 -- 172.21.15.63:0/2078510190 <== mgr.14164 v2:172.21.15.63:6800/3098785268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f99cc0ffa80 con 0x7f99b40387d0 2024-05-09T06:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:[{"addr": "172.21.15.63", "hostname": "smithi063", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-05-09T06:29:29.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f99b40387d0 msgr2=0x7f99b403ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:29.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 --2- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f99b40387d0 0x7f99b403ac90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f99c0006fd0 tx=0x7f99c0006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:29.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 msgr2=0x7f99cc100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:29.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 --2- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 0x7f99cc100b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009ed0 tx=0x7f99bc004690 comp rx=0 tx=0).stop 2024-05-09T06:29:29.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 shutdown_connections 2024-05-09T06:29:29.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 --2- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f99b40387d0 0x7f99b403ac90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 --2- 172.21.15.63:0/2078510190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99cc101330 0x7f99cc100b60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:29.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 >> 172.21.15.63:0/2078510190 conn(0x7f99cc0fad40 msgr2=0x7f99cc0fb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:29.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 shutdown_connections 2024-05-09T06:29:29.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:29.221+0000 7f99d278f700 1 -- 172.21.15.63:0/2078510190 wait complete. 2024-05-09T06:29:29.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:29 smithi063 ceph-mon[27171]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T06:29:30.223 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T06:29:30.223 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-09T06:29:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:29:30.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:30 smithi063 ceph-mon[27171]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:32.834 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:32 smithi063 ceph-mon[27171]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.307+0000 7fbdc6259700 1 -- 172.21.15.63:0/2499634984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 msgr2=0x7fbdc0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.307+0000 7fbdc6259700 1 --2- 172.21.15.63:0/2499634984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 0x7fbdc0100ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009910 tx=0x7fbdb0009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.307+0000 7fbdc6259700 1 -- 172.21.15.63:0/2499634984 shutdown_connections 2024-05-09T06:29:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.307+0000 7fbdc6259700 1 --2- 172.21.15.63:0/2499634984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 0x7fbdc0100ca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.307+0000 7fbdc6259700 1 -- 172.21.15.63:0/2499634984 >> 172.21.15.63:0/2499634984 conn(0x7fbdc00fbe40 msgr2=0x7fbdc00fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.307+0000 7fbdc6259700 1 -- 172.21.15.63:0/2499634984 shutdown_connections 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.307+0000 7fbdc6259700 1 -- 172.21.15.63:0/2499634984 wait complete. 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.308+0000 7fbdc6259700 1 Processor -- start 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.308+0000 7fbdc6259700 1 -- start start 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.308+0000 7fbdc6259700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 0x7fbdc0074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.308+0000 7fbdc6259700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00750a0 con 0x7fbdc0100880 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdbf7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 0x7fbdc0074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdbf7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 0x7fbdc0074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60930/0 (socket says 172.21.15.63:60930) 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdbf7fe700 1 -- 172.21.15.63:0/2407794954 learned_addr learned my addr 172.21.15.63:0/2407794954 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdbf7fe700 1 -- 172.21.15.63:0/2407794954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00095f0 con 0x7fbdc0100880 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdbf7fe700 1 --2- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 0x7fbdc0074b60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0006010 tx=0x7fbdb0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdbcff9700 1 -- 172.21.15.63:0/2407794954 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdb001d070 con 0x7fbdc0100880 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc00732e0 con 0x7fbdc0100880 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdbcff9700 1 -- 172.21.15.63:0/2407794954 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdb0005dc0 con 0x7fbdc0100880 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdbcff9700 1 -- 172.21.15.63:0/2407794954 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdb0022e90 con 0x7fbdc0100880 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.309+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc0073780 con 0x7fbdc0100880 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.310+0000 7fbdbcff9700 1 -- 172.21.15.63:0/2407794954 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fbdb000f4f0 con 0x7fbdc0100880 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.310+0000 7fbdbcff9700 1 --2- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbdac038760 0x7fbdac03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.310+0000 7fbdbcff9700 1 -- 172.21.15.63:0/2407794954 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbdb004c880 con 0x7fbdc0100880 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.310+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc0106fe0 con 0x7fbdc0100880 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.310+0000 7fbdbeffd700 1 --2- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbdac038760 0x7fbdac03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.311+0000 7fbdbeffd700 1 --2- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbdac038760 0x7fbdac03ac20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbda8006fd0 tx=0x7fbda8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:33.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.316+0000 7fbdbcff9700 1 -- 172.21.15.63:0/2407794954 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbdb0027070 con 0x7fbdc0100880 2024-05-09T06:29:33.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.528+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbdc0062520 con 0x7fbdc0100880 2024-05-09T06:29:33.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.607+0000 7fbdbcff9700 1 -- 172.21.15.63:0/2407794954 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbdb0034070 con 0x7fbdc0100880 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbdac038760 msgr2=0x7fbdac03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 --2- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbdac038760 0x7fbdac03ac20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbda8006fd0 tx=0x7fbda8006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 msgr2=0x7fbdc0074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 --2- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 0x7fbdc0074b60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0006010 tx=0x7fbdb0009f80 comp rx=0 tx=0).stop 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 shutdown_connections 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 --2- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fbdac038760 0x7fbdac03ac20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 --2- 172.21.15.63:0/2407794954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdc0100880 0x7fbdc0074b60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 >> 172.21.15.63:0/2407794954 conn(0x7fbdc00fbe40 msgr2=0x7fbdc00fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 shutdown_connections 2024-05-09T06:29:33.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:29:33.610+0000 7fbdc6259700 1 -- 172.21.15.63:0/2407794954 wait complete. 2024-05-09T06:29:33.593 INFO:teuthology.orchestra.run.smithi063.stderr:adjusted tunables profile to default 2024-05-09T06:29:33.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:33 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2407794954' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T06:29:34.460 INFO:tasks.cephadm:Adding mon.smithi063 on smithi063 2024-05-09T06:29:34.460 INFO:tasks.cephadm:Adding mon.smithi193 on smithi193 2024-05-09T06:29:34.460 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch apply mon '2;smithi063:172.21.15.63=smithi063;smithi193:172.21.15.193=smithi193' 2024-05-09T06:29:34.735 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:34.788 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:34.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:34 smithi063 ceph-mon[27171]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:34.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:34 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2407794954' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T06:29:34.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:34 smithi063 ceph-mon[27171]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.305+0000 7fcc873db700 1 -- 172.21.15.193:0/3852271197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 msgr2=0x7fcc80100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.305+0000 7fcc873db700 1 --2- 172.21.15.193:0/3852271197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 0x7fcc80100ca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009910 tx=0x7fcc7c009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.305+0000 7fcc873db700 1 -- 172.21.15.193:0/3852271197 shutdown_connections 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.305+0000 7fcc873db700 1 --2- 172.21.15.193:0/3852271197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 0x7fcc80100ca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.305+0000 7fcc873db700 1 -- 172.21.15.193:0/3852271197 >> 172.21.15.193:0/3852271197 conn(0x7fcc800fbe40 msgr2=0x7fcc800fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.305+0000 7fcc873db700 1 -- 172.21.15.193:0/3852271197 shutdown_connections 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.305+0000 7fcc873db700 1 -- 172.21.15.193:0/3852271197 wait complete. 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.306+0000 7fcc873db700 1 Processor -- start 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.306+0000 7fcc873db700 1 -- start start 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.306+0000 7fcc873db700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 0x7fcc80195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:36.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.306+0000 7fcc873db700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80196230 con 0x7fcc80100880 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.306+0000 7fcc85177700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 0x7fcc80195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.306+0000 7fcc85177700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 0x7fcc80195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:43788/0 (socket says 172.21.15.193:43788) 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.306+0000 7fcc85177700 1 -- 172.21.15.193:0/2033642848 learned_addr learned my addr 172.21.15.193:0/2033642848 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.307+0000 7fcc85177700 1 -- 172.21.15.193:0/2033642848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0095f0 con 0x7fcc80100880 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.307+0000 7fcc85177700 1 --2- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 0x7fcc80195cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c006010 tx=0x7fcc7c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.307+0000 7fcc767fc700 1 -- 172.21.15.193:0/2033642848 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc7c01d070 con 0x7fcc80100880 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.307+0000 7fcc767fc700 1 -- 172.21.15.193:0/2033642848 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc7c005dc0 con 0x7fcc80100880 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.307+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc80196430 con 0x7fcc80100880 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.307+0000 7fcc767fc700 1 -- 172.21.15.193:0/2033642848 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc7c022e90 con 0x7fcc80100880 2024-05-09T06:29:36.290 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.307+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc801968d0 con 0x7fcc80100880 2024-05-09T06:29:36.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.308+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8018f6b0 con 0x7fcc80100880 2024-05-09T06:29:36.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.308+0000 7fcc767fc700 1 -- 172.21.15.193:0/2033642848 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fcc7c022470 con 0x7fcc80100880 2024-05-09T06:29:36.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.308+0000 7fcc767fc700 1 --2- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fcc6c0387c0 0x7fcc6c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:36.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.308+0000 7fcc767fc700 1 -- 172.21.15.193:0/2033642848 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc7c04c800 con 0x7fcc80100880 2024-05-09T06:29:36.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.308+0000 7fcc84976700 1 --2- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fcc6c0387c0 0x7fcc6c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:36.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.309+0000 7fcc84976700 1 --2- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fcc6c0387c0 0x7fcc6c03ac80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc70006fd0 tx=0x7fcc70006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:36.295 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.313+0000 7fcc767fc700 1 -- 172.21.15.193:0/2033642848 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcc7c00b960 con 0x7fcc80100880 2024-05-09T06:29:36.497 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.516+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 --> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi063:172.21.15.63=smithi063;smithi193:172.21.15.193=smithi193", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc8002cdb0 con 0x7fcc6c0387c0 2024-05-09T06:29:36.504 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.523+0000 7fcc767fc700 1 -- 172.21.15.193:0/2033642848 <== mgr.14164 v2:172.21.15.63:6800/3098785268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcc8002cdb0 con 0x7fcc6c0387c0 2024-05-09T06:29:36.504 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-05-09T06:29:36.506 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fcc6c0387c0 msgr2=0x7fcc6c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.506 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 --2- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fcc6c0387c0 0x7fcc6c03ac80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc70006fd0 tx=0x7fcc70006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:36.507 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 msgr2=0x7fcc80195cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:36.507 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 --2- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 0x7fcc80195cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c006010 tx=0x7fcc7c009f80 comp rx=0 tx=0).stop 2024-05-09T06:29:36.507 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 shutdown_connections 2024-05-09T06:29:36.507 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 --2- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fcc6c0387c0 0x7fcc6c03ac80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.507 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 --2- 172.21.15.193:0/2033642848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc80100880 0x7fcc80195cf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:36.507 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 >> 172.21.15.193:0/2033642848 conn(0x7fcc800fbe40 msgr2=0x7fcc800fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:36.507 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 shutdown_connections 2024-05-09T06:29:36.507 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:36.525+0000 7fcc873db700 1 -- 172.21.15.193:0/2033642848 wait complete. 2024-05-09T06:29:36.779 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:36 smithi063 ceph-mon[27171]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:36.779 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:36 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:37.060 DEBUG:teuthology.orchestra.run.smithi193:mon.smithi193> sudo journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi193.service 2024-05-09T06:29:37.062 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:29:37.062 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:29:37.290 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:37.339 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:37.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:37 smithi063 ceph-mon[27171]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi063:172.21.15.63=smithi063;smithi193:172.21.15.193=smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:29:37.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:37 smithi063 ceph-mon[27171]: Saving service mon spec with placement smithi063:172.21.15.63=smithi063;smithi193:172.21.15.193=smithi193;count:2 2024-05-09T06:29:38.774 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.793+0000 7f7dc747b700 1 -- 172.21.15.193:0/3749758971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 msgr2=0x7f7dc0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:38.775 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.793+0000 7f7dc747b700 1 --2- 172.21.15.193:0/3749758971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 0x7f7dc0100ca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009910 tx=0x7f7dbc009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:38.775 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.793+0000 7f7dc747b700 1 -- 172.21.15.193:0/3749758971 shutdown_connections 2024-05-09T06:29:38.775 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.793+0000 7f7dc747b700 1 --2- 172.21.15.193:0/3749758971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 0x7f7dc0100ca0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:38.775 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.793+0000 7f7dc747b700 1 -- 172.21.15.193:0/3749758971 >> 172.21.15.193:0/3749758971 conn(0x7f7dc00fbe40 msgr2=0x7f7dc00fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:38.775 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.793+0000 7f7dc747b700 1 -- 172.21.15.193:0/3749758971 shutdown_connections 2024-05-09T06:29:38.775 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.793+0000 7f7dc747b700 1 -- 172.21.15.193:0/3749758971 wait complete. 2024-05-09T06:29:38.776 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.793+0000 7f7dc747b700 1 Processor -- start 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.794+0000 7f7dc747b700 1 -- start start 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.794+0000 7f7dc747b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 0x7f7dc0195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.794+0000 7f7dc747b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0196240 con 0x7f7dc0100880 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.794+0000 7f7dc5217700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 0x7f7dc0195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.794+0000 7f7dc5217700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 0x7f7dc0195d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:43804/0 (socket says 172.21.15.193:43804) 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.794+0000 7f7dc5217700 1 -- 172.21.15.193:0/2511171371 learned_addr learned my addr 172.21.15.193:0/2511171371 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.794+0000 7f7dc5217700 1 -- 172.21.15.193:0/2511171371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0095f0 con 0x7f7dc0100880 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.795+0000 7f7dc5217700 1 --2- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 0x7f7dc0195d00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc006010 tx=0x7f7dbc009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.795+0000 7f7db67fc700 1 -- 172.21.15.193:0/2511171371 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7dbc01d070 con 0x7f7dc0100880 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.795+0000 7f7db67fc700 1 -- 172.21.15.193:0/2511171371 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dbc005dc0 con 0x7f7dc0100880 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.795+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc0196440 con 0x7f7dc0100880 2024-05-09T06:29:38.777 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.795+0000 7f7db67fc700 1 -- 172.21.15.193:0/2511171371 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7dbc022e90 con 0x7f7dc0100880 2024-05-09T06:29:38.778 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.795+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc01968e0 con 0x7f7dc0100880 2024-05-09T06:29:38.778 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.796+0000 7f7db67fc700 1 -- 172.21.15.193:0/2511171371 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f7dbc00b960 con 0x7f7dc0100880 2024-05-09T06:29:38.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.796+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc018f6b0 con 0x7f7dc0100880 2024-05-09T06:29:38.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.796+0000 7f7db67fc700 1 --2- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f7dac038760 0x7f7dac03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:38.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.796+0000 7f7db67fc700 1 -- 172.21.15.193:0/2511171371 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7dbc04c800 con 0x7f7dc0100880 2024-05-09T06:29:38.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.796+0000 7f7dc4a16700 1 --2- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f7dac038760 0x7f7dac03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:38.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.797+0000 7f7dc4a16700 1 --2- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f7dac038760 0x7f7dac03ac20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7db0006fd0 tx=0x7f7db0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:38.782 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:38.801+0000 7f7db67fc700 1 -- 172.21.15.193:0/2511171371 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7dbc022750 con 0x7f7dc0100880 2024-05-09T06:29:39.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.071+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7dc0062520 con 0x7f7dc0100880 2024-05-09T06:29:39.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.072+0000 7f7db67fc700 1 -- 172.21.15.193:0/2511171371 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f7dc0062520 con 0x7f7dc0100880 2024-05-09T06:29:39.053 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:29:39.053 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:29:39.055 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f7dac038760 msgr2=0x7f7dac03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:39.055 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 --2- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f7dac038760 0x7f7dac03ac20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7db0006fd0 tx=0x7f7db0006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:39.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 msgr2=0x7f7dc0195d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:39.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 --2- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 0x7f7dc0195d00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc006010 tx=0x7f7dbc009f80 comp rx=0 tx=0).stop 2024-05-09T06:29:39.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 shutdown_connections 2024-05-09T06:29:39.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 --2- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f7dac038760 0x7f7dac03ac20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 --2- 172.21.15.193:0/2511171371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7dc0100880 0x7f7dc0195d00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:39.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 >> 172.21.15.193:0/2511171371 conn(0x7f7dc00fbe40 msgr2=0x7f7dc00fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:39.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 shutdown_connections 2024-05-09T06:29:39.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:39.074+0000 7f7dc747b700 1 -- 172.21.15.193:0/2511171371 wait complete. 2024-05-09T06:29:39.057 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:29:39.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:39.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: Regenerating cephadm self-signed grafana TLS certificates 2024-05-09T06:29:39.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-09T06:29:39.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-09T06:29:39.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: Deploying daemon grafana.smithi063 on smithi063 2024-05-09T06:29:39.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:38 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:39.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:39 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/2511171371' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:29:40.564 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:29:40.564 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:29:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:40.816 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:41.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:40 smithi063 ceph-mon[27171]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:42.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.186+0000 7f45f90f3700 1 -- 172.21.15.193:0/521217805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 msgr2=0x7f45f4100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.186+0000 7f45f90f3700 1 --2- 172.21.15.193:0/521217805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 0x7f45f4100ca0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009910 tx=0x7f45e4009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:42.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.187+0000 7f45f90f3700 1 -- 172.21.15.193:0/521217805 shutdown_connections 2024-05-09T06:29:42.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.187+0000 7f45f90f3700 1 --2- 172.21.15.193:0/521217805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 0x7f45f4100ca0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.187+0000 7f45f90f3700 1 -- 172.21.15.193:0/521217805 >> 172.21.15.193:0/521217805 conn(0x7f45f40fbe40 msgr2=0x7f45f40fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:42.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.187+0000 7f45f90f3700 1 -- 172.21.15.193:0/521217805 shutdown_connections 2024-05-09T06:29:42.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.187+0000 7f45f90f3700 1 -- 172.21.15.193:0/521217805 wait complete. 2024-05-09T06:29:42.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.187+0000 7f45f90f3700 1 Processor -- start 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.188+0000 7f45f90f3700 1 -- start start 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.188+0000 7f45f90f3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 0x7f45f4195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.188+0000 7f45f90f3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4196240 con 0x7f45f4100880 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.188+0000 7f45f2d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 0x7f45f4195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.188+0000 7f45f2d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 0x7f45f4195d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:37638/0 (socket says 172.21.15.193:37638) 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.188+0000 7f45f2d9d700 1 -- 172.21.15.193:0/1461072930 learned_addr learned my addr 172.21.15.193:0/1461072930 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.188+0000 7f45f2d9d700 1 -- 172.21.15.193:0/1461072930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e40095f0 con 0x7f45f4100880 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.188+0000 7f45f2d9d700 1 --2- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 0x7f45f4195d00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f45e4006010 tx=0x7f45e4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.188+0000 7f45ebfff700 1 -- 172.21.15.193:0/1461072930 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45e401d070 con 0x7f45f4100880 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.189+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f4196440 con 0x7f45f4100880 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.189+0000 7f45ebfff700 1 -- 172.21.15.193:0/1461072930 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45e4005dc0 con 0x7f45f4100880 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.189+0000 7f45ebfff700 1 -- 172.21.15.193:0/1461072930 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45e4022e90 con 0x7f45f4100880 2024-05-09T06:29:42.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.189+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f41968e0 con 0x7f45f4100880 2024-05-09T06:29:42.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.190+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f418f6b0 con 0x7f45f4100880 2024-05-09T06:29:42.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.190+0000 7f45ebfff700 1 -- 172.21.15.193:0/1461072930 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f45e400b960 con 0x7f45f4100880 2024-05-09T06:29:42.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.190+0000 7f45ebfff700 1 --2- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f45e0038760 0x7f45e003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:42.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.190+0000 7f45ebfff700 1 -- 172.21.15.193:0/1461072930 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f45e404c840 con 0x7f45f4100880 2024-05-09T06:29:42.174 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.190+0000 7f45f259c700 1 --2- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f45e0038760 0x7f45e003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:42.174 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.190+0000 7f45f259c700 1 --2- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f45e0038760 0x7f45e003ac20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45dc006fd0 tx=0x7f45dc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:42.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.199+0000 7f45ebfff700 1 -- 172.21.15.193:0/1461072930 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f45e4022750 con 0x7f45f4100880 2024-05-09T06:29:42.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.466+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f45f4062520 con 0x7f45f4100880 2024-05-09T06:29:42.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.467+0000 7f45ebfff700 1 -- 172.21.15.193:0/1461072930 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f45f4062520 con 0x7f45f4100880 2024-05-09T06:29:42.449 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:29:42.449 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f45e0038760 msgr2=0x7f45e003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 --2- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f45e0038760 0x7f45e003ac20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45dc006fd0 tx=0x7f45dc006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 msgr2=0x7f45f4195d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 --2- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 0x7f45f4195d00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f45e4006010 tx=0x7f45e4009f80 comp rx=0 tx=0).stop 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 shutdown_connections 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 --2- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f45e0038760 0x7f45e003ac20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 --2- 172.21.15.193:0/1461072930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45f4100880 0x7f45f4195d00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 >> 172.21.15.193:0/1461072930 conn(0x7f45f40fbe40 msgr2=0x7f45f40fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 shutdown_connections 2024-05-09T06:29:42.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:42.469+0000 7f45f90f3700 1 -- 172.21.15.193:0/1461072930 wait complete. 2024-05-09T06:29:42.452 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:29:43.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:42 smithi063 ceph-mon[27171]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:43.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:42 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/1461072930' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:29:43.959 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:29:43.959 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:29:44.168 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:44.218 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:45.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:44 smithi063 ceph-mon[27171]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:45.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 -- 172.21.15.193:0/93388040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 msgr2=0x7f65c8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:45.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 --2- 172.21.15.193:0/93388040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 0x7f65c8100ca0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009910 tx=0x7f65c4009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:45.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 -- 172.21.15.193:0/93388040 shutdown_connections 2024-05-09T06:29:45.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 --2- 172.21.15.193:0/93388040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 0x7f65c8100ca0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:45.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 -- 172.21.15.193:0/93388040 >> 172.21.15.193:0/93388040 conn(0x7f65c80fbe40 msgr2=0x7f65c80fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:45.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 -- 172.21.15.193:0/93388040 shutdown_connections 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 -- 172.21.15.193:0/93388040 wait complete. 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 Processor -- start 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 -- start start 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 0x7f65c8074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.608+0000 7f65cf718700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80750a0 con 0x7f65c8100880 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65cd4b4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 0x7f65c8074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65cd4b4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 0x7f65c8074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:37664/0 (socket says 172.21.15.193:37664) 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65cd4b4700 1 -- 172.21.15.193:0/3477626075 learned_addr learned my addr 172.21.15.193:0/3477626075 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65cd4b4700 1 -- 172.21.15.193:0/3477626075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c40095f0 con 0x7f65c8100880 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65cd4b4700 1 --2- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 0x7f65c8074b60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006010 tx=0x7f65c4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65be7fc700 1 -- 172.21.15.193:0/3477626075 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c401d070 con 0x7f65c8100880 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65be7fc700 1 -- 172.21.15.193:0/3477626075 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65c4005dc0 con 0x7f65c8100880 2024-05-09T06:29:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c80732e0 con 0x7f65c8100880 2024-05-09T06:29:45.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65be7fc700 1 -- 172.21.15.193:0/3477626075 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c4022e90 con 0x7f65c8100880 2024-05-09T06:29:45.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.609+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c8073780 con 0x7f65c8100880 2024-05-09T06:29:45.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.610+0000 7f65be7fc700 1 -- 172.21.15.193:0/3477626075 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f65c400b960 con 0x7f65c8100880 2024-05-09T06:29:45.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.610+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c8106fe0 con 0x7f65c8100880 2024-05-09T06:29:45.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.610+0000 7f65be7fc700 1 --2- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f65b40387c0 0x7f65b403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:45.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.610+0000 7f65be7fc700 1 -- 172.21.15.193:0/3477626075 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f65c404c840 con 0x7f65c8100880 2024-05-09T06:29:45.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.611+0000 7f65cccb3700 1 --2- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f65b40387c0 0x7f65b403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:45.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.611+0000 7f65cccb3700 1 --2- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f65b40387c0 0x7f65b403ac80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006fd0 tx=0x7f65b8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:45.597 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.615+0000 7f65be7fc700 1 -- 172.21.15.193:0/3477626075 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f65c4022750 con 0x7f65c8100880 2024-05-09T06:29:45.865 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.883+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f65c8062520 con 0x7f65c8100880 2024-05-09T06:29:45.865 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.883+0000 7f65be7fc700 1 -- 172.21.15.193:0/3477626075 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f65c4027020 con 0x7f65c8100880 2024-05-09T06:29:45.865 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:29:45.865 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:29:45.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f65b40387c0 msgr2=0x7f65b403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:45.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 --2- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f65b40387c0 0x7f65b403ac80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006fd0 tx=0x7f65b8006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:45.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 msgr2=0x7f65c8074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:45.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 --2- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 0x7f65c8074b60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006010 tx=0x7f65c4009f80 comp rx=0 tx=0).stop 2024-05-09T06:29:45.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 shutdown_connections 2024-05-09T06:29:45.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 --2- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f65b40387c0 0x7f65b403ac80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:45.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 --2- 172.21.15.193:0/3477626075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c8100880 0x7f65c8074b60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:45.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 >> 172.21.15.193:0/3477626075 conn(0x7f65c80fbe40 msgr2=0x7f65c80fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:45.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 shutdown_connections 2024-05-09T06:29:45.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:45.886+0000 7f65cf718700 1 -- 172.21.15.193:0/3477626075 wait complete. 2024-05-09T06:29:45.868 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:29:47.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:46 smithi063 ceph-mon[27171]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:47.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:46 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/3477626075' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:29:47.523 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:29:47.524 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:29:47.732 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:47.780 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.085+0000 7f536a33f700 1 -- 172.21.15.193:0/4080708207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 msgr2=0x7f53640ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.085+0000 7f536a33f700 1 --2- 172.21.15.193:0/4080708207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 0x7f53640ffb90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5354009910 tx=0x7f5354009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.085+0000 7f536a33f700 1 -- 172.21.15.193:0/4080708207 shutdown_connections 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.085+0000 7f536a33f700 1 --2- 172.21.15.193:0/4080708207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 0x7f53640ffb90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.085+0000 7f536a33f700 1 -- 172.21.15.193:0/4080708207 >> 172.21.15.193:0/4080708207 conn(0x7f53640fad10 msgr2=0x7f53640fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.085+0000 7f536a33f700 1 -- 172.21.15.193:0/4080708207 shutdown_connections 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.085+0000 7f536a33f700 1 -- 172.21.15.193:0/4080708207 wait complete. 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.085+0000 7f536a33f700 1 Processor -- start 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.085+0000 7f536a33f700 1 -- start start 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.086+0000 7f536a33f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 0x7f5364194c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.086+0000 7f536a33f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5364195180 con 0x7f53640ff770 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.086+0000 7f5363fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 0x7f5364194c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:49.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.086+0000 7f5363fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 0x7f5364194c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:37690/0 (socket says 172.21.15.193:37690) 2024-05-09T06:29:49.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.086+0000 7f5363fff700 1 -- 172.21.15.193:0/954483876 learned_addr learned my addr 172.21.15.193:0/954483876 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:29:49.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.086+0000 7f5363fff700 1 -- 172.21.15.193:0/954483876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540095f0 con 0x7f53640ff770 2024-05-09T06:29:49.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.086+0000 7f5363fff700 1 --2- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 0x7f5364194c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5354006010 tx=0x7f5354009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:49.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.086+0000 7f53617fa700 1 -- 172.21.15.193:0/954483876 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f535401d070 con 0x7f53640ff770 2024-05-09T06:29:49.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.087+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5364195380 con 0x7f53640ff770 2024-05-09T06:29:49.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.087+0000 7f53617fa700 1 -- 172.21.15.193:0/954483876 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5354005dc0 con 0x7f53640ff770 2024-05-09T06:29:49.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.087+0000 7f53617fa700 1 -- 172.21.15.193:0/954483876 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5354022e90 con 0x7f53640ff770 2024-05-09T06:29:49.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.087+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5364195820 con 0x7f53640ff770 2024-05-09T06:29:49.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.087+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536418e5a0 con 0x7f53640ff770 2024-05-09T06:29:49.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.087+0000 7f53617fa700 1 -- 172.21.15.193:0/954483876 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f535400b960 con 0x7f53640ff770 2024-05-09T06:29:49.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.088+0000 7f53617fa700 1 --2- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f534c038760 0x7f534c03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:49.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.088+0000 7f53617fa700 1 -- 172.21.15.193:0/954483876 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f535404c8c0 con 0x7f53640ff770 2024-05-09T06:29:49.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.088+0000 7f53637fe700 1 --2- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f534c038760 0x7f534c03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:49.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.088+0000 7f53637fe700 1 --2- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f534c038760 0x7f534c03ac20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5358006fd0 tx=0x7f5358006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:49.074 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.093+0000 7f53617fa700 1 -- 172.21.15.193:0/954483876 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5354022750 con 0x7f53640ff770 2024-05-09T06:29:49.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:48 smithi063 ceph-mon[27171]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:49.340 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.359+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5364062520 con 0x7f53640ff770 2024-05-09T06:29:49.341 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.359+0000 7f53617fa700 1 -- 172.21.15.193:0/954483876 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f5364062520 con 0x7f53640ff770 2024-05-09T06:29:49.341 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:29:49.341 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:29:49.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f534c038760 msgr2=0x7f534c03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:49.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 --2- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f534c038760 0x7f534c03ac20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5358006fd0 tx=0x7f5358006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:49.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 msgr2=0x7f5364194c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:49.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 --2- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 0x7f5364194c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5354006010 tx=0x7f5354009f80 comp rx=0 tx=0).stop 2024-05-09T06:29:49.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 shutdown_connections 2024-05-09T06:29:49.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 --2- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f534c038760 0x7f534c03ac20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:49.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 --2- 172.21.15.193:0/954483876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53640ff770 0x7f5364194c40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:49.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 >> 172.21.15.193:0/954483876 conn(0x7f53640fad10 msgr2=0x7f53640fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:49.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 shutdown_connections 2024-05-09T06:29:49.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:49.362+0000 7f536a33f700 1 -- 172.21.15.193:0/954483876 wait complete. 2024-05-09T06:29:49.344 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:29:49.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:49 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/954483876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:29:50.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:50 smithi063 ceph-mon[27171]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:50.884 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:29:50.884 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:29:51.079 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:51.127 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:52.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.489+0000 7f3406e3d700 1 -- 172.21.15.193:0/675196067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 msgr2=0x7f34000ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:52.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.489+0000 7f3406e3d700 1 --2- 172.21.15.193:0/675196067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 0x7f34000ffb90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009910 tx=0x7f33f8009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:52.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.489+0000 7f3406e3d700 1 -- 172.21.15.193:0/675196067 shutdown_connections 2024-05-09T06:29:52.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.489+0000 7f3406e3d700 1 --2- 172.21.15.193:0/675196067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 0x7f34000ffb90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:52.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.489+0000 7f3406e3d700 1 -- 172.21.15.193:0/675196067 >> 172.21.15.193:0/675196067 conn(0x7f34000fad10 msgr2=0x7f34000fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:52.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.489+0000 7f3406e3d700 1 -- 172.21.15.193:0/675196067 shutdown_connections 2024-05-09T06:29:52.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.489+0000 7f3406e3d700 1 -- 172.21.15.193:0/675196067 wait complete. 2024-05-09T06:29:52.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.490+0000 7f3406e3d700 1 Processor -- start 2024-05-09T06:29:52.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.490+0000 7f3406e3d700 1 -- start start 2024-05-09T06:29:52.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.490+0000 7f3406e3d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 0x7f34000fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:52.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.490+0000 7f3406e3d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000fd0c0 con 0x7f34000ff770 2024-05-09T06:29:52.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.490+0000 7f3404bd9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 0x7f34000fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:52.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.490+0000 7f3404bd9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 0x7f34000fcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:50894/0 (socket says 172.21.15.193:50894) 2024-05-09T06:29:52.475 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.490+0000 7f3404bd9700 1 -- 172.21.15.193:0/513556971 learned_addr learned my addr 172.21.15.193:0/513556971 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:29:52.475 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.490+0000 7f3404bd9700 1 -- 172.21.15.193:0/513556971 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80095f0 con 0x7f34000ff770 2024-05-09T06:29:52.475 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.491+0000 7f3404bd9700 1 --2- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 0x7f34000fcb80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f33f8006010 tx=0x7f33f8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:52.475 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.491+0000 7f33fdffb700 1 -- 172.21.15.193:0/513556971 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33f801d070 con 0x7f34000ff770 2024-05-09T06:29:52.475 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.491+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34000fb300 con 0x7f34000ff770 2024-05-09T06:29:52.475 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.491+0000 7f33fdffb700 1 -- 172.21.15.193:0/513556971 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33f8005dc0 con 0x7f34000ff770 2024-05-09T06:29:52.475 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.491+0000 7f33fdffb700 1 -- 172.21.15.193:0/513556971 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33f8022e90 con 0x7f34000ff770 2024-05-09T06:29:52.475 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.491+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34000fb7a0 con 0x7f34000ff770 2024-05-09T06:29:52.476 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.492+0000 7f33fdffb700 1 -- 172.21.15.193:0/513556971 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f33f800b960 con 0x7f34000ff770 2024-05-09T06:29:52.477 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.492+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3400105ed0 con 0x7f34000ff770 2024-05-09T06:29:52.477 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.492+0000 7f33fdffb700 1 --2- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f33ec038760 0x7f33ec03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:52.477 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.492+0000 7f33fdffb700 1 -- 172.21.15.193:0/513556971 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f33f804c800 con 0x7f34000ff770 2024-05-09T06:29:52.477 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.492+0000 7f33fffff700 1 --2- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f33ec038760 0x7f33ec03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:52.477 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.493+0000 7f33fffff700 1 --2- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f33ec038760 0x7f33ec03ac20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f33f000ad30 tx=0x7f33f00093f0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:52.479 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.498+0000 7f33fdffb700 1 -- 172.21.15.193:0/513556971 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f33f800f4f0 con 0x7f34000ff770 2024-05-09T06:29:52.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.766+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f340018f240 con 0x7f34000ff770 2024-05-09T06:29:52.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.767+0000 7f33fdffb700 1 -- 172.21.15.193:0/513556971 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f33f8027030 con 0x7f34000ff770 2024-05-09T06:29:52.748 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:29:52.749 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:29:52.750 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f33ec038760 msgr2=0x7f33ec03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 --2- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f33ec038760 0x7f33ec03ac20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f33f000ad30 tx=0x7f33f00093f0 comp rx=0 tx=0).stop 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 msgr2=0x7f34000fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 --2- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 0x7f34000fcb80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f33f8006010 tx=0x7f33f8009f80 comp rx=0 tx=0).stop 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 shutdown_connections 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 --2- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f33ec038760 0x7f33ec03ac20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 --2- 172.21.15.193:0/513556971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34000ff770 0x7f34000fcb80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 >> 172.21.15.193:0/513556971 conn(0x7f34000fad10 msgr2=0x7f34000690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 shutdown_connections 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:52.769+0000 7f3406e3d700 1 -- 172.21.15.193:0/513556971 wait complete. 2024-05-09T06:29:52.751 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:29:53.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:53.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:53.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:53.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:53.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:53.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:53.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:53.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:53.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:53.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:53 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/513556971' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:29:54.340 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:29:54.341 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:29:54.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:54 smithi063 ceph-mon[27171]: Deploying daemon prometheus.smithi063 on smithi063 2024-05-09T06:29:54.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:54 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:29:54.542 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:54.591 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:55.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:55 smithi063 ceph-mon[27171]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:56.002 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.019+0000 7f76afd89700 1 -- 172.21.15.193:0/2835176829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 msgr2=0x7f76a80ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:56.002 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.019+0000 7f76afd89700 1 --2- 172.21.15.193:0/2835176829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 0x7f76a80ffba0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f76a400b0a0 tx=0x7f76a400b3b0 comp rx=0 tx=0).stop 2024-05-09T06:29:56.002 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.019+0000 7f76afd89700 1 -- 172.21.15.193:0/2835176829 shutdown_connections 2024-05-09T06:29:56.002 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.019+0000 7f76afd89700 1 --2- 172.21.15.193:0/2835176829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 0x7f76a80ffba0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:56.002 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.019+0000 7f76afd89700 1 -- 172.21.15.193:0/2835176829 >> 172.21.15.193:0/2835176829 conn(0x7f76a80fad20 msgr2=0x7f76a80fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:56.002 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.020+0000 7f76afd89700 1 -- 172.21.15.193:0/2835176829 shutdown_connections 2024-05-09T06:29:56.002 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.020+0000 7f76afd89700 1 -- 172.21.15.193:0/2835176829 wait complete. 2024-05-09T06:29:56.002 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.020+0000 7f76afd89700 1 Processor -- start 2024-05-09T06:29:56.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.020+0000 7f76afd89700 1 -- start start 2024-05-09T06:29:56.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.020+0000 7f76afd89700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 0x7f76a8194c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:56.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.020+0000 7f76afd89700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8195160 con 0x7f76a80ff780 2024-05-09T06:29:56.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f76adb25700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 0x7f76a8194c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:56.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f76adb25700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 0x7f76a8194c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:50926/0 (socket says 172.21.15.193:50926) 2024-05-09T06:29:56.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f76adb25700 1 -- 172.21.15.193:0/1815111131 learned_addr learned my addr 172.21.15.193:0/1815111131 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:29:56.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f76adb25700 1 -- 172.21.15.193:0/1815111131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a4009d00 con 0x7f76a80ff780 2024-05-09T06:29:56.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f76adb25700 1 --2- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 0x7f76a8194c20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f76a40062a0 tx=0x7f76a400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:56.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f769effd700 1 -- 172.21.15.193:0/1815111131 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76a400e040 con 0x7f76a80ff780 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a8195360 con 0x7f76a80ff780 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f769effd700 1 -- 172.21.15.193:0/1815111131 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76a4003d70 con 0x7f76a80ff780 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f769effd700 1 -- 172.21.15.193:0/1815111131 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76a401c890 con 0x7f76a80ff780 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.021+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a8195800 con 0x7f76a80ff780 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.022+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a818e5b0 con 0x7f76a80ff780 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.022+0000 7f769effd700 1 -- 172.21.15.193:0/1815111131 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f76a401cb50 con 0x7f76a80ff780 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.022+0000 7f769effd700 1 --2- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f76940387c0 0x7f769403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.023+0000 7f769effd700 1 -- 172.21.15.193:0/1815111131 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f76a404ca20 con 0x7f76a80ff780 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.023+0000 7f76ad324700 1 --2- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f76940387c0 0x7f769403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:56.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.023+0000 7f76ad324700 1 --2- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f76940387c0 0x7f769403ac80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7698006fd0 tx=0x7f7698006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:56.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.028+0000 7f769effd700 1 -- 172.21.15.193:0/1815111131 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f76a401a3b0 con 0x7f76a80ff780 2024-05-09T06:29:56.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.301+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f76a8062520 con 0x7f76a80ff780 2024-05-09T06:29:56.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.301+0000 7f769effd700 1 -- 172.21.15.193:0/1815111131 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f76a401a3b0 con 0x7f76a80ff780 2024-05-09T06:29:56.283 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:29:56.283 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:29:56.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.303+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f76940387c0 msgr2=0x7f769403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:56.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.303+0000 7f76afd89700 1 --2- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f76940387c0 0x7f769403ac80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7698006fd0 tx=0x7f7698006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:56.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.303+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 msgr2=0x7f76a8194c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:56.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.303+0000 7f76afd89700 1 --2- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 0x7f76a8194c20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f76a40062a0 tx=0x7f76a400bd40 comp rx=0 tx=0).stop 2024-05-09T06:29:56.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.303+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 shutdown_connections 2024-05-09T06:29:56.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.303+0000 7f76afd89700 1 --2- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f76940387c0 0x7f769403ac80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:56.286 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.304+0000 7f76afd89700 1 --2- 172.21.15.193:0/1815111131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76a80ff780 0x7f76a8194c20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:56.286 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.304+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 >> 172.21.15.193:0/1815111131 conn(0x7f76a80fad20 msgr2=0x7f76a80fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:56.286 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.304+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 shutdown_connections 2024-05-09T06:29:56.286 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:56.304+0000 7f76afd89700 1 -- 172.21.15.193:0/1815111131 wait complete. 2024-05-09T06:29:56.286 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:29:57.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:57 smithi063 ceph-mon[27171]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:57.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:57 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/1815111131' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:29:57.970 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:29:57.971 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:29:58.175 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:58.223 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:29:59.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:29:59 smithi063 ceph-mon[27171]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:29:59.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.584+0000 7fa2fe534700 1 -- 172.21.15.193:0/1773109490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 msgr2=0x7fa2f8073320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:59.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.584+0000 7fa2fe534700 1 --2- 172.21.15.193:0/1773109490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 0x7fa2f8073320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009910 tx=0x7fa2e8009c20 comp rx=0 tx=0).stop 2024-05-09T06:29:59.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.584+0000 7fa2fe534700 1 -- 172.21.15.193:0/1773109490 shutdown_connections 2024-05-09T06:29:59.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.584+0000 7fa2fe534700 1 --2- 172.21.15.193:0/1773109490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 0x7fa2f8073320 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:59.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.584+0000 7fa2fe534700 1 -- 172.21.15.193:0/1773109490 >> 172.21.15.193:0/1773109490 conn(0x7fa2f80fbe00 msgr2=0x7fa2f80fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:59.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.584+0000 7fa2fe534700 1 -- 172.21.15.193:0/1773109490 shutdown_connections 2024-05-09T06:29:59.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.584+0000 7fa2fe534700 1 -- 172.21.15.193:0/1773109490 wait complete. 2024-05-09T06:29:59.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.585+0000 7fa2fe534700 1 Processor -- start 2024-05-09T06:29:59.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.585+0000 7fa2fe534700 1 -- start start 2024-05-09T06:29:59.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.585+0000 7fa2fe534700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 0x7fa2f8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:59.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.585+0000 7fa2fe534700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f8198440 con 0x7fa2f8074ec0 2024-05-09T06:29:59.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.585+0000 7fa2f7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 0x7fa2f8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:59.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.585+0000 7fa2f7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 0x7fa2f8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:50930/0 (socket says 172.21.15.193:50930) 2024-05-09T06:29:59.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.585+0000 7fa2f7fff700 1 -- 172.21.15.193:0/4176617577 learned_addr learned my addr 172.21.15.193:0/4176617577 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.585+0000 7fa2f7fff700 1 -- 172.21.15.193:0/4176617577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80095f0 con 0x7fa2f8074ec0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.586+0000 7fa2f7fff700 1 --2- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 0x7fa2f8197f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009ed0 tx=0x7fa2e8004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.586+0000 7fa2f5ffb700 1 -- 172.21.15.193:0/4176617577 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2e801d070 con 0x7fa2f8074ec0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.586+0000 7fa2f5ffb700 1 -- 172.21.15.193:0/4176617577 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2e800be20 con 0x7fa2f8074ec0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.586+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f8198640 con 0x7fa2f8074ec0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.586+0000 7fa2f5ffb700 1 -- 172.21.15.193:0/4176617577 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2e800f460 con 0x7fa2f8074ec0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.586+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f8198ae0 con 0x7fa2f8074ec0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.587+0000 7fa2f5ffb700 1 -- 172.21.15.193:0/4176617577 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7fa2e8022800 con 0x7fa2f8074ec0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.587+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f8191890 con 0x7fa2f8074ec0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.587+0000 7fa2f5ffb700 1 --2- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa2e40387d0 0x7fa2e403ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.587+0000 7fa2f5ffb700 1 -- 172.21.15.193:0/4176617577 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa2e804ccc0 con 0x7fa2f8074ec0 2024-05-09T06:29:59.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.587+0000 7fa2effff700 1 --2- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa2e40387d0 0x7fa2e403ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:29:59.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.588+0000 7fa2effff700 1 --2- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa2e40387d0 0x7fa2e403ac90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0006fd0 tx=0x7fa2e0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:29:59.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.592+0000 7fa2f5ffb700 1 -- 172.21.15.193:0/4176617577 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa2e8053080 con 0x7fa2f8074ec0 2024-05-09T06:29:59.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.858+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa2f8062520 con 0x7fa2f8074ec0 2024-05-09T06:29:59.840 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.859+0000 7fa2f5ffb700 1 -- 172.21.15.193:0/4176617577 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fa2f8062520 con 0x7fa2f8074ec0 2024-05-09T06:29:59.840 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:29:59.840 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:29:59.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa2e40387d0 msgr2=0x7fa2e403ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:59.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 --2- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa2e40387d0 0x7fa2e403ac90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0006fd0 tx=0x7fa2e0006e40 comp rx=0 tx=0).stop 2024-05-09T06:29:59.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 msgr2=0x7fa2f8197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:29:59.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 --2- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 0x7fa2f8197f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009ed0 tx=0x7fa2e8004690 comp rx=0 tx=0).stop 2024-05-09T06:29:59.842 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 shutdown_connections 2024-05-09T06:29:59.843 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 --2- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7fa2e40387d0 0x7fa2e403ac90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:59.843 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 --2- 172.21.15.193:0/4176617577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f8074ec0 0x7fa2f8197f00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:29:59.843 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 >> 172.21.15.193:0/4176617577 conn(0x7fa2f80fbe00 msgr2=0x7fa2f80fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:29:59.843 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 shutdown_connections 2024-05-09T06:29:59.843 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:29:59.861+0000 7fa2fe534700 1 -- 172.21.15.193:0/4176617577 wait complete. 2024-05-09T06:29:59.843 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:00.344 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:00 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/4176617577' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:01.266 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:01 smithi063 ceph-mon[27171]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:01.456 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:01.456 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:01.660 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:01.708 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:03.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.037+0000 7f1e82dd2700 1 -- 172.21.15.193:0/375812581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 msgr2=0x7f1e7c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:03.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.037+0000 7f1e82dd2700 1 --2- 172.21.15.193:0/375812581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 0x7f1e7c0ffbc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009910 tx=0x7f1e70009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:03.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.037+0000 7f1e82dd2700 1 -- 172.21.15.193:0/375812581 shutdown_connections 2024-05-09T06:30:03.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.037+0000 7f1e82dd2700 1 --2- 172.21.15.193:0/375812581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 0x7f1e7c0ffbc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:03.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.037+0000 7f1e82dd2700 1 -- 172.21.15.193:0/375812581 >> 172.21.15.193:0/375812581 conn(0x7f1e7c0fad20 msgr2=0x7f1e7c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:03.024 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.037+0000 7f1e82dd2700 1 -- 172.21.15.193:0/375812581 shutdown_connections 2024-05-09T06:30:03.024 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.037+0000 7f1e82dd2700 1 -- 172.21.15.193:0/375812581 wait complete. 2024-05-09T06:30:03.024 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.038+0000 7f1e82dd2700 1 Processor -- start 2024-05-09T06:30:03.024 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.038+0000 7f1e82dd2700 1 -- start start 2024-05-09T06:30:03.024 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.038+0000 7f1e82dd2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 0x7f1e7c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:03.024 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.038+0000 7f1e82dd2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c198450 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.024 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.038+0000 7f1e80b6e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 0x7f1e7c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:03.024 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.038+0000 7f1e80b6e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 0x7f1e7c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:47530/0 (socket says 172.21.15.193:47530) 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.038+0000 7f1e80b6e700 1 -- 172.21.15.193:0/3855589567 learned_addr learned my addr 172.21.15.193:0/3855589567 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.038+0000 7f1e80b6e700 1 -- 172.21.15.193:0/3855589567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700095f0 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.039+0000 7f1e80b6e700 1 --2- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 0x7f1e7c197f10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e70006010 tx=0x7f1e70009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.039+0000 7f1e79ffb700 1 -- 172.21.15.193:0/3855589567 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e7001d070 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.039+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c198650 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.039+0000 7f1e79ffb700 1 -- 172.21.15.193:0/3855589567 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e70005dc0 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.039+0000 7f1e79ffb700 1 -- 172.21.15.193:0/3855589567 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e70022e90 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.039+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c198af0 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.040+0000 7f1e79ffb700 1 -- 172.21.15.193:0/3855589567 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 45612+0+0 (secure 0 0 0) 0x7f1e7000b960 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.040+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c062520 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.040+0000 7f1e79ffb700 1 --2- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f1e6c038760 0x7f1e6c03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.040+0000 7f1e79ffb700 1 -- 172.21.15.193:0/3855589567 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1e7004c8c0 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.040+0000 7f1e7bfff700 1 --2- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f1e6c038760 0x7f1e6c03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:03.025 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.041+0000 7f1e7bfff700 1 --2- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f1e6c038760 0x7f1e6c03ac20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006fd0 tx=0x7f1e68006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:03.027 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.045+0000 7f1e79ffb700 1 -- 172.21.15.193:0/3855589567 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1e70022750 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.248 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:03 smithi063 ceph-mon[27171]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:03.297 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.315+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1e7c1057d0 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.297 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.316+0000 7f1e79ffb700 1 -- 172.21.15.193:0/3855589567 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1e70027020 con 0x7f1e7c0ff7a0 2024-05-09T06:30:03.298 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:03.298 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f1e6c038760 msgr2=0x7f1e6c03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 --2- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f1e6c038760 0x7f1e6c03ac20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006fd0 tx=0x7f1e68006e40 comp rx=0 tx=0).stop 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 msgr2=0x7f1e7c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 --2- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 0x7f1e7c197f10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e70006010 tx=0x7f1e70009f80 comp rx=0 tx=0).stop 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 shutdown_connections 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 --2- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f1e6c038760 0x7f1e6c03ac20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 --2- 172.21.15.193:0/3855589567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e7c0ff7a0 0x7f1e7c197f10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 >> 172.21.15.193:0/3855589567 conn(0x7f1e7c0fad20 msgr2=0x7f1e7c0fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 shutdown_connections 2024-05-09T06:30:03.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:03.318+0000 7f1e82dd2700 1 -- 172.21.15.193:0/3855589567 wait complete. 2024-05-09T06:30:03.301 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:04.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:04 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/3855589567' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:04.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:04 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:04.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:04 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:04.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:04 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:04.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:04 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-09T06:30:04.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:04 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:04.937 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:04.938 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:05.146 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:05.193 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:05.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:05 smithi063 ceph-mon[27171]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:05.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:05 smithi063 ceph-mon[27171]: from='mgr.14164 172.21.15.63:0/1493552534' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-09T06:30:05.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:05 smithi063 ceph-mon[27171]: mgrmap e14: smithi063.egfguy(active, since 60s) 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.650+0000 7f03454df700 1 -- 172.21.15.193:0/4033388079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 msgr2=0x7f03400ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.650+0000 7f03454df700 1 --2- 172.21.15.193:0/4033388079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 0x7f03400ffb90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0330009910 tx=0x7f0330009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.650+0000 7f03454df700 1 -- 172.21.15.193:0/4033388079 shutdown_connections 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.650+0000 7f03454df700 1 --2- 172.21.15.193:0/4033388079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 0x7f03400ffb90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.650+0000 7f03454df700 1 -- 172.21.15.193:0/4033388079 >> 172.21.15.193:0/4033388079 conn(0x7f03400fad10 msgr2=0x7f03400fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.650+0000 7f03454df700 1 -- 172.21.15.193:0/4033388079 shutdown_connections 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.650+0000 7f03454df700 1 -- 172.21.15.193:0/4033388079 wait complete. 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.650+0000 7f03454df700 1 Processor -- start 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.651+0000 7f03454df700 1 -- start start 2024-05-09T06:30:06.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.651+0000 7f03454df700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 0x7f0340069680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:06.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.651+0000 7f03454df700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0340069bc0 con 0x7f03400ff770 2024-05-09T06:30:06.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.651+0000 7f033effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 0x7f0340069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:06.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.651+0000 7f033effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 0x7f0340069680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:47542/0 (socket says 172.21.15.193:47542) 2024-05-09T06:30:06.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.651+0000 7f033effd700 1 -- 172.21.15.193:0/4008920965 learned_addr learned my addr 172.21.15.193:0/4008920965 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:06.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.651+0000 7f033effd700 1 -- 172.21.15.193:0/4008920965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03300095f0 con 0x7f03400ff770 2024-05-09T06:30:06.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.651+0000 7f033effd700 1 --2- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 0x7f0340069680 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0330006010 tx=0x7f0330009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:06.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.652+0000 7f0337fff700 1 -- 172.21.15.193:0/4008920965 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f033001d070 con 0x7f03400ff770 2024-05-09T06:30:06.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.652+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0340196880 con 0x7f03400ff770 2024-05-09T06:30:06.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.652+0000 7f0337fff700 1 -- 172.21.15.193:0/4008920965 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0330005dc0 con 0x7f03400ff770 2024-05-09T06:30:06.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.652+0000 7f0337fff700 1 -- 172.21.15.193:0/4008920965 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0330022e90 con 0x7f03400ff770 2024-05-09T06:30:06.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.652+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034019b0e0 con 0x7f03400ff770 2024-05-09T06:30:06.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.652+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034018e5a0 con 0x7f03400ff770 2024-05-09T06:30:06.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.653+0000 7f0337fff700 1 -- 172.21.15.193:0/4008920965 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 14) v1 ==== 45626+0+0 (secure 0 0 0) 0x7f033000b960 con 0x7f03400ff770 2024-05-09T06:30:06.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.653+0000 7f0337fff700 1 --2- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f032c0387b0 0x7f032c03ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:06.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.653+0000 7f0337fff700 1 -- 172.21.15.193:0/4008920965 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f033004c8f0 con 0x7f03400ff770 2024-05-09T06:30:06.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.653+0000 7f033e7fc700 1 -- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f032c0387b0 msgr2=0x7f032c03ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3098785268 2024-05-09T06:30:06.637 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.653+0000 7f033e7fc700 1 --2- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f032c0387b0 0x7f032c03ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:30:06.644 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.662+0000 7f0337fff700 1 -- 172.21.15.193:0/4008920965 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f033000f4f0 con 0x7f03400ff770 2024-05-09T06:30:06.836 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.854+0000 7f033e7fc700 1 -- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f032c0387b0 msgr2=0x7f032c03ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3098785268 2024-05-09T06:30:06.836 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.854+0000 7f033e7fc700 1 --2- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f032c0387b0 0x7f032c03ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T06:30:06.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.934+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0340062520 con 0x7f03400ff770 2024-05-09T06:30:06.918 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.934+0000 7f0337fff700 1 -- 172.21.15.193:0/4008920965 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f0330027030 con 0x7f03400ff770 2024-05-09T06:30:06.918 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:06.919 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:06.919 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f032c0387b0 msgr2=0x7f032c03ac70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:30:06.919 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 --2- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f032c0387b0 0x7f032c03ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:06.920 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 msgr2=0x7f0340069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:06.920 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 --2- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 0x7f0340069680 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0330006010 tx=0x7f0330009f80 comp rx=0 tx=0).stop 2024-05-09T06:30:06.920 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 shutdown_connections 2024-05-09T06:30:06.920 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 --2- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f032c0387b0 0x7f032c03ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:06.920 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 --2- 172.21.15.193:0/4008920965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03400ff770 0x7f0340069680 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:06.920 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 >> 172.21.15.193:0/4008920965 conn(0x7f03400fad10 msgr2=0x7f03400fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:06.920 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 shutdown_connections 2024-05-09T06:30:06.921 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:06.937+0000 7f03454df700 1 -- 172.21.15.193:0/4008920965 wait complete. 2024-05-09T06:30:06.921 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:07.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:06 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/4008920965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:08.483 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:08.483 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:08.676 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:08.723 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:10.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.120+0000 7f0ec2553700 1 -- 172.21.15.193:0/4104242330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 msgr2=0x7f0ebc0ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:10.102 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.120+0000 7f0ec2553700 1 --2- 172.21.15.193:0/4104242330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 0x7f0ebc0ffb90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009910 tx=0x7f0eac009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.120+0000 7f0ec2553700 1 -- 172.21.15.193:0/4104242330 shutdown_connections 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.120+0000 7f0ec2553700 1 --2- 172.21.15.193:0/4104242330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 0x7f0ebc0ffb90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.120+0000 7f0ec2553700 1 -- 172.21.15.193:0/4104242330 >> 172.21.15.193:0/4104242330 conn(0x7f0ebc0fad10 msgr2=0x7f0ebc0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.120+0000 7f0ec2553700 1 -- 172.21.15.193:0/4104242330 shutdown_connections 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.120+0000 7f0ec2553700 1 -- 172.21.15.193:0/4104242330 wait complete. 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.120+0000 7f0ec2553700 1 Processor -- start 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.120+0000 7f0ec2553700 1 -- start start 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.121+0000 7f0ec2553700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 0x7f0ebc0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.121+0000 7f0ec2553700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc069be0 con 0x7f0ebc0ff770 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.121+0000 7f0ebbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 0x7f0ebc0696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.121+0000 7f0ebbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 0x7f0ebc0696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:47566/0 (socket says 172.21.15.193:47566) 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.121+0000 7f0ebbfff700 1 -- 172.21.15.193:0/1700944613 learned_addr learned my addr 172.21.15.193:0/1700944613 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:10.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.121+0000 7f0ebbfff700 1 -- 172.21.15.193:0/1700944613 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0095f0 con 0x7f0ebc0ff770 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.121+0000 7f0ebbfff700 1 --2- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 0x7f0ebc0696a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0eac006010 tx=0x7f0eac009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.121+0000 7f0eb97fa700 1 -- 172.21.15.193:0/1700944613 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0eac01d070 con 0x7f0ebc0ff770 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.122+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ebc198a70 con 0x7f0ebc0ff770 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.122+0000 7f0eb97fa700 1 -- 172.21.15.193:0/1700944613 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0eac005dc0 con 0x7f0ebc0ff770 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.122+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ebc198f10 con 0x7f0ebc0ff770 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.122+0000 7f0eb97fa700 1 -- 172.21.15.193:0/1700944613 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0eac022e90 con 0x7f0ebc0ff770 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.122+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ebc18e5a0 con 0x7f0ebc0ff770 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.122+0000 7f0eb97fa700 1 -- 172.21.15.193:0/1700944613 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 14) v1 ==== 45626+0+0 (secure 0 0 0) 0x7f0eac00b960 con 0x7f0ebc0ff770 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.123+0000 7f0eb97fa700 1 --2- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0ea40387b0 0x7f0ea403ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:10.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.123+0000 7f0eb97fa700 1 -- 172.21.15.193:0/1700944613 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0eac04c8f0 con 0x7f0ebc0ff770 2024-05-09T06:30:10.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.123+0000 7f0ebb7fe700 1 -- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0ea40387b0 msgr2=0x7f0ea403ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3098785268 2024-05-09T06:30:10.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.123+0000 7f0ebb7fe700 1 --2- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0ea40387b0 0x7f0ea403ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:30:10.113 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.132+0000 7f0eb97fa700 1 -- 172.21.15.193:0/1700944613 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0eac00f570 con 0x7f0ebc0ff770 2024-05-09T06:30:10.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.324+0000 7f0ebb7fe700 1 -- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0ea40387b0 msgr2=0x7f0ea403ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3098785268 2024-05-09T06:30:10.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.324+0000 7f0ebb7fe700 1 --2- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0ea40387b0 0x7f0ea403ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T06:30:10.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.402+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0ebc062520 con 0x7f0ebc0ff770 2024-05-09T06:30:10.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.402+0000 7f0eb97fa700 1 -- 172.21.15.193:0/1700944613 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f0eac027030 con 0x7f0ebc0ff770 2024-05-09T06:30:10.384 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:10.384 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:10.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0ea40387b0 msgr2=0x7f0ea403ac70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:30:10.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 --2- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0ea40387b0 0x7f0ea403ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:10.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 msgr2=0x7f0ebc0696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:10.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 --2- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 0x7f0ebc0696a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0eac006010 tx=0x7f0eac009f80 comp rx=0 tx=0).stop 2024-05-09T06:30:10.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 shutdown_connections 2024-05-09T06:30:10.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 --2- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:6800/3098785268,v1:172.21.15.63:6801/3098785268] conn(0x7f0ea40387b0 0x7f0ea403ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:10.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 --2- 172.21.15.193:0/1700944613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ebc0ff770 0x7f0ebc0696a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:10.387 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 >> 172.21.15.193:0/1700944613 conn(0x7f0ebc0fad10 msgr2=0x7f0ebc0fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:10.387 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 shutdown_connections 2024-05-09T06:30:10.387 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:10.404+0000 7f0ec2553700 1 -- 172.21.15.193:0/1700944613 wait complete. 2024-05-09T06:30:10.387 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:10.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:10 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/1700944613' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: Active manager daemon smithi063.egfguy restarted 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: Activating manager daemon smithi063.egfguy 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: osdmap e5: 0 total, 0 up, 0 in 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: mgrmap e15: smithi063.egfguy(active, starting, since 0.00324822s) 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: Manager daemon smithi063.egfguy is now available 2024-05-09T06:30:11.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:30:11.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:11.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:30:11.987 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:11.988 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:12.193 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:12.242 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:12.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:12 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:30:12.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:12 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/trash_purge_schedule"}]: dispatch 2024-05-09T06:30:12.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:12 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:12.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:12 smithi063 ceph-mon[27171]: mgrmap e16: smithi063.egfguy(active, since 1.00561s) 2024-05-09T06:30:13.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 -- 172.21.15.193:0/483254025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 msgr2=0x7f53580ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:13.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 --2- 172.21.15.193:0/483254025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 0x7f53580ffba0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f534c009910 tx=0x7f534c009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:13.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 -- 172.21.15.193:0/483254025 shutdown_connections 2024-05-09T06:30:13.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 --2- 172.21.15.193:0/483254025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 0x7f53580ffba0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:13.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 -- 172.21.15.193:0/483254025 >> 172.21.15.193:0/483254025 conn(0x7f53580fad20 msgr2=0x7f53580fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:13.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 -- 172.21.15.193:0/483254025 shutdown_connections 2024-05-09T06:30:13.779 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 -- 172.21.15.193:0/483254025 wait complete. 2024-05-09T06:30:13.780 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 Processor -- start 2024-05-09T06:30:13.780 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 -- start start 2024-05-09T06:30:13.781 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 0x7f5358068ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:13.781 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535f23e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358069010 con 0x7f53580ff780 2024-05-09T06:30:13.781 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535cfda700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 0x7f5358068ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:13.781 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535cfda700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 0x7f5358068ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:47436/0 (socket says 172.21.15.193:47436) 2024-05-09T06:30:13.781 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.798+0000 7f535cfda700 1 -- 172.21.15.193:0/943394848 learned_addr learned my addr 172.21.15.193:0/943394848 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:13.781 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.799+0000 7f535cfda700 1 -- 172.21.15.193:0/943394848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c0095f0 con 0x7f53580ff780 2024-05-09T06:30:13.781 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.799+0000 7f535cfda700 1 --2- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 0x7f5358068ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f534c006010 tx=0x7f534c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:13.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.801+0000 7f5355ffb700 1 -- 172.21.15.193:0/943394848 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f534c01d070 con 0x7f53580ff780 2024-05-09T06:30:13.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.801+0000 7f5355ffb700 1 -- 172.21.15.193:0/943394848 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f534c005dc0 con 0x7f53580ff780 2024-05-09T06:30:13.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.801+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5358198a10 con 0x7f53580ff780 2024-05-09T06:30:13.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.801+0000 7f5355ffb700 1 -- 172.21.15.193:0/943394848 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f534c022e90 con 0x7f53580ff780 2024-05-09T06:30:13.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.802+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5358198eb0 con 0x7f53580ff780 2024-05-09T06:30:13.784 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.803+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535818e5b0 con 0x7f53580ff780 2024-05-09T06:30:13.784 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.803+0000 7f5355ffb700 1 -- 172.21.15.193:0/943394848 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f534c022470 con 0x7f53580ff780 2024-05-09T06:30:13.784 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.803+0000 7f5355ffb700 1 --2- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5348038850 0x7f534803ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:13.784 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.803+0000 7f5355ffb700 1 -- 172.21.15.193:0/943394848 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f534c04cae0 con 0x7f53580ff780 2024-05-09T06:30:13.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.803+0000 7f5357fff700 1 --2- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5348038850 0x7f534803ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:13.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.804+0000 7f5357fff700 1 --2- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5348038850 0x7f534803ad10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5344006fd0 tx=0x7f5344006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:13.789 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:13.808+0000 7f5355ffb700 1 -- 172.21.15.193:0/943394848 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f534c00b960 con 0x7f53580ff780 2024-05-09T06:30:13.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:13 smithi063 ceph-mon[27171]: [09/May/2024:06:30:12] ENGINE Bus STARTING 2024-05-09T06:30:13.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:13 smithi063 ceph-mon[27171]: [09/May/2024:06:30:12] ENGINE Serving on https://172.21.15.63:7150 2024-05-09T06:30:13.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:13 smithi063 ceph-mon[27171]: [09/May/2024:06:30:13] ENGINE Serving on http://172.21.15.63:8765 2024-05-09T06:30:13.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:13 smithi063 ceph-mon[27171]: [09/May/2024:06:30:13] ENGINE Bus STARTED 2024-05-09T06:30:14.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.094+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5358062520 con 0x7f53580ff780 2024-05-09T06:30:14.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.094+0000 7f5355ffb700 1 -- 172.21.15.193:0/943394848 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f5358062520 con 0x7f53580ff780 2024-05-09T06:30:14.077 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:14.077 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:14.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5348038850 msgr2=0x7f534803ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:14.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 --2- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5348038850 0x7f534803ad10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5344006fd0 tx=0x7f5344006e40 comp rx=0 tx=0).stop 2024-05-09T06:30:14.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 msgr2=0x7f5358068ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:14.080 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 --2- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 0x7f5358068ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f534c006010 tx=0x7f534c009f80 comp rx=0 tx=0).stop 2024-05-09T06:30:14.080 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 shutdown_connections 2024-05-09T06:30:14.080 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 --2- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5348038850 0x7f534803ad10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:14.080 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 --2- 172.21.15.193:0/943394848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53580ff780 0x7f5358068ad0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:14.080 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 >> 172.21.15.193:0/943394848 conn(0x7f53580fad20 msgr2=0x7f53580fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:14.080 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 shutdown_connections 2024-05-09T06:30:14.080 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:14.097+0000 7f535f23e700 1 -- 172.21.15.193:0/943394848 wait complete. 2024-05-09T06:30:14.081 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:14.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:14 smithi063 ceph-mon[27171]: mgrmap e17: smithi063.egfguy(active, since 2s) 2024-05-09T06:30:14.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:14.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:14 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/943394848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:15.696 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:15.697 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:15.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:15 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:16.124 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:16.536 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-09T06:30:18.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.561+0000 7f21f34d1700 1 -- 172.21.15.193:0/909567005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 msgr2=0x7f21ec0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.561+0000 7f21f34d1700 1 --2- 172.21.15.193:0/909567005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 0x7f21ec0ffba0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009910 tx=0x7f21dc009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.561+0000 7f21f34d1700 1 -- 172.21.15.193:0/909567005 shutdown_connections 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.561+0000 7f21f34d1700 1 --2- 172.21.15.193:0/909567005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 0x7f21ec0ffba0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.561+0000 7f21f34d1700 1 -- 172.21.15.193:0/909567005 >> 172.21.15.193:0/909567005 conn(0x7f21ec0fad20 msgr2=0x7f21ec0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.561+0000 7f21f34d1700 1 -- 172.21.15.193:0/909567005 shutdown_connections 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.561+0000 7f21f34d1700 1 -- 172.21.15.193:0/909567005 wait complete. 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.562+0000 7f21f34d1700 1 Processor -- start 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.562+0000 7f21f34d1700 1 -- start start 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.562+0000 7f21f34d1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 0x7f21ec193b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.562+0000 7f21f34d1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec194060 con 0x7f21ec0ff780 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.562+0000 7f21f126d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 0x7f21ec193b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.562+0000 7f21f126d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 0x7f21ec193b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:47450/0 (socket says 172.21.15.193:47450) 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.562+0000 7f21f126d700 1 -- 172.21.15.193:0/103352405 learned_addr learned my addr 172.21.15.193:0/103352405 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.563+0000 7f21f126d700 1 -- 172.21.15.193:0/103352405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc0095f0 con 0x7f21ec0ff780 2024-05-09T06:30:18.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.563+0000 7f21f126d700 1 --2- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 0x7f21ec193b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f21dc006010 tx=0x7f21dc009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.563+0000 7f21e27fc700 1 -- 172.21.15.193:0/103352405 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21dc01d070 con 0x7f21ec0ff780 2024-05-09T06:30:18.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.563+0000 7f21e27fc700 1 -- 172.21.15.193:0/103352405 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21dc005dc0 con 0x7f21ec0ff780 2024-05-09T06:30:18.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.563+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ec194260 con 0x7f21ec0ff780 2024-05-09T06:30:18.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.563+0000 7f21e27fc700 1 -- 172.21.15.193:0/103352405 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21dc022e90 con 0x7f21ec0ff780 2024-05-09T06:30:18.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.563+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ec194700 con 0x7f21ec0ff780 2024-05-09T06:30:18.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.564+0000 7f21e27fc700 1 -- 172.21.15.193:0/103352405 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f21dc00b960 con 0x7f21ec0ff780 2024-05-09T06:30:18.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.564+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ec18d4a0 con 0x7f21ec0ff780 2024-05-09T06:30:18.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.564+0000 7f21e27fc700 1 --2- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21d803aaf0 0x7f21d803cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:18.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.564+0000 7f21e27fc700 1 -- 172.21.15.193:0/103352405 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f21dc04ca40 con 0x7f21ec0ff780 2024-05-09T06:30:18.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.564+0000 7f21f0a6c700 1 --2- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21d803aaf0 0x7f21d803cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:18.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.566+0000 7f21f0a6c700 1 --2- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21d803aaf0 0x7f21d803cfb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21e8006fd0 tx=0x7f21e8006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:18.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.569+0000 7f21e27fc700 1 -- 172.21.15.193:0/103352405 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f21dc022750 con 0x7f21ec0ff780 2024-05-09T06:30:18.824 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.842+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f21ec062520 con 0x7f21ec0ff780 2024-05-09T06:30:18.824 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.843+0000 7f21e27fc700 1 -- 172.21.15.193:0/103352405 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f21ec062520 con 0x7f21ec0ff780 2024-05-09T06:30:18.824 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:18.825 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:18.826 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21d803aaf0 msgr2=0x7f21d803cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 --2- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21d803aaf0 0x7f21d803cfb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21e8006fd0 tx=0x7f21e8006e40 comp rx=0 tx=0).stop 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 msgr2=0x7f21ec193b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 --2- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 0x7f21ec193b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f21dc006010 tx=0x7f21dc009f80 comp rx=0 tx=0).stop 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 shutdown_connections 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 --2- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21d803aaf0 0x7f21d803cfb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 --2- 172.21.15.193:0/103352405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0ff780 0x7f21ec193b20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 >> 172.21.15.193:0/103352405 conn(0x7f21ec0fad20 msgr2=0x7f21ec0fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 shutdown_connections 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:18.845+0000 7f21f34d1700 1 -- 172.21.15.193:0/103352405 wait complete. 2024-05-09T06:30:18.827 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:19.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:19.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:19.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:19.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:19.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:30:20.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:19 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:20.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:19 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:20.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:19 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/103352405' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:20.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:19 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:20.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:19 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:20.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:19 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:30:20.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:19 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:20.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:19 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:30:20.411 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:20.411 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:20.609 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:21.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-09T06:30:21.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-09T06:30:21.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:21.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:21.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:30:21.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:30:21.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:30:21.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:30:21.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:21.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:21.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:21.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:21.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:21.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:30:21.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-09T06:30:21.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:20 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:22.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:21 smithi063 ceph-mon[27171]: Deploying daemon ceph-exporter.smithi193 on smithi193 2024-05-09T06:30:23.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.366+0000 7f58abe45700 1 -- 172.21.15.193:0/1103156615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 msgr2=0x7f58a40ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.366+0000 7f58abe45700 1 --2- 172.21.15.193:0/1103156615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 0x7f58a40ff350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009910 tx=0x7f58a0009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:23.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.366+0000 7f58abe45700 1 -- 172.21.15.193:0/1103156615 shutdown_connections 2024-05-09T06:30:23.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.366+0000 7f58abe45700 1 --2- 172.21.15.193:0/1103156615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 0x7f58a40ff350 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.366+0000 7f58abe45700 1 -- 172.21.15.193:0/1103156615 >> 172.21.15.193:0/1103156615 conn(0x7f58a40fa950 msgr2=0x7f58a40fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:23.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.366+0000 7f58abe45700 1 -- 172.21.15.193:0/1103156615 shutdown_connections 2024-05-09T06:30:23.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.366+0000 7f58abe45700 1 -- 172.21.15.193:0/1103156615 wait complete. 2024-05-09T06:30:23.350 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.367+0000 7f58abe45700 1 Processor -- start 2024-05-09T06:30:23.350 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.367+0000 7f58abe45700 1 -- start start 2024-05-09T06:30:23.350 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.367+0000 7f58abe45700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 0x7f58a40ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.350 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.367+0000 7f58abe45700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4101db0 con 0x7f58a40fef70 2024-05-09T06:30:23.350 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.367+0000 7f58a9be1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 0x7f58a40ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.350 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.367+0000 7f58a9be1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 0x7f58a40ffef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:48436/0 (socket says 172.21.15.193:48436) 2024-05-09T06:30:23.350 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.367+0000 7f58a9be1700 1 -- 172.21.15.193:0/3668351606 learned_addr learned my addr 172.21.15.193:0/3668351606 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:23.350 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.368+0000 7f58a9be1700 1 -- 172.21.15.193:0/3668351606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00095f0 con 0x7f58a40fef70 2024-05-09T06:30:23.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.368+0000 7f58a9be1700 1 --2- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 0x7f58a40ffef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f58a00098e0 tx=0x7f58a0005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.368+0000 7f589affd700 1 -- 172.21.15.193:0/3668351606 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58a001d070 con 0x7f58a40fef70 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.368+0000 7f589affd700 1 -- 172.21.15.193:0/3668351606 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58a00056b0 con 0x7f58a40fef70 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.368+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a4100430 con 0x7f58a40fef70 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.368+0000 7f589affd700 1 -- 172.21.15.193:0/3668351606 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58a0022d80 con 0x7f58a40fef70 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.368+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a4100850 con 0x7f58a40fef70 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.369+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a4062520 con 0x7f58a40fef70 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.369+0000 7f589affd700 1 -- 172.21.15.193:0/3668351606 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f58a0022470 con 0x7f58a40fef70 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.369+0000 7f589affd700 1 --2- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58900388b0 0x7f589003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.369+0000 7f589affd700 1 -- 172.21.15.193:0/3668351606 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f58a004ca10 con 0x7f58a40fef70 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.370+0000 7f58a93e0700 1 --2- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58900388b0 0x7f589003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:23.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.370+0000 7f58a93e0700 1 --2- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58900388b0 0x7f589003ad70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5894006fd0 tx=0x7f5894006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:23.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.375+0000 7f589affd700 1 -- 172.21.15.193:0/3668351606 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f58a002a360 con 0x7f58a40fef70 2024-05-09T06:30:23.658 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.676+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f58a4100a90 con 0x7f58a40fef70 2024-05-09T06:30:23.658 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.676+0000 7f589affd700 1 -- 172.21.15.193:0/3668351606 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f58a0027030 con 0x7f58a40fef70 2024-05-09T06:30:23.658 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:23.658 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:23.660 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58900388b0 msgr2=0x7f589003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.660 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 --2- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58900388b0 0x7f589003ad70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5894006fd0 tx=0x7f5894006e40 comp rx=0 tx=0).stop 2024-05-09T06:30:23.660 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 msgr2=0x7f58a40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:23.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 --2- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 0x7f58a40ffef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f58a00098e0 tx=0x7f58a0005520 comp rx=0 tx=0).stop 2024-05-09T06:30:23.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 shutdown_connections 2024-05-09T06:30:23.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 --2- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58900388b0 0x7f589003ad70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 --2- 172.21.15.193:0/3668351606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58a40fef70 0x7f58a40ffef0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:23.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 >> 172.21.15.193:0/3668351606 conn(0x7f58a40fa950 msgr2=0x7f58a4102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:23.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 shutdown_connections 2024-05-09T06:30:23.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:23.679+0000 7f58abe45700 1 -- 172.21.15.193:0/3668351606 wait complete. 2024-05-09T06:30:23.661 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:24.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:23 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/3668351606' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:25.260 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:25.260 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:25.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:25 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:25.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:25 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:25.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:25 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:25.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:25 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:25.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:25 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:30:25.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:25 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-09T06:30:25.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:25 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:25.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:25 smithi063 ceph-mon[27171]: Deploying daemon crash.smithi193 on smithi193 2024-05-09T06:30:26.008 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:27.587 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.601+0000 7f798e14b700 1 -- 172.21.15.193:0/257812994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 msgr2=0x7f7988106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.587 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.601+0000 7f798e14b700 1 --2- 172.21.15.193:0/257812994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 0x7f7988106be0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7978009910 tx=0x7f7978009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:27.587 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.601+0000 7f798e14b700 1 -- 172.21.15.193:0/257812994 shutdown_connections 2024-05-09T06:30:27.587 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.601+0000 7f798e14b700 1 --2- 172.21.15.193:0/257812994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 0x7f7988106be0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.587 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.601+0000 7f798e14b700 1 -- 172.21.15.193:0/257812994 >> 172.21.15.193:0/257812994 conn(0x7f79880fc310 msgr2=0x7f79880fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:27.588 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.601+0000 7f798e14b700 1 -- 172.21.15.193:0/257812994 shutdown_connections 2024-05-09T06:30:27.588 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.601+0000 7f798e14b700 1 -- 172.21.15.193:0/257812994 wait complete. 2024-05-09T06:30:27.588 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.602+0000 7f798e14b700 1 Processor -- start 2024-05-09T06:30:27.588 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.602+0000 7f798e14b700 1 -- start start 2024-05-09T06:30:27.588 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.602+0000 7f798e14b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 0x7f79880731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.588 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.602+0000 7f798e14b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880750b0 con 0x7f7988106800 2024-05-09T06:30:27.588 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f79877fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 0x7f79880731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f79877fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 0x7f79880731f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:48456/0 (socket says 172.21.15.193:48456) 2024-05-09T06:30:27.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f79877fe700 1 -- 172.21.15.193:0/1972472266 learned_addr learned my addr 172.21.15.193:0/1972472266 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:27.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f79877fe700 1 -- 172.21.15.193:0/1972472266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79780095f0 con 0x7f7988106800 2024-05-09T06:30:27.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f79877fe700 1 --2- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 0x7f79880731f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7978005490 tx=0x7f7978005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:27.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f7984ff9700 1 -- 172.21.15.193:0/1972472266 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f797801d070 con 0x7f7988106800 2024-05-09T06:30:27.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7988073730 con 0x7f7988106800 2024-05-09T06:30:27.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f7984ff9700 1 -- 172.21.15.193:0/1972472266 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7978005700 con 0x7f7988106800 2024-05-09T06:30:27.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7988073bb0 con 0x7f7988106800 2024-05-09T06:30:27.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.603+0000 7f7984ff9700 1 -- 172.21.15.193:0/1972472266 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7978003780 con 0x7f7988106800 2024-05-09T06:30:27.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.604+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7988101f40 con 0x7f7988106800 2024-05-09T06:30:27.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.604+0000 7f7984ff9700 1 -- 172.21.15.193:0/1972472266 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f797800b6f0 con 0x7f7988106800 2024-05-09T06:30:27.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.604+0000 7f7984ff9700 1 --2- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f79700388c0 0x7f797003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:27.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.604+0000 7f7984ff9700 1 -- 172.21.15.193:0/1972472266 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f797804c3a0 con 0x7f7988106800 2024-05-09T06:30:27.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.605+0000 7f7986ffd700 1 --2- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f79700388c0 0x7f797003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:27.590 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.605+0000 7f7986ffd700 1 --2- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f79700388c0 0x7f797003ad80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f797c006fd0 tx=0x7f797c006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:27.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.610+0000 7f7984ff9700 1 -- 172.21.15.193:0/1972472266 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7978004240 con 0x7f7988106800 2024-05-09T06:30:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:30:27.875 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.894+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7988073e50 con 0x7f7988106800 2024-05-09T06:30:27.876 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.894+0000 7f7984ff9700 1 -- 172.21.15.193:0/1972472266 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f7978026020 con 0x7f7988106800 2024-05-09T06:30:27.876 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:27.876 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:27.878 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f79700388c0 msgr2=0x7f797003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.878 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 --2- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f79700388c0 0x7f797003ad80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f797c006fd0 tx=0x7f797c006e40 comp rx=0 tx=0).stop 2024-05-09T06:30:27.878 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 msgr2=0x7f79880731f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:27.878 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 --2- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 0x7f79880731f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7978005490 tx=0x7f7978005570 comp rx=0 tx=0).stop 2024-05-09T06:30:27.878 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 shutdown_connections 2024-05-09T06:30:27.878 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 --2- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f79700388c0 0x7f797003ad80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.878 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 --2- 172.21.15.193:0/1972472266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7988106800 0x7f79880731f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:27.879 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 >> 172.21.15.193:0/1972472266 conn(0x7f79880fc310 msgr2=0x7f79880fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:27.879 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 shutdown_connections 2024-05-09T06:30:27.879 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:27.897+0000 7f798e14b700 1 -- 172.21.15.193:0/1972472266 wait complete. 2024-05-09T06:30:27.879 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:28 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/1972472266' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:28.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:28.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:28.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:28.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:28.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:28 smithi063 ceph-mon[27171]: Deploying daemon node-exporter.smithi193 on smithi193 2024-05-09T06:30:29.597 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:29.597 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:29.799 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:31.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.592+0000 7f1486993700 1 -- 172.21.15.193:0/854048961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 msgr2=0x7f1480106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.592+0000 7f1486993700 1 --2- 172.21.15.193:0/854048961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 0x7f1480106bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1470009910 tx=0x7f1470009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:31.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.593+0000 7f1486993700 1 -- 172.21.15.193:0/854048961 shutdown_connections 2024-05-09T06:30:31.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.593+0000 7f1486993700 1 --2- 172.21.15.193:0/854048961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 0x7f1480106bf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.593+0000 7f1486993700 1 -- 172.21.15.193:0/854048961 >> 172.21.15.193:0/854048961 conn(0x7f14800752a0 msgr2=0x7f14800756b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.593+0000 7f1486993700 1 -- 172.21.15.193:0/854048961 shutdown_connections 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.593+0000 7f1486993700 1 -- 172.21.15.193:0/854048961 wait complete. 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.593+0000 7f1486993700 1 Processor -- start 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.593+0000 7f1486993700 1 -- start start 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.593+0000 7f1486993700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 0x7f14800ff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.593+0000 7f1486993700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801016b0 con 0x7f1480106810 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.594+0000 7f147ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 0x7f14800ff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.594+0000 7f147ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 0x7f14800ff7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:44160/0 (socket says 172.21.15.193:44160) 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.594+0000 7f147ffff700 1 -- 172.21.15.193:0/1173149847 learned_addr learned my addr 172.21.15.193:0/1173149847 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.595+0000 7f147ffff700 1 -- 172.21.15.193:0/1173149847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14700095f0 con 0x7f1480106810 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.598+0000 7f147ffff700 1 --2- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 0x7f14800ff7f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1470005490 tx=0x7f1470005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.598+0000 7f147dffb700 1 -- 172.21.15.193:0/1173149847 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f147001d070 con 0x7f1480106810 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.599+0000 7f147dffb700 1 -- 172.21.15.193:0/1173149847 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1470005700 con 0x7f1480106810 2024-05-09T06:30:31.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.599+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14800ffd30 con 0x7f1480106810 2024-05-09T06:30:31.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.599+0000 7f147dffb700 1 -- 172.21.15.193:0/1173149847 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14700038e0 con 0x7f1480106810 2024-05-09T06:30:31.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.599+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14801001b0 con 0x7f1480106810 2024-05-09T06:30:31.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.599+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14800feee0 con 0x7f1480106810 2024-05-09T06:30:31.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.600+0000 7f147dffb700 1 -- 172.21.15.193:0/1173149847 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f1470003ba0 con 0x7f1480106810 2024-05-09T06:30:31.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.600+0000 7f147dffb700 1 --2- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f146c0388c0 0x7f146c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:31.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.600+0000 7f147dffb700 1 -- 172.21.15.193:0/1173149847 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f147004c6a0 con 0x7f1480106810 2024-05-09T06:30:31.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.600+0000 7f1477fff700 1 --2- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f146c0388c0 0x7f146c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:31.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.601+0000 7f1477fff700 1 --2- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f146c0388c0 0x7f146c03ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1468006fd0 tx=0x7f1468006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:31.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.605+0000 7f147dffb700 1 -- 172.21.15.193:0/1173149847 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f147002a980 con 0x7f1480106810 2024-05-09T06:30:31.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.911+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1480101f50 con 0x7f1480106810 2024-05-09T06:30:31.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.912+0000 7f147dffb700 1 -- 172.21.15.193:0/1173149847 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1470026030 con 0x7f1480106810 2024-05-09T06:30:31.893 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:31.894 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f146c0388c0 msgr2=0x7f146c03ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 --2- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f146c0388c0 0x7f146c03ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1468006fd0 tx=0x7f1468006e40 comp rx=0 tx=0).stop 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 msgr2=0x7f14800ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 --2- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 0x7f14800ff7f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1470005490 tx=0x7f1470005570 comp rx=0 tx=0).stop 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 shutdown_connections 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 --2- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f146c0388c0 0x7f146c03ad80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 --2- 172.21.15.193:0/1173149847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1480106810 0x7f14800ff7f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 >> 172.21.15.193:0/1173149847 conn(0x7f14800752a0 msgr2=0x7f14800fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 shutdown_connections 2024-05-09T06:30:31.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:31.914+0000 7f1486993700 1 -- 172.21.15.193:0/1173149847 wait complete. 2024-05-09T06:30:31.897 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:32.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:32 smithi063 ceph-mon[27171]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:32.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:32 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:32.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:32 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/1173149847' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:33.552 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:33.553 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:34.184 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:35.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:35 smithi063 ceph-mon[27171]: Deploying daemon mgr.smithi193.mmmqzh on smithi193 2024-05-09T06:30:35.658 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.676+0000 7f9dbbd39700 1 -- 172.21.15.193:0/3700715575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 msgr2=0x7f9db4105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.676+0000 7f9dbbd39700 1 --2- 172.21.15.193:0/3700715575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 0x7f9db4105b00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009910 tx=0x7f9db0009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.676+0000 7f9dbbd39700 1 -- 172.21.15.193:0/3700715575 shutdown_connections 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.676+0000 7f9dbbd39700 1 --2- 172.21.15.193:0/3700715575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 0x7f9db4105b00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.676+0000 7f9dbbd39700 1 -- 172.21.15.193:0/3700715575 >> 172.21.15.193:0/3700715575 conn(0x7f9db40fb250 msgr2=0x7f9db40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.676+0000 7f9dbbd39700 1 -- 172.21.15.193:0/3700715575 shutdown_connections 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.676+0000 7f9dbbd39700 1 -- 172.21.15.193:0/3700715575 wait complete. 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.676+0000 7f9dbbd39700 1 Processor -- start 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.676+0000 7f9dbbd39700 1 -- start start 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.677+0000 7f9dbbd39700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 0x7f9db4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.677+0000 7f9dbbd39700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db4069bc0 con 0x7f9db4105720 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.677+0000 7f9db9ad5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 0x7f9db4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.677+0000 7f9db9ad5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 0x7f9db4196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:44178/0 (socket says 172.21.15.193:44178) 2024-05-09T06:30:35.659 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.677+0000 7f9db9ad5700 1 -- 172.21.15.193:0/2862142128 learned_addr learned my addr 172.21.15.193:0/2862142128 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:35.660 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.677+0000 7f9db9ad5700 1 -- 172.21.15.193:0/2862142128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db00095f0 con 0x7f9db4105720 2024-05-09T06:30:35.660 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.677+0000 7f9db9ad5700 1 --2- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 0x7f9db4196f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9db0005490 tx=0x7f9db0005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:35.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.677+0000 7f9daaffd700 1 -- 172.21.15.193:0/2862142128 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9db001d070 con 0x7f9db4105720 2024-05-09T06:30:35.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.678+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db41974d0 con 0x7f9db4105720 2024-05-09T06:30:35.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.678+0000 7f9daaffd700 1 -- 172.21.15.193:0/2862142128 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9db0005700 con 0x7f9db4105720 2024-05-09T06:30:35.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.678+0000 7f9daaffd700 1 -- 172.21.15.193:0/2862142128 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9db0003950 con 0x7f9db4105720 2024-05-09T06:30:35.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.678+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db4197950 con 0x7f9db4105720 2024-05-09T06:30:35.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.678+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db4100e60 con 0x7f9db4105720 2024-05-09T06:30:35.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.678+0000 7f9daaffd700 1 -- 172.21.15.193:0/2862142128 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f9db0004240 con 0x7f9db4105720 2024-05-09T06:30:35.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.679+0000 7f9daaffd700 1 --2- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9da00388c0 0x7f9da003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:35.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.679+0000 7f9daaffd700 1 -- 172.21.15.193:0/2862142128 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f9db004c5f0 con 0x7f9db4105720 2024-05-09T06:30:35.662 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.679+0000 7f9db92d4700 1 --2- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9da00388c0 0x7f9da003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:35.662 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.680+0000 7f9db92d4700 1 --2- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9da00388c0 0x7f9da003ad80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9da4006fd0 tx=0x7f9da4006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:35.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.684+0000 7f9daaffd700 1 -- 172.21.15.193:0/2862142128 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9db002a980 con 0x7f9db4105720 2024-05-09T06:30:35.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.975+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9db4197c50 con 0x7f9db4105720 2024-05-09T06:30:35.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.976+0000 7f9daaffd700 1 -- 172.21.15.193:0/2862142128 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f9db0026020 con 0x7f9db4105720 2024-05-09T06:30:35.958 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:35.958 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:35.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9da00388c0 msgr2=0x7f9da003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 --2- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9da00388c0 0x7f9da003ad80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9da4006fd0 tx=0x7f9da4006e40 comp rx=0 tx=0).stop 2024-05-09T06:30:35.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 msgr2=0x7f9db4196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:35.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 --2- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 0x7f9db4196f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9db0005490 tx=0x7f9db0005570 comp rx=0 tx=0).stop 2024-05-09T06:30:35.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 shutdown_connections 2024-05-09T06:30:35.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 --2- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9da00388c0 0x7f9da003ad80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 --2- 172.21.15.193:0/2862142128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db4105720 0x7f9db4196f90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:35.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 >> 172.21.15.193:0/2862142128 conn(0x7f9db40fb250 msgr2=0x7f9db41073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:35.961 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 shutdown_connections 2024-05-09T06:30:35.961 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:35.978+0000 7f9dbbd39700 1 -- 172.21.15.193:0/2862142128 wait complete. 2024-05-09T06:30:35.961 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:36.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:36 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/2862142128' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:37.541 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:37.542 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:37 smithi063 ceph-mon[27171]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:37 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:38.183 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:39.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:39 smithi063 ceph-mon[27171]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:39.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:39.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:39.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:39.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:39.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:30:39.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:39.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:39 smithi063 ceph-mon[27171]: Deploying daemon mon.smithi193 on smithi193 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.736+0000 7f0352c5b700 1 -- 172.21.15.193:0/3853123794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 msgr2=0x7f034c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.736+0000 7f0352c5b700 1 --2- 172.21.15.193:0/3853123794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 0x7f034c106be0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0340009910 tx=0x7f0340009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.736+0000 7f0352c5b700 1 -- 172.21.15.193:0/3853123794 shutdown_connections 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.736+0000 7f0352c5b700 1 --2- 172.21.15.193:0/3853123794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 0x7f034c106be0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.736+0000 7f0352c5b700 1 -- 172.21.15.193:0/3853123794 >> 172.21.15.193:0/3853123794 conn(0x7f034c0fc2f0 msgr2=0x7f034c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.736+0000 7f0352c5b700 1 -- 172.21.15.193:0/3853123794 shutdown_connections 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.736+0000 7f0352c5b700 1 -- 172.21.15.193:0/3853123794 wait complete. 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.736+0000 7f0352c5b700 1 Processor -- start 2024-05-09T06:30:39.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.736+0000 7f0352c5b700 1 -- start start 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.737+0000 7f0352c5b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 0x7f034c1960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.737+0000 7f0352c5b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c196600 con 0x7f034c106800 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.737+0000 7f03509f7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 0x7f034c1960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.737+0000 7f03509f7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 0x7f034c1960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:44226/0 (socket says 172.21.15.193:44226) 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.737+0000 7f03509f7700 1 -- 172.21.15.193:0/3293126628 learned_addr learned my addr 172.21.15.193:0/3293126628 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.737+0000 7f03509f7700 1 -- 172.21.15.193:0/3293126628 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03400095f0 con 0x7f034c106800 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.738+0000 7f03509f7700 1 --2- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 0x7f034c1960c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0340005490 tx=0x7f0340005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.738+0000 7f0349ffb700 1 -- 172.21.15.193:0/3293126628 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f034001d070 con 0x7f034c106800 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.738+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c196860 con 0x7f034c106800 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.738+0000 7f0349ffb700 1 -- 172.21.15.193:0/3293126628 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0340005700 con 0x7f034c106800 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.738+0000 7f0349ffb700 1 -- 172.21.15.193:0/3293126628 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03400038e0 con 0x7f034c106800 2024-05-09T06:30:39.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.738+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c196ce0 con 0x7f034c106800 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.739+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c101f40 con 0x7f034c106800 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.739+0000 7f0349ffb700 1 -- 172.21.15.193:0/3293126628 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 45669+0+0 (secure 0 0 0) 0x7f0340003b50 con 0x7f034c106800 2024-05-09T06:30:39.721 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.739+0000 7f0349ffb700 1 --2- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f033c038910 0x7f033c03add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:39.722 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.739+0000 7f0349ffb700 1 -- 172.21.15.193:0/3293126628 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f034004c650 con 0x7f034c106800 2024-05-09T06:30:39.722 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.740+0000 7f034bfff700 1 --2- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f033c038910 0x7f033c03add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:39.722 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.740+0000 7f034bfff700 1 --2- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f033c038910 0x7f033c03add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0338006fd0 tx=0x7f0338006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:39.726 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:39.745+0000 7f0349ffb700 1 -- 172.21.15.193:0/3293126628 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f034002a590 con 0x7f034c106800 2024-05-09T06:30:40.015 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.034+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f034c10abf0 con 0x7f034c106800 2024-05-09T06:30:40.016 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.035+0000 7f0349ffb700 1 -- 172.21.15.193:0/3293126628 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f0340026070 con 0x7f034c106800 2024-05-09T06:30:40.016 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:40.016 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:27:48.192363Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T06:30:40.018 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f033c038910 msgr2=0x7f033c03add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.018 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 --2- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f033c038910 0x7f033c03add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0338006fd0 tx=0x7f0338006e40 comp rx=0 tx=0).stop 2024-05-09T06:30:40.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 msgr2=0x7f034c1960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:40.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 --2- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 0x7f034c1960c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0340005490 tx=0x7f0340005570 comp rx=0 tx=0).stop 2024-05-09T06:30:40.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 shutdown_connections 2024-05-09T06:30:40.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 --2- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f033c038910 0x7f033c03add0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 --2- 172.21.15.193:0/3293126628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f034c106800 0x7f034c1960c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:40.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 >> 172.21.15.193:0/3293126628 conn(0x7f034c0fc2f0 msgr2=0x7f034c1083d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:40.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 shutdown_connections 2024-05-09T06:30:40.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:40.037+0000 7f0352c5b700 1 -- 172.21.15.193:0/3293126628 wait complete. 2024-05-09T06:30:40.019 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-09T06:30:40.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:40 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/3293126628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:41.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:41 smithi063 ceph-mon[27171]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:42.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:30:42.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:43.100 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-09T06:30:43.101 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T06:30:43.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:42 smithi193 systemd[1]: Starting Ceph mon.smithi193 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:30:43.374 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 podman[35077]: 2024-05-09 06:30:43.11834807 +0000 UTC m=+0.018244300 image pull 196422f1b64117f056e4ad8be5d26eeefdfc0c98363316c4529effef2079a699 quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f 2024-05-09T06:30:43.375 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 podman[35077]: 2024-05-09 06:30:43.220258752 +0000 UTC m=+0.120154957 container create bb042f0315eb935cfb36dca64555b1b7784f5bbb2412aeb9d906c2e1f7600a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-1758e8e, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T06:30:43.562 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi193/config 2024-05-09T06:30:43.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:43 smithi063 ceph-mon[27171]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:43.647 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 podman[35077]: 2024-05-09 06:30:43.434351995 +0000 UTC m=+0.334248211 container init bb042f0315eb935cfb36dca64555b1b7784f5bbb2412aeb9d906c2e1f7600a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=reef-1758e8e, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T06:30:43.647 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 podman[35077]: 2024-05-09 06:30:43.439851047 +0000 UTC m=+0.339747260 container start bb042f0315eb935cfb36dca64555b1b7784f5bbb2412aeb9d906c2e1f7600a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, maintainer=Guillaume Abrioux , RELEASE=reef-1758e8e, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T06:30:43.647 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T06:30:43.647 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable), process ceph-mon, pid 2 2024-05-09T06:30:43.647 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: pidfile_write: ignore empty --pid-file 2024-05-09T06:30:43.647 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: load: jerasure load: lrc 2024-05-09T06:30:43.647 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: RocksDB version: 7.9.2 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Git sha 0 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Compile date 2024-05-04 20:27:46 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: DB SUMMARY 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: DB Session ID: 0W7NWTD1O886C9JS5LR2 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: CURRENT file: CURRENT 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi193/store.db dir, Total Num: 0, files: 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi193/store.db: 000004.log size: 511 ; 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.error_if_exists: 0 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.create_if_missing: 0 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.paranoid_checks: 1 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T06:30:43.648 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.env: 0x5642db5e6820 2024-05-09T06:30:43.649 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T06:30:43.649 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.info_log: 0x5642ddf373c0 2024-05-09T06:30:43.649 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T06:30:43.649 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.statistics: (nil) 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.use_fsync: 0 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_log_file_size: 0 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.allow_fallocate: 1 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.use_direct_reads: 0 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T06:30:43.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.db_log_dir: 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.wal_dir: 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.write_buffer_manager: 0x5642dd1ac140 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T06:30:43.651 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.unordered_write: 0 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.row_cache: None 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.wal_filter: None 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T06:30:43.652 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.two_write_queues: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.wal_compression: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.atomic_flush: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.log_readahead_size: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_background_jobs: 2 2024-05-09T06:30:43.653 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_background_compactions: -1 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_subcompactions: 1 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_open_files: -1 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T06:30:43.654 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_background_flushes: -1 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Compression algorithms supported: 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: kZSTD supported: 0 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: kXpressCompression supported: 0 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: kZlibCompression supported: 1 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: kSnappyCompression supported: 1 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: kLZ4Compression supported: 1 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: kBZip2Compression supported: 0 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000005 2024-05-09T06:30:43.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.merge_operator: 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_filter: None 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5642ddf374e0) 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_type: 0 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_index_type: 0 2024-05-09T06:30:43.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_shortening: 1 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: checksum: 4 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: no_block_cache: 0 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache: 0x5642dd24f1f0 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_options: 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: capacity : 536870912 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: num_shard_bits : 4 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: strict_capacity_limit : 0 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_compressed: (nil) 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: persistent_cache: (nil) 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size: 4096 2024-05-09T06:30:43.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size_deviation: 10 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_restart_interval: 16 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_block_restart_interval: 1 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: metadata_block_size: 4096 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: partition_filters: 0 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: use_delta_encoding: 1 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: filter_policy: bloomfilter 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: whole_key_filtering: 1 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: verify_compression: 0 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: format_version: 5 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_index_compression: 1 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_align: 0 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_auto_readahead_size: 262144 2024-05-09T06:30:43.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: prepopulate_block_cache: 0 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: initial_auto_readahead_size: 8192 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression: NoCompression 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.num_levels: 7 2024-05-09T06:30:43.659 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T06:30:43.660 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T06:30:43.661 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T06:30:43.662 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T06:30:43.662 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T06:30:43.662 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T06:30:43.663 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T06:30:43.664 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T06:30:43.664 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T06:30:43.664 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T06:30:43.664 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T06:30:43.664 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T06:30:43.664 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T06:30:43.664 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.table_properties_collectors: 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.inplace_update_support: 0 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.bloom_locality: 0 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.max_successive_merges: 0 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.ttl: 2592000 2024-05-09T06:30:43.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.enable_blob_files: false 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.min_blob_size: 0 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-09T06:30:43.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7e83f109-0ae1-4fa8-8338-a0979228845b 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236243470080, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236243470501, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715236243, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7e83f109-0ae1-4fa8-8338-a0979228845b", "db_session_id": "0W7NWTD1O886C9JS5LR2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236243470580, "job": 1, "event": "recovery_finished"} 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi193/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5642dd2ec000 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: DB pointer 0x5642dd2da000 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** DB Stats ** 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T06:30:43.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-05-09T06:30:43.668 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T06:30:43.669 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Block cache BinnedLRUCache@0x5642dd24f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193 does not exist in monmap, will attempt to join an existing cluster 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: starting mon.smithi193 rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi193 fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(???) e0 preinit fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).mds e1 new map 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).mds e1 print_map 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: e1 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: legacy client fscid: -1 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:30:43.670 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: No filesystems configured 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:43.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: Deploying daemon mon.smithi193 on smithi193 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: from='client.? 172.21.15.193:0/3293126628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 ceph-mon[35141]: mon.smithi193@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 bash[35077]: bb042f0315eb935cfb36dca64555b1b7784f5bbb2412aeb9d906c2e1f7600a9b 2024-05-09T06:30:43.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:43 smithi193 systemd[1]: Started Ceph mon.smithi193 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:30:48.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.748+0000 7f74a7e3c700 1 -- 172.21.15.193:0/2606919682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 msgr2=0x7f748c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:48.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.748+0000 7f74a7e3c700 1 --2- 172.21.15.193:0/2606919682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 0x7f748c005a50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f74a00780c0 tx=0x7f7490014ec0 comp rx=0 tx=0).stop 2024-05-09T06:30:48.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.748+0000 7f74a7e3c700 1 -- 172.21.15.193:0/2606919682 shutdown_connections 2024-05-09T06:30:48.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.748+0000 7f74a7e3c700 1 --2- 172.21.15.193:0/2606919682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 0x7f748c005a50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:48.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.748+0000 7f74a7e3c700 1 -- 172.21.15.193:0/2606919682 >> 172.21.15.193:0/2606919682 conn(0x7f74a0076df0 msgr2=0x7f74a0077200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:48.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.748+0000 7f74a7e3c700 1 -- 172.21.15.193:0/2606919682 shutdown_connections 2024-05-09T06:30:48.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.748+0000 7f74a7e3c700 1 -- 172.21.15.193:0/2606919682 wait complete. 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.749+0000 7f74a7e3c700 1 Processor -- start 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a7e3c700 1 -- start start 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a7e3c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 0x7f74a01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a7e3c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 0x7f74a019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a7e3c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a0199000 con 0x7f748c0035c0 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a7e3c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a0199170 con 0x7f74a01989e0 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a53d7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 0x7f74a019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a5bd8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 0x7f74a01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a5bd8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 0x7f74a01984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.193:34824/0 (socket says 172.21.15.193:34824) 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a53d7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 0x7f74a019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:47502/0 (socket says 172.21.15.193:47502) 2024-05-09T06:30:48.733 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a5bd8700 1 -- 172.21.15.193:0/1547372847 learned_addr learned my addr 172.21.15.193:0/1547372847 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a53d7700 1 -- 172.21.15.193:0/1547372847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 msgr2=0x7f74a019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a53d7700 1 -- 172.21.15.193:0/1547372847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 msgr2=0x7f74a019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a53d7700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 0x7f74a019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a53d7700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 0x7f74a019cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a5bd8700 1 -- 172.21.15.193:0/1547372847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 msgr2=0x7f74a019cd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a5bd8700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 0x7f74a019cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.750+0000 7f74a5bd8700 1 -- 172.21.15.193:0/1547372847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7490014910 con 0x7f748c0035c0 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.751+0000 7f74a5bd8700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 0x7f74a01984a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7490001eb0 tx=0x7f749000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.751+0000 7f7496ffd700 1 -- 172.21.15.193:0/1547372847 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7490023070 con 0x7f748c0035c0 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.751+0000 7f7496ffd700 1 -- 172.21.15.193:0/1547372847 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f749000fec0 con 0x7f748c0035c0 2024-05-09T06:30:48.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.751+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a019d390 con 0x7f748c0035c0 2024-05-09T06:30:48.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.751+0000 7f7496ffd700 1 -- 172.21.15.193:0/1547372847 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7490019040 con 0x7f748c0035c0 2024-05-09T06:30:48.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.751+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a019d8e0 con 0x7f748c0035c0 2024-05-09T06:30:48.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.752+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a0068290 con 0x7f748c0035c0 2024-05-09T06:30:48.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.753+0000 7f7496ffd700 1 -- 172.21.15.193:0/1547372847 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f7490004900 con 0x7f748c0035c0 2024-05-09T06:30:48.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.753+0000 7f7496ffd700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f748406cdb0 0x7f748406f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:48.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.753+0000 7f7496ffd700 1 -- 172.21.15.193:0/1547372847 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7490000e40 con 0x7f748c0035c0 2024-05-09T06:30:48.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.754+0000 7f74a53d7700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f748406cdb0 0x7f748406f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:48.736 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.754+0000 7f74a53d7700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f748406cdb0 0x7f748406f270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f749c009850 tx=0x7f749c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:48.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:48.758+0000 7f7496ffd700 1 -- 172.21.15.193:0/1547372847 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7490058b30 con 0x7f748c0035c0 2024-05-09T06:30:48.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:30:48.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: mon.smithi063 calling monitor election 2024-05-09T06:30:48.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:48.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/4222280275' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: mon.smithi193 calling monitor election 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: mon.smithi063 is new leader, mons smithi063,smithi193 in quorum (ranks 0,1) 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: monmap e2: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi193=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: fsmap 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: osdmap e5: 0 total, 0 up, 0 in 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: mgrmap e17: smithi063.egfguy(active, since 36s) 2024-05-09T06:30:48.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: overall HEALTH_OK 2024-05-09T06:30:48.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:30:48.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/4222280275' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:30:48.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:48.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/4222280275' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:30:48.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/4222280275' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:30:48.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:48.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:48.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:48.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:48.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:30:48.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: mon.smithi063 calling monitor election 2024-05-09T06:30:48.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:48.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/4222280275' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: mon.smithi193 calling monitor election 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: mon.smithi063 is new leader, mons smithi063,smithi193 in quorum (ranks 0,1) 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: monmap e2: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi193=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-09T06:30:48.874 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: fsmap 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: osdmap e5: 0 total, 0 up, 0 in 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: mgrmap e17: smithi063.egfguy(active, since 36s) 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: overall HEALTH_OK 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/4222280275' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/4222280275' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/4222280275' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:48.875 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:49.030 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.049+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f74a0062520 con 0x7f748c0035c0 2024-05-09T06:30:49.031 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.050+0000 7f7496ffd700 1 -- 172.21.15.193:0/1547372847 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1022 (secure 0 0 0) 0x7f749008cdc0 con 0x7f748c0035c0 2024-05-09T06:30:49.031 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-09T06:30:49.031 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","modified":"2024-05-09T06:30:43.500943Z","created":"2024-05-09T06:27:48.192363Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi193","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f748406cdb0 msgr2=0x7f748406f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f748406cdb0 0x7f748406f270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f749c009850 tx=0x7f749c008040 comp rx=0 tx=0).stop 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 msgr2=0x7f74a01984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 0x7f74a01984a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7490001eb0 tx=0x7f749000fc60 comp rx=0 tx=0).stop 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 shutdown_connections 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f748c0035c0 0x7f74a01984a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f748406cdb0 0x7f748406f270 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 --2- 172.21.15.193:0/1547372847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74a01989e0 0x7f74a019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 >> 172.21.15.193:0/1547372847 conn(0x7f74a0076df0 msgr2=0x7f74a00fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 shutdown_connections 2024-05-09T06:30:49.034 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:30:49.052+0000 7f74a7e3c700 1 -- 172.21.15.193:0/1547372847 wait complete. 2024-05-09T06:30:49.035 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-05-09T06:30:49.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:49 smithi193 ceph-mon[35141]: mgrmap e18: smithi063.egfguy(active, since 37s), standbys: smithi193.mmmqzh 2024-05-09T06:30:49.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:49 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mmmqzh", "id": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:30:49.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:49 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:30:49.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:49 smithi193 ceph-mon[35141]: from='client.? 172.21.15.193:0/1547372847' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:49.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:49 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:49.588 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T06:30:49.589 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T06:30:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:30:49.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:49 smithi063 ceph-mon[27171]: mgrmap e18: smithi063.egfguy(active, since 37s), standbys: smithi193.mmmqzh 2024-05-09T06:30:49.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:49 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mmmqzh", "id": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:30:49.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:49 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:30:49.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:49 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/1547372847' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T06:30:49.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:49 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:30:50.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:50 smithi063 ceph-mon[27171]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:50.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:50 smithi193 ceph-mon[35141]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:51.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.461+0000 7f9054c29700 1 -- 172.21.15.63:0/2798518089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 msgr2=0x7f90500ff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:51.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.461+0000 7f9054c29700 1 --2- 172.21.15.63:0/2798518089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 0x7f90500ff4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9038009910 tx=0x7f9038009c20 comp rx=0 tx=0).stop 2024-05-09T06:30:51.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.462+0000 7f9054c29700 1 -- 172.21.15.63:0/2798518089 shutdown_connections 2024-05-09T06:30:51.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.462+0000 7f9054c29700 1 --2- 172.21.15.63:0/2798518089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 0x7f90500ff4c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:51.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.462+0000 7f9054c29700 1 -- 172.21.15.63:0/2798518089 >> 172.21.15.63:0/2798518089 conn(0x7f90500fa950 msgr2=0x7f90500fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:51.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.462+0000 7f9054c29700 1 -- 172.21.15.63:0/2798518089 shutdown_connections 2024-05-09T06:30:51.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.462+0000 7f9054c29700 1 -- 172.21.15.63:0/2798518089 wait complete. 2024-05-09T06:30:51.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f9054c29700 1 Processor -- start 2024-05-09T06:30:51.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f9054c29700 1 -- start start 2024-05-09T06:30:51.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f9054c29700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 0x7f9050100b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:51.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f9054c29700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90501010c0 0x7f9050101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:51.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f9054c29700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9050102ad0 con 0x7f90500ff0e0 2024-05-09T06:30:51.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f9054c29700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90501a24f0 con 0x7f90501010c0 2024-05-09T06:30:51.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f904e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 0x7f9050100b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:51.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f904e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 0x7f9050100b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49566/0 (socket says 172.21.15.63:49566) 2024-05-09T06:30:51.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f904e59c700 1 -- 172.21.15.63:0/2615180941 learned_addr learned my addr 172.21.15.63:0/2615180941 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:30:51.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f904dd9b700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90501010c0 0x7f9050101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f904dd9b700 1 -- 172.21.15.63:0/2615180941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90501010c0 msgr2=0x7f9050101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T06:30:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f904dd9b700 1 -- 172.21.15.63:0/2615180941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90501010c0 msgr2=0x7f9050101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T06:30:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.463+0000 7f904dd9b700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90501010c0 0x7f9050101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:30:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f904dd9b700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90501010c0 0x7f9050101540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:30:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f904e59c700 1 -- 172.21.15.63:0/2615180941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90501010c0 msgr2=0x7f9050101540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:30:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f904e59c700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90501010c0 0x7f9050101540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f904e59c700 1 -- 172.21.15.63:0/2615180941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90380095f0 con 0x7f90500ff0e0 2024-05-09T06:30:51.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f904e59c700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 0x7f9050100b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9038005b40 tx=0x7f903800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:51.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f90477fe700 1 -- 172.21.15.63:0/2615180941 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903801d070 con 0x7f90500ff0e0 2024-05-09T06:30:51.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f90477fe700 1 -- 172.21.15.63:0/2615180941 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9038003d50 con 0x7f90500ff0e0 2024-05-09T06:30:51.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90501a2690 con 0x7f90500ff0e0 2024-05-09T06:30:51.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f90477fe700 1 -- 172.21.15.63:0/2615180941 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9038017440 con 0x7f90500ff0e0 2024-05-09T06:30:51.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.464+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90501a2b00 con 0x7f90500ff0e0 2024-05-09T06:30:51.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.465+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90501005c0 con 0x7f90500ff0e0 2024-05-09T06:30:51.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.466+0000 7f90477fe700 1 -- 172.21.15.63:0/2615180941 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f90380176b0 con 0x7f90500ff0e0 2024-05-09T06:30:51.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.466+0000 7f90477fe700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f903c06cdb0 0x7f903c06f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:30:51.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.466+0000 7f90477fe700 1 -- 172.21.15.63:0/2615180941 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f903808c430 con 0x7f90500ff0e0 2024-05-09T06:30:51.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.466+0000 7f904dd9b700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f903c06cdb0 0x7f903c06f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:30:51.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.467+0000 7f904dd9b700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f903c06cdb0 0x7f903c06f270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9040009850 tx=0x7f9040008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:30:51.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.471+0000 7f90477fe700 1 -- 172.21.15.63:0/2615180941 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9038057950 con 0x7f90500ff0e0 2024-05-09T06:30:51.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.706+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9050109be0 con 0x7f90500ff0e0 2024-05-09T06:30:51.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.707+0000 7f90477fe700 1 -- 172.21.15.63:0/2615180941 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f9038026100 con 0x7f90500ff0e0 2024-05-09T06:30:51.688 INFO:teuthology.orchestra.run.smithi063.stdout:# minimal ceph.conf for 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:30:51.688 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-09T06:30:51.688 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:30:51.688 INFO:teuthology.orchestra.run.smithi063.stdout: mon_host = [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-05-09T06:30:51.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f903c06cdb0 msgr2=0x7f903c06f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f903c06cdb0 0x7f903c06f270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9040009850 tx=0x7f9040008040 comp rx=0 tx=0).stop 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 msgr2=0x7f9050100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 0x7f9050100b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9038005b40 tx=0x7f903800bdc0 comp rx=0 tx=0).stop 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 shutdown_connections 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90500ff0e0 0x7f9050100b80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f903c06cdb0 0x7f903c06f270 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 --2- 172.21.15.63:0/2615180941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90501010c0 0x7f9050101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 >> 172.21.15.63:0/2615180941 conn(0x7f90500fa950 msgr2=0x7f90501088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 shutdown_connections 2024-05-09T06:30:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:30:51.709+0000 7f9054c29700 1 -- 172.21.15.63:0/2615180941 wait complete. 2024-05-09T06:30:52.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:52.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:30:52.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-09T06:30:52.272 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2615180941' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:30:52.273 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:52.293 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T06:30:52.293 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:30:52.293 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T06:30:52.323 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:30:52.323 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:30:52.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:52.374 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2615180941' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:52.374 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:30:52.374 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:30:52.374 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:52.389 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:30:52.389 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T06:30:52.418 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:30:52.418 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:30:52.484 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T06:30:52.484 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:30:52.484 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T06:30:52.501 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T06:30:52.501 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-05-09T06:30:52.558 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T06:30:52.559 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:30:52.559 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-09T06:30:52.559 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:30:52.559 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:30:52.559 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-09 06:30:02.709934259 +0000 2024-05-09T06:30:52.559 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-09 06:24:02.824018901 +0000 2024-05-09T06:30:52.559 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-09 06:24:02.824018901 +0000 2024-05-09T06:30:52.560 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-09 06:24:02.824018901 +0000 2024-05-09T06:30:52.560 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T06:30:52.624 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-09T06:30:52.625 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-09T06:30:52.625 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000348344 s, 1.5 MB/s 2024-05-09T06:30:52.626 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T06:30:52.683 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-05-09T06:30:52.739 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T06:30:52.739 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:30:52.740 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T06:30:52.740 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:30:52.740 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:30:52.740 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-09 06:30:02.709934259 +0000 2024-05-09T06:30:52.740 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-09 06:24:03.274014643 +0000 2024-05-09T06:30:52.740 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-09 06:24:03.274014643 +0000 2024-05-09T06:30:52.740 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-09 06:24:03.274014643 +0000 2024-05-09T06:30:52.740 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T06:30:52.805 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-09T06:30:52.805 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-09T06:30:52.805 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000197298 s, 2.6 MB/s 2024-05-09T06:30:52.807 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T06:30:52.864 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-05-09T06:30:52.920 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T06:30:52.920 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:30:52.920 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-09T06:30:52.920 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:30:52.921 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:30:52.921 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-09 06:30:02.710934249 +0000 2024-05-09T06:30:52.921 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-09 06:24:03.626011312 +0000 2024-05-09T06:30:52.921 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-09 06:24:03.626011312 +0000 2024-05-09T06:30:52.921 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-09 06:24:03.626011312 +0000 2024-05-09T06:30:52.921 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T06:30:52.984 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-09T06:30:52.984 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-09T06:30:52.985 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000212937 s, 2.4 MB/s 2024-05-09T06:30:52.986 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T06:30:53.044 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-05-09T06:30:53.100 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T06:30:53.100 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:30:53.100 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-09T06:30:53.101 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:30:53.101 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:30:53.101 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-09 06:30:02.710934249 +0000 2024-05-09T06:30:53.101 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-09 06:24:03.971008047 +0000 2024-05-09T06:30:53.101 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-09 06:24:03.971008047 +0000 2024-05-09T06:30:53.101 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-09 06:24:03.971008047 +0000 2024-05-09T06:30:53.101 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T06:30:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-09T06:30:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-09T06:30:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000185489 s, 2.8 MB/s 2024-05-09T06:30:53.166 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T06:30:53.223 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:30:53.223 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T06:30:53.241 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T06:30:53.241 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-05-09T06:30:53.298 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T06:30:53.298 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:30:53.298 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-09T06:30:53.298 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:30:53.298 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:30:53.298 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-09 06:30:42.928548585 +0000 2024-05-09T06:30:53.298 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-09 06:23:54.718241885 +0000 2024-05-09T06:30:53.298 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-09 06:23:54.718241885 +0000 2024-05-09T06:30:53.298 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-09 06:23:54.718241885 +0000 2024-05-09T06:30:53.298 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T06:30:53.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:53 smithi063 ceph-mon[27171]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:53.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:53 smithi063 ceph-mon[27171]: Reconfiguring mon.smithi063 (unknown last config time)... 2024-05-09T06:30:53.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:53 smithi063 ceph-mon[27171]: Reconfiguring daemon mon.smithi063 on smithi063 2024-05-09T06:30:53.360 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:53 smithi193 ceph-mon[35141]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:53.360 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:53 smithi193 ceph-mon[35141]: Reconfiguring mon.smithi063 (unknown last config time)... 2024-05-09T06:30:53.361 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:53 smithi193 ceph-mon[35141]: Reconfiguring daemon mon.smithi063 on smithi063 2024-05-09T06:30:53.363 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-09T06:30:53.363 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-09T06:30:53.364 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000252289 s, 2.0 MB/s 2024-05-09T06:30:53.364 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T06:30:53.423 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-05-09T06:30:53.479 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T06:30:53.479 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:30:53.480 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-09T06:30:53.480 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:30:53.480 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:30:53.480 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-09 06:30:42.929548570 +0000 2024-05-09T06:30:53.480 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-09 06:23:55.050236547 +0000 2024-05-09T06:30:53.480 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-09 06:23:55.050236547 +0000 2024-05-09T06:30:53.480 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-09 06:23:55.050236547 +0000 2024-05-09T06:30:53.480 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T06:30:53.548 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-09T06:30:53.549 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-09T06:30:53.549 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000317604 s, 1.6 MB/s 2024-05-09T06:30:53.550 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T06:30:53.608 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-05-09T06:30:53.664 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T06:30:53.664 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:30:53.664 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-09T06:30:53.664 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:30:53.664 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:30:53.664 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-09 06:30:42.929548570 +0000 2024-05-09T06:30:53.665 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-09 06:23:55.395231000 +0000 2024-05-09T06:30:53.665 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-09 06:23:55.395231000 +0000 2024-05-09T06:30:53.665 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-09 06:23:55.395231000 +0000 2024-05-09T06:30:53.665 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T06:30:53.726 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-09T06:30:53.727 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-09T06:30:53.727 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00020198 s, 2.5 MB/s 2024-05-09T06:30:53.728 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T06:30:53.784 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-05-09T06:30:53.839 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T06:30:53.840 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:30:53.840 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-09T06:30:53.840 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:30:53.840 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:30:53.840 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-09 06:30:42.929548570 +0000 2024-05-09T06:30:53.840 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-09 06:23:55.743225405 +0000 2024-05-09T06:30:53.840 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-09 06:23:55.743225405 +0000 2024-05-09T06:30:53.840 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-09 06:23:55.743225405 +0000 2024-05-09T06:30:53.840 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T06:30:53.904 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-09T06:30:53.904 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-09T06:30:53.904 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000236863 s, 2.2 MB/s 2024-05-09T06:30:53.905 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T06:30:53.962 INFO:tasks.cephadm:Deploying osd.0 on smithi063 with /dev/vg_nvme/lv_4... 2024-05-09T06:30:53.963 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T06:30:54.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:54 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:54.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:54 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:54.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:54 smithi063 ceph-mon[27171]: Reconfiguring mgr.smithi063.egfguy (unknown last config time)... 2024-05-09T06:30:54.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:54 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.egfguy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:30:54.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:54 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:30:54.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:54 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:54.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:54 smithi063 ceph-mon[27171]: Reconfiguring daemon mgr.smithi063.egfguy on smithi063 2024-05-09T06:30:54.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:54 smithi063 ceph-mon[27171]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:54.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:54 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:54.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:54 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:54.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:54 smithi193 ceph-mon[35141]: Reconfiguring mgr.smithi063.egfguy (unknown last config time)... 2024-05-09T06:30:54.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:54 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.egfguy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:30:54.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:54 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:30:54.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:54 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:54.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:54 smithi193 ceph-mon[35141]: Reconfiguring daemon mgr.smithi063.egfguy on smithi063 2024-05-09T06:30:54.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:54 smithi193 ceph-mon[35141]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:55.037 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:30:56.252 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:56.252 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:56.252 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:56 smithi063 ceph-mon[27171]: Reconfiguring ceph-exporter.smithi063 (monmap changed)... 2024-05-09T06:30:56.252 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:30:56.252 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:56.252 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:56 smithi063 ceph-mon[27171]: Reconfiguring daemon ceph-exporter.smithi063 on smithi063 2024-05-09T06:30:56.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:56.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:56.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:56 smithi193 ceph-mon[35141]: Reconfiguring ceph-exporter.smithi063 (monmap changed)... 2024-05-09T06:30:56.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:30:56.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:56.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:56 smithi193 ceph-mon[35141]: Reconfiguring daemon ceph-exporter.smithi063 on smithi063 2024-05-09T06:30:57.063 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T06:30:57.063 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T06:30:57.063 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-09T06:30:57.063 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-09T06:30:57.064 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0138261 s, 758 MB/s 2024-05-09T06:30:57.064 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 2024-05-09T06:30:57.064 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-09T06:30:57.167 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:57 smithi063 ceph-mon[27171]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:57.167 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:30:57.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:57 smithi193 ceph-mon[35141]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:57.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:30:58.156 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_4 2024-05-09T06:30:59.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:58 smithi063 ceph-mon[27171]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:59.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:58 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:59.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:58 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:59.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:58 smithi063 ceph-mon[27171]: Reconfiguring crash.smithi063 (monmap changed)... 2024-05-09T06:30:59.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:58 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:30:59.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:58 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:59.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:30:58 smithi063 ceph-mon[27171]: Reconfiguring daemon crash.smithi063 on smithi063 2024-05-09T06:30:59.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:58 smithi193 ceph-mon[35141]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:30:59.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:58 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:59.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:58 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:30:59.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:58 smithi193 ceph-mon[35141]: Reconfiguring crash.smithi063 (monmap changed)... 2024-05-09T06:30:59.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:58 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:30:59.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:58 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:30:59.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:30:58 smithi193 ceph-mon[35141]: Reconfiguring daemon crash.smithi063 on smithi063 2024-05-09T06:30:59.337 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:31:00.570 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:00 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:00.570 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:00 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:00.570 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:00 smithi063 ceph-mon[27171]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-05-09T06:31:00.570 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:00 smithi063 ceph-mon[27171]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-05-09T06:31:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:00 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:00 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:00 smithi193 ceph-mon[35141]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-05-09T06:31:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:00 smithi193 ceph-mon[35141]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-05-09T06:31:01.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:01 smithi063 ceph-mon[27171]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:01.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.630+0000 7fa125046700 1 -- 172.21.15.63:0/3616680279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120101e40 msgr2=0x7fa1201022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:01.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.630+0000 7fa125046700 1 --2- 172.21.15.63:0/3616680279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120101e40 0x7fa1201022a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa108009910 tx=0x7fa108009c20 comp rx=0 tx=0).stop 2024-05-09T06:31:01.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.630+0000 7fa125046700 1 -- 172.21.15.63:0/3616680279 shutdown_connections 2024-05-09T06:31:01.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.630+0000 7fa125046700 1 --2- 172.21.15.63:0/3616680279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120101e40 0x7fa1201022a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:01.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.630+0000 7fa125046700 1 --2- 172.21.15.63:0/3616680279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa120100c40 0x7fa120101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:01.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.630+0000 7fa125046700 1 -- 172.21.15.63:0/3616680279 >> 172.21.15.63:0/3616680279 conn(0x7fa1200fc1e0 msgr2=0x7fa1200fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:01.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.630+0000 7fa125046700 1 -- 172.21.15.63:0/3616680279 shutdown_connections 2024-05-09T06:31:01.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.630+0000 7fa125046700 1 -- 172.21.15.63:0/3616680279 wait complete. 2024-05-09T06:31:01.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa125046700 1 Processor -- start 2024-05-09T06:31:01.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa125046700 1 -- start start 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa125046700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120100c40 0x7fa120074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa125046700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa120101e40 0x7fa1200732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa125046700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa120073820 con 0x7fa120100c40 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa125046700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa120073960 con 0x7fa120101e40 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa11ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120100c40 0x7fa120074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa11ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120100c40 0x7fa120074bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45274/0 (socket says 172.21.15.63:45274) 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa11ed9d700 1 -- 172.21.15.63:0/4153586519 learned_addr learned my addr 172.21.15.63:0/4153586519 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.631+0000 7fa11e59c700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa120101e40 0x7fa1200732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.632+0000 7fa11ed9d700 1 -- 172.21.15.63:0/4153586519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa120101e40 msgr2=0x7fa1200732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.632+0000 7fa11ed9d700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa120101e40 0x7fa1200732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.632+0000 7fa11ed9d700 1 -- 172.21.15.63:0/4153586519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1080095f0 con 0x7fa120100c40 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.632+0000 7fa11ed9d700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120100c40 0x7fa120074bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b9c0 tx=0x7fa11000bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.632+0000 7fa117fff700 1 -- 172.21.15.63:0/4153586519 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa11000d8b0 con 0x7fa120100c40 2024-05-09T06:31:01.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.632+0000 7fa117fff700 1 -- 172.21.15.63:0/4153586519 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa110015420 con 0x7fa120100c40 2024-05-09T06:31:01.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.632+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa120073c40 con 0x7fa120100c40 2024-05-09T06:31:01.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.632+0000 7fa117fff700 1 -- 172.21.15.63:0/4153586519 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa11000c610 con 0x7fa120100c40 2024-05-09T06:31:01.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.632+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12019e7f0 con 0x7fa120100c40 2024-05-09T06:31:01.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.633+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa120190860 con 0x7fa120100c40 2024-05-09T06:31:01.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.634+0000 7fa117fff700 1 -- 172.21.15.63:0/4153586519 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7fa110015590 con 0x7fa120100c40 2024-05-09T06:31:01.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.634+0000 7fa117fff700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa10c06cc60 0x7fa10c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:01.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.634+0000 7fa117fff700 1 -- 172.21.15.63:0/4153586519 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa110011070 con 0x7fa120100c40 2024-05-09T06:31:01.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.634+0000 7fa11e59c700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa10c06cc60 0x7fa10c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:01.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.635+0000 7fa11e59c700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa10c06cc60 0x7fa10c06f120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa108005090 tx=0x7fa108005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:01.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.639+0000 7fa117fff700 1 -- 172.21.15.63:0/4153586519 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa11004fc50 con 0x7fa120100c40 2024-05-09T06:31:01.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:01 smithi193 ceph-mon[35141]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:01.853+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa120061330 con 0x7fa10c06cc60 2024-05-09T06:31:02.524 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:02 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:31:02.524 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:02 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:31:02.524 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:02 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:02.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:02 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:31:02.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:02 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:31:02.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:02 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:03.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:03 smithi063 ceph-mon[27171]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:03.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:03 smithi063 ceph-mon[27171]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:31:03.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:03 smithi193 ceph-mon[35141]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:03.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:03 smithi193 ceph-mon[35141]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:31:05.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:05 smithi063 ceph-mon[27171]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:05.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:05 smithi193 ceph-mon[35141]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:07.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:07 smithi063 ceph-mon[27171]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:07.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:07 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:07.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:07 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:07.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:07 smithi063 ceph-mon[27171]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-05-09T06:31:07.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:07 smithi063 ceph-mon[27171]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-05-09T06:31:07.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:07 smithi193 ceph-mon[35141]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:07.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:07 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:07.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:07 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:07.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:07 smithi193 ceph-mon[35141]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-05-09T06:31:07.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:07 smithi193 ceph-mon[35141]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-05-09T06:31:08.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:08 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2620619150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eea045d9-973d-4960-a34e-689f9403d55f"}]: dispatch 2024-05-09T06:31:08.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:08 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2620619150' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eea045d9-973d-4960-a34e-689f9403d55f"}]': finished 2024-05-09T06:31:08.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:08 smithi063 ceph-mon[27171]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T06:31:08.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:08.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:08 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2067227056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:31:08.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:08 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2620619150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eea045d9-973d-4960-a34e-689f9403d55f"}]: dispatch 2024-05-09T06:31:08.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:08 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2620619150' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eea045d9-973d-4960-a34e-689f9403d55f"}]': finished 2024-05-09T06:31:08.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:08 smithi193 ceph-mon[35141]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T06:31:08.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:08.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:08 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2067227056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:31:09.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:09 smithi063 ceph-mon[27171]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:09 smithi193 ceph-mon[35141]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:11.596 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:11 smithi063 ceph-mon[27171]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:11.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:11 smithi193 ceph-mon[35141]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:12.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:12 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:31:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:12 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:31:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:13 smithi063 ceph-mon[27171]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:13.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:13 smithi193 ceph-mon[35141]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:15.495 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:15 smithi063 ceph-mon[27171]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:15.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:15 smithi193 ceph-mon[35141]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:17.574 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:17 smithi063 ceph-mon[27171]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:17.574 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:17 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:17.574 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:17 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:17.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:17 smithi193 ceph-mon[35141]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:17.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:17 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:17.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:17 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:18.423 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:18 smithi063 ceph-mon[27171]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-05-09T06:31:18.423 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:18 smithi063 ceph-mon[27171]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-05-09T06:31:18.423 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T06:31:18.423 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:18.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:18 smithi193 ceph-mon[35141]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-05-09T06:31:18.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:18 smithi193 ceph-mon[35141]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-05-09T06:31:18.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T06:31:18.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:19.457 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:19 smithi063 ceph-mon[27171]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:19.457 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:19 smithi063 ceph-mon[27171]: Deploying daemon osd.0 on smithi063 2024-05-09T06:31:19.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:19 smithi193 ceph-mon[35141]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:19.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:19 smithi193 ceph-mon[35141]: Deploying daemon osd.0 on smithi063 2024-05-09T06:31:21.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:21 smithi063 ceph-mon[27171]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:21.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:21 smithi193 ceph-mon[35141]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:23 smithi193 ceph-mon[35141]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:23 smithi193 ceph-mon[35141]: Reconfiguring ceph-exporter.smithi193 (monmap changed)... 2024-05-09T06:31:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:31:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:23 smithi193 ceph-mon[35141]: Reconfiguring daemon ceph-exporter.smithi193 on smithi193 2024-05-09T06:31:23.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:23 smithi063 ceph-mon[27171]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:23 smithi063 ceph-mon[27171]: Reconfiguring ceph-exporter.smithi193 (monmap changed)... 2024-05-09T06:31:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:31:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:23 smithi063 ceph-mon[27171]: Reconfiguring daemon ceph-exporter.smithi193 on smithi193 2024-05-09T06:31:25.010 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:24 smithi193 ceph-mon[35141]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:25.010 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:24 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:25.010 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:24 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:25.011 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:24 smithi193 ceph-mon[35141]: Reconfiguring crash.smithi193 (monmap changed)... 2024-05-09T06:31:25.011 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:24 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:31:25.011 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:24 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:25.011 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:24 smithi193 ceph-mon[35141]: Reconfiguring daemon crash.smithi193 on smithi193 2024-05-09T06:31:25.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:24 smithi063 ceph-mon[27171]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:25.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:24 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:25.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:24 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:25.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:24 smithi063 ceph-mon[27171]: Reconfiguring crash.smithi193 (monmap changed)... 2024-05-09T06:31:25.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:24 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:31:25.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:24 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:25.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:24 smithi063 ceph-mon[27171]: Reconfiguring daemon crash.smithi193 on smithi193 2024-05-09T06:31:26.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:26 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:26.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:26 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:26.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:26 smithi063 ceph-mon[27171]: Reconfiguring mgr.smithi193.mmmqzh (monmap changed)... 2024-05-09T06:31:26.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:26 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:31:26.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:26 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:31:26.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:26 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:26.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:26 smithi063 ceph-mon[27171]: Reconfiguring daemon mgr.smithi193.mmmqzh on smithi193 2024-05-09T06:31:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:26 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:26.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:26 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:26.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:26 smithi193 ceph-mon[35141]: Reconfiguring mgr.smithi193.mmmqzh (monmap changed)... 2024-05-09T06:31:26.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:26 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:31:26.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:26 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:31:26.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:26 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:26.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:26 smithi193 ceph-mon[35141]: Reconfiguring daemon mgr.smithi193.mmmqzh on smithi193 2024-05-09T06:31:27.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:27 smithi193 ceph-mon[35141]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:27.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:31:27.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:27.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:27.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:31:27.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:31:27.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:27.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:27.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:27 smithi063 ceph-mon[27171]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:27.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:31:27.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:27.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:27.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:31:27.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:31:27.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:27.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:27.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: Reconfiguring mon.smithi193 (monmap changed)... 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: Reconfiguring daemon mon.smithi193 on smithi193 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: Reconfiguring mon.smithi193 (monmap changed)... 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: Reconfiguring daemon mon.smithi193 on smithi193 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:31:28.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-05-09T06:31:28.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:28.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:31:29.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:29 smithi193 ceph-mon[35141]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:29.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:29 smithi193 ceph-mon[35141]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-09T06:31:29.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:29 smithi193 ceph-mon[35141]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-05-09T06:31:29.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:29 smithi193 ceph-mon[35141]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-09T06:31:29.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:29 smithi193 ceph-mon[35141]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-05-09T06:31:29.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:29 smithi193 ceph-mon[35141]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:31:29.588 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:29 smithi193 ceph-mon[35141]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-05-09T06:31:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:29 smithi063 ceph-mon[27171]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:29 smithi063 ceph-mon[27171]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-09T06:31:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:29 smithi063 ceph-mon[27171]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-05-09T06:31:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:29 smithi063 ceph-mon[27171]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-09T06:31:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:29 smithi063 ceph-mon[27171]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-05-09T06:31:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:29 smithi063 ceph-mon[27171]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:31:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:29 smithi063 ceph-mon[27171]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-05-09T06:31:30.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.178+0000 7fa117fff700 1 -- 172.21.15.63:0/4153586519 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa120061330 con 0x7fa10c06cc60 2024-05-09T06:31:30.160 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 0 on host 'smithi063' 2024-05-09T06:31:30.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa10c06cc60 msgr2=0x7fa10c06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa10c06cc60 0x7fa10c06f120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa108005090 tx=0x7fa108005960 comp rx=0 tx=0).stop 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120100c40 msgr2=0x7fa120074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120100c40 0x7fa120074bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b9c0 tx=0x7fa11000bcd0 comp rx=0 tx=0).stop 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 shutdown_connections 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa120100c40 0x7fa120074bf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa10c06cc60 0x7fa10c06f120 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 --2- 172.21.15.63:0/4153586519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa120101e40 0x7fa1200732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 >> 172.21.15.63:0/4153586519 conn(0x7fa1200fc1e0 msgr2=0x7fa120105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 shutdown_connections 2024-05-09T06:31:30.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:30.180+0000 7fa125046700 1 -- 172.21.15.63:0/4153586519 wait complete. 2024-05-09T06:31:31.079 DEBUG:teuthology.orchestra.run.smithi063:osd.0> sudo journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.0.service 2024-05-09T06:31:31.082 INFO:tasks.cephadm:Deploying osd.1 on smithi063 with /dev/vg_nvme/lv_3... 2024-05-09T06:31:31.082 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T06:31:31.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:31 smithi063 ceph-mon[27171]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:31.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:31 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:31.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:31 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:31.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:31 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:31.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:31 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:31.357 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:31:31 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[45817]: 2024-05-09T06:31:31.201+0000 7eff789a17c0 -1 osd.0 0 log_to_monitors true 2024-05-09T06:31:31.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:31 smithi193 ceph-mon[35141]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:31.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:31 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:31.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:31 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:31.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:31 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:31.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:31 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:32.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:32 smithi063 ceph-mon[27171]: from='osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T06:31:32.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:32 smithi193 ceph-mon[35141]: from='osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T06:31:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:31:33.356 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:31:33 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[45817]: 2024-05-09T06:31:33.126+0000 7eff6d804700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T06:31:33.357 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:31:33 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[45817]: 2024-05-09T06:31:33.129+0000 7eff69dfa700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:31:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:33 smithi063 ceph-mon[27171]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:33 smithi063 ceph-mon[27171]: from='osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T06:31:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:33 smithi063 ceph-mon[27171]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T06:31:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:33 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:33 smithi063 ceph-mon[27171]: from='osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:31:33.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:33 smithi193 ceph-mon[35141]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:33.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:33 smithi193 ceph-mon[35141]: from='osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T06:31:33.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:33 smithi193 ceph-mon[35141]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T06:31:33.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:33 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:33.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:33 smithi193 ceph-mon[35141]: from='osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:31:34.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:34 smithi063 ceph-mon[27171]: from='osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-09T06:31:34.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:34 smithi063 ceph-mon[27171]: osdmap e8: 1 total, 0 up, 1 in 2024-05-09T06:31:34.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:34.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:34.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:34 smithi193 ceph-mon[35141]: from='osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-09T06:31:34.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:34 smithi193 ceph-mon[35141]: osdmap e8: 1 total, 0 up, 1 in 2024-05-09T06:31:34.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:34.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:35.102 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T06:31:35.102 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T06:31:35.102 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-09T06:31:35.102 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-09T06:31:35.103 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124277 s, 844 MB/s 2024-05-09T06:31:35.103 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 2024-05-09T06:31:35.103 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-09T06:31:35.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:35 smithi063 ceph-mon[27171]: purged_snaps scrub starts 2024-05-09T06:31:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:35 smithi063 ceph-mon[27171]: purged_snaps scrub ok 2024-05-09T06:31:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:35 smithi063 ceph-mon[27171]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:35 smithi063 ceph-mon[27171]: osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] boot 2024-05-09T06:31:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:35 smithi063 ceph-mon[27171]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T06:31:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:35.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:35 smithi193 ceph-mon[35141]: purged_snaps scrub starts 2024-05-09T06:31:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:35 smithi193 ceph-mon[35141]: purged_snaps scrub ok 2024-05-09T06:31:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:35 smithi193 ceph-mon[35141]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T06:31:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:35 smithi193 ceph-mon[35141]: osd.0 [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] boot 2024-05-09T06:31:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:35 smithi193 ceph-mon[35141]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T06:31:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:35 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:31:35.928 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_3 2024-05-09T06:31:36.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:36 smithi063 ceph-mon[27171]: osdmap e10: 1 total, 1 up, 1 in 2024-05-09T06:31:36.385 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:31:36.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:36 smithi193 ceph-mon[35141]: osdmap e10: 1 total, 1 up, 1 in 2024-05-09T06:31:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:37 smithi193 ceph-mon[35141]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:37.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:37 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:37.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:37 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:37.698 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:37 smithi063 ceph-mon[27171]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:37.698 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:37 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:37.698 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:37 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.951+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/2684175532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc101730 msgr2=0x7f9abc103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.951+0000 7f9ac0aa4700 1 --2- 172.21.15.63:0/2684175532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc101730 0x7f9abc103b20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009910 tx=0x7f9aac009c20 comp rx=0 tx=0).stop 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.951+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/2684175532 shutdown_connections 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.951+0000 7f9ac0aa4700 1 --2- 172.21.15.63:0/2684175532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9abc104060 0x7f9abc106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.951+0000 7f9ac0aa4700 1 --2- 172.21.15.63:0/2684175532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc101730 0x7f9abc103b20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.951+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/2684175532 >> 172.21.15.63:0/2684175532 conn(0x7f9abc0fb120 msgr2=0x7f9abc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.951+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/2684175532 shutdown_connections 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.951+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/2684175532 wait complete. 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9ac0aa4700 1 Processor -- start 2024-05-09T06:31:37.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9ac0aa4700 1 -- start start 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9ac0aa4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9abc101730 0x7f9abc196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9ac0aa4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc104060 0x7f9abc196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9ac0aa4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc197190 con 0x7f9abc104060 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9ac0aa4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc1972d0 con 0x7f9abc101730 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9ab9d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc104060 0x7f9abc196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9ab9d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc104060 0x7f9abc196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44794/0 (socket says 172.21.15.63:44794) 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9ab9d9b700 1 -- 172.21.15.63:0/228514186 learned_addr learned my addr 172.21.15.63:0/228514186 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.952+0000 7f9aba59c700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9abc101730 0x7f9abc196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.953+0000 7f9ab9d9b700 1 -- 172.21.15.63:0/228514186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9abc101730 msgr2=0x7f9abc196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.953+0000 7f9ab9d9b700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9abc101730 0x7f9abc196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.953+0000 7f9ab9d9b700 1 -- 172.21.15.63:0/228514186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0095f0 con 0x7f9abc104060 2024-05-09T06:31:37.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.953+0000 7f9ab9d9b700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc104060 0x7f9abc196b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e920 tx=0x7f9ab000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:37.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.953+0000 7f9aab7fe700 1 -- 172.21.15.63:0/228514186 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab000c9d0 con 0x7f9abc104060 2024-05-09T06:31:37.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.953+0000 7f9aab7fe700 1 -- 172.21.15.63:0/228514186 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ab0010430 con 0x7f9abc104060 2024-05-09T06:31:37.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.953+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9abc19bd30 con 0x7f9abc104060 2024-05-09T06:31:37.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.953+0000 7f9aab7fe700 1 -- 172.21.15.63:0/228514186 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab0018630 con 0x7f9abc104060 2024-05-09T06:31:37.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.953+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9abc19c280 con 0x7f9abc104060 2024-05-09T06:31:37.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.954+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9abc04ebb0 con 0x7f9abc104060 2024-05-09T06:31:37.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.955+0000 7f9aab7fe700 1 -- 172.21.15.63:0/228514186 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f9ab00105a0 con 0x7f9abc104060 2024-05-09T06:31:37.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.955+0000 7f9aab7fe700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9aa406cc60 0x7f9aa406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:31:37.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.956+0000 7f9aab7fe700 1 -- 172.21.15.63:0/228514186 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f9ab0014070 con 0x7f9abc104060 2024-05-09T06:31:37.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.956+0000 7f9aba59c700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9aa406cc60 0x7f9aa406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:31:37.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.956+0000 7f9aba59c700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9aa406cc60 0x7f9aa406f120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b800 tx=0x7f9aac009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:31:37.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:37.960+0000 7f9aab7fe700 1 -- 172.21.15.63:0/228514186 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9ab0052c50 con 0x7f9abc104060 2024-05-09T06:31:38.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:38.182+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9abc105f10 con 0x7f9aa406cc60 2024-05-09T06:31:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:31:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:31:38.658 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:38.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:31:38.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:31:38.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:39.544 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:39 smithi063 ceph-mon[27171]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:39.544 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:39 smithi063 ceph-mon[27171]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:31:39.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:39 smithi193 ceph-mon[35141]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:39.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:39 smithi193 ceph-mon[35141]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:31:41.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:41 smithi063 ceph-mon[27171]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:41.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:41.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:41.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:31:41.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:41.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:41.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:31:41.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:41.450 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:31:41.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:41 smithi193 ceph-mon[35141]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:41.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:41.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:41.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:31:41.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:41.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:41.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:31:41.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:41.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:31:42.657 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:42 smithi193 ceph-mon[35141]: Detected new or changed devices on smithi063 2024-05-09T06:31:42.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:42 smithi193 ceph-mon[35141]: Adjusting osd_memory_target on smithi063 to 11840M 2024-05-09T06:31:42.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:31:42.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:42 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3585370762' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da187f97-762d-40d3-94d0-fef449ce1dd9"}]: dispatch 2024-05-09T06:31:42.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:42 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3585370762' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da187f97-762d-40d3-94d0-fef449ce1dd9"}]': finished 2024-05-09T06:31:42.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:42 smithi193 ceph-mon[35141]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T06:31:42.658 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:31:42.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:42 smithi063 ceph-mon[27171]: Detected new or changed devices on smithi063 2024-05-09T06:31:42.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:42 smithi063 ceph-mon[27171]: Adjusting osd_memory_target on smithi063 to 11840M 2024-05-09T06:31:42.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:31:42.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:42 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3585370762' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da187f97-762d-40d3-94d0-fef449ce1dd9"}]: dispatch 2024-05-09T06:31:42.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:42 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3585370762' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da187f97-762d-40d3-94d0-fef449ce1dd9"}]': finished 2024-05-09T06:31:42.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:42 smithi063 ceph-mon[27171]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T06:31:42.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:31:43.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:43 smithi063 ceph-mon[27171]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:43.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:43 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:43.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:43 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:43.802 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:43 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3331755588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:31:43.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:43 smithi193 ceph-mon[35141]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:43.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:43.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:43.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:43 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3331755588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:31:45.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:45 smithi063 ceph-mon[27171]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:45.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:45 smithi193 ceph-mon[35141]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:46.596 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:46.596 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:46.596 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:46.596 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:31:46.596 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:46.596 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:46 smithi063 ceph-mon[27171]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:46.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:46.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:46.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:46.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:31:46.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:46.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:46 smithi193 ceph-mon[35141]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:48.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:48 smithi063 ceph-mon[27171]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:48.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:48 smithi193 ceph-mon[35141]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:50.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:50 smithi063 ceph-mon[27171]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:50.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:50 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T06:31:50.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:50 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:50.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:50 smithi063 ceph-mon[27171]: Deploying daemon osd.1 on smithi063 2024-05-09T06:31:50.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:50 smithi193 ceph-mon[35141]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:50.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:50 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T06:31:50.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:50 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:31:50.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:50 smithi193 ceph-mon[35141]: Deploying daemon osd.1 on smithi063 2024-05-09T06:31:52.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:52 smithi063 ceph-mon[27171]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:52 smithi193 ceph-mon[35141]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:54.681 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:54 smithi063 ceph-mon[27171]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:54.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:54 smithi193 ceph-mon[35141]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:56.318 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:56.319 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:56.319 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:31:56.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:56.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:56.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:31:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:57 smithi063 ceph-mon[27171]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:31:57.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:57.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:57.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:57 smithi193 ceph-mon[35141]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:57.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:31:57.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:57.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:58.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.688+0000 7f9aab7fe700 1 -- 172.21.15.63:0/228514186 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9abc105f10 con 0x7f9aa406cc60 2024-05-09T06:31:58.669 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 1 on host 'smithi063' 2024-05-09T06:31:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9aa406cc60 msgr2=0x7f9aa406f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9aa406cc60 0x7f9aa406f120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b800 tx=0x7f9aac009f70 comp rx=0 tx=0).stop 2024-05-09T06:31:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc104060 msgr2=0x7f9abc196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:31:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc104060 0x7f9abc196b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e920 tx=0x7f9ab000ec30 comp rx=0 tx=0).stop 2024-05-09T06:31:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 shutdown_connections 2024-05-09T06:31:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f9aa406cc60 0x7f9aa406f120 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9abc101730 0x7f9abc196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 --2- 172.21.15.63:0/228514186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9abc104060 0x7f9abc196b70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:31:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 >> 172.21.15.63:0/228514186 conn(0x7f9abc0fb120 msgr2=0x7f9abc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:31:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 shutdown_connections 2024-05-09T06:31:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:31:58.690+0000 7f9ac0aa4700 1 -- 172.21.15.63:0/228514186 wait complete. 2024-05-09T06:31:59.280 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:59 smithi063 ceph-mon[27171]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:59.280 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:59 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:59.280 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:59 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:59.281 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:31:59 smithi063 ceph-mon[27171]: from='osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T06:31:59.304 DEBUG:teuthology.orchestra.run.smithi063:osd.1> sudo journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.1.service 2024-05-09T06:31:59.307 INFO:tasks.cephadm:Deploying osd.2 on smithi063 with /dev/vg_nvme/lv_2... 2024-05-09T06:31:59.307 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T06:31:59.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:59 smithi193 ceph-mon[35141]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:31:59.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:59 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:59.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:59 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:31:59.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:31:59 smithi193 ceph-mon[35141]: from='osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T06:31:59.647 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:32:01.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:00 smithi063 ceph-mon[27171]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:32:01.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:00 smithi063 ceph-mon[27171]: from='osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T06:32:01.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:00 smithi063 ceph-mon[27171]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T06:32:01.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:00 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:01.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:00 smithi063 ceph-mon[27171]: from='osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:32:01.107 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:00 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[53735]: 2024-05-09T06:32:00.669+0000 7f4b5bbf2700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T06:32:01.107 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:00 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[53735]: 2024-05-09T06:32:00.674+0000 7f4b581e8700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:32:01.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:00 smithi193 ceph-mon[35141]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:32:01.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:00 smithi193 ceph-mon[35141]: from='osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T06:32:01.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:00 smithi193 ceph-mon[35141]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T06:32:01.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:00 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:01.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:00 smithi193 ceph-mon[35141]: from='osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:32:01.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:01 smithi063 ceph-mon[27171]: from='osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-09T06:32:01.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:01 smithi063 ceph-mon[27171]: osdmap e13: 2 total, 1 up, 2 in 2024-05-09T06:32:01.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:01 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:01.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:01 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:02.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:01 smithi193 ceph-mon[35141]: from='osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-09T06:32:02.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:01 smithi193 ceph-mon[35141]: osdmap e13: 2 total, 1 up, 2 in 2024-05-09T06:32:02.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:01 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:02.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:01 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:02.676 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T06:32:02.676 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T06:32:02.676 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-09T06:32:02.677 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-09T06:32:02.677 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0139394 s, 752 MB/s 2024-05-09T06:32:02.677 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-09T06:32:02.903 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:02 smithi063 ceph-mon[27171]: purged_snaps scrub starts 2024-05-09T06:32:02.903 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:02 smithi063 ceph-mon[27171]: purged_snaps scrub ok 2024-05-09T06:32:02.903 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:02 smithi063 ceph-mon[27171]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:32:02.903 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:02 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:02.903 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:02 smithi063 ceph-mon[27171]: osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] boot 2024-05-09T06:32:02.903 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:02 smithi063 ceph-mon[27171]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T06:32:02.903 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:02 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:03.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:02 smithi193 ceph-mon[35141]: purged_snaps scrub starts 2024-05-09T06:32:03.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:02 smithi193 ceph-mon[35141]: purged_snaps scrub ok 2024-05-09T06:32:03.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:02 smithi193 ceph-mon[35141]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T06:32:03.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:02 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:03.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:02 smithi193 ceph-mon[35141]: osd.1 [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] boot 2024-05-09T06:32:03.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:02 smithi193 ceph-mon[35141]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T06:32:03.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:02 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:32:03.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:03 smithi063 ceph-mon[27171]: osdmap e15: 2 total, 2 up, 2 in 2024-05-09T06:32:03.830 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_2 2024-05-09T06:32:04.064 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:32:04.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:03 smithi193 ceph-mon[35141]: osdmap e15: 2 total, 2 up, 2 in 2024-05-09T06:32:05.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:04 smithi063 ceph-mon[27171]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:05.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:04 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:05.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:04 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:05.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:04 smithi193 ceph-mon[35141]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:05.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:04 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:05.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:04 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 -- 172.21.15.63:0/576520109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e240ffba0 msgr2=0x7f6e240fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 --2- 172.21.15.63:0/576520109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e240ffba0 0x7f6e240fffc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400b0a0 tx=0x7f6e1400b3b0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 -- 172.21.15.63:0/576520109 shutdown_connections 2024-05-09T06:32:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 --2- 172.21.15.63:0/576520109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e24100da0 0x7f6e24101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 --2- 172.21.15.63:0/576520109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e240ffba0 0x7f6e240fffc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 -- 172.21.15.63:0/576520109 >> 172.21.15.63:0/576520109 conn(0x7f6e240fb120 msgr2=0x7f6e240fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 -- 172.21.15.63:0/576520109 shutdown_connections 2024-05-09T06:32:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 -- 172.21.15.63:0/576520109 wait complete. 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 Processor -- start 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 -- start start 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e240ffba0 0x7f6e24103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24100da0 0x7f6e24102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e241026c0 con 0x7f6e24100da0 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.783+0000 7f6e2af80700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24102800 con 0x7f6e240ffba0 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e23fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24100da0 0x7f6e24102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e23fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24100da0 0x7f6e24102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58228/0 (socket says 172.21.15.63:58228) 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e23fff700 1 -- 172.21.15.63:0/433040146 learned_addr learned my addr 172.21.15.63:0/433040146 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e28d1c700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e240ffba0 0x7f6e24103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e23fff700 1 -- 172.21.15.63:0/433040146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e240ffba0 msgr2=0x7f6e24103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e23fff700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e240ffba0 0x7f6e24103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e23fff700 1 -- 172.21.15.63:0/433040146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14009d00 con 0x7f6e24100da0 2024-05-09T06:32:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e23fff700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24100da0 0x7f6e24102180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00c8a0 tx=0x7f6e0c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e21ffb700 1 -- 172.21.15.63:0/433040146 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0c00cea0 con 0x7f6e24100da0 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e21ffb700 1 -- 172.21.15.63:0/433040146 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e0c004830 con 0x7f6e24100da0 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e24102ae0 con 0x7f6e24100da0 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.784+0000 7f6e21ffb700 1 -- 172.21.15.63:0/433040146 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0c0055b0 con 0x7f6e24100da0 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.785+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2419e790 con 0x7f6e24100da0 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.785+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2404ebb0 con 0x7f6e24100da0 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.786+0000 7f6e21ffb700 1 -- 172.21.15.63:0/433040146 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f6e0c0049a0 con 0x7f6e24100da0 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.786+0000 7f6e21ffb700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6e1006cc50 0x7f6e1006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.787+0000 7f6e21ffb700 1 -- 172.21.15.63:0/433040146 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f6e0c08b1f0 con 0x7f6e24100da0 2024-05-09T06:32:06.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.787+0000 7f6e28d1c700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6e1006cc50 0x7f6e1006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:06.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.787+0000 7f6e28d1c700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6e1006cc50 0x7f6e1006f110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009fd0 tx=0x7f6e1400b7b0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:06.791+0000 7f6e21ffb700 1 -- 172.21.15.63:0/433040146 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6e0c059d20 con 0x7f6e24100da0 2024-05-09T06:32:06.995 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:06 smithi063 ceph-mon[27171]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:06.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:07.014+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e24068dd0 con 0x7f6e1006cc50 2024-05-09T06:32:07.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:06 smithi193 ceph-mon[35141]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:08.093 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:07 smithi063 ceph-mon[27171]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:32:08.093 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:07 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:32:08.093 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:07 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:32:08.094 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:07 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:08.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:07 smithi193 ceph-mon[35141]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:32:08.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:07 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:32:08.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:07 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:32:08.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:07 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:09.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:09.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:09.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:09.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:09.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:09.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:09.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:09.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:09.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:09.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:08 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:09.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:09.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:09.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:09.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:09.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:09.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:09.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:09.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:09.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:09.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:08 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:10.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:09 smithi063 ceph-mon[27171]: Detected new or changed devices on smithi063 2024-05-09T06:32:10.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:09 smithi063 ceph-mon[27171]: Adjusting osd_memory_target on smithi063 to 5920M 2024-05-09T06:32:10.345 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:09 smithi193 ceph-mon[35141]: Detected new or changed devices on smithi063 2024-05-09T06:32:10.345 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:09 smithi193 ceph-mon[35141]: Adjusting osd_memory_target on smithi063 to 5920M 2024-05-09T06:32:11.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:10 smithi063 ceph-mon[27171]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:11.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:10 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/256140501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1"}]: dispatch 2024-05-09T06:32:11.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:10 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/256140501' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1"}]': finished 2024-05-09T06:32:11.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:10 smithi063 ceph-mon[27171]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T06:32:11.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:11.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:11.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:10 smithi193 ceph-mon[35141]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:11.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:10 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/256140501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1"}]: dispatch 2024-05-09T06:32:11.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:10 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/256140501' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1"}]': finished 2024-05-09T06:32:11.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:10 smithi193 ceph-mon[35141]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T06:32:11.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:11.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:11.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:12.274 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:11 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/952514227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:32:12.274 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:11 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:32:12.285 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:11 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/952514227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:32:12.285 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:32:13.023 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:13 smithi063 ceph-mon[27171]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:13.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:13 smithi193 ceph-mon[35141]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:14.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:14.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:14.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:14.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:14.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:14.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:14.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:14.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:14.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:14.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:15.701 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:15 smithi063 ceph-mon[27171]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:15.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:15 smithi193 ceph-mon[35141]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:17.821 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:17 smithi063 ceph-mon[27171]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:17.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:17 smithi193 ceph-mon[35141]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:18.666 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T06:32:18.667 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:18.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T06:32:18.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:19.698 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:19 smithi063 ceph-mon[27171]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:19.699 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:19 smithi063 ceph-mon[27171]: Deploying daemon osd.2 on smithi063 2024-05-09T06:32:19.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:19 smithi193 ceph-mon[35141]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:19.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:19 smithi193 ceph-mon[35141]: Deploying daemon osd.2 on smithi063 2024-05-09T06:32:21.599 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:21 smithi063 ceph-mon[27171]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:21.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:21 smithi193 ceph-mon[35141]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:23.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:23 smithi063 ceph-mon[27171]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:23.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:23.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:23.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:23.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:23 smithi193 ceph-mon[35141]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:23.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:23.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:23.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:25.770 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:25 smithi063 ceph-mon[27171]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:25.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:25 smithi193 ceph-mon[35141]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:26.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:26 smithi063 ceph-mon[27171]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:26.747 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 2 on host 'smithi063' 2024-05-09T06:32:26.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.766+0000 7f6e21ffb700 1 -- 172.21.15.63:0/433040146 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6e24068dd0 con 0x7f6e1006cc50 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.768+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6e1006cc50 msgr2=0x7f6e1006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.768+0000 7f6e2af80700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6e1006cc50 0x7f6e1006f110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009fd0 tx=0x7f6e1400b7b0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.769+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24100da0 msgr2=0x7f6e24102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.769+0000 7f6e2af80700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24100da0 0x7f6e24102180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00c8a0 tx=0x7f6e0c00cc60 comp rx=0 tx=0).stop 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.769+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 shutdown_connections 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.769+0000 7f6e2af80700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6e1006cc50 0x7f6e1006f110 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.769+0000 7f6e2af80700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e240ffba0 0x7f6e24103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.769+0000 7f6e2af80700 1 --2- 172.21.15.63:0/433040146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e24100da0 0x7f6e24102180 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.769+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 >> 172.21.15.63:0/433040146 conn(0x7f6e240fb120 msgr2=0x7f6e240686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.769+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 shutdown_connections 2024-05-09T06:32:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:32:26.769+0000 7f6e2af80700 1 -- 172.21.15.63:0/433040146 wait complete. 2024-05-09T06:32:26.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:26 smithi193 ceph-mon[35141]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:27.390 DEBUG:teuthology.orchestra.run.smithi063:osd.2> sudo journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.2.service 2024-05-09T06:32:27.392 INFO:tasks.cephadm:Deploying osd.3 on smithi193 with /dev/vg_nvme/lv_4... 2024-05-09T06:32:27.392 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T06:32:27.608 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi193/config 2024-05-09T06:32:27.731 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:32:27.731 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:27.731 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:27.732 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:27 smithi193 ceph-mon[35141]: from='osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T06:32:27.761 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:32:27.762 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:27.762 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:27.762 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:27 smithi063 ceph-mon[27171]: from='osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T06:32:28.988 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:28 smithi193 ceph-mon[35141]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:28.988 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:28 smithi193 ceph-mon[35141]: from='osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T06:32:28.988 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:28 smithi193 ceph-mon[35141]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T06:32:28.988 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:28 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:28.988 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:28 smithi193 ceph-mon[35141]: from='osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:32:29.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:28 smithi063 ceph-mon[27171]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:29.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:28 smithi063 ceph-mon[27171]: from='osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T06:32:29.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:28 smithi063 ceph-mon[27171]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T06:32:29.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:28 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:29.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:28 smithi063 ceph-mon[27171]: from='osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:32:29.107 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:28 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[61576]: 2024-05-09T06:32:28.743+0000 7ff1c8562700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T06:32:29.107 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:28 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[61576]: 2024-05-09T06:32:28.752+0000 7ff1c4b58700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:32:29.692 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T06:32:29.692 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T06:32:29.692 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-09T06:32:29.692 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-09T06:32:29.692 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113949 s, 920 MB/s 2024-05-09T06:32:29.693 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-09T06:32:29.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:29 smithi193 ceph-mon[35141]: from='osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-09T06:32:29.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:29 smithi193 ceph-mon[35141]: osdmap e18: 3 total, 2 up, 3 in 2024-05-09T06:32:29.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:29 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:29.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:29 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:30.011 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:29 smithi063 ceph-mon[27171]: from='osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-09T06:32:30.011 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:29 smithi063 ceph-mon[27171]: osdmap e18: 3 total, 2 up, 3 in 2024-05-09T06:32:30.011 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:29 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:30.012 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:29 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:30.254 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-05-09T06:32:30.463 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi193/config 2024-05-09T06:32:30.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:30 smithi193 ceph-mon[35141]: purged_snaps scrub starts 2024-05-09T06:32:30.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:30 smithi193 ceph-mon[35141]: purged_snaps scrub ok 2024-05-09T06:32:30.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:30 smithi193 ceph-mon[35141]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:30.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:30 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:30.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:30 smithi193 ceph-mon[35141]: osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] boot 2024-05-09T06:32:30.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:30 smithi193 ceph-mon[35141]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T06:32:30.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:30 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:30.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:30 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:30.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:30 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:30.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:30 smithi063 ceph-mon[27171]: purged_snaps scrub starts 2024-05-09T06:32:30.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:30 smithi063 ceph-mon[27171]: purged_snaps scrub ok 2024-05-09T06:32:30.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:30 smithi063 ceph-mon[27171]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T06:32:30.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:30 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:30.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:30 smithi063 ceph-mon[27171]: osd.2 [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] boot 2024-05-09T06:32:30.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:30 smithi063 ceph-mon[27171]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T06:32:30.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:30 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:32:30.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:30 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:30.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:30 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:32.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 -- 172.21.15.193:0/4080858137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c0fef20 msgr2=0x7f898c0ff340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 --2- 172.21.15.193:0/4080858137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c0fef20 0x7f898c0ff340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f897c009910 tx=0x7f897c009c20 comp rx=0 tx=0).stop 2024-05-09T06:32:32.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 -- 172.21.15.193:0/4080858137 shutdown_connections 2024-05-09T06:32:32.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 --2- 172.21.15.193:0/4080858137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f898c101f00 0x7f898c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 --2- 172.21.15.193:0/4080858137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c0fef20 0x7f898c0ff340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 -- 172.21.15.193:0/4080858137 >> 172.21.15.193:0/4080858137 conn(0x7f898c0fa8a0 msgr2=0x7f898c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 -- 172.21.15.193:0/4080858137 shutdown_connections 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 -- 172.21.15.193:0/4080858137 wait complete. 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 Processor -- start 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.025+0000 7f89924a9700 1 -- start start 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f89924a9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f898c0fef20 0x7f898c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f89924a9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c101f00 0x7f898c0ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f89924a9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c1003f0 con 0x7f898c0fef20 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f89924a9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c100530 con 0x7f898c101f00 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f898b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c101f00 0x7f898c0ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f898b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c101f00 0x7f898c0ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:42404/0 (socket says 172.21.15.193:42404) 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f898b7fe700 1 -- 172.21.15.193:0/410364649 learned_addr learned my addr 172.21.15.193:0/410364649 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f898bfff700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f898c0fef20 0x7f898c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f898b7fe700 1 -- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f898c0fef20 msgr2=0x7f898c101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:32.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f898b7fe700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f898c0fef20 0x7f898c101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:32.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f898b7fe700 1 -- 172.21.15.193:0/410364649 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0095f0 con 0x7f898c101f00 2024-05-09T06:32:32.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f898b7fe700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c101f00 0x7f898c0ffeb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f898000e920 tx=0x7f898000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f89897fa700 1 -- 172.21.15.193:0/410364649 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f898000c9d0 con 0x7f898c101f00 2024-05-09T06:32:32.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f89897fa700 1 -- 172.21.15.193:0/410364649 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8980010430 con 0x7f898c101f00 2024-05-09T06:32:32.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c100810 con 0x7f898c101f00 2024-05-09T06:32:32.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.026+0000 7f89897fa700 1 -- 172.21.15.193:0/410364649 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8980018630 con 0x7f898c101f00 2024-05-09T06:32:32.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.027+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c19e6e0 con 0x7f898c101f00 2024-05-09T06:32:32.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.027+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c1907e0 con 0x7f898c101f00 2024-05-09T06:32:32.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.028+0000 7f89897fa700 1 -- 172.21.15.193:0/410364649 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f89800105a0 con 0x7f898c101f00 2024-05-09T06:32:32.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.028+0000 7f89897fa700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f897406cc50 0x7f897406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:32.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.028+0000 7f89897fa700 1 -- 172.21.15.193:0/410364649 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2846+0+0 (secure 0 0 0) 0x7f8980014070 con 0x7f898c101f00 2024-05-09T06:32:32.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.028+0000 7f898bfff700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f897406cc50 0x7f897406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:32.011 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.029+0000 7f898bfff700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f897406cc50 0x7f897406f110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f897c00b800 tx=0x7f897c009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:32.015 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.033+0000 7f89897fa700 1 -- 172.21.15.193:0/410364649 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8980052c50 con 0x7f898c101f00 2024-05-09T06:32:32.216 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:32.235+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f898c061330 con 0x7f897406cc50 2024-05-09T06:32:32.241 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:32 smithi063 ceph-mon[27171]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T06:32:32.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:32 smithi193 ceph-mon[35141]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T06:32:33.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:33 smithi063 ceph-mon[27171]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:33 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:33 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:33 smithi063 ceph-mon[27171]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:33 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:33 smithi063 ceph-mon[27171]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:33 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:33 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:33 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:33 smithi193 ceph-mon[35141]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:33.357 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:33 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T06:32:33.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:33 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T06:32:33.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:33 smithi193 ceph-mon[35141]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T06:32:33.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:33 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T06:32:33.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:33 smithi193 ceph-mon[35141]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:32:33.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:33 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:32:33.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:33 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:32:33.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:33 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:33.766 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:32:33 smithi063 sudo[65723]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T06:32:33.767 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:32:33 smithi063 sudo[65723]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:33.767 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:32:33 smithi063 sudo[65723]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:33.767 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:32:33 smithi063 sudo[65723]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:34.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:34.104+0000 7f89897fa700 1 -- 172.21.15.193:0/410364649 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8980056c60 con 0x7f898c101f00 2024-05-09T06:32:34.356 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:32:34 smithi063 sudo[65824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T06:32:34.356 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:32:34 smithi063 sudo[65824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:34.357 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:32:34 smithi063 sudo[65824]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:34.357 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:32:34 smithi063 sudo[65824]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:34.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:34 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:34.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T06:32:34.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:34.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:34 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:34.965 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:34 smithi063 sudo[65913]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T06:32:34.965 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:34 smithi063 sudo[65913]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:34.965 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:34 smithi063 sudo[65913]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:34.965 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:34 smithi063 sudo[65913]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:35.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:35 smithi063 sudo[65949]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T06:32:35.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:35 smithi063 sudo[65949]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:35.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:35 smithi063 sudo[65949]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:35.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:32:35 smithi063 sudo[65949]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: Detected new or changed devices on smithi063 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: Adjusting osd_memory_target on smithi063 to 3946M 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: osdmap e23: 3 total, 3 up, 3 in 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: mgrmap e19: smithi063.egfguy(active, since 2m), standbys: smithi193.mmmqzh 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/1458515195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1a9d2a-4481-4e61-aaa5-e5046f3c0055"}]: dispatch 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1a9d2a-4481-4e61-aaa5-e5046f3c0055"}]: dispatch 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1a9d2a-4481-4e61-aaa5-e5046f3c0055"}]': finished 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T06:32:35.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:35 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:35.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:35.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: Detected new or changed devices on smithi063 2024-05-09T06:32:35.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: Adjusting osd_memory_target on smithi063 to 3946M 2024-05-09T06:32:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: osdmap e23: 3 total, 3 up, 3 in 2024-05-09T06:32:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: mgrmap e19: smithi063.egfguy(active, since 2m), standbys: smithi193.mmmqzh 2024-05-09T06:32:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: from='client.? 172.21.15.193:0/1458515195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1a9d2a-4481-4e61-aaa5-e5046f3c0055"}]: dispatch 2024-05-09T06:32:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1a9d2a-4481-4e61-aaa5-e5046f3c0055"}]: dispatch 2024-05-09T06:32:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1a9d2a-4481-4e61-aaa5-e5046f3c0055"}]': finished 2024-05-09T06:32:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T06:32:35.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:35 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:35.856 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:35 smithi063 sudo[66052]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T06:32:35.856 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:35 smithi063 sudo[66052]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:35.856 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:35 smithi063 sudo[66052]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:35.856 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:35 smithi063 sudo[66052]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:36.356 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66083]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T06:32:36.356 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66083]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:36.356 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66083]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:36.356 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66083]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:36.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66138]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T06:32:36.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66138]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:36.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66138]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:36.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66138]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T06:32:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66187]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:36 smithi063 sudo[66187]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:37.446 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 sudo[39730]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T06:32:37.446 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 sudo[39730]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:37.446 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 sudo[39730]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:37.446 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 sudo[39730]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:37.827 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:37 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/3883307991' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:32:37.827 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:37 smithi063 ceph-mon[27171]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T06:32:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 sudo[39917]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T06:32:37.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 sudo[39917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T06:32:37.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 sudo[39917]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T06:32:37.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 sudo[39917]: pam_unix(sudo:session): session closed for user root 2024-05-09T06:32:37.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 ceph-mon[35141]: from='client.? 172.21.15.193:0/3883307991' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:32:37.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:37 smithi193 ceph-mon[35141]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T06:32:38.704 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:38.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:38 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:38.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:38.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T06:32:38.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:32:38.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:32:38.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T06:32:38.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:32:38.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:32:38.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T06:32:38.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:38.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:38.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:38.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:38 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:39.751 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:39 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:39.751 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:39 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:39.751 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:39 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:39.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:39.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:39.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:40.829 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:40 smithi193 ceph-mon[35141]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:40.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:40 smithi063 ceph-mon[27171]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:41.753 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:32:41.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:32:42.644 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:42 smithi193 ceph-mon[35141]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:42.644 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T06:32:42.644 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:42.645 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:42 smithi193 ceph-mon[35141]: Deploying daemon osd.3 on smithi193 2024-05-09T06:32:42.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:42 smithi063 ceph-mon[27171]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:42.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T06:32:42.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:42.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:42 smithi063 ceph-mon[27171]: Deploying daemon osd.3 on smithi193 2024-05-09T06:32:44.842 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:44 smithi193 ceph-mon[35141]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:44.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:44 smithi063 ceph-mon[27171]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:46.847 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:46 smithi193 ceph-mon[35141]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:47.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:46 smithi063 ceph-mon[27171]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:48.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:48.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:48.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:48.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:48.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:48.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:49.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:49 smithi063 ceph-mon[27171]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:49.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:49 smithi193 ceph-mon[35141]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:50.480 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.499+0000 7f89897fa700 1 -- 172.21.15.193:0/410364649 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f898c061330 con 0x7f897406cc50 2024-05-09T06:32:50.481 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 3 on host 'smithi193' 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f897406cc50 msgr2=0x7f897406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f897406cc50 0x7f897406f110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f897c00b800 tx=0x7f897c009f70 comp rx=0 tx=0).stop 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c101f00 msgr2=0x7f898c0ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c101f00 0x7f898c0ffeb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f898000e920 tx=0x7f898000ec30 comp rx=0 tx=0).stop 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 shutdown_connections 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f898c0fef20 0x7f898c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f897406cc50 0x7f897406f110 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 --2- 172.21.15.193:0/410364649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f898c101f00 0x7f898c0ffeb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 >> 172.21.15.193:0/410364649 conn(0x7f898c0fa8a0 msgr2=0x7f898c105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 shutdown_connections 2024-05-09T06:32:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:50.501+0000 7f89924a9700 1 -- 172.21.15.193:0/410364649 wait complete. 2024-05-09T06:32:51.300 DEBUG:teuthology.orchestra.run.smithi193:osd.3> sudo journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.3.service 2024-05-09T06:32:51.302 INFO:tasks.cephadm:Deploying osd.4 on smithi193 with /dev/vg_nvme/lv_3... 2024-05-09T06:32:51.303 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T06:32:51.548 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi193/config 2024-05-09T06:32:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:51 smithi193 ceph-mon[35141]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:51 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:51 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:51 smithi193 ceph-mon[35141]: from='osd.3 [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T06:32:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:51 smithi193 ceph-mon[35141]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T06:32:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:51 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:51 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:51.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:51 smithi063 ceph-mon[27171]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:51.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:51 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:51.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:51 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:51.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:51 smithi063 ceph-mon[27171]: from='osd.3 [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T06:32:51.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:51 smithi063 ceph-mon[27171]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T06:32:51.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:51 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:51.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:51 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:52.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:52 smithi193 ceph-mon[35141]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T06:32:52.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:52 smithi193 ceph-mon[35141]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T06:32:52.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:52 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:52.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:52 smithi193 ceph-mon[35141]: from='osd.3 [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:32:52.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:52 smithi193 ceph-mon[35141]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:32:52.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:52 smithi193 ceph-mon[35141]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:52.623 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:32:52 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[41190]: 2024-05-09T06:32:52.482+0000 7f520d1bf700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T06:32:52.623 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:32:52 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[41190]: 2024-05-09T06:32:52.488+0000 7f52097b5700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:32:52.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:52 smithi063 ceph-mon[27171]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T06:32:52.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:52 smithi063 ceph-mon[27171]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T06:32:52.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:52 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:52.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:52 smithi063 ceph-mon[27171]: from='osd.3 [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:32:52.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:52 smithi063 ceph-mon[27171]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:32:52.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:52 smithi063 ceph-mon[27171]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:53.512 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T06:32:53.513 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T06:32:53.513 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-09T06:32:53.513 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-09T06:32:53.513 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127308 s, 824 MB/s 2024-05-09T06:32:53.513 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-09T06:32:53.537 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:53 smithi193 ceph-mon[35141]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-09T06:32:53.537 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:53 smithi193 ceph-mon[35141]: osdmap e26: 4 total, 3 up, 4 in 2024-05-09T06:32:53.537 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:53 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:53.537 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:53 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:53.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:53 smithi063 ceph-mon[27171]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-09T06:32:53.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:53 smithi063 ceph-mon[27171]: osdmap e26: 4 total, 3 up, 4 in 2024-05-09T06:32:53.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:53 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:53.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:53 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:54.604 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:54 smithi193 ceph-mon[35141]: purged_snaps scrub starts 2024-05-09T06:32:54.604 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:54 smithi193 ceph-mon[35141]: purged_snaps scrub ok 2024-05-09T06:32:54.604 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:54 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:54.604 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:54 smithi193 ceph-mon[35141]: osd.3 [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] boot 2024-05-09T06:32:54.604 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:54 smithi193 ceph-mon[35141]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T06:32:54.604 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:54 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:54.604 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:54 smithi193 ceph-mon[35141]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:54.714 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-05-09T06:32:54.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:54 smithi063 ceph-mon[27171]: purged_snaps scrub starts 2024-05-09T06:32:54.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:54 smithi063 ceph-mon[27171]: purged_snaps scrub ok 2024-05-09T06:32:54.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:54 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:54.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:54 smithi063 ceph-mon[27171]: osd.3 [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] boot 2024-05-09T06:32:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:54 smithi063 ceph-mon[27171]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T06:32:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:54 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:32:54.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:54 smithi063 ceph-mon[27171]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-09T06:32:55.130 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi193/config 2024-05-09T06:32:55.569 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:55 smithi193 ceph-mon[35141]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T06:32:55.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:55 smithi063 ceph-mon[27171]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T06:32:56.768 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T06:32:56.768 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:32:56.768 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: Detected new or changed devices on smithi193 2024-05-09T06:32:56.768 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:56.768 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:56.768 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:56.768 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: Adjusting osd_memory_target on smithi193 to 14912M 2024-05-09T06:32:56.768 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:56.769 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:56.769 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:56.769 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:56 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:56.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: Detected new or changed devices on smithi193 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: Adjusting osd_memory_target on smithi193 to 14912M 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:56.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:56 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.277+0000 7f32f15ab700 1 -- 172.21.15.193:0/757997699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 msgr2=0x7f32ec102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.277+0000 7f32f15ab700 1 --2- 172.21.15.193:0/757997699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 0x7f32ec102640 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009910 tx=0x7f32dc009c20 comp rx=0 tx=0).stop 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.277+0000 7f32f15ab700 1 -- 172.21.15.193:0/757997699 shutdown_connections 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.277+0000 7f32f15ab700 1 --2- 172.21.15.193:0/757997699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec0feca0 0x7f32ec0ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.277+0000 7f32f15ab700 1 --2- 172.21.15.193:0/757997699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 0x7f32ec102640 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.277+0000 7f32f15ab700 1 -- 172.21.15.193:0/757997699 >> 172.21.15.193:0/757997699 conn(0x7f32ec0fa860 msgr2=0x7f32ec0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.277+0000 7f32f15ab700 1 -- 172.21.15.193:0/757997699 shutdown_connections 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.277+0000 7f32f15ab700 1 -- 172.21.15.193:0/757997699 wait complete. 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32f15ab700 1 Processor -- start 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32f15ab700 1 -- start start 2024-05-09T06:32:57.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32f15ab700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec0feca0 0x7f32ec1018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32f15ab700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 0x7f32ec0fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32f15ab700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1004b0 con 0x7f32ec0feca0 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32f15ab700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1005f0 con 0x7f32ec102220 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32ea7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 0x7f32ec0fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32ea7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 0x7f32ec0fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:38672/0 (socket says 172.21.15.193:38672) 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32ea7fc700 1 -- 172.21.15.193:0/3440847904 learned_addr learned my addr 172.21.15.193:0/3440847904 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32eaffd700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec0feca0 0x7f32ec1018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32ea7fc700 1 -- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec0feca0 msgr2=0x7f32ec1018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32ea7fc700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec0feca0 0x7f32ec1018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32ea7fc700 1 -- 172.21.15.193:0/3440847904 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc0095f0 con 0x7f32ec102220 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.278+0000 7f32eaffd700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec0feca0 0x7f32ec1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.279+0000 7f32ea7fc700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 0x7f32ec0fff70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f32e000c8a0 tx=0x7f32e000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.279+0000 7f32d3fff700 1 -- 172.21.15.193:0/3440847904 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32e000cea0 con 0x7f32ec102220 2024-05-09T06:32:57.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.279+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec1008d0 con 0x7f32ec102220 2024-05-09T06:32:57.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.279+0000 7f32d3fff700 1 -- 172.21.15.193:0/3440847904 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32e0004830 con 0x7f32ec102220 2024-05-09T06:32:57.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.279+0000 7f32d3fff700 1 -- 172.21.15.193:0/3440847904 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32e00055b0 con 0x7f32ec102220 2024-05-09T06:32:57.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.279+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec19e6a0 con 0x7f32ec102220 2024-05-09T06:32:57.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.280+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec190770 con 0x7f32ec102220 2024-05-09T06:32:57.264 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.280+0000 7f32d3fff700 1 -- 172.21.15.193:0/3440847904 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f32e00049a0 con 0x7f32ec102220 2024-05-09T06:32:57.264 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.281+0000 7f32d3fff700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f32d406cce0 0x7f32d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:32:57.264 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.281+0000 7f32d3fff700 1 -- 172.21.15.193:0/3440847904 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f32e008b860 con 0x7f32ec102220 2024-05-09T06:32:57.264 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.281+0000 7f32eaffd700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f32d406cce0 0x7f32d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:32:57.264 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.281+0000 7f32eaffd700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f32d406cce0 0x7f32d406f1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f32dc006010 tx=0x7f32dc005920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:32:57.266 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.286+0000 7f32d3fff700 1 -- 172.21.15.193:0/3440847904 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f32e0059d60 con 0x7f32ec102220 2024-05-09T06:32:57.468 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:32:57.487+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f32ec02ce50 con 0x7f32d406cce0 2024-05-09T06:32:57.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:57 smithi063 ceph-mon[27171]: osdmap e30: 4 total, 4 up, 4 in 2024-05-09T06:32:57.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:32:57.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:57.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:57.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:57.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:57.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:32:57.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:32:57.859 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:57.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:57 smithi193 ceph-mon[35141]: osdmap e30: 4 total, 4 up, 4 in 2024-05-09T06:32:57.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:32:57.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:32:57.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:57.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:32:57.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:32:57.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:32:57.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:32:57.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:32:58.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:58 smithi193 ceph-mon[35141]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:32:58.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:58 smithi193 ceph-mon[35141]: osdmap e31: 4 total, 4 up, 4 in 2024-05-09T06:32:58.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:32:58 smithi193 ceph-mon[35141]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:32:58.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:58 smithi063 ceph-mon[27171]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:32:58.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:58 smithi063 ceph-mon[27171]: osdmap e31: 4 total, 4 up, 4 in 2024-05-09T06:32:58.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:32:58 smithi063 ceph-mon[27171]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:00.600 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:00 smithi193 ceph-mon[35141]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-05-09T06:33:00.600 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:00 smithi193 ceph-mon[35141]: from='client.? 172.21.15.193:0/2006327883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23f0fabc-4f38-438c-a691-307209bb0f31"}]: dispatch 2024-05-09T06:33:00.600 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:00 smithi193 ceph-mon[35141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23f0fabc-4f38-438c-a691-307209bb0f31"}]: dispatch 2024-05-09T06:33:00.600 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:00 smithi193 ceph-mon[35141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23f0fabc-4f38-438c-a691-307209bb0f31"}]': finished 2024-05-09T06:33:00.600 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:00 smithi193 ceph-mon[35141]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T06:33:00.600 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:00 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:00.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:00 smithi063 ceph-mon[27171]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-05-09T06:33:00.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:00 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/2006327883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23f0fabc-4f38-438c-a691-307209bb0f31"}]: dispatch 2024-05-09T06:33:00.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:00 smithi063 ceph-mon[27171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23f0fabc-4f38-438c-a691-307209bb0f31"}]: dispatch 2024-05-09T06:33:00.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:00 smithi063 ceph-mon[27171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23f0fabc-4f38-438c-a691-307209bb0f31"}]': finished 2024-05-09T06:33:00.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:00 smithi063 ceph-mon[27171]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T06:33:00.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:00 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:01.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:01 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/578714893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:33:01.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:01 smithi193 ceph-mon[35141]: from='client.? 172.21.15.193:0/578714893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:33:02.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:02 smithi063 ceph-mon[27171]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T06:33:02.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:02 smithi193 ceph-mon[35141]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T06:33:04.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:04 smithi063 ceph-mon[27171]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T06:33:04.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:04 smithi193 ceph-mon[35141]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T06:33:06.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:06 smithi063 ceph-mon[27171]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T06:33:06.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:06 smithi193 ceph-mon[35141]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T06:33:07.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:07 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T06:33:07.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:07 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:07.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:07 smithi063 ceph-mon[27171]: Deploying daemon osd.4 on smithi193 2024-05-09T06:33:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:07 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T06:33:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:07 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:07 smithi193 ceph-mon[35141]: Deploying daemon osd.4 on smithi193 2024-05-09T06:33:08.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:08 smithi193 ceph-mon[35141]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T06:33:09.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:08 smithi063 ceph-mon[27171]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T06:33:10.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:10 smithi193 ceph-mon[35141]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:11.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:10 smithi063 ceph-mon[27171]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:11.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:11 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:33:12.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:33:13.465 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:13 smithi193 ceph-mon[35141]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:13.466 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:13 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:13.466 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:13 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:13.466 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:13 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:33:13.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:13 smithi063 ceph-mon[27171]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:13.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:13 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:13.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:13 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:13.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:13 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:33:15.813 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:15 smithi193 ceph-mon[35141]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:15.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:15 smithi063 ceph-mon[27171]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:15.883 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 4 on host 'smithi193' 2024-05-09T06:33:15.884 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.902+0000 7f32d3fff700 1 -- 172.21.15.193:0/3440847904 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f32ec02ce50 con 0x7f32d406cce0 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.904+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f32d406cce0 msgr2=0x7f32d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.904+0000 7f32f15ab700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f32d406cce0 0x7f32d406f1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f32dc006010 tx=0x7f32dc005920 comp rx=0 tx=0).stop 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.904+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 msgr2=0x7f32ec0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.904+0000 7f32f15ab700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 0x7f32ec0fff70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f32e000c8a0 tx=0x7f32e000cc60 comp rx=0 tx=0).stop 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.904+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 shutdown_connections 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.904+0000 7f32f15ab700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec0feca0 0x7f32ec1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.904+0000 7f32f15ab700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f32d406cce0 0x7f32d406f1a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.904+0000 7f32f15ab700 1 --2- 172.21.15.193:0/3440847904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32ec102220 0x7f32ec0fff70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.904+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 >> 172.21.15.193:0/3440847904 conn(0x7f32ec0fa860 msgr2=0x7f32ec1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.905+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 shutdown_connections 2024-05-09T06:33:15.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:15.905+0000 7f32f15ab700 1 -- 172.21.15.193:0/3440847904 wait complete. 2024-05-09T06:33:16.492 DEBUG:teuthology.orchestra.run.smithi193:osd.4> sudo journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.4.service 2024-05-09T06:33:16.495 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_2... 2024-05-09T06:33:16.495 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T06:33:17.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:16 smithi063 ceph-mon[27171]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:17.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:16 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:17.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:16 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:17.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:16 smithi063 ceph-mon[27171]: from='osd.4 [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T06:33:17.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:16 smithi063 ceph-mon[27171]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T06:33:17.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:16 smithi193 ceph-mon[35141]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:17.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:16 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:17.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:16 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:17.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:16 smithi193 ceph-mon[35141]: from='osd.4 [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T06:33:17.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:16 smithi193 ceph-mon[35141]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T06:33:17.300 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi193/config 2024-05-09T06:33:17.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:17 smithi193 ceph-mon[35141]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T06:33:17.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:17 smithi193 ceph-mon[35141]: osdmap e33: 5 total, 4 up, 5 in 2024-05-09T06:33:17.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:17 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:17.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:17 smithi193 ceph-mon[35141]: from='osd.4 [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:33:17.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:17 smithi193 ceph-mon[35141]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:33:17.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:17 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:17.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:17 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:18.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:17 smithi063 ceph-mon[27171]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T06:33:18.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:17 smithi063 ceph-mon[27171]: osdmap e33: 5 total, 4 up, 5 in 2024-05-09T06:33:18.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:17 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:18.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:17 smithi063 ceph-mon[27171]: from='osd.4 [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:33:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:17 smithi063 ceph-mon[27171]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:33:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:17 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:18.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:17 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:18.867 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:33:18 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[48066]: 2024-05-09T06:33:18.701+0000 7f484aaf2700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T06:33:18.867 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:33:18 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[48066]: 2024-05-09T06:33:18.711+0000 7f48430e0700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:33:19.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:18 smithi193 ceph-mon[35141]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:19.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:18 smithi193 ceph-mon[35141]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-09T06:33:19.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:18 smithi193 ceph-mon[35141]: osdmap e34: 5 total, 4 up, 5 in 2024-05-09T06:33:19.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:19.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:19.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:18 smithi193 ceph-mon[35141]: from='osd.4 ' entity='osd.4' 2024-05-09T06:33:19.156 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T06:33:19.156 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T06:33:19.156 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-09T06:33:19.156 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-09T06:33:19.156 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129325 s, 811 MB/s 2024-05-09T06:33:19.156 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-09T06:33:19.289 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:18 smithi063 ceph-mon[27171]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-09T06:33:19.289 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:18 smithi063 ceph-mon[27171]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-09T06:33:19.289 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:18 smithi063 ceph-mon[27171]: osdmap e34: 5 total, 4 up, 5 in 2024-05-09T06:33:19.289 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:19.289 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:19.289 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:18 smithi063 ceph-mon[27171]: from='osd.4 ' entity='osd.4' 2024-05-09T06:33:20.054 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:19 smithi193 ceph-mon[35141]: purged_snaps scrub starts 2024-05-09T06:33:20.054 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:19 smithi193 ceph-mon[35141]: purged_snaps scrub ok 2024-05-09T06:33:20.054 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:19 smithi193 ceph-mon[35141]: osd.4 [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] boot 2024-05-09T06:33:20.054 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:19 smithi193 ceph-mon[35141]: osdmap e35: 5 total, 5 up, 5 in 2024-05-09T06:33:20.054 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:19 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:20.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:19 smithi063 ceph-mon[27171]: purged_snaps scrub starts 2024-05-09T06:33:20.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:19 smithi063 ceph-mon[27171]: purged_snaps scrub ok 2024-05-09T06:33:20.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:19 smithi063 ceph-mon[27171]: osd.4 [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] boot 2024-05-09T06:33:20.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:19 smithi063 ceph-mon[27171]: osdmap e35: 5 total, 5 up, 5 in 2024-05-09T06:33:20.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:19 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:33:20.343 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-05-09T06:33:20.815 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi193/config 2024-05-09T06:33:21.233 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:20 smithi193 ceph-mon[35141]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 535 MiB used, 446 GiB / 447 GiB avail 2024-05-09T06:33:21.233 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:20 smithi193 ceph-mon[35141]: osdmap e36: 5 total, 5 up, 5 in 2024-05-09T06:33:21.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:20 smithi063 ceph-mon[27171]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 535 MiB used, 446 GiB / 447 GiB avail 2024-05-09T06:33:21.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:20 smithi063 ceph-mon[27171]: osdmap e36: 5 total, 5 up, 5 in 2024-05-09T06:33:23.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.106+0000 7f2c43ae4700 1 -- 172.21.15.193:0/2652737835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c100d20 msgr2=0x7f2c3c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:23.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.106+0000 7f2c43ae4700 1 --2- 172.21.15.193:0/2652737835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c100d20 0x7f2c3c101180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009910 tx=0x7f2c2c009c20 comp rx=0 tx=0).stop 2024-05-09T06:33:23.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.106+0000 7f2c43ae4700 1 -- 172.21.15.193:0/2652737835 shutdown_connections 2024-05-09T06:33:23.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.106+0000 7f2c43ae4700 1 --2- 172.21.15.193:0/2652737835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c100d20 0x7f2c3c101180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:23.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.106+0000 7f2c43ae4700 1 --2- 172.21.15.193:0/2652737835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c3c0ffb20 0x7f2c3c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:23.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.106+0000 7f2c43ae4700 1 -- 172.21.15.193:0/2652737835 >> 172.21.15.193:0/2652737835 conn(0x7f2c3c0fb0c0 msgr2=0x7f2c3c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:23.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.107+0000 7f2c43ae4700 1 -- 172.21.15.193:0/2652737835 shutdown_connections 2024-05-09T06:33:23.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.107+0000 7f2c43ae4700 1 -- 172.21.15.193:0/2652737835 wait complete. 2024-05-09T06:33:23.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.107+0000 7f2c43ae4700 1 Processor -- start 2024-05-09T06:33:23.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.107+0000 7f2c43ae4700 1 -- start start 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c43ae4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c0ffb20 0x7f2c3c069c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c43ae4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c3c100d20 0x7f2c3c19b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c43ae4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c19ba10 con 0x7f2c3c100d20 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c43ae4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c19bb50 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c41880700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c0ffb20 0x7f2c3c069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c41880700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c0ffb20 0x7f2c3c069c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:58948/0 (socket says 172.21.15.193:58948) 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c41880700 1 -- 172.21.15.193:0/1820488559 learned_addr learned my addr 172.21.15.193:0/1820488559 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c4107f700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c3c100d20 0x7f2c3c19b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c41880700 1 -- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c3c100d20 msgr2=0x7f2c3c19b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c41880700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c3c100d20 0x7f2c3c19b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c41880700 1 -- 172.21.15.193:0/1820488559 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c0095f0 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c41880700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c0ffb20 0x7f2c3c069c80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b480 tx=0x7f2c3800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:23.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c32ffd700 1 -- 172.21.15.193:0/1820488559 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c380118c0 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c32ffd700 1 -- 172.21.15.193:0/1820488559 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c38010380 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c19be00 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c32ffd700 1 -- 172.21.15.193:0/1820488559 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c3800f4f0 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.108+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c19c350 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.109+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3c18f740 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.110+0000 7f2c32ffd700 1 -- 172.21.15.193:0/1820488559 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2c38011a20 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.110+0000 7f2c32ffd700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2c2806cce0 0x7f2c2806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:23.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.110+0000 7f2c32ffd700 1 -- 172.21.15.193:0/1820488559 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f2c3808b9d0 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.110+0000 7f2c4107f700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2c2806cce0 0x7f2c2806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:23.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.111+0000 7f2c4107f700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2c2806cce0 0x7f2c2806f1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0051c0 tx=0x7f2c2c005920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:23.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.115+0000 7f2c32ffd700 1 -- 172.21.15.193:0/1820488559 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2c38059d20 con 0x7f2c3c0ffb20 2024-05-09T06:33:23.299 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:23.318+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c3c02cea0 con 0x7f2c2806cce0 2024-05-09T06:33:23.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 535 MiB used, 446 GiB / 447 GiB avail 2024-05-09T06:33:23.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: Detected new or changed devices on smithi193 2024-05-09T06:33:23.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: Adjusting osd_memory_target on smithi193 to 7456M 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:33:23.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:23.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 535 MiB used, 446 GiB / 447 GiB avail 2024-05-09T06:33:23.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: Detected new or changed devices on smithi193 2024-05-09T06:33:23.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: Adjusting osd_memory_target on smithi193 to 7456M 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:33:23.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:24.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:24 smithi193 ceph-mon[35141]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:33:24.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:24 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:33:24.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:24 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:33:24.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:24 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:24.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:24 smithi063 ceph-mon[27171]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:33:24.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:24 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T06:33:24.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:24 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T06:33:24.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:24 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:25.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:25 smithi193 ceph-mon[35141]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 535 MiB used, 446 GiB / 447 GiB avail 2024-05-09T06:33:25.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:25 smithi063 ceph-mon[27171]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 535 MiB used, 446 GiB / 447 GiB avail 2024-05-09T06:33:26.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:26 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/811257765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb6226de-63c7-4200-8978-4ee9491c1427"}]: dispatch 2024-05-09T06:33:26.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:26 smithi063 ceph-mon[27171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb6226de-63c7-4200-8978-4ee9491c1427"}]: dispatch 2024-05-09T06:33:26.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:26 smithi063 ceph-mon[27171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb6226de-63c7-4200-8978-4ee9491c1427"}]': finished 2024-05-09T06:33:26.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:26 smithi063 ceph-mon[27171]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T06:33:26.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:26 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:26 smithi193 ceph-mon[35141]: from='client.? 172.21.15.193:0/811257765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb6226de-63c7-4200-8978-4ee9491c1427"}]: dispatch 2024-05-09T06:33:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:26 smithi193 ceph-mon[35141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb6226de-63c7-4200-8978-4ee9491c1427"}]: dispatch 2024-05-09T06:33:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:26 smithi193 ceph-mon[35141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb6226de-63c7-4200-8978-4ee9491c1427"}]': finished 2024-05-09T06:33:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:26 smithi193 ceph-mon[35141]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T06:33:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:26 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:27 smithi063 ceph-mon[27171]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:27 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/793331410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:33:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:33:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:27 smithi193 ceph-mon[35141]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:27 smithi193 ceph-mon[35141]: from='client.? 172.21.15.193:0/793331410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T06:33:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:33:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:29 smithi063 ceph-mon[27171]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:29.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:29 smithi193 ceph-mon[35141]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:31.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:31 smithi193 ceph-mon[35141]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:31.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:31 smithi063 ceph-mon[27171]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:33.351 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:33 smithi193 ceph-mon[35141]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:33.351 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:33 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T06:33:33.351 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:33 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:33.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:33 smithi063 ceph-mon[27171]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:33.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:33 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T06:33:33.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:33 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:34.595 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:34 smithi193 ceph-mon[35141]: Deploying daemon osd.5 on smithi193 2024-05-09T06:33:34.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:34 smithi063 ceph-mon[27171]: Deploying daemon osd.5 on smithi193 2024-05-09T06:33:35.601 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:35 smithi193 ceph-mon[35141]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:35.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:35 smithi063 ceph-mon[27171]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:37 smithi063 ceph-mon[27171]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:37 smithi193 ceph-mon[35141]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:39.817 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:39 smithi193 ceph-mon[35141]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:39.817 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:39 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:39.817 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:39 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:39.817 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:39 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:33:39.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:39 smithi063 ceph-mon[27171]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:39.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:39.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:39.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:39 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:33:41.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:41 smithi193 ceph-mon[35141]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:41.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:41 smithi063 ceph-mon[27171]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:42.066 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.085+0000 7f2c32ffd700 1 -- 172.21.15.193:0/1820488559 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2c3c02cea0 con 0x7f2c2806cce0 2024-05-09T06:33:42.066 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2c2806cce0 msgr2=0x7f2c2806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2c2806cce0 0x7f2c2806f1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0051c0 tx=0x7f2c2c005920 comp rx=0 tx=0).stop 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c0ffb20 msgr2=0x7f2c3c069c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c0ffb20 0x7f2c3c069c80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b480 tx=0x7f2c3800b840 comp rx=0 tx=0).stop 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 shutdown_connections 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2c2806cce0 0x7f2c2806f1a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c3c0ffb20 0x7f2c3c069c80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 --2- 172.21.15.193:0/1820488559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c3c100d20 0x7f2c3c19b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:42.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 >> 172.21.15.193:0/1820488559 conn(0x7f2c3c0fb0c0 msgr2=0x7f2c3c104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 shutdown_connections 2024-05-09T06:33:42.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:42.087+0000 7f2c43ae4700 1 -- 172.21.15.193:0/1820488559 wait complete. 2024-05-09T06:33:42.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:33:42.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:42.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:42.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:42 smithi193 ceph-mon[35141]: from='osd.5 [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T06:33:42.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:42 smithi193 ceph-mon[35141]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T06:33:42.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:33:42.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:42.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:42.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:42 smithi063 ceph-mon[27171]: from='osd.5 [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T06:33:42.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:42 smithi063 ceph-mon[27171]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T06:33:42.789 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.5.service 2024-05-09T06:33:42.792 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-05-09T06:33:42.792 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T06:33:43.025 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:33:43.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:43 smithi193 ceph-mon[35141]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:43.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:43 smithi193 ceph-mon[35141]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T06:33:43.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:43 smithi193 ceph-mon[35141]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T06:33:43.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:43.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:43 smithi193 ceph-mon[35141]: from='osd.5 [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:33:43.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:43 smithi193 ceph-mon[35141]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:33:43.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:43.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:43 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:43.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:43 smithi063 ceph-mon[27171]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:43.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:43 smithi063 ceph-mon[27171]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T06:33:43.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:43 smithi063 ceph-mon[27171]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T06:33:43.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:43 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:43.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:43 smithi063 ceph-mon[27171]: from='osd.5 [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:33:43.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:43 smithi063 ceph-mon[27171]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:33:43.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:43 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:43.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:43 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:44.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.645+0000 7fba21793700 1 -- 172.21.15.63:0/663513956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c101e50 msgr2=0x7fba1c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.645+0000 7fba21793700 1 --2- 172.21.15.63:0/663513956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c101e50 0x7fba1c1022b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba10009910 tx=0x7fba10009c20 comp rx=0 tx=0).stop 2024-05-09T06:33:44.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.645+0000 7fba21793700 1 -- 172.21.15.63:0/663513956 shutdown_connections 2024-05-09T06:33:44.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.645+0000 7fba21793700 1 --2- 172.21.15.63:0/663513956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c101e50 0x7fba1c1022b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.645+0000 7fba21793700 1 --2- 172.21.15.63:0/663513956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba1c100c50 0x7fba1c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.645+0000 7fba21793700 1 -- 172.21.15.63:0/663513956 >> 172.21.15.63:0/663513956 conn(0x7fba1c0fc210 msgr2=0x7fba1c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:44.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.645+0000 7fba21793700 1 -- 172.21.15.63:0/663513956 shutdown_connections 2024-05-09T06:33:44.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.645+0000 7fba21793700 1 -- 172.21.15.63:0/663513956 wait complete. 2024-05-09T06:33:44.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba21793700 1 Processor -- start 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba21793700 1 -- start start 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba21793700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c100c50 0x7fba1c074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba21793700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba1c101e50 0x7fba1c0732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba21793700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c073820 con 0x7fba1c100c50 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba21793700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c073960 con 0x7fba1c101e50 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba1affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c100c50 0x7fba1c074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba1affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c100c50 0x7fba1c074bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36558/0 (socket says 172.21.15.63:36558) 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba1affd700 1 -- 172.21.15.63:0/4183787512 learned_addr learned my addr 172.21.15.63:0/4183787512 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.646+0000 7fba1a7fc700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba1c101e50 0x7fba1c0732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.647+0000 7fba1affd700 1 -- 172.21.15.63:0/4183787512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba1c101e50 msgr2=0x7fba1c0732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.647+0000 7fba1affd700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba1c101e50 0x7fba1c0732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.647+0000 7fba1affd700 1 -- 172.21.15.63:0/4183787512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba100095f0 con 0x7fba1c100c50 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.647+0000 7fba1affd700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c100c50 0x7fba1c074bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00d900 tx=0x7fba0c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:44.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.647+0000 7fba03fff700 1 -- 172.21.15.63:0/4183787512 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0c00b920 con 0x7fba1c100c50 2024-05-09T06:33:44.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.647+0000 7fba03fff700 1 -- 172.21.15.63:0/4183787512 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba0c010460 con 0x7fba1c100c50 2024-05-09T06:33:44.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.647+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c073c40 con 0x7fba1c100c50 2024-05-09T06:33:44.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.647+0000 7fba03fff700 1 -- 172.21.15.63:0/4183787512 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0c00f590 con 0x7fba1c100c50 2024-05-09T06:33:44.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.647+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c19e670 con 0x7fba1c100c50 2024-05-09T06:33:44.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.648+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c1907d0 con 0x7fba1c100c50 2024-05-09T06:33:44.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.649+0000 7fba03fff700 1 -- 172.21.15.63:0/4183787512 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fba0c0105d0 con 0x7fba1c100c50 2024-05-09T06:33:44.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.649+0000 7fba03fff700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fba0406ccf0 0x7fba0406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:44.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.649+0000 7fba03fff700 1 -- 172.21.15.63:0/4183787512 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4358+0+0 (secure 0 0 0) 0x7fba0c08b9a0 con 0x7fba1c100c50 2024-05-09T06:33:44.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.649+0000 7fba1a7fc700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fba0406ccf0 0x7fba0406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:44.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.650+0000 7fba1a7fc700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fba0406ccf0 0x7fba0406f1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba10006010 tx=0x7fba10005180 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:44.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.654+0000 7fba03fff700 1 -- 172.21.15.63:0/4183787512 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fba0c059cd0 con 0x7fba1c100c50 2024-05-09T06:33:44.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.856+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fba1c04ebb0 con 0x7fba1c100c50 2024-05-09T06:33:44.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.857+0000 7fba03fff700 1 -- 172.21.15.63:0/4183787512 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7fba0c059860 con 0x7fba1c100c50 2024-05-09T06:33:44.838 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.859+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fba0406ccf0 msgr2=0x7fba0406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.859+0000 7fba21793700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fba0406ccf0 0x7fba0406f1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba10006010 tx=0x7fba10005180 comp rx=0 tx=0).stop 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.859+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c100c50 msgr2=0x7fba1c074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.859+0000 7fba21793700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c100c50 0x7fba1c074bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00d900 tx=0x7fba0c00dc10 comp rx=0 tx=0).stop 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.860+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 shutdown_connections 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.860+0000 7fba21793700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba1c100c50 0x7fba1c074bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.860+0000 7fba21793700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fba0406ccf0 0x7fba0406f1b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.860+0000 7fba21793700 1 --2- 172.21.15.63:0/4183787512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba1c101e50 0x7fba1c0732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.860+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 >> 172.21.15.63:0/4183787512 conn(0x7fba1c0fc210 msgr2=0x7fba1c1053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.860+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 shutdown_connections 2024-05-09T06:33:44.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:44.860+0000 7fba21793700 1 -- 172.21.15.63:0/4183787512 wait complete. 2024-05-09T06:33:45.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:45 smithi063 ceph-mon[27171]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:45.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:45 smithi063 ceph-mon[27171]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-09T06:33:45.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:45 smithi063 ceph-mon[27171]: osdmap e39: 6 total, 5 up, 6 in 2024-05-09T06:33:45.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:45 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:45.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:45 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:45.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:45 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/4183787512' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T06:33:45.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:45 smithi063 ceph-mon[27171]: from='osd.5 ' entity='osd.5' 2024-05-09T06:33:45.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:45 smithi193 ceph-mon[35141]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:45.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:45 smithi193 ceph-mon[35141]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-09T06:33:45.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:45 smithi193 ceph-mon[35141]: osdmap e39: 6 total, 5 up, 6 in 2024-05-09T06:33:45.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:45 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:45.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:45 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:45.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:45 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/4183787512' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T06:33:45.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:45 smithi193 ceph-mon[35141]: from='osd.5 ' entity='osd.5' 2024-05-09T06:33:45.123 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:33:44 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[54915]: 2024-05-09T06:33:44.885+0000 7fca97a7f700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T06:33:45.123 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:33:44 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[54915]: 2024-05-09T06:33:44.901+0000 7fca9006d700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:33:45.541 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1715236398,"num_in_osds":6,"osd_in_since":1715236405,"num_remapped_pgs":0} 2024-05-09T06:33:46.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:46 smithi063 ceph-mon[27171]: purged_snaps scrub starts 2024-05-09T06:33:46.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:46 smithi063 ceph-mon[27171]: purged_snaps scrub ok 2024-05-09T06:33:46.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:46.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:46 smithi063 ceph-mon[27171]: osd.5 [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] boot 2024-05-09T06:33:46.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:46 smithi063 ceph-mon[27171]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T06:33:46.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:46.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:46 smithi193 ceph-mon[35141]: purged_snaps scrub starts 2024-05-09T06:33:46.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:46 smithi193 ceph-mon[35141]: purged_snaps scrub ok 2024-05-09T06:33:46.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:46.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:46 smithi193 ceph-mon[35141]: osd.5 [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] boot 2024-05-09T06:33:46.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:46 smithi193 ceph-mon[35141]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T06:33:46.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:33:46.542 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T06:33:46.761 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:33:47.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:47 smithi063 ceph-mon[27171]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:47 smithi193 ceph-mon[35141]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-09T06:33:48.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T06:33:48.180 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: Detected new or changed devices on smithi193 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: Adjusting osd_memory_target on smithi193 to 4970M 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:48.181 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:48 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:33:48.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T06:33:48.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: Detected new or changed devices on smithi193 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: Adjusting osd_memory_target on smithi193 to 4970M 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:48.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:48 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.492+0000 7f5dae700700 1 -- 172.21.15.63:0/815839436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 msgr2=0x7f5da8100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.492+0000 7f5dae700700 1 --2- 172.21.15.63:0/815839436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 0x7f5da8100250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009910 tx=0x7f5d90009c20 comp rx=0 tx=0).stop 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.492+0000 7f5dae700700 1 -- 172.21.15.63:0/815839436 shutdown_connections 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.492+0000 7f5dae700700 1 --2- 172.21.15.63:0/815839436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 0x7f5da8100250 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.492+0000 7f5dae700700 1 --2- 172.21.15.63:0/815839436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5da81010b0 0x7f5da81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.492+0000 7f5dae700700 1 -- 172.21.15.63:0/815839436 >> 172.21.15.63:0/815839436 conn(0x7f5da80fa840 msgr2=0x7f5da80fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.492+0000 7f5dae700700 1 -- 172.21.15.63:0/815839436 shutdown_connections 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.492+0000 7f5dae700700 1 -- 172.21.15.63:0/815839436 wait complete. 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.493+0000 7f5dae700700 1 Processor -- start 2024-05-09T06:33:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.493+0000 7f5dae700700 1 -- start start 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.493+0000 7f5dae700700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 0x7f5da8103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.493+0000 7f5dae700700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5da81010b0 0x7f5da81021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.493+0000 7f5dae700700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da8102720 con 0x7f5da80ffdf0 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.493+0000 7f5dae700700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da8102860 con 0x7f5da81010b0 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.493+0000 7f5da7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 0x7f5da8103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.493+0000 7f5da7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 0x7f5da8103b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60266/0 (socket says 172.21.15.63:60266) 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.493+0000 7f5da7fff700 1 -- 172.21.15.63:0/2882225784 learned_addr learned my addr 172.21.15.63:0/2882225784 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.494+0000 7f5da7fff700 1 -- 172.21.15.63:0/2882225784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5da81010b0 msgr2=0x7f5da81021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.494+0000 7f5da7fff700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5da81010b0 0x7f5da81021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.494+0000 7f5da7fff700 1 -- 172.21.15.63:0/2882225784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d900095f0 con 0x7f5da80ffdf0 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.494+0000 7f5da7fff700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 0x7f5da8103b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d98007e90 tx=0x7f5d9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.494+0000 7f5da5ffb700 1 -- 172.21.15.63:0/2882225784 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d98017070 con 0x7f5da80ffdf0 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.494+0000 7f5da5ffb700 1 -- 172.21.15.63:0/2882225784 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d9800f040 con 0x7f5da80ffdf0 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.494+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da8102b40 con 0x7f5da80ffdf0 2024-05-09T06:33:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.494+0000 7f5da5ffb700 1 -- 172.21.15.63:0/2882225784 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d980048f0 con 0x7f5da80ffdf0 2024-05-09T06:33:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.494+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da819e6c0 con 0x7f5da80ffdf0 2024-05-09T06:33:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.495+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da804ebb0 con 0x7f5da80ffdf0 2024-05-09T06:33:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.496+0000 7f5da5ffb700 1 -- 172.21.15.63:0/2882225784 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5d98004120 con 0x7f5da80ffdf0 2024-05-09T06:33:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.496+0000 7f5da5ffb700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5d9406cc80 0x7f5d9406f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.496+0000 7f5da5ffb700 1 -- 172.21.15.63:0/2882225784 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5d9808b980 con 0x7f5da80ffdf0 2024-05-09T06:33:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.496+0000 7f5d9f7fe700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5d9406cc80 0x7f5d9406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.497+0000 7f5d9f7fe700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5d9406cc80 0x7f5d9406f140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b5c0 tx=0x7f5d90005160 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.501+0000 7f5da5ffb700 1 -- 172.21.15.63:0/2882225784 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5d98059b20 con 0x7f5da80ffdf0 2024-05-09T06:33:48.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.709+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5da8108880 con 0x7f5da80ffdf0 2024-05-09T06:33:48.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.710+0000 7f5da5ffb700 1 -- 172.21.15.63:0/2882225784 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f5d980596b0 con 0x7f5da80ffdf0 2024-05-09T06:33:48.691 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:33:48.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5d9406cc80 msgr2=0x7f5d9406f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5d9406cc80 0x7f5d9406f140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b5c0 tx=0x7f5d90005160 comp rx=0 tx=0).stop 2024-05-09T06:33:48.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 msgr2=0x7f5da8103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:48.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 0x7f5da8103b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d98007e90 tx=0x7f5d9800d3b0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 shutdown_connections 2024-05-09T06:33:48.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5da80ffdf0 0x7f5da8103b40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5d9406cc80 0x7f5d9406f140 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 --2- 172.21.15.63:0/2882225784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5da81010b0 0x7f5da81021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:48.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 >> 172.21.15.63:0/2882225784 conn(0x7f5da80fa840 msgr2=0x7f5da80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:48.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 shutdown_connections 2024-05-09T06:33:48.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:48.712+0000 7f5dae700700 1 -- 172.21.15.63:0/2882225784 wait complete. 2024-05-09T06:33:49.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:49 smithi063 ceph-mon[27171]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:49.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:49 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2882225784' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T06:33:49.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:49 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:49.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:49 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:33:49.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:49 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:49.371 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1715236425,"num_in_osds":6,"osd_in_since":1715236405,"num_remapped_pgs":0} 2024-05-09T06:33:49.372 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T06:33:49.588 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:33:49.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:49 smithi193 ceph-mon[35141]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:49.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:49 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2882225784' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T06:33:49.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:49 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:33:49.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:49 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:33:49.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:49 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:33:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.200+0000 7fad158aa700 1 -- 172.21.15.63:0/3228849547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 msgr2=0x7fad10105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.200+0000 7fad158aa700 1 --2- 172.21.15.63:0/3228849547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 0x7fad10105830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fad04009910 tx=0x7fad04009c20 comp rx=0 tx=0).stop 2024-05-09T06:33:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.200+0000 7fad158aa700 1 -- 172.21.15.63:0/3228849547 shutdown_connections 2024-05-09T06:33:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.200+0000 7fad158aa700 1 --2- 172.21.15.63:0/3228849547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 0x7fad10105830 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.200+0000 7fad158aa700 1 --2- 172.21.15.63:0/3228849547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad10100e30 0x7fad10101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.200+0000 7fad158aa700 1 -- 172.21.15.63:0/3228849547 >> 172.21.15.63:0/3228849547 conn(0x7fad100fc6e0 msgr2=0x7fad100feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.200+0000 7fad158aa700 1 -- 172.21.15.63:0/3228849547 shutdown_connections 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.200+0000 7fad158aa700 1 -- 172.21.15.63:0/3228849547 wait complete. 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.201+0000 7fad158aa700 1 Processor -- start 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.201+0000 7fad158aa700 1 -- start start 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.201+0000 7fad158aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad10100e30 0x7fad1019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.201+0000 7fad158aa700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 0x7fad1019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.201+0000 7fad158aa700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019d4d0 con 0x7fad101017e0 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.201+0000 7fad158aa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad10196ab0 con 0x7fad10100e30 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad0e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 0x7fad1019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad0e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 0x7fad1019ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60276/0 (socket says 172.21.15.63:60276) 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad0e7fc700 1 -- 172.21.15.63:0/3750773793 learned_addr learned my addr 172.21.15.63:0/3750773793 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:33:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad0effd700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad10100e30 0x7fad1019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad0e7fc700 1 -- 172.21.15.63:0/3750773793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad10100e30 msgr2=0x7fad1019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad0e7fc700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad10100e30 0x7fad1019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad0e7fc700 1 -- 172.21.15.63:0/3750773793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad040095f0 con 0x7fad101017e0 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad0effd700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad10100e30 0x7fad1019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad0e7fc700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 0x7fad1019ceb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fad040092e0 tx=0x7fad0400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad148a8700 1 -- 172.21.15.63:0/3750773793 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad0401d070 con 0x7fad101017e0 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad148a8700 1 -- 172.21.15.63:0/3750773793 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad0400f460 con 0x7fad101017e0 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad10196d30 con 0x7fad101017e0 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad148a8700 1 -- 172.21.15.63:0/3750773793 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad04017610 con 0x7fad101017e0 2024-05-09T06:33:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.202+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad10197220 con 0x7fad101017e0 2024-05-09T06:33:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.203+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad10109180 con 0x7fad101017e0 2024-05-09T06:33:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.204+0000 7fad148a8700 1 -- 172.21.15.63:0/3750773793 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fad0400f5d0 con 0x7fad101017e0 2024-05-09T06:33:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.204+0000 7fad148a8700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7facf806ccf0 0x7facf806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.204+0000 7fad148a8700 1 -- 172.21.15.63:0/3750773793 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fad0408d3f0 con 0x7fad101017e0 2024-05-09T06:33:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.204+0000 7fad0effd700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7facf806ccf0 0x7facf806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.205+0000 7fad0effd700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7facf806ccf0 0x7facf806f1b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad000099d0 tx=0x7fad00008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:51.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.209+0000 7fad148a8700 1 -- 172.21.15.63:0/3750773793 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fad0405b590 con 0x7fad101017e0 2024-05-09T06:33:51.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:51 smithi063 ceph-mon[27171]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:51.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.414+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad10102310 con 0x7fad101017e0 2024-05-09T06:33:51.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.415+0000 7fad148a8700 1 -- 172.21.15.63:0/3750773793 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11324 (secure 0 0 0) 0x7fad0408d030 con 0x7fad101017e0 2024-05-09T06:33:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:33:51.397 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":41,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","created":"2024-05-09T06:27:53.398729+0000","modified":"2024-05-09T06:33:47.077331+0000","last_up_change":"2024-05-09T06:33:45.888973+0000","last_in_change":"2024-05-09T06:33:25.853191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:32:32.042216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eea045d9-973d-4960-a34e-689f9403d55f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6803","nonce":1596324997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6805","nonce":1596324997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6809","nonce":1596324997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6807","nonce":1596324997}]},"public_addr":"172.21.15.63:6803/1596324997","cluster_addr":"172.21.15.63:6805/1596324997","heartbeat_back_addr":"172.21.15.63:6809/1596324997","heartbeat_front_addr":"172.21.15.63:6807/1596324997","state":["exists","up"]},{"osd":1,"uuid":"da187f97-762d-40d3-94d0-fef449ce1dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6811","nonce":2457092004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6813","nonce":2457092004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6817","nonce":2457092004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6815","nonce":2457092004}]},"public_addr":"172.21.15.63:6811/2457092004","cluster_addr":"172.21.15.63:6813/2457092004","heartbeat_back_addr":"172.21.15.63:6817/2457092004","heartbeat_front_addr":"172.21.15.63:6815/2457092004","state":["exists","up"]},{"osd":2,"uuid":"6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6819","nonce":3467870918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6821","nonce":3467870918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6825","nonce":3467870918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6823","nonce":3467870918}]},"public_addr":"172.21.15.63:6819/3467870918","cluster_addr":"172.21.15.63:6821/3467870918","heartbeat_back_addr":"172.21.15.63:6825/3467870918","heartbeat_front_addr":"172.21.15.63:6823/3467870918","state":["exists","up"]},{"osd":3,"uuid":"ff1a9d2a-4481-4e61-aaa5-e5046f3c0055","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6801","nonce":974699773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6803","nonce":974699773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6807","nonce":974699773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6805","nonce":974699773}]},"public_addr":"172.21.15.193:6801/974699773","cluster_addr":"172.21.15.193:6803/974699773","heartbeat_back_addr":"172.21.15.193:6807/974699773","heartbeat_front_addr":"172.21.15.193:6805/974699773","state":["exists","up"]},{"osd":4,"uuid":"23f0fabc-4f38-438c-a691-307209bb0f31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6809","nonce":1396318320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6811","nonce":1396318320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6815","nonce":1396318320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6813","nonce":1396318320}]},"public_addr":"172.21.15.193:6809/1396318320","cluster_addr":"172.21.15.193:6811/1396318320","heartbeat_back_addr":"172.21.15.193:6815/1396318320","heartbeat_front_addr":"172.21.15.193:6813/1396318320","state":["exists","up"]},{"osd":5,"uuid":"eb6226de-63c7-4200-8978-4ee9491c1427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6817","nonce":449909828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6819","nonce":449909828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6823","nonce":449909828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6821","nonce":449909828}]},"public_addr":"172.21.15.193:6817/449909828","cluster_addr":"172.21.15.193:6819/449909828","heartbeat_back_addr":"172.21.15.193:6823/449909828","heartbeat_front_addr":"172.21.15.193:6821/449909828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:32.202943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:59.919211+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:28.132162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:52.038182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:17.334991+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:43.406303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6800/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/1465547870":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/3349953626":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6801/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/576235286":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/750127211":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6801/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:6801/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/1971919126":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/4061231873":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/1507850411":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/494791920":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/3728039632":"2024-05-10T06:28:29.800161+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:33:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7facf806ccf0 msgr2=0x7facf806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7facf806ccf0 0x7facf806f1b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad000099d0 tx=0x7fad00008040 comp rx=0 tx=0).stop 2024-05-09T06:33:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 msgr2=0x7fad1019ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 0x7fad1019ceb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fad040092e0 tx=0x7fad0400be20 comp rx=0 tx=0).stop 2024-05-09T06:33:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 shutdown_connections 2024-05-09T06:33:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7facf806ccf0 0x7facf806f1b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad10100e30 0x7fad1019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 --2- 172.21.15.63:0/3750773793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad101017e0 0x7fad1019ceb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 >> 172.21.15.63:0/3750773793 conn(0x7fad100fc6e0 msgr2=0x7fad100fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:51.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 shutdown_connections 2024-05-09T06:33:51.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:51.418+0000 7fad158aa700 1 -- 172.21.15.63:0/3750773793 wait complete. 2024-05-09T06:33:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:51 smithi193 ceph-mon[35141]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:52.186 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:32:32.042216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:33:52.187 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T06:33:52.402 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:33:52.506 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:52 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3750773793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:33:52.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:52 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3750773793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:33:53.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:53 smithi063 ceph-mon[27171]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:53.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:53 smithi193 ceph-mon[35141]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:54.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.200+0000 7fe722006700 1 -- 172.21.15.63:0/546603551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c105c90 msgr2=0x7fe71c106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.200+0000 7fe722006700 1 --2- 172.21.15.63:0/546603551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c105c90 0x7fe71c106070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00b0a0 tx=0x7fe70c00b3b0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.200+0000 7fe722006700 1 -- 172.21.15.63:0/546603551 shutdown_connections 2024-05-09T06:33:54.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.200+0000 7fe722006700 1 --2- 172.21.15.63:0/546603551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe71c100560 0x7fe71c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.200+0000 7fe722006700 1 --2- 172.21.15.63:0/546603551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c105c90 0x7fe71c106070 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.200+0000 7fe722006700 1 -- 172.21.15.63:0/546603551 >> 172.21.15.63:0/546603551 conn(0x7fe71c074d60 msgr2=0x7fe71c075170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:54.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.200+0000 7fe722006700 1 -- 172.21.15.63:0/546603551 shutdown_connections 2024-05-09T06:33:54.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.200+0000 7fe722006700 1 -- 172.21.15.63:0/546603551 wait complete. 2024-05-09T06:33:54.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.200+0000 7fe722006700 1 Processor -- start 2024-05-09T06:33:54.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe722006700 1 -- start start 2024-05-09T06:33:54.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe722006700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c100560 0x7fe71c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe722006700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe71c105c90 0x7fe71c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe722006700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c198460 con 0x7fe71c100560 2024-05-09T06:33:54.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe722006700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1985a0 con 0x7fe71c105c90 2024-05-09T06:33:54.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe71b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c100560 0x7fe71c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe71b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c100560 0x7fe71c1979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60294/0 (socket says 172.21.15.63:60294) 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe71b7fe700 1 -- 172.21.15.63:0/3244201302 learned_addr learned my addr 172.21.15.63:0/3244201302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe71b7fe700 1 -- 172.21.15.63:0/3244201302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe71c105c90 msgr2=0x7fe71c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe71b7fe700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe71c105c90 0x7fe71c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe71b7fe700 1 -- 172.21.15.63:0/3244201302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c009d00 con 0x7fe71c100560 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.201+0000 7fe71b7fe700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c100560 0x7fe71c1979e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe70c015040 tx=0x7fe70c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.202+0000 7fe718ff9700 1 -- 172.21.15.63:0/3244201302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe70c00e040 con 0x7fe71c100560 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.202+0000 7fe718ff9700 1 -- 172.21.15.63:0/3244201302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe70c007cd0 con 0x7fe71c100560 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.202+0000 7fe718ff9700 1 -- 172.21.15.63:0/3244201302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe70c01d6b0 con 0x7fe71c100560 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.202+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c198820 con 0x7fe71c100560 2024-05-09T06:33:54.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.202+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c1a0e00 con 0x7fe71c100560 2024-05-09T06:33:54.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.203+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c108120 con 0x7fe71c100560 2024-05-09T06:33:54.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.203+0000 7fe718ff9700 1 -- 172.21.15.63:0/3244201302 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe70c004040 con 0x7fe71c100560 2024-05-09T06:33:54.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.204+0000 7fe718ff9700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe70806cce0 0x7fe70806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:54.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.204+0000 7fe718ff9700 1 -- 172.21.15.63:0/3244201302 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe70c08d5a0 con 0x7fe71c100560 2024-05-09T06:33:54.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.204+0000 7fe71affd700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe70806cce0 0x7fe70806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:54.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.204+0000 7fe71affd700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe70806cce0 0x7fe70806f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0ffbf0 tx=0x7fe704006d20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:54.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.209+0000 7fe718ff9700 1 -- 172.21.15.63:0/3244201302 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe70c057d30 con 0x7fe71c100560 2024-05-09T06:33:54.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.410+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe71c04ebe0 con 0x7fe71c100560 2024-05-09T06:33:54.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.411+0000 7fe718ff9700 1 -- 172.21.15.63:0/3244201302 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe70c05b350 con 0x7fe71c100560 2024-05-09T06:33:54.391 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe70806cce0 msgr2=0x7fe70806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe70806cce0 0x7fe70806f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0ffbf0 tx=0x7fe704006d20 comp rx=0 tx=0).stop 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c100560 msgr2=0x7fe71c1979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c100560 0x7fe71c1979e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe70c015040 tx=0x7fe70c003ce0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 shutdown_connections 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71c100560 0x7fe71c1979e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe70806cce0 0x7fe70806f1a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 --2- 172.21.15.63:0/3244201302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe71c105c90 0x7fe71c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 >> 172.21.15.63:0/3244201302 conn(0x7fe71c074d60 msgr2=0x7fe71c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 shutdown_connections 2024-05-09T06:33:54.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:54.413+0000 7fe722006700 1 -- 172.21.15.63:0/3244201302 wait complete. 2024-05-09T06:33:55.151 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T06:33:55.152 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T06:33:55.368 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:33:55.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:55 smithi193 ceph-mon[35141]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:55.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:55 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3244201302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:33:55.490 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:55 smithi063 ceph-mon[27171]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:55.490 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:55 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3244201302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:33:56.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.998+0000 7fb08cfec700 1 -- 172.21.15.63:0/3829533143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb088101120 msgr2=0x7fb088101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.998+0000 7fb08cfec700 1 --2- 172.21.15.63:0/3829533143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb088101120 0x7fb088101540 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb078009910 tx=0x7fb078009c20 comp rx=0 tx=0).stop 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.998+0000 7fb08cfec700 1 -- 172.21.15.63:0/3829533143 shutdown_connections 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.998+0000 7fb08cfec700 1 --2- 172.21.15.63:0/3829533143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0880ffdd0 0x7fb088100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.998+0000 7fb08cfec700 1 --2- 172.21.15.63:0/3829533143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb088101120 0x7fb088101540 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.998+0000 7fb08cfec700 1 -- 172.21.15.63:0/3829533143 >> 172.21.15.63:0/3829533143 conn(0x7fb0880fa860 msgr2=0x7fb0880fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.998+0000 7fb08cfec700 1 -- 172.21.15.63:0/3829533143 shutdown_connections 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.998+0000 7fb08cfec700 1 -- 172.21.15.63:0/3829533143 wait complete. 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08cfec700 1 Processor -- start 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08cfec700 1 -- start start 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08cfec700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0880ffdd0 0x7fb088196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08cfec700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101120 0x7fb088196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08cfec700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb088197280 con 0x7fb0880ffdd0 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08cfec700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0881973c0 con 0x7fb088101120 2024-05-09T06:33:56.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0880ffdd0 0x7fb088196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0880ffdd0 0x7fb088196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34824/0 (socket says 172.21.15.63:34824) 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08659c700 1 -- 172.21.15.63:0/1802686514 learned_addr learned my addr 172.21.15.63:0/1802686514 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb085d9b700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101120 0x7fb088196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08659c700 1 -- 172.21.15.63:0/1802686514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101120 msgr2=0x7fb088196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08659c700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101120 0x7fb088196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08659c700 1 -- 172.21.15.63:0/1802686514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0780095f0 con 0x7fb0880ffdd0 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb085d9b700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101120 0x7fb088196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:56.999+0000 7fb08659c700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0880ffdd0 0x7fb088196720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb07800bd70 tx=0x7fb07800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.000+0000 7fb0777fe700 1 -- 172.21.15.63:0/1802686514 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07801d070 con 0x7fb0880ffdd0 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.000+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08819bdc0 con 0x7fb0880ffdd0 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.000+0000 7fb0777fe700 1 -- 172.21.15.63:0/1802686514 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb07800f460 con 0x7fb0880ffdd0 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.000+0000 7fb0777fe700 1 -- 172.21.15.63:0/1802686514 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb078021600 con 0x7fb0880ffdd0 2024-05-09T06:33:56.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.000+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08819c2d0 con 0x7fb0880ffdd0 2024-05-09T06:33:56.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.001+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08804ebb0 con 0x7fb0880ffdd0 2024-05-09T06:33:56.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.001+0000 7fb0777fe700 1 -- 172.21.15.63:0/1802686514 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb07800fa50 con 0x7fb0880ffdd0 2024-05-09T06:33:56.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.002+0000 7fb0777fe700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb07006ccf0 0x7fb07006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:56.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.002+0000 7fb0777fe700 1 -- 172.21.15.63:0/1802686514 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb07808d230 con 0x7fb0880ffdd0 2024-05-09T06:33:56.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.002+0000 7fb085d9b700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb07006ccf0 0x7fb07006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:56.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.002+0000 7fb085d9b700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb07006ccf0 0x7fb07006f1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0099d0 tx=0x7fb07c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:56.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.006+0000 7fb0777fe700 1 -- 172.21.15.63:0/1802686514 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb07805b3d0 con 0x7fb0880ffdd0 2024-05-09T06:33:57.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.294+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb088108720 con 0x7fb0880ffdd0 2024-05-09T06:33:57.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.297+0000 7fb0777fe700 1 -- 172.21.15.63:0/1802686514 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb07805af60 con 0x7fb0880ffdd0 2024-05-09T06:33:57.278 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-05-09T06:33:57.278 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBVbjxmoIKcERAAlmmAb2GpMyeB67JtL34MHw== 2024-05-09T06:33:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.299+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb07006ccf0 msgr2=0x7fb07006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb07006ccf0 0x7fb07006f1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0099d0 tx=0x7fb07c008040 comp rx=0 tx=0).stop 2024-05-09T06:33:57.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0880ffdd0 msgr2=0x7fb088196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:57.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0880ffdd0 0x7fb088196720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb07800bd70 tx=0x7fb07800be50 comp rx=0 tx=0).stop 2024-05-09T06:33:57.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 shutdown_connections 2024-05-09T06:33:57.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0880ffdd0 0x7fb088196720 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb07006ccf0 0x7fb07006f1b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 --2- 172.21.15.63:0/1802686514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101120 0x7fb088196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:57.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 >> 172.21.15.63:0/1802686514 conn(0x7fb0880fa860 msgr2=0x7fb0880fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:57.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 shutdown_connections 2024-05-09T06:33:57.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:33:57.300+0000 7fb08cfec700 1 -- 172.21.15.63:0/1802686514 wait complete. 2024-05-09T06:33:57.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:57 smithi063 ceph-mon[27171]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:57.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:33:57.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:57 smithi193 ceph-mon[35141]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:57.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:33:57.874 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:33:57.875 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T06:33:57.875 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T06:33:57.914 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T06:33:58.131 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi193/config 2024-05-09T06:33:58.254 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:58 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1802686514' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T06:33:58.254 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:58 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1802686514' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T06:33:58.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:58 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1802686514' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T06:33:58.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:58 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1802686514' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T06:33:59.544 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:33:59 smithi193 ceph-mon[35141]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:59.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:33:59 smithi063 ceph-mon[27171]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:33:59.800 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.819+0000 7f2658869700 1 -- 172.21.15.193:0/1160891749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 msgr2=0x7f26541011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.800 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.819+0000 7f2658869700 1 --2- 172.21.15.193:0/1160891749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 0x7f26541011e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f263c009910 tx=0x7f263c009c20 comp rx=0 tx=0).stop 2024-05-09T06:33:59.800 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.819+0000 7f2658869700 1 -- 172.21.15.193:0/1160891749 shutdown_connections 2024-05-09T06:33:59.800 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.819+0000 7f2658869700 1 --2- 172.21.15.193:0/1160891749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 0x7f26541011e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.800 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.819+0000 7f2658869700 1 --2- 172.21.15.193:0/1160891749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2654100040 0x7f2654100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.801 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.819+0000 7f2658869700 1 -- 172.21.15.193:0/1160891749 >> 172.21.15.193:0/1160891749 conn(0x7f26540fa840 msgr2=0x7f26540fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:33:59.801 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.819+0000 7f2658869700 1 -- 172.21.15.193:0/1160891749 shutdown_connections 2024-05-09T06:33:59.801 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.819+0000 7f2658869700 1 -- 172.21.15.193:0/1160891749 wait complete. 2024-05-09T06:33:59.802 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f2658869700 1 Processor -- start 2024-05-09T06:33:59.802 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f2658869700 1 -- start start 2024-05-09T06:33:59.802 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f2658869700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2654100040 0x7f26540ff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.802 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f2658869700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 0x7f265419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.802 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f2658869700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265419b950 con 0x7f2654100040 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f2658869700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265419ba90 con 0x7f2654100d60 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f2649d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 0x7f265419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f2649d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 0x7f265419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:49086/0 (socket says 172.21.15.193:49086) 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f265259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2654100040 0x7f26540ff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.820+0000 7f2649d9b700 1 -- 172.21.15.193:0/3752649062 learned_addr learned my addr 172.21.15.193:0/3752649062 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f2649d9b700 1 -- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2654100040 msgr2=0x7f26540ff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f2649d9b700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2654100040 0x7f26540ff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f2649d9b700 1 -- 172.21.15.193:0/3752649062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c0095f0 con 0x7f2654100d60 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f265259c700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2654100040 0x7f26540ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f2649d9b700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 0x7f265419b380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f263c005f00 tx=0x7f263c00b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f264bfff700 1 -- 172.21.15.193:0/3752649062 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f263c01d070 con 0x7f2654100d60 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f264bfff700 1 -- 172.21.15.193:0/3752649062 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f263c0052a0 con 0x7f2654100d60 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265419bcc0 con 0x7f2654100d60 2024-05-09T06:33:59.803 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f264bfff700 1 -- 172.21.15.193:0/3752649062 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f263c017400 con 0x7f2654100d60 2024-05-09T06:33:59.804 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.821+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265419c230 con 0x7f2654100d60 2024-05-09T06:33:59.804 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.822+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265418f6d0 con 0x7f2654100d60 2024-05-09T06:33:59.805 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.823+0000 7f264bfff700 1 -- 172.21.15.193:0/3752649062 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f263c003680 con 0x7f2654100d60 2024-05-09T06:33:59.805 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.823+0000 7f264bfff700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f264006cce0 0x7f264006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:33:59.805 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.823+0000 7f264bfff700 1 -- 172.21.15.193:0/3752649062 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f263c08d2b0 con 0x7f2654100d60 2024-05-09T06:33:59.805 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.823+0000 7f265259c700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f264006cce0 0x7f264006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:33:59.805 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.824+0000 7f265259c700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f264006cce0 0x7f264006f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2654068140 tx=0x7f2644009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:33:59.809 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:33:59.828+0000 7f264bfff700 1 -- 172.21.15.193:0/3752649062 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f263c05b450 con 0x7f2654100d60 2024-05-09T06:34:00.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.097+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2654108750 con 0x7f2654100d60 2024-05-09T06:34:00.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.102+0000 7f264bfff700 1 -- 172.21.15.193:0/3752649062 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f263c05afe0 con 0x7f2654100d60 2024-05-09T06:34:00.083 INFO:teuthology.orchestra.run.smithi193.stdout:[client.1] 2024-05-09T06:34:00.084 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQBYbjxmUDj3BRAA2Is/E0b8qC/SEkkblRE5NQ== 2024-05-09T06:34:00.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.104+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f264006cce0 msgr2=0x7f264006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:00.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.104+0000 7f2658869700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f264006cce0 0x7f264006f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2654068140 tx=0x7f2644009260 comp rx=0 tx=0).stop 2024-05-09T06:34:00.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.104+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 msgr2=0x7f265419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:00.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.104+0000 7f2658869700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 0x7f265419b380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f263c005f00 tx=0x7f263c00b5e0 comp rx=0 tx=0).stop 2024-05-09T06:34:00.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.105+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 shutdown_connections 2024-05-09T06:34:00.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.105+0000 7f2658869700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2654100040 0x7f26540ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:00.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.105+0000 7f2658869700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f264006cce0 0x7f264006f1a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:00.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.105+0000 7f2658869700 1 --2- 172.21.15.193:0/3752649062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100d60 0x7f265419b380 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:00.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.105+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 >> 172.21.15.193:0/3752649062 conn(0x7f26540fa840 msgr2=0x7f26541043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:00.087 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.105+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 shutdown_connections 2024-05-09T06:34:00.087 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:34:00.105+0000 7f2658869700 1 -- 172.21.15.193:0/3752649062 wait complete. 2024-05-09T06:34:00.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:00 smithi193 ceph-mon[35141]: from='client.? 172.21.15.193:0/3752649062' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T06:34:00.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:00 smithi193 ceph-mon[35141]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T06:34:00.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:00 smithi193 ceph-mon[35141]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T06:34:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:00 smithi063 ceph-mon[27171]: from='client.? 172.21.15.193:0/3752649062' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T06:34:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:00 smithi063 ceph-mon[27171]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T06:34:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:00 smithi063 ceph-mon[27171]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T06:34:00.723 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:34:00.723 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-05-09T06:34:00.723 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-09T06:34:00.760 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T06:34:00.761 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T06:34:00.761 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T06:34:00.988 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:01.430 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:01 smithi063 ceph-mon[27171]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:01.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:01 smithi193 ceph-mon[35141]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:02.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 -- 172.21.15.63:0/3997982905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec105b90 msgr2=0x7fb2ec105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:02.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 --2- 172.21.15.63:0/3997982905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec105b90 0x7fb2ec105f70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009910 tx=0x7fb2d4009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:02.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 -- 172.21.15.63:0/3997982905 shutdown_connections 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 --2- 172.21.15.63:0/3997982905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ec0ffb00 0x7fb2ec0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 --2- 172.21.15.63:0/3997982905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec105b90 0x7fb2ec105f70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 -- 172.21.15.63:0/3997982905 >> 172.21.15.63:0/3997982905 conn(0x7fb2ec0fb650 msgr2=0x7fb2ec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 -- 172.21.15.63:0/3997982905 shutdown_connections 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 -- 172.21.15.63:0/3997982905 wait complete. 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 Processor -- start 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.574+0000 7fb2f3149700 1 -- start start 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f3149700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec0ffb00 0x7fb2ec0fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f3149700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ec105b90 0x7fb2ec0fc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f3149700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0fde10 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f3149700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0fc8c0 con 0x7fb2ec105b90 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f0ee5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec0ffb00 0x7fb2ec0fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f0ee5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec0ffb00 0x7fb2ec0fbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34848/0 (socket says 172.21.15.63:34848) 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f0ee5700 1 -- 172.21.15.63:0/4285295154 learned_addr learned my addr 172.21.15.63:0/4285295154 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2ebfff700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ec105b90 0x7fb2ec0fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f0ee5700 1 -- 172.21.15.63:0/4285295154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ec105b90 msgr2=0x7fb2ec0fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:02.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f0ee5700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ec105b90 0x7fb2ec0fc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f0ee5700 1 -- 172.21.15.63:0/4285295154 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d40095f0 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2f0ee5700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec0ffb00 0x7fb2ec0fbe40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40092e0 tx=0x7fb2d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:02.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2e9ffb700 1 -- 172.21.15.63:0/4285295154 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d401d070 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.575+0000 7fb2e9ffb700 1 -- 172.21.15.63:0/4285295154 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2d400f460 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.576+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec0fcb10 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.576+0000 7fb2e9ffb700 1 -- 172.21.15.63:0/4285295154 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2d4017630 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.576+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec1a2ef0 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.577+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec109110 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.577+0000 7fb2e9ffb700 1 -- 172.21.15.63:0/4285295154 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb2d400f5d0 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.578+0000 7fb2e9ffb700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb2d806ccf0 0x7fb2d806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:02.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.578+0000 7fb2ebfff700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb2d806ccf0 0x7fb2d806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:02.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.578+0000 7fb2e9ffb700 1 -- 172.21.15.63:0/4285295154 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb2d408d370 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.578+0000 7fb2ebfff700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb2d806ccf0 0x7fb2d806f1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0fd570 tx=0x7fb2dc005d50 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:02.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.583+0000 7fb2e9ffb700 1 -- 172.21.15.63:0/4285295154 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb2d4057b00 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.845+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb2ec0fd3c0 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.849+0000 7fb2e9ffb700 1 -- 172.21.15.63:0/4285295154 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+174012 (secure 0 0 0) 0x7fb2d405b120 con 0x7fb2ec0ffb00 2024-05-09T06:34:02.832 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:34:02.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.851+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb2d806ccf0 msgr2=0x7fb2d806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:02.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.851+0000 7fb2f3149700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb2d806ccf0 0x7fb2d806f1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0fd570 tx=0x7fb2dc005d50 comp rx=0 tx=0).stop 2024-05-09T06:34:02.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.851+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec0ffb00 msgr2=0x7fb2ec0fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:02.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.851+0000 7fb2f3149700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec0ffb00 0x7fb2ec0fbe40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40092e0 tx=0x7fb2d400be20 comp rx=0 tx=0).stop 2024-05-09T06:34:02.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.851+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 shutdown_connections 2024-05-09T06:34:02.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.851+0000 7fb2f3149700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ec0ffb00 0x7fb2ec0fbe40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.851+0000 7fb2f3149700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb2d806ccf0 0x7fb2d806f1b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.851+0000 7fb2f3149700 1 --2- 172.21.15.63:0/4285295154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ec105b90 0x7fb2ec0fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:02.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.851+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 >> 172.21.15.63:0/4285295154 conn(0x7fb2ec0fb650 msgr2=0x7fb2ec069620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:02.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.852+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 shutdown_connections 2024-05-09T06:34:02.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:02.852+0000 7fb2f3149700 1 -- 172.21.15.63:0/4285295154 wait complete. 2024-05-09T06:34:03.407 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:03 smithi063 ceph-mon[27171]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:03.408 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:03 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/4285295154' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T06:34:03.431 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":19,"flags":0,"active_gid":14215,"active_name":"smithi063.egfguy","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":4130975635},{"type":"v1","addr":"172.21.15.63:6801","nonce":4130975635}]},"active_addr":"172.21.15.63:6801/4130975635","active_change":"2024-05-09T06:30:11.518386+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14240,"name":"smithi193.mmmqzh","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.63:8443/","prometheus":"http://172.21.15.63:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":452783656}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":2209960091}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":4215281381}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":2254623220}]}]} 2024-05-09T06:34:03.435 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T06:34:03.435 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T06:34:03.435 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T06:34:03.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:03 smithi193 ceph-mon[35141]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:03.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:03 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/4285295154' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T06:34:03.654 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:05.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.159+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1136365861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 msgr2=0x7f21f80fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.159+0000 7f21fdbc9700 1 --2- 172.21.15.63:0/1136365861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 0x7f21f80fff40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009910 tx=0x7f21ec009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:05.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.159+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1136365861 shutdown_connections 2024-05-09T06:34:05.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.159+0000 7f21fdbc9700 1 --2- 172.21.15.63:0/1136365861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 0x7f21f80fff40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.159+0000 7f21fdbc9700 1 --2- 172.21.15.63:0/1136365861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f8105ac0 0x7f21f8105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.159+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1136365861 >> 172.21.15.63:0/1136365861 conn(0x7f21f80fb5f0 msgr2=0x7f21f80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:05.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.159+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1136365861 shutdown_connections 2024-05-09T06:34:05.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.159+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1136365861 wait complete. 2024-05-09T06:34:05.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.160+0000 7f21fdbc9700 1 Processor -- start 2024-05-09T06:34:05.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.160+0000 7f21fdbc9700 1 -- start start 2024-05-09T06:34:05.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.160+0000 7f21fdbc9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 0x7f21f819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:05.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.160+0000 7f21fdbc9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f8105ac0 0x7f21f819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:05.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.160+0000 7f21fdbc9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f819d360 con 0x7f21f80ffac0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.160+0000 7f21fdbc9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f8196940 con 0x7f21f8105ac0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.160+0000 7f21f77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 0x7f21f819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.160+0000 7f21f77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 0x7f21f819c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35104/0 (socket says 172.21.15.63:35104) 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.160+0000 7f21f77fe700 1 -- 172.21.15.63:0/1436355690 learned_addr learned my addr 172.21.15.63:0/1436355690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21f6ffd700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f8105ac0 0x7f21f819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21f77fe700 1 -- 172.21.15.63:0/1436355690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f8105ac0 msgr2=0x7f21f819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21f77fe700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f8105ac0 0x7f21f819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21f77fe700 1 -- 172.21.15.63:0/1436355690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ec0095f0 con 0x7f21f80ffac0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21f77fe700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 0x7f21f819c800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c8a0 tx=0x7f21e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21f4ff9700 1 -- 172.21.15.63:0/1436355690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21e800cea0 con 0x7f21f80ffac0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f8196c20 con 0x7f21f80ffac0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21f4ff9700 1 -- 172.21.15.63:0/1436355690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21e8004830 con 0x7f21f80ffac0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21f4ff9700 1 -- 172.21.15.63:0/1436355690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21e80055b0 con 0x7f21f80ffac0 2024-05-09T06:34:05.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.161+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f8197170 con 0x7f21f80ffac0 2024-05-09T06:34:05.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.162+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f81090f0 con 0x7f21f80ffac0 2024-05-09T06:34:05.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.162+0000 7f21f4ff9700 1 -- 172.21.15.63:0/1436355690 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f21e80049a0 con 0x7f21f80ffac0 2024-05-09T06:34:05.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.163+0000 7f21f4ff9700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21e006cce0 0x7f21e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:05.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.163+0000 7f21f4ff9700 1 -- 172.21.15.63:0/1436355690 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f21e808baf0 con 0x7f21f80ffac0 2024-05-09T06:34:05.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.163+0000 7f21f6ffd700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21e006cce0 0x7f21e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:05.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.163+0000 7f21f6ffd700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21e006cce0 0x7f21e006f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b5c0 tx=0x7f21ec005380 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:05.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.167+0000 7f21f4ff9700 1 -- 172.21.15.63:0/1436355690 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f21e8059d10 con 0x7f21f80ffac0 2024-05-09T06:34:05.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.369+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21f8101200 con 0x7f21f80ffac0 2024-05-09T06:34:05.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.370+0000 7f21f4ff9700 1 -- 172.21.15.63:0/1436355690 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11324 (secure 0 0 0) 0x7f21e801d090 con 0x7f21f80ffac0 2024-05-09T06:34:05.351 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:34:05.352 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":41,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","created":"2024-05-09T06:27:53.398729+0000","modified":"2024-05-09T06:33:47.077331+0000","last_up_change":"2024-05-09T06:33:45.888973+0000","last_in_change":"2024-05-09T06:33:25.853191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:32:32.042216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eea045d9-973d-4960-a34e-689f9403d55f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6803","nonce":1596324997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6805","nonce":1596324997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6809","nonce":1596324997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6807","nonce":1596324997}]},"public_addr":"172.21.15.63:6803/1596324997","cluster_addr":"172.21.15.63:6805/1596324997","heartbeat_back_addr":"172.21.15.63:6809/1596324997","heartbeat_front_addr":"172.21.15.63:6807/1596324997","state":["exists","up"]},{"osd":1,"uuid":"da187f97-762d-40d3-94d0-fef449ce1dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6811","nonce":2457092004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6813","nonce":2457092004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6817","nonce":2457092004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6815","nonce":2457092004}]},"public_addr":"172.21.15.63:6811/2457092004","cluster_addr":"172.21.15.63:6813/2457092004","heartbeat_back_addr":"172.21.15.63:6817/2457092004","heartbeat_front_addr":"172.21.15.63:6815/2457092004","state":["exists","up"]},{"osd":2,"uuid":"6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6819","nonce":3467870918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6821","nonce":3467870918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6825","nonce":3467870918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6823","nonce":3467870918}]},"public_addr":"172.21.15.63:6819/3467870918","cluster_addr":"172.21.15.63:6821/3467870918","heartbeat_back_addr":"172.21.15.63:6825/3467870918","heartbeat_front_addr":"172.21.15.63:6823/3467870918","state":["exists","up"]},{"osd":3,"uuid":"ff1a9d2a-4481-4e61-aaa5-e5046f3c0055","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6801","nonce":974699773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6803","nonce":974699773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6807","nonce":974699773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6805","nonce":974699773}]},"public_addr":"172.21.15.193:6801/974699773","cluster_addr":"172.21.15.193:6803/974699773","heartbeat_back_addr":"172.21.15.193:6807/974699773","heartbeat_front_addr":"172.21.15.193:6805/974699773","state":["exists","up"]},{"osd":4,"uuid":"23f0fabc-4f38-438c-a691-307209bb0f31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6809","nonce":1396318320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6811","nonce":1396318320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6815","nonce":1396318320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6813","nonce":1396318320}]},"public_addr":"172.21.15.193:6809/1396318320","cluster_addr":"172.21.15.193:6811/1396318320","heartbeat_back_addr":"172.21.15.193:6815/1396318320","heartbeat_front_addr":"172.21.15.193:6813/1396318320","state":["exists","up"]},{"osd":5,"uuid":"eb6226de-63c7-4200-8978-4ee9491c1427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6817","nonce":449909828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6819","nonce":449909828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6823","nonce":449909828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6821","nonce":449909828}]},"public_addr":"172.21.15.193:6817/449909828","cluster_addr":"172.21.15.193:6819/449909828","heartbeat_back_addr":"172.21.15.193:6823/449909828","heartbeat_front_addr":"172.21.15.193:6821/449909828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:32.202943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:59.919211+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:28.132162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:52.038182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:17.334991+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:43.406303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6800/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/1465547870":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/3349953626":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6801/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/576235286":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/750127211":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6801/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:6801/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/1971919126":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/4061231873":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/1507850411":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/494791920":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/3728039632":"2024-05-10T06:28:29.800161+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:34:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21e006cce0 msgr2=0x7f21e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21e006cce0 0x7f21e006f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b5c0 tx=0x7f21ec005380 comp rx=0 tx=0).stop 2024-05-09T06:34:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 msgr2=0x7f21f819c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 0x7f21f819c800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c8a0 tx=0x7f21e800cc60 comp rx=0 tx=0).stop 2024-05-09T06:34:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 shutdown_connections 2024-05-09T06:34:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21f80ffac0 0x7f21f819c800 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f21e006cce0 0x7f21e006f1a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 --2- 172.21.15.63:0/1436355690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f8105ac0 0x7f21f819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 >> 172.21.15.63:0/1436355690 conn(0x7f21f80fb5f0 msgr2=0x7f21f80693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 shutdown_connections 2024-05-09T06:34:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:05.372+0000 7f21fdbc9700 1 -- 172.21.15.63:0/1436355690 wait complete. 2024-05-09T06:34:05.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:05 smithi063 ceph-mon[27171]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:05.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:05 smithi193 ceph-mon[35141]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:06.040 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T06:34:06.040 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T06:34:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:06.582 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:06 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1436355690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:34:06.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:06 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1436355690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:34:07.598 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:07 smithi063 ceph-mon[27171]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:07.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:07 smithi193 ceph-mon[35141]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:07.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.823+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1784810383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 msgr2=0x7f78d8101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.823+0000 7f78dd6f8700 1 --2- 172.21.15.63:0/1784810383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 0x7f78d8101210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009910 tx=0x7f78c8009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:07.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.823+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1784810383 shutdown_connections 2024-05-09T06:34:07.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.823+0000 7f78dd6f8700 1 --2- 172.21.15.63:0/1784810383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d81017e0 0x7f78d8105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.823+0000 7f78dd6f8700 1 --2- 172.21.15.63:0/1784810383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 0x7f78d8101210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.823+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1784810383 >> 172.21.15.63:0/1784810383 conn(0x7f78d80fc6e0 msgr2=0x7f78d80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.823+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1784810383 shutdown_connections 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.823+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1784810383 wait complete. 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.823+0000 7f78dd6f8700 1 Processor -- start 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78dd6f8700 1 -- start start 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78dd6f8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 0x7f78d819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78dd6f8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d81017e0 0x7f78d819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78dd6f8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d819d3b0 con 0x7f78d8100e30 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78dd6f8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d81969e0 con 0x7f78d81017e0 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78d6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 0x7f78d819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78d6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 0x7f78d819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35122/0 (socket says 172.21.15.63:35122) 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78d6ffd700 1 -- 172.21.15.63:0/1606640611 learned_addr learned my addr 172.21.15.63:0/1606640611 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78d67fc700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d81017e0 0x7f78d819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78d6ffd700 1 -- 172.21.15.63:0/1606640611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d81017e0 msgr2=0x7f78d819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78d6ffd700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d81017e0 0x7f78d819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78d6ffd700 1 -- 172.21.15.63:0/1606640611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c80095f0 con 0x7f78d8100e30 2024-05-09T06:34:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78d6ffd700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 0x7f78d819c850 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f78c800bd70 tx=0x7f78c800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.824+0000 7f78bffff700 1 -- 172.21.15.63:0/1606640611 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c801d070 con 0x7f78d8100e30 2024-05-09T06:34:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.825+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d8196c60 con 0x7f78d8100e30 2024-05-09T06:34:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.825+0000 7f78bffff700 1 -- 172.21.15.63:0/1606640611 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78c80056e0 con 0x7f78d8100e30 2024-05-09T06:34:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.825+0000 7f78bffff700 1 -- 172.21.15.63:0/1606640611 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c800fe40 con 0x7f78d8100e30 2024-05-09T06:34:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.825+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d8197150 con 0x7f78d8100e30 2024-05-09T06:34:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.826+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d81091d0 con 0x7f78d8100e30 2024-05-09T06:34:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.826+0000 7f78bffff700 1 -- 172.21.15.63:0/1606640611 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f78c800f680 con 0x7f78d8100e30 2024-05-09T06:34:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.827+0000 7f78bffff700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f78c006cc80 0x7f78c006f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.827+0000 7f78bffff700 1 -- 172.21.15.63:0/1606640611 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f78c808d300 con 0x7f78d8100e30 2024-05-09T06:34:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.827+0000 7f78d67fc700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f78c006cc80 0x7f78c006f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.827+0000 7f78d67fc700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f78c006cc80 0x7f78c006f140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78d8197b50 tx=0x7f78cc00b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:07.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:07.831+0000 7f78bffff700 1 -- 172.21.15.63:0/1606640611 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f78c805b4a0 con 0x7f78d8100e30 2024-05-09T06:34:08.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.038+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78d8197e50 con 0x7f78d8100e30 2024-05-09T06:34:08.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.038+0000 7f78bffff700 1 -- 172.21.15.63:0/1606640611 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11324 (secure 0 0 0) 0x7f78c805b030 con 0x7f78d8100e30 2024-05-09T06:34:08.019 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:34:08.020 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":41,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","created":"2024-05-09T06:27:53.398729+0000","modified":"2024-05-09T06:33:47.077331+0000","last_up_change":"2024-05-09T06:33:45.888973+0000","last_in_change":"2024-05-09T06:33:25.853191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:32:32.042216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eea045d9-973d-4960-a34e-689f9403d55f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6803","nonce":1596324997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6805","nonce":1596324997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6809","nonce":1596324997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6807","nonce":1596324997}]},"public_addr":"172.21.15.63:6803/1596324997","cluster_addr":"172.21.15.63:6805/1596324997","heartbeat_back_addr":"172.21.15.63:6809/1596324997","heartbeat_front_addr":"172.21.15.63:6807/1596324997","state":["exists","up"]},{"osd":1,"uuid":"da187f97-762d-40d3-94d0-fef449ce1dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6811","nonce":2457092004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6813","nonce":2457092004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6817","nonce":2457092004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6815","nonce":2457092004}]},"public_addr":"172.21.15.63:6811/2457092004","cluster_addr":"172.21.15.63:6813/2457092004","heartbeat_back_addr":"172.21.15.63:6817/2457092004","heartbeat_front_addr":"172.21.15.63:6815/2457092004","state":["exists","up"]},{"osd":2,"uuid":"6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6819","nonce":3467870918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6821","nonce":3467870918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6825","nonce":3467870918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6823","nonce":3467870918}]},"public_addr":"172.21.15.63:6819/3467870918","cluster_addr":"172.21.15.63:6821/3467870918","heartbeat_back_addr":"172.21.15.63:6825/3467870918","heartbeat_front_addr":"172.21.15.63:6823/3467870918","state":["exists","up"]},{"osd":3,"uuid":"ff1a9d2a-4481-4e61-aaa5-e5046f3c0055","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6801","nonce":974699773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6803","nonce":974699773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6807","nonce":974699773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6805","nonce":974699773}]},"public_addr":"172.21.15.193:6801/974699773","cluster_addr":"172.21.15.193:6803/974699773","heartbeat_back_addr":"172.21.15.193:6807/974699773","heartbeat_front_addr":"172.21.15.193:6805/974699773","state":["exists","up"]},{"osd":4,"uuid":"23f0fabc-4f38-438c-a691-307209bb0f31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6809","nonce":1396318320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6811","nonce":1396318320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6815","nonce":1396318320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6813","nonce":1396318320}]},"public_addr":"172.21.15.193:6809/1396318320","cluster_addr":"172.21.15.193:6811/1396318320","heartbeat_back_addr":"172.21.15.193:6815/1396318320","heartbeat_front_addr":"172.21.15.193:6813/1396318320","state":["exists","up"]},{"osd":5,"uuid":"eb6226de-63c7-4200-8978-4ee9491c1427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6817","nonce":449909828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6819","nonce":449909828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6823","nonce":449909828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6821","nonce":449909828}]},"public_addr":"172.21.15.193:6817/449909828","cluster_addr":"172.21.15.193:6819/449909828","heartbeat_back_addr":"172.21.15.193:6823/449909828","heartbeat_front_addr":"172.21.15.193:6821/449909828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:32.202943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:59.919211+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:28.132162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:52.038182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:17.334991+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:43.406303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6800/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/1465547870":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/3349953626":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6801/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/576235286":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/750127211":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6801/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:6801/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/1971919126":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/4061231873":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/1507850411":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/494791920":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/3728039632":"2024-05-10T06:28:29.800161+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:34:08.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.040+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f78c006cc80 msgr2=0x7f78c006f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.040+0000 7f78dd6f8700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f78c006cc80 0x7f78c006f140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78d8197b50 tx=0x7f78cc00b410 comp rx=0 tx=0).stop 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.040+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 msgr2=0x7f78d819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.040+0000 7f78dd6f8700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 0x7f78d819c850 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f78c800bd70 tx=0x7f78c800bda0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.041+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 shutdown_connections 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.041+0000 7f78dd6f8700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78d8100e30 0x7f78d819c850 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.041+0000 7f78dd6f8700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f78c006cc80 0x7f78c006f140 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.041+0000 7f78dd6f8700 1 --2- 172.21.15.63:0/1606640611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d81017e0 0x7f78d819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.041+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 >> 172.21.15.63:0/1606640611 conn(0x7f78d80fc6e0 msgr2=0x7f78d80fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.041+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 shutdown_connections 2024-05-09T06:34:08.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:08.041+0000 7f78dd6f8700 1 -- 172.21.15.63:0/1606640611 wait complete. 2024-05-09T06:34:08.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:08 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1606640611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:34:08.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:08 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1606640611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:34:08.686 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T06:34:08.687 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T06:34:08.687 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T06:34:08.688 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T06:34:08.688 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T06:34:08.689 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T06:34:08.972 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:09.233 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:09.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:09 smithi063 ceph-mon[27171]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:09 smithi193 ceph-mon[35141]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:09.646 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:11.106 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:11.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:11 smithi063 ceph-mon[27171]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:11.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:11 smithi193 ceph-mon[35141]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:12.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.112+0000 7fed5eaee700 1 -- 172.21.15.63:0/1009833076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 msgr2=0x7fed5810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.112+0000 7fed5eaee700 1 --2- 172.21.15.63:0/1009833076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 0x7fed5810ac00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009940 tx=0x7fed4c009c50 comp rx=0 tx=0).stop 2024-05-09T06:34:12.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.112+0000 7fed5eaee700 1 -- 172.21.15.63:0/1009833076 shutdown_connections 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.112+0000 7fed5eaee700 1 --2- 172.21.15.63:0/1009833076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 0x7fed5810ac00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.112+0000 7fed5eaee700 1 --2- 172.21.15.63:0/1009833076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed58101df0 0x7fed581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.112+0000 7fed5eaee700 1 -- 172.21.15.63:0/1009833076 >> 172.21.15.63:0/1009833076 conn(0x7fed58076750 msgr2=0x7fed580fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.112+0000 7fed5eaee700 1 -- 172.21.15.63:0/1009833076 shutdown_connections 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.112+0000 7fed5eaee700 1 -- 172.21.15.63:0/1009833076 wait complete. 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed5eaee700 1 Processor -- start 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed5eaee700 1 -- start start 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed5eaee700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed58101df0 0x7fed580ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed5eaee700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 0x7fed580fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed5eaee700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed581004d0 con 0x7fed58102710 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed5eaee700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed58100610 con 0x7fed58101df0 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed57fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 0x7fed580fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed57fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 0x7fed580fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35138/0 (socket says 172.21.15.63:35138) 2024-05-09T06:34:12.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed57fff700 1 -- 172.21.15.63:0/476696363 learned_addr learned my addr 172.21.15.63:0/476696363 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:12.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed5c88a700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed58101df0 0x7fed580ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed57fff700 1 -- 172.21.15.63:0/476696363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed58101df0 msgr2=0x7fed580ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed57fff700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed58101df0 0x7fed580ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed57fff700 1 -- 172.21.15.63:0/476696363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c0095f0 con 0x7fed58102710 2024-05-09T06:34:12.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.113+0000 7fed5c88a700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed58101df0 0x7fed580ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:12.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.114+0000 7fed57fff700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 0x7fed580fff90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0055f0 tx=0x7fed4c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:12.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.114+0000 7fed55ffb700 1 -- 172.21.15.63:0/476696363 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed4c01d070 con 0x7fed58102710 2024-05-09T06:34:12.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.114+0000 7fed55ffb700 1 -- 172.21.15.63:0/476696363 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed4c00bed0 con 0x7fed58102710 2024-05-09T06:34:12.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.114+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed58100890 con 0x7fed58102710 2024-05-09T06:34:12.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.114+0000 7fed55ffb700 1 -- 172.21.15.63:0/476696363 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed4c021410 con 0x7fed58102710 2024-05-09T06:34:12.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.114+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed581a3030 con 0x7fed58102710 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.115+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed5810e110 con 0x7fed58102710 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.116+0000 7fed55ffb700 1 -- 172.21.15.63:0/476696363 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fed4c00f7f0 con 0x7fed58102710 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.116+0000 7fed55ffb700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fed4006cce0 0x7fed4006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.116+0000 7fed55ffb700 1 -- 172.21.15.63:0/476696363 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fed4c08d010 con 0x7fed58102710 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.116+0000 7fed5c88a700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fed4006cce0 0x7fed4006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.116+0000 7fed55ffb700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] conn(0x7fed400729c0 0x7fed40074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.116+0000 7fed55ffb700 1 -- 172.21.15.63:0/476696363 --> [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed40075490 con 0x7fed400729c0 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.116+0000 7fed55ffb700 1 -- 172.21.15.63:0/476696363 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed4c08f6e0 con 0x7fed58102710 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.116+0000 7fed5d08b700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] conn(0x7fed400729c0 0x7fed40074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.117+0000 7fed5c88a700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fed4006cce0 0x7fed4006f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed48007900 tx=0x7fed48008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.117+0000 7fed5d08b700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] conn(0x7fed400729c0 0x7fed40074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:12.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.117+0000 7fed55ffb700 1 -- 172.21.15.63:0/476696363 <== osd.0 v2:172.21.15.63:6802/1596324997 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fed40075490 con 0x7fed400729c0 2024-05-09T06:34:12.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 --> [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed58061570 con 0x7fed400729c0 2024-05-09T06:34:12.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed55ffb700 1 -- 172.21.15.63:0/476696363 <== osd.0 v2:172.21.15.63:6802/1596324997 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed58061570 con 0x7fed400729c0 2024-05-09T06:34:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] conn(0x7fed400729c0 msgr2=0x7fed40074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] conn(0x7fed400729c0 0x7fed40074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fed4006cce0 msgr2=0x7fed4006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fed4006cce0 0x7fed4006f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed48007900 tx=0x7fed48008040 comp rx=0 tx=0).stop 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 msgr2=0x7fed580fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 0x7fed580fff90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0055f0 tx=0x7fed4c0056d0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 shutdown_connections 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6802/1596324997,v1:172.21.15.63:6803/1596324997] conn(0x7fed400729c0 0x7fed40074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fed4006cce0 0x7fed4006f1a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed58101df0 0x7fed580ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 --2- 172.21.15.63:0/476696363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed58102710 0x7fed580fff90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 >> 172.21.15.63:0/476696363 conn(0x7fed58076750 msgr2=0x7fed581030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 shutdown_connections 2024-05-09T06:34:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:12.134+0000 7fed5eaee700 1 -- 172.21.15.63:0/476696363 wait complete. 2024-05-09T06:34:12.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:12 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:34:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:12 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:34:12.795 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:13.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:13 smithi193 ceph-mon[35141]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:13.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:13 smithi063 ceph-mon[27171]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:14.151 INFO:teuthology.orchestra.run.smithi063.stdout:38654705699 2024-05-09T06:34:14.151 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T06:34:15.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.381+0000 7f75acda2700 1 -- 172.21.15.63:0/108331216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a8101bc0 msgr2=0x7f75a8101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.381+0000 7f75acda2700 1 --2- 172.21.15.63:0/108331216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a8101bc0 0x7f75a8101fa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7598009910 tx=0x7f7598009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.381+0000 7f75acda2700 1 -- 172.21.15.63:0/108331216 shutdown_connections 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.381+0000 7f75acda2700 1 --2- 172.21.15.63:0/108331216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a81024e0 0x7f75a810a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.381+0000 7f75acda2700 1 --2- 172.21.15.63:0/108331216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a8101bc0 0x7f75a8101fa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.381+0000 7f75acda2700 1 -- 172.21.15.63:0/108331216 >> 172.21.15.63:0/108331216 conn(0x7f75a80fb450 msgr2=0x7f75a80fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.381+0000 7f75acda2700 1 -- 172.21.15.63:0/108331216 shutdown_connections 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.381+0000 7f75acda2700 1 -- 172.21.15.63:0/108331216 wait complete. 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.381+0000 7f75acda2700 1 Processor -- start 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75acda2700 1 -- start start 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75acda2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a8101bc0 0x7f75a80ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75acda2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a81024e0 0x7f75a80ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75acda2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a81002a0 con 0x7f75a81024e0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75acda2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a81003e0 con 0x7f75a8101bc0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75a6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a81024e0 0x7f75a80ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75a6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a81024e0 0x7f75a80ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58026/0 (socket says 172.21.15.63:58026) 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75a6ffd700 1 -- 172.21.15.63:0/743348487 learned_addr learned my addr 172.21.15.63:0/743348487 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75a77fe700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a8101bc0 0x7f75a80ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75a6ffd700 1 -- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a8101bc0 msgr2=0x7f75a80ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75a6ffd700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a8101bc0 0x7f75a80ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75a6ffd700 1 -- 172.21.15.63:0/743348487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980095f0 con 0x7f75a81024e0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.382+0000 7f75a6ffd700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a81024e0 0x7f75a80ffd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f759c00d6d0 tx=0x7f759c00d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.383+0000 7f75a4ff9700 1 -- 172.21.15.63:0/743348487 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f759c00f860 con 0x7f75a81024e0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.383+0000 7f75a4ff9700 1 -- 172.21.15.63:0/743348487 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f759c00fe80 con 0x7f75a81024e0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.383+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a81a2b00 con 0x7f75a81024e0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.383+0000 7f75a4ff9700 1 -- 172.21.15.63:0/743348487 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f759c010ea0 con 0x7f75a81024e0 2024-05-09T06:34:15.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.383+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a81a2e60 con 0x7f75a81024e0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.383+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75a810df40 con 0x7f75a81024e0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.384+0000 7f75a4ff9700 1 -- 172.21.15.63:0/743348487 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f759c010460 con 0x7f75a81024e0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.385+0000 7f75a4ff9700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f759006cce0 0x7f759006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.385+0000 7f75a4ff9700 1 -- 172.21.15.63:0/743348487 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f759c08bb50 con 0x7f75a81024e0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.385+0000 7f75a77fe700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f759006cce0 0x7f759006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.385+0000 7f75a4ff9700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] conn(0x7f75900729c0 0x7f7590074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.385+0000 7f75a4ff9700 1 -- 172.21.15.63:0/743348487 --> [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7590075490 con 0x7f75900729c0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.385+0000 7f75a4ff9700 1 -- 172.21.15.63:0/743348487 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f759c0902a0 con 0x7f75a81024e0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.385+0000 7f75a7fff700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] conn(0x7f75900729c0 0x7f7590074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.385+0000 7f75a77fe700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f759006cce0 0x7f759006f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7598000c00 tx=0x7f759801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.385+0000 7f75a7fff700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] conn(0x7f75900729c0 0x7f7590074de0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:15.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.386+0000 7f75a4ff9700 1 -- 172.21.15.63:0/743348487 <== osd.4 v2:172.21.15.193:6808/1396318320 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7590075490 con 0x7f75900729c0 2024-05-09T06:34:15.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:15 smithi193 ceph-mon[35141]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:15.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.402+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 --> [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f75a80615b0 con 0x7f75900729c0 2024-05-09T06:34:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75a4ff9700 1 -- 172.21.15.63:0/743348487 <== osd.4 v2:172.21.15.193:6808/1396318320 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f75a80615b0 con 0x7f75900729c0 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] conn(0x7f75900729c0 msgr2=0x7f7590074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] conn(0x7f75900729c0 0x7f7590074de0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f759006cce0 msgr2=0x7f759006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f759006cce0 0x7f759006f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7598000c00 tx=0x7f759801a040 comp rx=0 tx=0).stop 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a81024e0 msgr2=0x7f75a80ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a81024e0 0x7f75a80ffd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f759c00d6d0 tx=0x7f759c00d9e0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 shutdown_connections 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:6808/1396318320,v1:172.21.15.193:6809/1396318320] conn(0x7f75900729c0 0x7f7590074de0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f759006cce0 0x7f759006f1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75a8101bc0 0x7f75a80ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 --2- 172.21.15.63:0/743348487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75a81024e0 0x7f75a80ffd60 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f759c00d6d0 tx=0x7f759c00d9e0 comp rx=0 tx=0).stop 2024-05-09T06:34:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.403+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 >> 172.21.15.63:0/743348487 conn(0x7f75a80fb450 msgr2=0x7f75a8102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.404+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 shutdown_connections 2024-05-09T06:34:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:15.404+0000 7f75acda2700 1 -- 172.21.15.63:0/743348487 wait complete. 2024-05-09T06:34:15.507 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:15 smithi063 ceph-mon[27171]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:16.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.724+0000 7ff28fd99700 1 -- 172.21.15.63:0/1719615291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 msgr2=0x7ff288104f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.724+0000 7ff28fd99700 1 --2- 172.21.15.63:0/1719615291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 0x7ff288104f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff27800b0a0 tx=0x7ff27800b3b0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.724+0000 7ff28fd99700 1 -- 172.21.15.63:0/1719615291 shutdown_connections 2024-05-09T06:34:16.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.724+0000 7ff28fd99700 1 --2- 172.21.15.63:0/1719615291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2881002f0 0x7ff288100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.724+0000 7ff28fd99700 1 --2- 172.21.15.63:0/1719615291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 0x7ff288104f20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.724+0000 7ff28fd99700 1 -- 172.21.15.63:0/1719615291 >> 172.21.15.63:0/1719615291 conn(0x7ff288075f10 msgr2=0x7ff2880fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.724+0000 7ff28fd99700 1 -- 172.21.15.63:0/1719615291 shutdown_connections 2024-05-09T06:34:16.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.724+0000 7ff28fd99700 1 -- 172.21.15.63:0/1719615291 wait complete. 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28fd99700 1 Processor -- start 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28fd99700 1 -- start start 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28fd99700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2881002f0 0x7ff2881979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28fd99700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 0x7ff288197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28fd99700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881984e0 con 0x7ff288104b40 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28fd99700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff288198620 con 0x7ff2881002f0 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28d334700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 0x7ff288197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28d334700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 0x7ff288197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58046/0 (socket says 172.21.15.63:58046) 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28d334700 1 -- 172.21.15.63:0/3117088850 learned_addr learned my addr 172.21.15.63:0/3117088850 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.725+0000 7ff28db35700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2881002f0 0x7ff2881979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff28d334700 1 -- 172.21.15.63:0/3117088850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2881002f0 msgr2=0x7ff2881979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff28d334700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2881002f0 0x7ff2881979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff28d334700 1 -- 172.21.15.63:0/3117088850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278009d00 con 0x7ff288104b40 2024-05-09T06:34:16.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff28db35700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2881002f0 0x7ff2881979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:16.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff28d334700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 0x7ff288197f10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00d8d0 tx=0x7ff27c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff286ffd700 1 -- 172.21.15.63:0/3117088850 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff27c009940 con 0x7ff288104b40 2024-05-09T06:34:16.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2881a0a40 con 0x7ff288104b40 2024-05-09T06:34:16.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff286ffd700 1 -- 172.21.15.63:0/3117088850 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff27c010460 con 0x7ff288104b40 2024-05-09T06:34:16.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff286ffd700 1 -- 172.21.15.63:0/3117088850 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff27c00f5f0 con 0x7ff288104b40 2024-05-09T06:34:16.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.726+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2881a0eb0 con 0x7ff288104b40 2024-05-09T06:34:16.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.727+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff28810d0b0 con 0x7ff288104b40 2024-05-09T06:34:16.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.728+0000 7ff286ffd700 1 -- 172.21.15.63:0/3117088850 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff27c0105d0 con 0x7ff288104b40 2024-05-09T06:34:16.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.728+0000 7ff286ffd700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff27006cce0 0x7ff27006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.728+0000 7ff286ffd700 1 -- 172.21.15.63:0/3117088850 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff27c08bf20 con 0x7ff288104b40 2024-05-09T06:34:16.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.728+0000 7ff28db35700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff27006cce0 0x7ff27006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.728+0000 7ff286ffd700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] conn(0x7ff2700729c0 0x7ff270074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.728+0000 7ff286ffd700 1 -- 172.21.15.63:0/3117088850 --> [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff270075490 con 0x7ff2700729c0 2024-05-09T06:34:16.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.728+0000 7ff286ffd700 1 -- 172.21.15.63:0/3117088850 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff27c08e5f0 con 0x7ff288104b40 2024-05-09T06:34:16.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.729+0000 7ff28e336700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] conn(0x7ff2700729c0 0x7ff270074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.729+0000 7ff28db35700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff27006cce0 0x7ff27006f1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff278009480 tx=0x7ff2780097c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.729+0000 7ff28e336700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] conn(0x7ff2700729c0 0x7ff270074de0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.729+0000 7ff286ffd700 1 -- 172.21.15.63:0/3117088850 <== osd.1 v2:172.21.15.63:6810/2457092004 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff270075490 con 0x7ff2700729c0 2024-05-09T06:34:16.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.745+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 --> [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff288061570 con 0x7ff2700729c0 2024-05-09T06:34:16.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.745+0000 7ff286ffd700 1 -- 172.21.15.63:0/3117088850 <== osd.1 v2:172.21.15.63:6810/2457092004 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff288061570 con 0x7ff2700729c0 2024-05-09T06:34:16.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] conn(0x7ff2700729c0 msgr2=0x7ff270074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] conn(0x7ff2700729c0 0x7ff270074de0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff27006cce0 msgr2=0x7ff27006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff27006cce0 0x7ff27006f1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff278009480 tx=0x7ff2780097c0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 msgr2=0x7ff288197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 0x7ff288197f10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00d8d0 tx=0x7ff27c00dc90 comp rx=0 tx=0).stop 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 shutdown_connections 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6810/2457092004,v1:172.21.15.63:6811/2457092004] conn(0x7ff2700729c0 0x7ff270074de0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff27006cce0 0x7ff27006f1a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2881002f0 0x7ff2881979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 --2- 172.21.15.63:0/3117088850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff288104b40 0x7ff288197f10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 >> 172.21.15.63:0/3117088850 conn(0x7ff288075f10 msgr2=0x7ff2880fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 shutdown_connections 2024-05-09T06:34:16.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.746+0000 7ff28fd99700 1 -- 172.21.15.63:0/3117088850 wait complete. 2024-05-09T06:34:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.943+0000 7fef2838c700 1 -- 172.21.15.63:0/112612050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef201026f0 msgr2=0x7fef2010abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.943+0000 7fef2838c700 1 --2- 172.21.15.63:0/112612050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef201026f0 0x7fef2010abe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fef10009940 tx=0x7fef10009c50 comp rx=0 tx=0).stop 2024-05-09T06:34:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.943+0000 7fef2838c700 1 -- 172.21.15.63:0/112612050 shutdown_connections 2024-05-09T06:34:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.943+0000 7fef2838c700 1 --2- 172.21.15.63:0/112612050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef201026f0 0x7fef2010abe0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.943+0000 7fef2838c700 1 --2- 172.21.15.63:0/112612050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef20101dd0 0x7fef201021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.943+0000 7fef2838c700 1 -- 172.21.15.63:0/112612050 >> 172.21.15.63:0/112612050 conn(0x7fef20076750 msgr2=0x7fef200fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.943+0000 7fef2838c700 1 -- 172.21.15.63:0/112612050 shutdown_connections 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.943+0000 7fef2838c700 1 -- 172.21.15.63:0/112612050 wait complete. 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.944+0000 7fef2838c700 1 Processor -- start 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.944+0000 7fef2838c700 1 -- start start 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.944+0000 7fef2838c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef20101dd0 0x7fef200ffa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.944+0000 7fef2838c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef201026f0 0x7fef200fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.944+0000 7fef2838c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef201004b0 con 0x7fef20101dd0 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.944+0000 7fef2838c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef201005f0 con 0x7fef201026f0 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.944+0000 7fef26128700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef20101dd0 0x7fef200ffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.944+0000 7fef26128700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef20101dd0 0x7fef200ffa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58062/0 (socket says 172.21.15.63:58062) 2024-05-09T06:34:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.944+0000 7fef26128700 1 -- 172.21.15.63:0/1219100430 learned_addr learned my addr 172.21.15.63:0/1219100430 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.945+0000 7fef26128700 1 -- 172.21.15.63:0/1219100430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef201026f0 msgr2=0x7fef200fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.945+0000 7fef26128700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef201026f0 0x7fef200fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.945+0000 7fef26128700 1 -- 172.21.15.63:0/1219100430 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100095f0 con 0x7fef20101dd0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.945+0000 7fef26128700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef20101dd0 0x7fef200ffa30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00b4b0 tx=0x7fef1c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.945+0000 7fef177fe700 1 -- 172.21.15.63:0/1219100430 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef1c011800 con 0x7fef20101dd0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.945+0000 7fef177fe700 1 -- 172.21.15.63:0/1219100430 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef1c011e20 con 0x7fef20101dd0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.945+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef201a2db0 con 0x7fef20101dd0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.945+0000 7fef177fe700 1 -- 172.21.15.63:0/1219100430 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef1c00f2e0 con 0x7fef20101dd0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.945+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef201a3110 con 0x7fef20101dd0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.946+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef2010e150 con 0x7fef20101dd0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.947+0000 7fef177fe700 1 -- 172.21.15.63:0/1219100430 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fef1c010380 con 0x7fef20101dd0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.947+0000 7fef177fe700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fef0c06cce0 0x7fef0c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.947+0000 7fef177fe700 1 -- 172.21.15.63:0/1219100430 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fef1c08c000 con 0x7fef20101dd0 2024-05-09T06:34:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.947+0000 7fef25927700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fef0c06cce0 0x7fef0c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.947+0000 7fef177fe700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] conn(0x7fef0c0729c0 0x7fef0c074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.947+0000 7fef177fe700 1 -- 172.21.15.63:0/1219100430 --> [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef0c075490 con 0x7fef0c0729c0 2024-05-09T06:34:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.947+0000 7fef177fe700 1 -- 172.21.15.63:0/1219100430 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef1c08c460 con 0x7fef20101dd0 2024-05-09T06:34:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.947+0000 7fef25927700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fef0c06cce0 0x7fef0c06f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef10006010 tx=0x7fef10000bc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.948+0000 7fef26929700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] conn(0x7fef0c0729c0 0x7fef0c074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.948+0000 7fef26929700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] conn(0x7fef0c0729c0 0x7fef0c074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.949+0000 7fef177fe700 1 -- 172.21.15.63:0/1219100430 <== osd.5 v2:172.21.15.193:6816/449909828 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fef0c075490 con 0x7fef0c0729c0 2024-05-09T06:34:16.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.965+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 --> [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef20061570 con 0x7fef0c0729c0 2024-05-09T06:34:16.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.965+0000 7fef177fe700 1 -- 172.21.15.63:0/1219100430 <== osd.5 v2:172.21.15.193:6816/449909828 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fef20061570 con 0x7fef0c0729c0 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.965+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 >> [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] conn(0x7fef0c0729c0 msgr2=0x7fef0c074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.965+0000 7fef2838c700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] conn(0x7fef0c0729c0 0x7fef0c074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.965+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fef0c06cce0 msgr2=0x7fef0c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.965+0000 7fef2838c700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fef0c06cce0 0x7fef0c06f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef10006010 tx=0x7fef10000bc0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.965+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef20101dd0 msgr2=0x7fef200ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.965+0000 7fef2838c700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef20101dd0 0x7fef200ffa30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00b4b0 tx=0x7fef1c00b870 comp rx=0 tx=0).stop 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.966+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 shutdown_connections 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.966+0000 7fef2838c700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.193:6816/449909828,v1:172.21.15.193:6817/449909828] conn(0x7fef0c0729c0 0x7fef0c074de0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.966+0000 7fef2838c700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef20101dd0 0x7fef200ffa30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.966+0000 7fef2838c700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fef0c06cce0 0x7fef0c06f1a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.966+0000 7fef2838c700 1 --2- 172.21.15.63:0/1219100430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef201026f0 0x7fef200fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:16.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.966+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 >> 172.21.15.63:0/1219100430 conn(0x7fef20076750 msgr2=0x7fef201030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:16.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.966+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 shutdown_connections 2024-05-09T06:34:16.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:16.966+0000 7fef2838c700 1 -- 172.21.15.63:0/1219100430 wait complete. 2024-05-09T06:34:17.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:17 smithi063 ceph-mon[27171]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:17.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:17 smithi193 ceph-mon[35141]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:18.518 INFO:teuthology.orchestra.run.smithi063.stdout:150323855374 2024-05-09T06:34:18.518 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T06:34:19.372 INFO:teuthology.orchestra.run.smithi063.stdout:171798691849 2024-05-09T06:34:19.373 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T06:34:19.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:19 smithi063 ceph-mon[27171]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:19.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:19 smithi193 ceph-mon[35141]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:19.731 INFO:teuthology.orchestra.run.smithi063.stdout:60129542174 2024-05-09T06:34:19.732 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T06:34:20.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.242+0000 7fa936130700 1 -- 172.21.15.63:0/3638713653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa930101dd0 msgr2=0x7fa9301021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.242+0000 7fa936130700 1 --2- 172.21.15.63:0/3638713653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa930101dd0 0x7fa9301021b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa92000b0d0 tx=0x7fa92000b3e0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.242+0000 7fa936130700 1 -- 172.21.15.63:0/3638713653 shutdown_connections 2024-05-09T06:34:20.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.242+0000 7fa936130700 1 --2- 172.21.15.63:0/3638713653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9301026f0 0x7fa93010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.242+0000 7fa936130700 1 --2- 172.21.15.63:0/3638713653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa930101dd0 0x7fa9301021b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.242+0000 7fa936130700 1 -- 172.21.15.63:0/3638713653 >> 172.21.15.63:0/3638713653 conn(0x7fa930076750 msgr2=0x7fa9300fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:20.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.242+0000 7fa936130700 1 -- 172.21.15.63:0/3638713653 shutdown_connections 2024-05-09T06:34:20.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.242+0000 7fa936130700 1 -- 172.21.15.63:0/3638713653 wait complete. 2024-05-09T06:34:20.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.243+0000 7fa936130700 1 Processor -- start 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.243+0000 7fa936130700 1 -- start start 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa936130700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa930101dd0 0x7fa93019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa936130700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9301026f0 0x7fa93019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa936130700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93019d370 con 0x7fa9301026f0 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa936130700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa930196950 con 0x7fa930101dd0 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9301026f0 0x7fa93019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9301026f0 0x7fa93019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58090/0 (socket says 172.21.15.63:58090) 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92effd700 1 -- 172.21.15.63:0/226964883 learned_addr learned my addr 172.21.15.63:0/226964883 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92f7fe700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa930101dd0 0x7fa93019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92effd700 1 -- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa930101dd0 msgr2=0x7fa93019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92effd700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa930101dd0 0x7fa93019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92effd700 1 -- 172.21.15.63:0/226964883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920009d00 con 0x7fa9301026f0 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92effd700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9301026f0 0x7fa93019cd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa91800d900 tx=0x7fa91800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92cff9700 1 -- 172.21.15.63:0/226964883 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa918009900 con 0x7fa9301026f0 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa930196c30 con 0x7fa9301026f0 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.244+0000 7fa92cff9700 1 -- 172.21.15.63:0/226964883 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa918010460 con 0x7fa9301026f0 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.245+0000 7fa92cff9700 1 -- 172.21.15.63:0/226964883 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa91800f5f0 con 0x7fa9301026f0 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.245+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa930197180 con 0x7fa9301026f0 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.245+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa93010e0f0 con 0x7fa9301026f0 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.246+0000 7fa92cff9700 1 -- 172.21.15.63:0/226964883 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa9180105d0 con 0x7fa9301026f0 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.247+0000 7fa92cff9700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa91c06cce0 0x7fa91c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.247+0000 7fa92f7fe700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa91c06cce0 0x7fa91c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.247+0000 7fa92cff9700 1 -- 172.21.15.63:0/226964883 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa91808beb0 con 0x7fa9301026f0 2024-05-09T06:34:20.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.247+0000 7fa92cff9700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] conn(0x7fa91c0729c0 0x7fa91c074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.247+0000 7fa92cff9700 1 -- 172.21.15.63:0/226964883 --> [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91c075490 con 0x7fa91c0729c0 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.247+0000 7fa92cff9700 1 -- 172.21.15.63:0/226964883 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa91808c310 con 0x7fa9301026f0 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.247+0000 7fa92f7fe700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa91c06cce0 0x7fa91c06f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa92000b820 tx=0x7fa920006000 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.247+0000 7fa92ffff700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] conn(0x7fa91c0729c0 0x7fa91c074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.248+0000 7fa92ffff700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] conn(0x7fa91c0729c0 0x7fa91c074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.249+0000 7fa92cff9700 1 -- 172.21.15.63:0/226964883 <== osd.3 v2:172.21.15.193:6800/974699773 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa91c075490 con 0x7fa91c0729c0 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.265+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 --> [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9301975e0 con 0x7fa91c0729c0 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa92cff9700 1 -- 172.21.15.63:0/226964883 <== osd.3 v2:172.21.15.193:6800/974699773 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa9301975e0 con 0x7fa91c0729c0 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] conn(0x7fa91c0729c0 msgr2=0x7fa91c074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] conn(0x7fa91c0729c0 0x7fa91c074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa91c06cce0 msgr2=0x7fa91c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa91c06cce0 0x7fa91c06f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa92000b820 tx=0x7fa920006000 comp rx=0 tx=0).stop 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9301026f0 msgr2=0x7fa93019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9301026f0 0x7fa93019cd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa91800d900 tx=0x7fa91800dc10 comp rx=0 tx=0).stop 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 shutdown_connections 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa91c06cce0 0x7fa91c06f1a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa930101dd0 0x7fa93019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.193:6800/974699773,v1:172.21.15.193:6801/974699773] conn(0x7fa91c0729c0 0x7fa91c074de0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 --2- 172.21.15.63:0/226964883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9301026f0 0x7fa93019cd50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 >> 172.21.15.63:0/226964883 conn(0x7fa930076750 msgr2=0x7fa9301030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 shutdown_connections 2024-05-09T06:34:20.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.266+0000 7fa936130700 1 -- 172.21.15.63:0/226964883 wait complete. 2024-05-09T06:34:20.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 -- 172.21.15.63:0/817055788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 msgr2=0x7efeac105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 --2- 172.21.15.63:0/817055788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 0x7efeac105850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efea0009910 tx=0x7efea0009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:20.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 -- 172.21.15.63:0/817055788 shutdown_connections 2024-05-09T06:34:20.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 --2- 172.21.15.63:0/817055788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 0x7efeac105850 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 --2- 172.21.15.63:0/817055788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeac100e50 0x7efeac101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 -- 172.21.15.63:0/817055788 >> 172.21.15.63:0/817055788 conn(0x7efeac0fc720 msgr2=0x7efeac0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:20.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 -- 172.21.15.63:0/817055788 shutdown_connections 2024-05-09T06:34:20.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 -- 172.21.15.63:0/817055788 wait complete. 2024-05-09T06:34:20.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 Processor -- start 2024-05-09T06:34:20.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.895+0000 7efeb16b1700 1 -- start start 2024-05-09T06:34:20.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeb16b1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeac100e50 0x7efeac19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeb16b1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 0x7efeac19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeb16b1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac19d360 con 0x7efeac101800 2024-05-09T06:34:20.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeb16b1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac196950 con 0x7efeac100e50 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeaa7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 0x7efeac19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeaa7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 0x7efeac19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58116/0 (socket says 172.21.15.63:58116) 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeaa7fc700 1 -- 172.21.15.63:0/3026716782 learned_addr learned my addr 172.21.15.63:0/3026716782 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeaaffd700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeac100e50 0x7efeac19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeaa7fc700 1 -- 172.21.15.63:0/3026716782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeac100e50 msgr2=0x7efeac19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeaa7fc700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeac100e50 0x7efeac19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeaa7fc700 1 -- 172.21.15.63:0/3026716782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00095f0 con 0x7efeac101800 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efeaa7fc700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 0x7efeac19cc80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efea000bd70 tx=0x7efea000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.896+0000 7efe93fff700 1 -- 172.21.15.63:0/3026716782 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efea001d070 con 0x7efeac101800 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.897+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac196bd0 con 0x7efeac101800 2024-05-09T06:34:20.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.897+0000 7efe93fff700 1 -- 172.21.15.63:0/3026716782 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efea00056c0 con 0x7efeac101800 2024-05-09T06:34:20.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.897+0000 7efe93fff700 1 -- 172.21.15.63:0/3026716782 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efea0017400 con 0x7efeac101800 2024-05-09T06:34:20.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.897+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac1970c0 con 0x7efeac101800 2024-05-09T06:34:20.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.897+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efeac10e0f0 con 0x7efeac101800 2024-05-09T06:34:20.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.898+0000 7efe93fff700 1 -- 172.21.15.63:0/3026716782 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7efea0017560 con 0x7efeac101800 2024-05-09T06:34:20.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.899+0000 7efe93fff700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7efe9406cce0 0x7efe9406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.899+0000 7efe93fff700 1 -- 172.21.15.63:0/3026716782 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efea008d5e0 con 0x7efeac101800 2024-05-09T06:34:20.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.899+0000 7efeaaffd700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7efe9406cce0 0x7efe9406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.899+0000 7efe93fff700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] conn(0x7efe940729c0 0x7efe94074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:20.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.899+0000 7efe93fff700 1 -- 172.21.15.63:0/3026716782 --> [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe94075490 con 0x7efe940729c0 2024-05-09T06:34:20.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.899+0000 7efe93fff700 1 -- 172.21.15.63:0/3026716782 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea008da40 con 0x7efeac101800 2024-05-09T06:34:20.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.899+0000 7efeab7fe700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] conn(0x7efe940729c0 0x7efe94074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:20.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.899+0000 7efeaaffd700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7efe9406cce0 0x7efe9406f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005950 tx=0x7efe9c00a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.899+0000 7efeab7fe700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] conn(0x7efe940729c0 0x7efe94074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:20.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.900+0000 7efe93fff700 1 -- 172.21.15.63:0/3026716782 <== osd.2 v2:172.21.15.63:6818/3467870918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7efe94075490 con 0x7efe940729c0 2024-05-09T06:34:20.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 --> [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efeac197570 con 0x7efe940729c0 2024-05-09T06:34:20.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efe93fff700 1 -- 172.21.15.63:0/3026716782 <== osd.2 v2:172.21.15.63:6818/3467870918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efeac197570 con 0x7efe940729c0 2024-05-09T06:34:20.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] conn(0x7efe940729c0 msgr2=0x7efe94074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] conn(0x7efe940729c0 0x7efe94074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7efe9406cce0 msgr2=0x7efe9406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7efe9406cce0 0x7efe9406f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005950 tx=0x7efe9c00a300 comp rx=0 tx=0).stop 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 msgr2=0x7efeac19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 0x7efeac19cc80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efea000bd70 tx=0x7efea000bda0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 shutdown_connections 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6818/3467870918,v1:172.21.15.63:6819/3467870918] conn(0x7efe940729c0 0x7efe94074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7efe9406cce0 0x7efe9406f1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efeac100e50 0x7efeac19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 --2- 172.21.15.63:0/3026716782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efeac101800 0x7efeac19cc80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.916+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 >> 172.21.15.63:0/3026716782 conn(0x7efeac0fc720 msgr2=0x7efeac0fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.917+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 shutdown_connections 2024-05-09T06:34:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:20.917+0000 7efeb16b1700 1 -- 172.21.15.63:0/3026716782 wait complete. 2024-05-09T06:34:21.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:21 smithi063 ceph-mon[27171]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:21.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:21 smithi193 ceph-mon[35141]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:21.899 INFO:teuthology.orchestra.run.smithi063.stdout:115964117013 2024-05-09T06:34:21.899 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T06:34:21.990 INFO:teuthology.orchestra.run.smithi063.stdout:81604378650 2024-05-09T06:34:21.991 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T06:34:22.039 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:22.039 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:22.096 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:22.097 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:23.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:23 smithi063 ceph-mon[27171]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:23.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:23 smithi193 ceph-mon[35141]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:24.759 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:24.759 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:25.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:25 smithi063 ceph-mon[27171]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:25.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:25 smithi193 ceph-mon[35141]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:27.488 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:27 smithi063 ceph-mon[27171]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:27.488 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:34:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:27 smithi193 ceph-mon[35141]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:34:28.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1060292705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280ff5e0 msgr2=0x7fb2280ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:28.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 --2- 172.21.15.63:0/1060292705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280ff5e0 0x7fb2280ff9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb224009910 tx=0x7fb224009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:28.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1060292705 shutdown_connections 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 --2- 172.21.15.63:0/1060292705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2280fff90 0x7fb228103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 --2- 172.21.15.63:0/1060292705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280ff5e0 0x7fb2280ff9c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1060292705 >> 172.21.15.63:0/1060292705 conn(0x7fb228074e00 msgr2=0x7fb228075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1060292705 shutdown_connections 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1060292705 wait complete. 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 Processor -- start 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.037+0000 7fb22fa4f700 1 -- start start 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22fa4f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2280ff5e0 0x7fb228197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22fa4f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280fff90 0x7fb228197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22fa4f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281985b0 con 0x7fb2280fff90 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22fa4f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281986f0 con 0x7fb2280ff5e0 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22cfea700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280fff90 0x7fb228197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22cfea700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280fff90 0x7fb228197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59080/0 (socket says 172.21.15.63:59080) 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22cfea700 1 -- 172.21.15.63:0/1628563273 learned_addr learned my addr 172.21.15.63:0/1628563273 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22d7eb700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2280ff5e0 0x7fb228197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:28.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22cfea700 1 -- 172.21.15.63:0/1628563273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2280ff5e0 msgr2=0x7fb228197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:28.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22cfea700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2280ff5e0 0x7fb228197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:28.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22cfea700 1 -- 172.21.15.63:0/1628563273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240095f0 con 0x7fb2280fff90 2024-05-09T06:34:28.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.038+0000 7fb22cfea700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280fff90 0x7fb228197fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb21800b480 tx=0x7fb21800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:28.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.039+0000 7fb21e7fc700 1 -- 172.21.15.63:0/1628563273 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb218011800 con 0x7fb2280fff90 2024-05-09T06:34:28.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.039+0000 7fb21e7fc700 1 -- 172.21.15.63:0/1628563273 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb218011e20 con 0x7fb2280fff90 2024-05-09T06:34:28.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.039+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2281a0b10 con 0x7fb2280fff90 2024-05-09T06:34:28.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.039+0000 7fb21e7fc700 1 -- 172.21.15.63:0/1628563273 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb218010ea0 con 0x7fb2280fff90 2024-05-09T06:34:28.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.039+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2281a0f20 con 0x7fb2280fff90 2024-05-09T06:34:28.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.040+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb228100980 con 0x7fb2280fff90 2024-05-09T06:34:28.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.040+0000 7fb21e7fc700 1 -- 172.21.15.63:0/1628563273 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb218010380 con 0x7fb2280fff90 2024-05-09T06:34:28.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.041+0000 7fb21e7fc700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb21406ccf0 0x7fb21406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:28.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.041+0000 7fb21e7fc700 1 -- 172.21.15.63:0/1628563273 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb21808b960 con 0x7fb2280fff90 2024-05-09T06:34:28.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.041+0000 7fb22d7eb700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb21406ccf0 0x7fb21406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:28.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.041+0000 7fb22d7eb700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb21406ccf0 0x7fb21406f1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb224006010 tx=0x7fb22401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:28.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.045+0000 7fb21e7fc700 1 -- 172.21.15.63:0/1628563273 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb2180560f0 con 0x7fb2280fff90 2024-05-09T06:34:28.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.255+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb22804ebb0 con 0x7fb2280fff90 2024-05-09T06:34:28.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.255+0000 7fb21e7fc700 1 -- 172.21.15.63:0/1628563273 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb218059710 con 0x7fb2280fff90 2024-05-09T06:34:28.237 INFO:teuthology.orchestra.run.smithi063.stdout:60129542176 2024-05-09T06:34:28.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.257+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb21406ccf0 msgr2=0x7fb21406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.257+0000 7fb22fa4f700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb21406ccf0 0x7fb21406f1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb224006010 tx=0x7fb22401a040 comp rx=0 tx=0).stop 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.257+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280fff90 msgr2=0x7fb228197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.257+0000 7fb22fa4f700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280fff90 0x7fb228197fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb21800b480 tx=0x7fb21800b790 comp rx=0 tx=0).stop 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.258+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 shutdown_connections 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.258+0000 7fb22fa4f700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fb21406ccf0 0x7fb21406f1b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.258+0000 7fb22fa4f700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2280ff5e0 0x7fb228197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.258+0000 7fb22fa4f700 1 --2- 172.21.15.63:0/1628563273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2280fff90 0x7fb228197fe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.258+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 >> 172.21.15.63:0/1628563273 conn(0x7fb228074e00 msgr2=0x7fb228102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.258+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 shutdown_connections 2024-05-09T06:34:28.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:28.258+0000 7fb22fa4f700 1 -- 172.21.15.63:0/1628563273 wait complete. 2024-05-09T06:34:28.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:28 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1628563273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T06:34:28.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:28 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1628563273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T06:34:29.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:29 smithi063 ceph-mon[27171]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:29.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:29 smithi193 ceph-mon[35141]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:29.827 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542176 for osd.1 2024-05-09T06:34:29.827 DEBUG:teuthology.parallel:result is None 2024-05-09T06:34:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.786+0000 7fbca79ec700 1 -- 172.21.15.63:0/237240030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 msgr2=0x7fbca0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.786+0000 7fbca79ec700 1 --2- 172.21.15.63:0/237240030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 0x7fbca0102150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009910 tx=0x7fbc9c009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.786+0000 7fbca79ec700 1 -- 172.21.15.63:0/237240030 shutdown_connections 2024-05-09T06:34:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.786+0000 7fbca79ec700 1 --2- 172.21.15.63:0/237240030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca0102690 0x7fbca010ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.786+0000 7fbca79ec700 1 --2- 172.21.15.63:0/237240030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 0x7fbca0102150 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.786+0000 7fbca79ec700 1 -- 172.21.15.63:0/237240030 >> 172.21.15.63:0/237240030 conn(0x7fbca0076750 msgr2=0x7fbca00fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.787+0000 7fbca79ec700 1 -- 172.21.15.63:0/237240030 shutdown_connections 2024-05-09T06:34:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.787+0000 7fbca79ec700 1 -- 172.21.15.63:0/237240030 wait complete. 2024-05-09T06:34:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.787+0000 7fbca79ec700 1 Processor -- start 2024-05-09T06:34:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.787+0000 7fbca79ec700 1 -- start start 2024-05-09T06:34:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.787+0000 7fbca79ec700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 0x7fbca019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.787+0000 7fbca79ec700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca0102690 0x7fbca019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.787+0000 7fbca79ec700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca019d370 con 0x7fbca0101d70 2024-05-09T06:34:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.787+0000 7fbca79ec700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0196950 con 0x7fbca0102690 2024-05-09T06:34:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca5788700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 0x7fbca019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca5788700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 0x7fbca019c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59136/0 (socket says 172.21.15.63:59136) 2024-05-09T06:34:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca5788700 1 -- 172.21.15.63:0/1411561379 learned_addr learned my addr 172.21.15.63:0/1411561379 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca4f87700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca0102690 0x7fbca019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca5788700 1 -- 172.21.15.63:0/1411561379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca0102690 msgr2=0x7fbca019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca5788700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca0102690 0x7fbca019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca5788700 1 -- 172.21.15.63:0/1411561379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0095f0 con 0x7fbca0101d70 2024-05-09T06:34:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca5788700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 0x7fbca019c810 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c006010 tx=0x7fbc9c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:30.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbc967fc700 1 -- 172.21.15.63:0/1411561379 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9c01d070 con 0x7fbca0101d70 2024-05-09T06:34:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbc967fc700 1 -- 172.21.15.63:0/1411561379 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc9c0056e0 con 0x7fbca0101d70 2024-05-09T06:34:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca0196bd0 con 0x7fbca0101d70 2024-05-09T06:34:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.788+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca01970c0 con 0x7fbca0101d70 2024-05-09T06:34:30.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.789+0000 7fbc967fc700 1 -- 172.21.15.63:0/1411561379 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9c017400 con 0x7fbca0101d70 2024-05-09T06:34:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.789+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc840052f0 con 0x7fbca0101d70 2024-05-09T06:34:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.790+0000 7fbc967fc700 1 -- 172.21.15.63:0/1411561379 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbc9c00f6f0 con 0x7fbca0101d70 2024-05-09T06:34:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.791+0000 7fbc967fc700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbc8c06cd30 0x7fbc8c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.791+0000 7fbc967fc700 1 -- 172.21.15.63:0/1411561379 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbc9c08d4b0 con 0x7fbca0101d70 2024-05-09T06:34:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.791+0000 7fbca4f87700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbc8c06cd30 0x7fbc8c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.792+0000 7fbca4f87700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbc8c06cd30 0x7fbc8c06f1f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc90005950 tx=0x7fbc900058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:30.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.796+0000 7fbc967fc700 1 -- 172.21.15.63:0/1411561379 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbc9c05b6b0 con 0x7fbca0101d70 2024-05-09T06:34:30.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.014+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbc84005160 con 0x7fbca0101d70 2024-05-09T06:34:30.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.014+0000 7fbc967fc700 1 -- 172.21.15.63:0/1411561379 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbc9c05b240 con 0x7fbca0101d70 2024-05-09T06:34:30.996 INFO:teuthology.orchestra.run.smithi063.stdout:38654705702 2024-05-09T06:34:30.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbc8c06cd30 msgr2=0x7fbc8c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:30.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbc8c06cd30 0x7fbc8c06f1f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc90005950 tx=0x7fbc900058e0 comp rx=0 tx=0).stop 2024-05-09T06:34:30.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 msgr2=0x7fbca019c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:30.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 0x7fbca019c810 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c006010 tx=0x7fbc9c00be20 comp rx=0 tx=0).stop 2024-05-09T06:34:30.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 shutdown_connections 2024-05-09T06:34:30.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbca0101d70 0x7fbca019c810 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:30.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbc8c06cd30 0x7fbc8c06f1f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:30.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 --2- 172.21.15.63:0/1411561379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca0102690 0x7fbca019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:30.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 >> 172.21.15.63:0/1411561379 conn(0x7fbca0076750 msgr2=0x7fbca0103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:30.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 shutdown_connections 2024-05-09T06:34:30.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.016+0000 7fbca79ec700 1 -- 172.21.15.63:0/1411561379 wait complete. 2024-05-09T06:34:31.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.579+0000 7f80142c3700 1 -- 172.21.15.63:0/182383337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 msgr2=0x7f800c105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:31.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.579+0000 7f80142c3700 1 --2- 172.21.15.63:0/182383337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 0x7f800c105ea0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8008009910 tx=0x7f8008009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:31.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.579+0000 7f80142c3700 1 -- 172.21.15.63:0/182383337 shutdown_connections 2024-05-09T06:34:31.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.579+0000 7f80142c3700 1 --2- 172.21.15.63:0/182383337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c0ffac0 0x7f800c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:31.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.579+0000 7f80142c3700 1 --2- 172.21.15.63:0/182383337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 0x7f800c105ea0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:31.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.579+0000 7f80142c3700 1 -- 172.21.15.63:0/182383337 >> 172.21.15.63:0/182383337 conn(0x7f800c0fb5f0 msgr2=0x7f800c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:31.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.579+0000 7f80142c3700 1 -- 172.21.15.63:0/182383337 shutdown_connections 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.579+0000 7f80142c3700 1 -- 172.21.15.63:0/182383337 wait complete. 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.580+0000 7f80142c3700 1 Processor -- start 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.580+0000 7f80142c3700 1 -- start start 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.580+0000 7f80142c3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c0ffac0 0x7f800c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.580+0000 7f80142c3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 0x7f800c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.580+0000 7f80142c3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c199920 con 0x7f800c105ac0 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.580+0000 7f80142c3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c199a60 con 0x7f800c0ffac0 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f801185e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 0x7f800c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f801185e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 0x7f800c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59096/0 (socket says 172.21.15.63:59096) 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f801185e700 1 -- 172.21.15.63:0/1300687216 learned_addr learned my addr 172.21.15.63:0/1300687216 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f801205f700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c0ffac0 0x7f800c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f801185e700 1 -- 172.21.15.63:0/1300687216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c0ffac0 msgr2=0x7f800c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f801185e700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c0ffac0 0x7f800c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f801185e700 1 -- 172.21.15.63:0/1300687216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080095f0 con 0x7f800c105ac0 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f801185e700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 0x7f800c1992d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c930 tx=0x7f7ffc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f80037fe700 1 -- 172.21.15.63:0/1300687216 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ffc007ad0 con 0x7f800c105ac0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f80037fe700 1 -- 172.21.15.63:0/1300687216 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ffc00ce80 con 0x7f800c105ac0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f80037fe700 1 -- 172.21.15.63:0/1300687216 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ffc018630 con 0x7f800c105ac0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c19d7f0 con 0x7f800c105ac0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.581+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c19dd40 con 0x7f800c105ac0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.582+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c102d60 con 0x7f800c105ac0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.583+0000 7f80037fe700 1 -- 172.21.15.63:0/1300687216 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7ffc00f450 con 0x7f800c105ac0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.584+0000 7f80037fe700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7ff806cce0 0x7f7ff806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.584+0000 7f80037fe700 1 -- 172.21.15.63:0/1300687216 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7ffc08be10 con 0x7f800c105ac0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.584+0000 7f801205f700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7ff806cce0 0x7f7ff806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.584+0000 7f801205f700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7ff806cce0 0x7f7ff806f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c00 tx=0x7f800801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.589+0000 7f80037fe700 1 -- 172.21.15.63:0/1300687216 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7ffc0565a0 con 0x7f800c105ac0 2024-05-09T06:34:31.116 INFO:teuthology.orchestra.run.smithi063.stdout:171798691851 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.803+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f800c101200 con 0x7f800c105ac0 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.803+0000 7f80037fe700 1 -- 172.21.15.63:0/1300687216 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ffc059bc0 con 0x7f800c105ac0 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.805+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7ff806cce0 msgr2=0x7f7ff806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.805+0000 7f80142c3700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7ff806cce0 0x7f7ff806f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c00 tx=0x7f800801a040 comp rx=0 tx=0).stop 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.805+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 msgr2=0x7f800c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.805+0000 7f80142c3700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 0x7f800c1992d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c930 tx=0x7f7ffc00cc40 comp rx=0 tx=0).stop 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.805+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 shutdown_connections 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.805+0000 7f80142c3700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7ff806cce0 0x7f7ff806f1a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.805+0000 7f80142c3700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c0ffac0 0x7f800c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.806+0000 7f80142c3700 1 --2- 172.21.15.63:0/1300687216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f800c105ac0 0x7f800c1992d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.806+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 >> 172.21.15.63:0/1300687216 conn(0x7f800c0fb5f0 msgr2=0x7f800c103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.806+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 shutdown_connections 2024-05-09T06:34:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.806+0000 7f80142c3700 1 -- 172.21.15.63:0/1300687216 wait complete. 2024-05-09T06:34:31.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:31 smithi063 ceph-mon[27171]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:31.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:31 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1300687216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T06:34:31.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:31 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2781608079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T06:34:31.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:31 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1411561379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T06:34:31.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:31 smithi193 ceph-mon[35141]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:31.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:31 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1300687216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T06:34:31.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:31 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2781608079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T06:34:31.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:31 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1411561379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T06:34:31.751 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691851 for osd.5 2024-05-09T06:34:31.752 DEBUG:teuthology.parallel:result is None 2024-05-09T06:34:31.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.987+0000 7f3a13198700 1 -- 172.21.15.63:0/2497344187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 msgr2=0x7f3a0c0ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:31.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.987+0000 7f3a13198700 1 --2- 172.21.15.63:0/2497344187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 0x7f3a0c0ff7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009910 tx=0x7f39fc009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:31.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.987+0000 7f3a13198700 1 -- 172.21.15.63:0/2497344187 shutdown_connections 2024-05-09T06:34:31.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.987+0000 7f3a13198700 1 --2- 172.21.15.63:0/2497344187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c0ffd70 0x7f3a0c1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:31.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.987+0000 7f3a13198700 1 --2- 172.21.15.63:0/2497344187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 0x7f3a0c0ff7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:31.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.987+0000 7f3a13198700 1 -- 172.21.15.63:0/2497344187 >> 172.21.15.63:0/2497344187 conn(0x7f3a0c075f10 msgr2=0x7f3a0c0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.987+0000 7f3a13198700 1 -- 172.21.15.63:0/2497344187 shutdown_connections 2024-05-09T06:34:31.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.987+0000 7f3a13198700 1 -- 172.21.15.63:0/2497344187 wait complete. 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a13198700 1 Processor -- start 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a13198700 1 -- start start 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a13198700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 0x7f3a0c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a13198700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c0ffd70 0x7f3a0c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a13198700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c19d3e0 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a13198700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c1969c0 con 0x7f3a0c0ffd70 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a10f34700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 0x7f3a0c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a10f34700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 0x7f3a0c19c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59162/0 (socket says 172.21.15.63:59162) 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a10f34700 1 -- 172.21.15.63:0/1108952776 learned_addr learned my addr 172.21.15.63:0/1108952776 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:31.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a0bfff700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c0ffd70 0x7f3a0c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a10f34700 1 -- 172.21.15.63:0/1108952776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c0ffd70 msgr2=0x7f3a0c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a10f34700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c0ffd70 0x7f3a0c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a10f34700 1 -- 172.21.15.63:0/1108952776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0095f0 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a0bfff700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c0ffd70 0x7f3a0c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.988+0000 7f3a10f34700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 0x7f3a0c19c880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f39fc000c00 tx=0x7f39fc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.989+0000 7f3a09ffb700 1 -- 172.21.15.63:0/1108952776 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39fc01d070 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.989+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c196c40 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.989+0000 7f3a09ffb700 1 -- 172.21.15.63:0/1108952776 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39fc0056e0 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.989+0000 7f3a09ffb700 1 -- 172.21.15.63:0/1108952776 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39fc00fe80 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.989+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c197130 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.990+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c109250 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.991+0000 7f3a09ffb700 1 -- 172.21.15.63:0/1108952776 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f39fc00f6f0 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.991+0000 7f3a09ffb700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806ccf0 0x7f39f806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.991+0000 7f3a09ffb700 1 -- 172.21.15.63:0/1108952776 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f39fc08d340 con 0x7f3a0c0ff3c0 2024-05-09T06:34:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.991+0000 7f3a0bfff700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806ccf0 0x7f39f806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:31.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.992+0000 7f3a0bfff700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806ccf0 0x7f39f806f1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c197b80 tx=0x7f39f4009040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:31.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:31.996+0000 7f3a09ffb700 1 -- 172.21.15.63:0/1108952776 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f39fc05b4e0 con 0x7f3a0c0ff3c0 2024-05-09T06:34:32.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.201+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3a0c197e80 con 0x7f3a0c0ff3c0 2024-05-09T06:34:32.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.201+0000 7f3a09ffb700 1 -- 172.21.15.63:0/1108952776 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39fc05b070 con 0x7f3a0c0ff3c0 2024-05-09T06:34:32.182 INFO:teuthology.orchestra.run.smithi063.stdout:150323855377 2024-05-09T06:34:32.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806ccf0 msgr2=0x7f39f806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806ccf0 0x7f39f806f1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c197b80 tx=0x7f39f4009040 comp rx=0 tx=0).stop 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 msgr2=0x7f3a0c19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 0x7f3a0c19c880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f39fc000c00 tx=0x7f39fc00be20 comp rx=0 tx=0).stop 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 shutdown_connections 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a0c0ff3c0 0x7f3a0c19c880 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806ccf0 0x7f39f806f1b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 --2- 172.21.15.63:0/1108952776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c0ffd70 0x7f3a0c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 >> 172.21.15.63:0/1108952776 conn(0x7f3a0c075f10 msgr2=0x7f3a0c104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 shutdown_connections 2024-05-09T06:34:32.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:32.204+0000 7f3a13198700 1 -- 172.21.15.63:0/1108952776 wait complete. 2024-05-09T06:34:32.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.678+0000 7f3efb8ee700 1 -- 172.21.15.63:0/3185858089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 msgr2=0x7f3ef41020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:32.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.678+0000 7f3efb8ee700 1 --2- 172.21.15.63:0/3185858089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 0x7f3ef41020d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009910 tx=0x7f3ef0009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:32.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.679+0000 7f3efb8ee700 1 -- 172.21.15.63:0/3185858089 shutdown_connections 2024-05-09T06:34:32.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.679+0000 7f3efb8ee700 1 --2- 172.21.15.63:0/3185858089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef41026a0 0x7f3ef410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:32.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.679+0000 7f3efb8ee700 1 --2- 172.21.15.63:0/3185858089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 0x7f3ef41020d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:32.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.679+0000 7f3efb8ee700 1 -- 172.21.15.63:0/3185858089 >> 172.21.15.63:0/3185858089 conn(0x7f3ef40fb5f0 msgr2=0x7f3ef40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:32.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.679+0000 7f3efb8ee700 1 -- 172.21.15.63:0/3185858089 shutdown_connections 2024-05-09T06:34:32.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.679+0000 7f3efb8ee700 1 -- 172.21.15.63:0/3185858089 wait complete. 2024-05-09T06:34:32.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.679+0000 7f3efb8ee700 1 Processor -- start 2024-05-09T06:34:32.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.679+0000 7f3efb8ee700 1 -- start start 2024-05-09T06:34:32.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3efb8ee700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 0x7f3ef40ff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:32.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3efb8ee700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef41026a0 0x7f3ef40fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:32.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3efb8ee700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4100460 con 0x7f3ef4101cf0 2024-05-09T06:34:32.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3efb8ee700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef41005a0 con 0x7f3ef41026a0 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3ef968a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 0x7f3ef40ff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3ef968a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 0x7f3ef40ff9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59112/0 (socket says 172.21.15.63:59112) 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3ef968a700 1 -- 172.21.15.63:0/2781608079 learned_addr learned my addr 172.21.15.63:0/2781608079 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3ef8e89700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef41026a0 0x7f3ef40fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3ef968a700 1 -- 172.21.15.63:0/2781608079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef41026a0 msgr2=0x7f3ef40fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3ef968a700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef41026a0 0x7f3ef40fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3ef968a700 1 -- 172.21.15.63:0/2781608079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00095f0 con 0x7f3ef4101cf0 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3ef968a700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 0x7f3ef40ff9e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00092e0 tx=0x7f3ef000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.680+0000 7f3eea7fc700 1 -- 172.21.15.63:0/2781608079 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ef001d070 con 0x7f3ef4101cf0 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.681+0000 7f3eea7fc700 1 -- 172.21.15.63:0/2781608079 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ef000f460 con 0x7f3ef4101cf0 2024-05-09T06:34:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.681+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef4100820 con 0x7f3ef4101cf0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.681+0000 7f3eea7fc700 1 -- 172.21.15.63:0/2781608079 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ef0017630 con 0x7f3ef4101cf0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.681+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef41a2f70 con 0x7f3ef4101cf0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.682+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef4069740 con 0x7f3ef4101cf0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.683+0000 7f3eea7fc700 1 -- 172.21.15.63:0/2781608079 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3ef000f5d0 con 0x7f3ef4101cf0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.684+0000 7f3eea7fc700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3ee006ccf0 0x7f3ee006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.684+0000 7f3eea7fc700 1 -- 172.21.15.63:0/2781608079 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3ef008d390 con 0x7f3ef4101cf0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.684+0000 7f3ef8e89700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3ee006ccf0 0x7f3ee006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.685+0000 7f3ef8e89700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3ee006ccf0 0x7f3ee006f1b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ef41010c0 tx=0x7f3ee400a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.689+0000 7f3eea7fc700 1 -- 172.21.15.63:0/2781608079 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3ef0057aa0 con 0x7f3ef4101cf0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.901+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3ef4105530 con 0x7f3ef4101cf0 2024-05-09T06:34:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.901+0000 7f3eea7fc700 1 -- 172.21.15.63:0/2781608079 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ef005b0c0 con 0x7f3ef4101cf0 2024-05-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi063.stdout:115964117015 2024-05-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.903+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3ee006ccf0 msgr2=0x7f3ee006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.903+0000 7f3efb8ee700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3ee006ccf0 0x7f3ee006f1b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ef41010c0 tx=0x7f3ee400a300 comp rx=0 tx=0).stop 2024-05-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.903+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 msgr2=0x7f3ef40ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.903+0000 7f3efb8ee700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 0x7f3ef40ff9e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00092e0 tx=0x7f3ef000bd70 comp rx=0 tx=0).stop 2024-05-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.903+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 shutdown_connections 2024-05-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.903+0000 7f3efb8ee700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ef4101cf0 0x7f3ef40ff9e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.903+0000 7f3efb8ee700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3ee006ccf0 0x7f3ee006f1b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.903+0000 7f3efb8ee700 1 --2- 172.21.15.63:0/2781608079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef41026a0 0x7f3ef40fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.903+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 >> 172.21.15.63:0/2781608079 conn(0x7f3ef40fb5f0 msgr2=0x7f3ef4103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.904+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 shutdown_connections 2024-05-09T06:34:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:30.904+0000 7f3efb8ee700 1 -- 172.21.15.63:0/2781608079 wait complete. 2024-05-09T06:34:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:32 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1108952776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T06:34:32.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:32 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1108952776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T06:34:32.873 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117013 got 115964117015 for osd.3 2024-05-09T06:34:32.873 DEBUG:teuthology.parallel:result is None 2024-05-09T06:34:33.466 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:33 smithi063 ceph-mon[27171]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:33.587 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705702 for osd.0 2024-05-09T06:34:33.587 DEBUG:teuthology.parallel:result is None 2024-05-09T06:34:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:33 smithi193 ceph-mon[35141]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.788+0000 7f58f9ffc700 1 -- 172.21.15.63:0/4094190505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f41026c0 msgr2=0x7f58f410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.788+0000 7f58f9ffc700 1 --2- 172.21.15.63:0/4094190505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f41026c0 0x7f58f410abb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009910 tx=0x7f58e8009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.788+0000 7f58f9ffc700 1 -- 172.21.15.63:0/4094190505 shutdown_connections 2024-05-09T06:34:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.788+0000 7f58f9ffc700 1 --2- 172.21.15.63:0/4094190505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f41026c0 0x7f58f410abb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.788+0000 7f58f9ffc700 1 --2- 172.21.15.63:0/4094190505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f4101da0 0x7f58f4102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.788+0000 7f58f9ffc700 1 -- 172.21.15.63:0/4094190505 >> 172.21.15.63:0/4094190505 conn(0x7f58f40fb610 msgr2=0x7f58f40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.788+0000 7f58f9ffc700 1 -- 172.21.15.63:0/4094190505 shutdown_connections 2024-05-09T06:34:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.788+0000 7f58f9ffc700 1 -- 172.21.15.63:0/4094190505 wait complete. 2024-05-09T06:34:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.788+0000 7f58f9ffc700 1 Processor -- start 2024-05-09T06:34:33.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f9ffc700 1 -- start start 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f9ffc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f4101da0 0x7f58f4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f9ffc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f41026c0 0x7f58f4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f9ffc700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4199990 con 0x7f58f4101da0 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f9ffc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4199ad0 con 0x7f58f41026c0 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f4101da0 0x7f58f4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f4101da0 0x7f58f4198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59176/0 (socket says 172.21.15.63:59176) 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f37fe700 1 -- 172.21.15.63:0/744558896 learned_addr learned my addr 172.21.15.63:0/744558896 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f2ffd700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f41026c0 0x7f58f4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f37fe700 1 -- 172.21.15.63:0/744558896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f41026c0 msgr2=0x7f58f4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f37fe700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f41026c0 0x7f58f4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f37fe700 1 -- 172.21.15.63:0/744558896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80095f0 con 0x7f58f4101da0 2024-05-09T06:34:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f2ffd700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f41026c0 0x7f58f4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.789+0000 7f58f37fe700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f4101da0 0x7f58f4198e00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58e400c930 tx=0x7f58e400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.790+0000 7f58f0ff9700 1 -- 172.21.15.63:0/744558896 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e4007ad0 con 0x7f58f4101da0 2024-05-09T06:34:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.790+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f419d860 con 0x7f58f4101da0 2024-05-09T06:34:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.790+0000 7f58f0ff9700 1 -- 172.21.15.63:0/744558896 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58e4007c30 con 0x7f58f4101da0 2024-05-09T06:34:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.790+0000 7f58f0ff9700 1 -- 172.21.15.63:0/744558896 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e4018610 con 0x7f58f4101da0 2024-05-09T06:34:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.790+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f419ddb0 con 0x7f58f4101da0 2024-05-09T06:34:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.791+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f4069a40 con 0x7f58f4101da0 2024-05-09T06:34:33.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.791+0000 7f58f0ff9700 1 -- 172.21.15.63:0/744558896 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f58e400f450 con 0x7f58f4101da0 2024-05-09T06:34:33.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.792+0000 7f58f0ff9700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58dc06cce0 0x7f58dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:33.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.792+0000 7f58f0ff9700 1 -- 172.21.15.63:0/744558896 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f58e408c140 con 0x7f58f4101da0 2024-05-09T06:34:33.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.792+0000 7f58f2ffd700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58dc06cce0 0x7f58dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:33.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.792+0000 7f58f2ffd700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58dc06cce0 0x7f58dc06f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f58f419a360 tx=0x7f58e801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:33.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.797+0000 7f58f0ff9700 1 -- 172.21.15.63:0/744558896 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f58e4056850 con 0x7f58f4101da0 2024-05-09T06:34:33.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.999+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f58f404ebb0 con 0x7f58f4101da0 2024-05-09T06:34:33.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:33.999+0000 7f58f0ff9700 1 -- 172.21.15.63:0/744558896 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f58e4059e70 con 0x7f58f4101da0 2024-05-09T06:34:33.980 INFO:teuthology.orchestra.run.smithi063.stdout:81604378652 2024-05-09T06:34:33.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.001+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58dc06cce0 msgr2=0x7f58dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:33.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.001+0000 7f58f9ffc700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58dc06cce0 0x7f58dc06f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f58f419a360 tx=0x7f58e801a040 comp rx=0 tx=0).stop 2024-05-09T06:34:33.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.001+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f4101da0 msgr2=0x7f58f4198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:33.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.001+0000 7f58f9ffc700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f4101da0 0x7f58f4198e00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58e400c930 tx=0x7f58e400ccf0 comp rx=0 tx=0).stop 2024-05-09T06:34:33.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.002+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 shutdown_connections 2024-05-09T06:34:33.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.002+0000 7f58f9ffc700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58f4101da0 0x7f58f4198e00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:33.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.002+0000 7f58f9ffc700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f58dc06cce0 0x7f58dc06f1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:33.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.002+0000 7f58f9ffc700 1 --2- 172.21.15.63:0/744558896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f41026c0 0x7f58f4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:33.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.002+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 >> 172.21.15.63:0/744558896 conn(0x7f58f40fb610 msgr2=0x7f58f410a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:33.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.002+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 shutdown_connections 2024-05-09T06:34:33.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:34.002+0000 7f58f9ffc700 1 -- 172.21.15.63:0/744558896 wait complete. 2024-05-09T06:34:34.464 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855374 got 150323855377 for osd.4 2024-05-09T06:34:34.464 DEBUG:teuthology.parallel:result is None 2024-05-09T06:34:34.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:34 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/744558896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T06:34:34.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:34 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/744558896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T06:34:35.072 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378650 got 81604378652 for osd.2 2024-05-09T06:34:35.073 DEBUG:teuthology.parallel:result is None 2024-05-09T06:34:35.073 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T06:34:35.073 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T06:34:35.300 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:35.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:35 smithi063 ceph-mon[27171]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:35.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:35 smithi193 ceph-mon[35141]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.007+0000 7fedf9da2700 1 -- 172.21.15.63:0/994597152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf4105ab0 msgr2=0x7fedf4105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.007+0000 7fedf9da2700 1 --2- 172.21.15.63:0/994597152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf4105ab0 0x7fedf4105e90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fede4009910 tx=0x7fede4009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.007+0000 7fedf9da2700 1 -- 172.21.15.63:0/994597152 shutdown_connections 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.007+0000 7fedf9da2700 1 --2- 172.21.15.63:0/994597152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf40ffab0 0x7fedf40fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.007+0000 7fedf9da2700 1 --2- 172.21.15.63:0/994597152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf4105ab0 0x7fedf4105e90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.007+0000 7fedf9da2700 1 -- 172.21.15.63:0/994597152 >> 172.21.15.63:0/994597152 conn(0x7fedf4076750 msgr2=0x7fedf40fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.007+0000 7fedf9da2700 1 -- 172.21.15.63:0/994597152 shutdown_connections 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.007+0000 7fedf9da2700 1 -- 172.21.15.63:0/994597152 wait complete. 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.007+0000 7fedf9da2700 1 Processor -- start 2024-05-09T06:34:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf9da2700 1 -- start start 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf9da2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf40ffab0 0x7fedf41958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf9da2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf4105ab0 0x7fedf4195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf9da2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf4069560 con 0x7fedf40ffab0 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf9da2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf4196340 con 0x7fedf4105ab0 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf40ffab0 0x7fedf41958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf40ffab0 0x7fedf41958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33862/0 (socket says 172.21.15.63:33862) 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf37fe700 1 -- 172.21.15.63:0/1579725725 learned_addr learned my addr 172.21.15.63:0/1579725725 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf2ffd700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf4105ab0 0x7fedf4195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf37fe700 1 -- 172.21.15.63:0/1579725725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf4105ab0 msgr2=0x7fedf4195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf37fe700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf4105ab0 0x7fedf4195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf37fe700 1 -- 172.21.15.63:0/1579725725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede40095f0 con 0x7fedf40ffab0 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.008+0000 7fedf2ffd700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf4105ab0 0x7fedf4195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.009+0000 7fedf37fe700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf40ffab0 0x7fedf41958c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fede40092e0 tx=0x7fede400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:36.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.009+0000 7fedf0ff9700 1 -- 172.21.15.63:0/1579725725 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede401d070 con 0x7fedf40ffab0 2024-05-09T06:34:36.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.009+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf41965c0 con 0x7fedf40ffab0 2024-05-09T06:34:36.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.009+0000 7fedf0ff9700 1 -- 172.21.15.63:0/1579725725 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fede400f460 con 0x7fedf40ffab0 2024-05-09T06:34:36.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.009+0000 7fedf0ff9700 1 -- 172.21.15.63:0/1579725725 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede4017630 con 0x7fedf40ffab0 2024-05-09T06:34:36.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.009+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf41a3100 con 0x7fedf40ffab0 2024-05-09T06:34:36.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.010+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf41080b0 con 0x7fedf40ffab0 2024-05-09T06:34:36.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.010+0000 7fedf0ff9700 1 -- 172.21.15.63:0/1579725725 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fede400f5d0 con 0x7fedf40ffab0 2024-05-09T06:34:36.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.011+0000 7fedf0ff9700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7feddc06ccf0 0x7feddc06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:36.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.011+0000 7fedf0ff9700 1 -- 172.21.15.63:0/1579725725 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fede408d430 con 0x7fedf40ffab0 2024-05-09T06:34:36.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.011+0000 7fedf2ffd700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7feddc06ccf0 0x7feddc06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:36.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.011+0000 7fedf2ffd700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7feddc06ccf0 0x7feddc06f1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedf4068e70 tx=0x7fede8008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:36.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.017+0000 7fedf0ff9700 1 -- 172.21.15.63:0/1579725725 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fede405b5d0 con 0x7fedf40ffab0 2024-05-09T06:34:37.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.223+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fedf4068be0 con 0x7feddc06ccf0 2024-05-09T06:34:37.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.225+0000 7fedf0ff9700 1 -- 172.21.15.63:0/1579725725 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28229 (secure 0 0 0) 0x7fedf4068be0 con 0x7feddc06ccf0 2024-05-09T06:34:37.206 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7feddc06ccf0 msgr2=0x7feddc06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7feddc06ccf0 0x7feddc06f1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedf4068e70 tx=0x7fede8008040 comp rx=0 tx=0).stop 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf40ffab0 msgr2=0x7fedf41958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf40ffab0 0x7fedf41958c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fede40092e0 tx=0x7fede400be20 comp rx=0 tx=0).stop 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 shutdown_connections 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedf40ffab0 0x7fedf41958c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7feddc06ccf0 0x7feddc06f1b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 --2- 172.21.15.63:0/1579725725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf4105ab0 0x7fedf4195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 >> 172.21.15.63:0/1579725725 conn(0x7fedf4076750 msgr2=0x7fedf40fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 shutdown_connections 2024-05-09T06:34:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:37.227+0000 7fedf9da2700 1 -- 172.21.15.63:0/1579725725 wait complete. 2024-05-09T06:34:37.209 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-09T06:34:37.566 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:37 smithi063 ceph-mon[27171]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:37 smithi193 ceph-mon[35141]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:37.839 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-05-09T06:34:35.565890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":174256,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":562296656,"statfs":{"total":575970213888,"available":575791775744,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":174316198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002365"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-09T06:33:45.892317+0000","last_change":"2024-05-09T06:33:18.887245+0000","last_active":"2024-05-09T06:33:45.892317+0000","last_peered":"2024-05-09T06:33:45.892317+0000","last_clean":"2024-05-09T06:33:45.892317+0000","last_became_active":"2024-05-09T06:33:18.886826+0000","last_became_peered":"2024-05-09T06:33:18.886826+0000","last_unstale":"2024-05-09T06:33:45.892317+0000","last_undegraded":"2024-05-09T06:33:45.892317+0000","last_fullsized":"2024-05-09T06:33:45.892317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:32.092347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:32.092347+0000","last_clean_scrub_stamp":"2024-05-09T06:32:32.092347+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:22.518475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31480,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713672,"statfs":{"total":95995035648,"available":95962800128,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31484,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713668,"statfs":{"total":95995035648,"available":95962796032,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu May 9 06:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.746},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu May 9 06:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu May 9 06:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu May 9 06:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu May 9 06:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu May 9 06:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu May 9 06:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.434},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu May 9 06:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.38300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu May 9 06:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.254}]},{"osd":1,"last update":"Thu May 9 06:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu May 9 06:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu May 9 06:34:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":19.359999999999999,"5min":19.359999999999999,"15min":19.359999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu May 9 06:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.27800000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48399999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.35999999999999999,"15min":0.36399999999999999},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53100000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu May 9 06:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.374,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.46400000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu May 9 06:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu May 9 06:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.62},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu May 9 06:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35399999999999998},"min":{"1min":0.249,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.35999999999999999},"min":{"1min":0.26200000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu May 9 06:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu May 9 06:33:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.499}]},{"osd":4,"last update":"Thu May 9 06:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:34:37.840 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T06:34:38.055 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:38.461 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:38 smithi063 ceph-mon[27171]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T06:34:38.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:38 smithi193 ceph-mon[35141]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T06:34:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.601+0000 7f59336ae700 1 -- 172.21.15.63:0/4060383672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c105ac0 msgr2=0x7f592c105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.601+0000 7f59336ae700 1 --2- 172.21.15.63:0/4060383672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c105ac0 0x7f592c105ea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5928009910 tx=0x7f5928009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.601+0000 7f59336ae700 1 -- 172.21.15.63:0/4060383672 shutdown_connections 2024-05-09T06:34:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.601+0000 7f59336ae700 1 --2- 172.21.15.63:0/4060383672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f592c0ffac0 0x7f592c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:39.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.601+0000 7f59336ae700 1 --2- 172.21.15.63:0/4060383672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c105ac0 0x7f592c105ea0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:39.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.601+0000 7f59336ae700 1 -- 172.21.15.63:0/4060383672 >> 172.21.15.63:0/4060383672 conn(0x7f592c0fb5f0 msgr2=0x7f592c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:39.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.601+0000 7f59336ae700 1 -- 172.21.15.63:0/4060383672 shutdown_connections 2024-05-09T06:34:39.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.601+0000 7f59336ae700 1 -- 172.21.15.63:0/4060383672 wait complete. 2024-05-09T06:34:39.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.601+0000 7f59336ae700 1 Processor -- start 2024-05-09T06:34:39.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f59336ae700 1 -- start start 2024-05-09T06:34:39.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f59336ae700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c0ffac0 0x7f592c1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f59336ae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f592c105ac0 0x7f592c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f59336ae700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1984b0 con 0x7f592c0ffac0 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f59336ae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1985f0 con 0x7f592c105ac0 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f593144a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c0ffac0 0x7f592c1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f593144a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c0ffac0 0x7f592c1979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33882/0 (socket says 172.21.15.63:33882) 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f593144a700 1 -- 172.21.15.63:0/870490176 learned_addr learned my addr 172.21.15.63:0/870490176 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f5930c49700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f592c105ac0 0x7f592c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f593144a700 1 -- 172.21.15.63:0/870490176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f592c105ac0 msgr2=0x7f592c197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f593144a700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f592c105ac0 0x7f592c197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f593144a700 1 -- 172.21.15.63:0/870490176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59280095f0 con 0x7f592c0ffac0 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f5930c49700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f592c105ac0 0x7f592c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.602+0000 7f593144a700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c0ffac0 0x7f592c1979a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f592800bd70 tx=0x7f592800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.603+0000 7f59227fc700 1 -- 172.21.15.63:0/870490176 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f592801d070 con 0x7f592c0ffac0 2024-05-09T06:34:39.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.603+0000 7f59227fc700 1 -- 172.21.15.63:0/870490176 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59280056e0 con 0x7f592c0ffac0 2024-05-09T06:34:39.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.603+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c1a0a10 con 0x7f592c0ffac0 2024-05-09T06:34:39.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.603+0000 7f59227fc700 1 -- 172.21.15.63:0/870490176 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5928017400 con 0x7f592c0ffac0 2024-05-09T06:34:39.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.603+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c1a0d70 con 0x7f592c0ffac0 2024-05-09T06:34:39.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.604+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c1080c0 con 0x7f592c0ffac0 2024-05-09T06:34:39.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.604+0000 7f59227fc700 1 -- 172.21.15.63:0/870490176 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5928003a60 con 0x7f592c0ffac0 2024-05-09T06:34:39.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.605+0000 7f59227fc700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f591806cce0 0x7f591806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:39.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.605+0000 7f59227fc700 1 -- 172.21.15.63:0/870490176 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f592808d370 con 0x7f592c0ffac0 2024-05-09T06:34:39.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.605+0000 7f5930c49700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f591806cce0 0x7f591806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:39.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.605+0000 7f5930c49700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f591806cce0 0x7f591806f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f591c005950 tx=0x7f591c0058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:39.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.609+0000 7f59227fc700 1 -- 172.21.15.63:0/870490176 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5928057a80 con 0x7f592c0ffac0 2024-05-09T06:34:39.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:39 smithi193 ceph-mon[35141]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.809+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f592c0694e0 con 0x7f591806cce0 2024-05-09T06:34:39.790 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:39 smithi063 ceph-mon[27171]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:39.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.811+0000 7f59227fc700 1 -- 172.21.15.63:0/870490176 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28173 (secure 0 0 0) 0x7f592c0694e0 con 0x7f591806cce0 2024-05-09T06:34:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f591806cce0 msgr2=0x7f591806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f591806cce0 0x7f591806f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f591c005950 tx=0x7f591c0058e0 comp rx=0 tx=0).stop 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c0ffac0 msgr2=0x7f592c1979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c0ffac0 0x7f592c1979a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f592800bd70 tx=0x7f592800be50 comp rx=0 tx=0).stop 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 shutdown_connections 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f592c0ffac0 0x7f592c1979a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f591806cce0 0x7f591806f1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 --2- 172.21.15.63:0/870490176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f592c105ac0 0x7f592c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 >> 172.21.15.63:0/870490176 conn(0x7f592c0fb5f0 msgr2=0x7f592c0fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 shutdown_connections 2024-05-09T06:34:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:39.814+0000 7f59336ae700 1 -- 172.21.15.63:0/870490176 wait complete. 2024-05-09T06:34:39.796 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-09T06:34:40.401 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-05-09T06:34:39.566650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":174256,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":562296656,"statfs":{"total":575970213888,"available":575791775744,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":174316198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002374"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-09T06:33:45.892317+0000","last_change":"2024-05-09T06:33:18.887245+0000","last_active":"2024-05-09T06:33:45.892317+0000","last_peered":"2024-05-09T06:33:45.892317+0000","last_clean":"2024-05-09T06:33:45.892317+0000","last_became_active":"2024-05-09T06:33:18.886826+0000","last_became_peered":"2024-05-09T06:33:18.886826+0000","last_unstale":"2024-05-09T06:33:45.892317+0000","last_undegraded":"2024-05-09T06:33:45.892317+0000","last_fullsized":"2024-05-09T06:33:45.892317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:32:32.092347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:32:32.092347+0000","last_clean_scrub_stamp":"2024-05-09T06:32:32.092347+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:22.518475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31480,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713672,"statfs":{"total":95995035648,"available":95962800128,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31484,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713668,"statfs":{"total":95995035648,"available":95962796032,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu May 9 06:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.442},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu May 9 06:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu May 9 06:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.374}]},{"osd":3,"last update":"Thu May 9 06:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.41499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117017,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu May 9 06:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu May 9 06:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.438},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu May 9 06:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu May 9 06:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.28799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu May 9 06:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.371,"15min":0.38100000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48899999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.254,"5min":0.248,"15min":0.248},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.27400000000000002}]},{"osd":1,"last update":"Thu May 9 06:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.246,"5min":0.215,"15min":0.215},"max":{"1min":0.48299999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.39600000000000002,"15min":0.40100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55900000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.28299999999999997}]},{"osd":3,"last update":"Thu May 9 06:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.501},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu May 9 06:34:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":19.359999999999999,"5min":19.359999999999999,"15min":19.359999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu May 9 06:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34399999999999997,"15min":0.34499999999999997},"min":{"1min":0.27800000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48399999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.35999999999999999,"15min":0.36399999999999999},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53100000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu May 9 06:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.374,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.46400000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu May 9 06:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu May 9 06:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.62},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu May 9 06:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35399999999999998},"min":{"1min":0.249,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.35999999999999999},"min":{"1min":0.26200000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu May 9 06:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38400000000000001,"15min":0.39000000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.57199999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39000000000000001,"15min":0.39100000000000001},"min":{"1min":0.255,"5min":0.248,"15min":0.248},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu May 9 06:33:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu May 9 06:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:34:40.402 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T06:34:40.402 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T06:34:40.402 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T06:34:40.402 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-09T06:34:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:41.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:41 smithi193 ceph-mon[35141]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:41.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:41 smithi193 ceph-mon[35141]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T06:34:41.643 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:41 smithi063 ceph-mon[27171]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:41.643 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:41 smithi063 ceph-mon[27171]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T06:34:42.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.206+0000 7ff987e42700 1 -- 172.21.15.63:0/1647682542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff980105ab0 msgr2=0x7ff980105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.206+0000 7ff987e42700 1 --2- 172.21.15.63:0/1647682542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff980105ab0 0x7ff980105e90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff97000b0a0 tx=0x7ff97000b3b0 comp rx=0 tx=0).stop 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.206+0000 7ff987e42700 1 -- 172.21.15.63:0/1647682542 shutdown_connections 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.206+0000 7ff987e42700 1 --2- 172.21.15.63:0/1647682542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9800ffab0 0x7ff9800fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.206+0000 7ff987e42700 1 --2- 172.21.15.63:0/1647682542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff980105ab0 0x7ff980105e90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.206+0000 7ff987e42700 1 -- 172.21.15.63:0/1647682542 >> 172.21.15.63:0/1647682542 conn(0x7ff980076750 msgr2=0x7ff9800fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.206+0000 7ff987e42700 1 -- 172.21.15.63:0/1647682542 shutdown_connections 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.206+0000 7ff987e42700 1 -- 172.21.15.63:0/1647682542 wait complete. 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.207+0000 7ff987e42700 1 Processor -- start 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.207+0000 7ff987e42700 1 -- start start 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.207+0000 7ff987e42700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9800ffab0 0x7ff98019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.207+0000 7ff987e42700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff980105ab0 0x7ff98019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.207+0000 7ff987e42700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019d370 con 0x7ff9800ffab0 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.207+0000 7ff987e42700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9801969a0 con 0x7ff980105ab0 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.207+0000 7ff985bde700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9800ffab0 0x7ff98019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff985bde700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9800ffab0 0x7ff98019c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33906/0 (socket says 172.21.15.63:33906) 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff985bde700 1 -- 172.21.15.63:0/1623981688 learned_addr learned my addr 172.21.15.63:0/1623981688 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff9853dd700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff980105ab0 0x7ff98019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff985bde700 1 -- 172.21.15.63:0/1623981688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff980105ab0 msgr2=0x7ff98019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff985bde700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff980105ab0 0x7ff98019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff985bde700 1 -- 172.21.15.63:0/1623981688 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970009d00 con 0x7ff9800ffab0 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff9853dd700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff980105ab0 0x7ff98019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff985bde700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9800ffab0 0x7ff98019c810 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff970000f80 tx=0x7ff970007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff97effd700 1 -- 172.21.15.63:0/1623981688 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97000e040 con 0x7ff9800ffab0 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff980196c20 con 0x7ff9800ffab0 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff97effd700 1 -- 172.21.15.63:0/1623981688 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff970003c10 con 0x7ff9800ffab0 2024-05-09T06:34:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff97effd700 1 -- 172.21.15.63:0/1623981688 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97001d6b0 con 0x7ff9800ffab0 2024-05-09T06:34:42.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.208+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff980197110 con 0x7ff9800ffab0 2024-05-09T06:34:42.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.209+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9801011f0 con 0x7ff9800ffab0 2024-05-09T06:34:42.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.210+0000 7ff97effd700 1 -- 172.21.15.63:0/1623981688 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff970004040 con 0x7ff9800ffab0 2024-05-09T06:34:42.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.210+0000 7ff97effd700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff96806cce0 0x7ff96806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:42.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.210+0000 7ff97effd700 1 -- 172.21.15.63:0/1623981688 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff97008d690 con 0x7ff9800ffab0 2024-05-09T06:34:42.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.210+0000 7ff9853dd700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff96806cce0 0x7ff96806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:42.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.211+0000 7ff9853dd700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff96806cce0 0x7ff96806f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff974005fd0 tx=0x7ff974005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:42.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.215+0000 7ff97effd700 1 -- 172.21.15.63:0/1623981688 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff97005b830 con 0x7ff9800ffab0 2024-05-09T06:34:42.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.464+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff98004ebb0 con 0x7ff9800ffab0 2024-05-09T06:34:42.445 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:34:42.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.464+0000 7ff97effd700 1 -- 172.21.15.63:0/1623981688 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff97005b3c0 con 0x7ff9800ffab0 2024-05-09T06:34:42.446 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:34:42.446 INFO:teuthology.orchestra.run.smithi063.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T06:34:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff96806cce0 msgr2=0x7ff96806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff96806cce0 0x7ff96806f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff974005fd0 tx=0x7ff974005ee0 comp rx=0 tx=0).stop 2024-05-09T06:34:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9800ffab0 msgr2=0x7ff98019c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9800ffab0 0x7ff98019c810 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff970000f80 tx=0x7ff970007bd0 comp rx=0 tx=0).stop 2024-05-09T06:34:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 shutdown_connections 2024-05-09T06:34:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9800ffab0 0x7ff98019c810 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff96806cce0 0x7ff96806f1a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 --2- 172.21.15.63:0/1623981688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff980105ab0 0x7ff98019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 >> 172.21.15.63:0/1623981688 conn(0x7ff980076750 msgr2=0x7ff9800fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 shutdown_connections 2024-05-09T06:34:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:42.466+0000 7ff987e42700 1 -- 172.21.15.63:0/1623981688 wait complete. 2024-05-09T06:34:42.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:34:43.039 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T06:34:43.039 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T06:34:43.040 INFO:teuthology.run_tasks:Running task print... 2024-05-09T06:34:43.053 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-05-09T06:34:43.053 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T06:34:43.067 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:34:43.067 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-05-09T06:34:43.285 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:43.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:43 smithi193 ceph-mon[35141]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:43.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:43 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1623981688' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T06:34:43.653 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:43 smithi063 ceph-mon[27171]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:43.653 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:43 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1623981688' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T06:34:44.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.933+0000 7f1c23c27700 1 -- 172.21.15.63:0/4051594512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c1016b0 msgr2=0x7f1c1c103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:44.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.933+0000 7f1c23c27700 1 --2- 172.21.15.63:0/4051594512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c1016b0 0x7f1c1c103aa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009910 tx=0x7f1c18009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:44.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.933+0000 7f1c23c27700 1 -- 172.21.15.63:0/4051594512 shutdown_connections 2024-05-09T06:34:44.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.933+0000 7f1c23c27700 1 --2- 172.21.15.63:0/4051594512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c103fe0 0x7f1c1c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.933+0000 7f1c23c27700 1 --2- 172.21.15.63:0/4051594512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c1016b0 0x7f1c1c103aa0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.933+0000 7f1c23c27700 1 -- 172.21.15.63:0/4051594512 >> 172.21.15.63:0/4051594512 conn(0x7f1c1c0fb0c0 msgr2=0x7f1c1c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.933+0000 7f1c23c27700 1 -- 172.21.15.63:0/4051594512 shutdown_connections 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.933+0000 7f1c23c27700 1 -- 172.21.15.63:0/4051594512 wait complete. 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c23c27700 1 Processor -- start 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c23c27700 1 -- start start 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c23c27700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c1016b0 0x7f1c1c1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c23c27700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c103fe0 0x7f1c1c196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c23c27700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c197250 con 0x7f1c1c103fe0 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c23c27700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c197390 con 0x7f1c1c1016b0 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c211c2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c103fe0 0x7f1c1c196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c211c2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c103fe0 0x7f1c1c196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39512/0 (socket says 172.21.15.63:39512) 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c211c2700 1 -- 172.21.15.63:0/2908017482 learned_addr learned my addr 172.21.15.63:0/2908017482 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:44.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.934+0000 7f1c219c3700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c1016b0 0x7f1c1c1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c211c2700 1 -- 172.21.15.63:0/2908017482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c1016b0 msgr2=0x7f1c1c1966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c211c2700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c1016b0 0x7f1c1c1966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c211c2700 1 -- 172.21.15.63:0/2908017482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180095f0 con 0x7f1c1c103fe0 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c219c3700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c1016b0 0x7f1c1c1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c211c2700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c103fe0 0x7f1c1c196c30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00b4b0 tx=0x7f1c0c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c12ffd700 1 -- 172.21.15.63:0/2908017482 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c0c011800 con 0x7f1c1c103fe0 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c19bdf0 con 0x7f1c1c103fe0 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c12ffd700 1 -- 172.21.15.63:0/2908017482 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c0c011e20 con 0x7f1c1c103fe0 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c12ffd700 1 -- 172.21.15.63:0/2908017482 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c0c00f2e0 con 0x7f1c1c103fe0 2024-05-09T06:34:44.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.935+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c19c390 con 0x7f1c1c103fe0 2024-05-09T06:34:44.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.936+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c04ebb0 con 0x7f1c1c103fe0 2024-05-09T06:34:44.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.936+0000 7f1c12ffd700 1 -- 172.21.15.63:0/2908017482 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1c0c010380 con 0x7f1c1c103fe0 2024-05-09T06:34:44.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.937+0000 7f1c12ffd700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1c0806ccf0 0x7f1c0806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:44.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.937+0000 7f1c12ffd700 1 -- 172.21.15.63:0/2908017482 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1c0c08bac0 con 0x7f1c1c103fe0 2024-05-09T06:34:44.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.937+0000 7f1c219c3700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1c0806ccf0 0x7f1c0806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:44.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.937+0000 7f1c219c3700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1c0806ccf0 0x7f1c0806f1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c18006010 tx=0x7f1c1800b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:44.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:44.941+0000 7f1c12ffd700 1 -- 172.21.15.63:0/2908017482 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1c0c059c60 con 0x7f1c1c103fe0 2024-05-09T06:34:45.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.145+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f1c1c109800 con 0x7f1c1c103fe0 2024-05-09T06:34:45.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.149+0000 7f1c12ffd700 1 -- 172.21.15.63:0/2908017482 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v22)=0 v22) v1 ==== 143+0+0 (secure 0 0 0) 0x7f1c0c0597f0 con 0x7f1c1c103fe0 2024-05-09T06:34:45.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1c0806ccf0 msgr2=0x7f1c0806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:45.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1c0806ccf0 0x7f1c0806f1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c18006010 tx=0x7f1c1800b560 comp rx=0 tx=0).stop 2024-05-09T06:34:45.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c103fe0 msgr2=0x7f1c1c196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:45.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c103fe0 0x7f1c1c196c30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00b4b0 tx=0x7f1c0c00b870 comp rx=0 tx=0).stop 2024-05-09T06:34:45.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 shutdown_connections 2024-05-09T06:34:45.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1c0806ccf0 0x7f1c0806f1b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:45.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c1c1016b0 0x7f1c1c1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:45.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 --2- 172.21.15.63:0/2908017482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1c103fe0 0x7f1c1c196c30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:45.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 >> 172.21.15.63:0/2908017482 conn(0x7f1c1c0fb0c0 msgr2=0x7f1c1c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:45.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 shutdown_connections 2024-05-09T06:34:45.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:45.152+0000 7f1c23c27700 1 -- 172.21.15.63:0/2908017482 wait complete. 2024-05-09T06:34:45.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:45 smithi193 ceph-mon[35141]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:45.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:45 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2908017482' entity='client.admin' 2024-05-09T06:34:45.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:45 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:34:45.813 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:45 smithi063 ceph-mon[27171]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:45.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:45 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2908017482' entity='client.admin' 2024-05-09T06:34:45.814 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:45 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:34:45.836 INFO:teuthology.run_tasks:Running task print... 2024-05-09T06:34:45.846 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-05-09T06:34:45.847 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T06:34:45.901 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:34:45.902 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T06:34:46.122 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:46.524 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:34:46.524 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:34:46.525 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:34:46.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:34:46.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:34:46.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:34:47.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 -- 172.21.15.63:0/732735250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc0ffb50 msgr2=0x7f4dfc0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:47.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 --2- 172.21.15.63:0/732735250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc0ffb50 0x7f4dfc0fff70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009910 tx=0x7f4df8009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:47.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 -- 172.21.15.63:0/732735250 shutdown_connections 2024-05-09T06:34:47.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 --2- 172.21.15.63:0/732735250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dfc100d50 0x7f4dfc1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:47.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 --2- 172.21.15.63:0/732735250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc0ffb50 0x7f4dfc0fff70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:47.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 -- 172.21.15.63:0/732735250 >> 172.21.15.63:0/732735250 conn(0x7f4dfc0fb0f0 msgr2=0x7f4dfc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:47.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 -- 172.21.15.63:0/732735250 shutdown_connections 2024-05-09T06:34:47.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 -- 172.21.15.63:0/732735250 wait complete. 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 Processor -- start 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 -- start start 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dfc0ffb50 0x7f4dfc198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.600+0000 7f4e03849700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc100d50 0x7f4dfc198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e03849700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc1993a0 con 0x7f4dfc100d50 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e03849700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc1994e0 con 0x7f4dfc0ffb50 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e00de4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc100d50 0x7f4dfc198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e00de4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc100d50 0x7f4dfc198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39534/0 (socket says 172.21.15.63:39534) 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e00de4700 1 -- 172.21.15.63:0/319325623 learned_addr learned my addr 172.21.15.63:0/319325623 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e015e5700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dfc0ffb50 0x7f4dfc198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e00de4700 1 -- 172.21.15.63:0/319325623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dfc0ffb50 msgr2=0x7f4dfc198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e00de4700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dfc0ffb50 0x7f4dfc198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:47.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e00de4700 1 -- 172.21.15.63:0/319325623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80095f0 con 0x7f4dfc100d50 2024-05-09T06:34:47.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e00de4700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc100d50 0x7f4dfc198d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00d8d0 tx=0x7f4dec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:47.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4df27fc700 1 -- 172.21.15.63:0/319325623 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dec009940 con 0x7f4dfc100d50 2024-05-09T06:34:47.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dfc19df40 con 0x7f4dfc100d50 2024-05-09T06:34:47.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc19e4e0 con 0x7f4dfc100d50 2024-05-09T06:34:47.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4df27fc700 1 -- 172.21.15.63:0/319325623 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dec010460 con 0x7f4dfc100d50 2024-05-09T06:34:47.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.601+0000 7f4df27fc700 1 -- 172.21.15.63:0/319325623 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dec00f5d0 con 0x7f4dfc100d50 2024-05-09T06:34:47.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.602+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de00052f0 con 0x7f4dfc100d50 2024-05-09T06:34:47.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.603+0000 7f4df27fc700 1 -- 172.21.15.63:0/319325623 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4dec009aa0 con 0x7f4dfc100d50 2024-05-09T06:34:47.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.603+0000 7f4df27fc700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f4de806cce0 0x7f4de806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:47.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.603+0000 7f4e015e5700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f4de806cce0 0x7f4de806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:47.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.603+0000 7f4df27fc700 1 -- 172.21.15.63:0/319325623 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4dec08bd60 con 0x7f4dfc100d50 2024-05-09T06:34:47.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.604+0000 7f4e015e5700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f4de806cce0 0x7f4de806f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4df8006010 tx=0x7f4df800b580 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:47.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.608+0000 7f4df27fc700 1 -- 172.21.15.63:0/319325623 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4dec059c70 con 0x7f4dfc100d50 2024-05-09T06:34:47.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:47 smithi063 ceph-mon[27171]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:47.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.833+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4de0000bc0 con 0x7f4de806cce0 2024-05-09T06:34:47.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.834+0000 7f4df27fc700 1 -- 172.21.15.63:0/319325623 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4de0000bc0 con 0x7f4de806cce0 2024-05-09T06:34:47.815 INFO:teuthology.orchestra.run.smithi063.stdout:Backend: cephadm 2024-05-09T06:34:47.815 INFO:teuthology.orchestra.run.smithi063.stdout:Available: Yes 2024-05-09T06:34:47.815 INFO:teuthology.orchestra.run.smithi063.stdout:Paused: No 2024-05-09T06:34:47.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.836+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f4de806cce0 msgr2=0x7f4de806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:47.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.836+0000 7f4e03849700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f4de806cce0 0x7f4de806f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4df8006010 tx=0x7f4df800b580 comp rx=0 tx=0).stop 2024-05-09T06:34:47.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.836+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc100d50 msgr2=0x7f4dfc198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:47.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.836+0000 7f4e03849700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc100d50 0x7f4dfc198d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00d8d0 tx=0x7f4dec00dc90 comp rx=0 tx=0).stop 2024-05-09T06:34:47.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.836+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 shutdown_connections 2024-05-09T06:34:47.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.836+0000 7f4e03849700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f4de806cce0 0x7f4de806f1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:47.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.836+0000 7f4e03849700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dfc0ffb50 0x7f4dfc198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:47.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.836+0000 7f4e03849700 1 --2- 172.21.15.63:0/319325623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4dfc100d50 0x7f4dfc198d80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:47.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.836+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 >> 172.21.15.63:0/319325623 conn(0x7f4dfc0fb0f0 msgr2=0x7f4dfc1042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:47.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.837+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 shutdown_connections 2024-05-09T06:34:47.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:47.837+0000 7f4e03849700 1 -- 172.21.15.63:0/319325623 wait complete. 2024-05-09T06:34:47.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:47 smithi193 ceph-mon[35141]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:48.424 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T06:34:48.643 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:49.841 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:49 smithi063 ceph-mon[27171]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:49.841 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:49 smithi063 ceph-mon[27171]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:34:49.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:49 smithi193 ceph-mon[35141]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:49.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:49 smithi193 ceph-mon[35141]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:34:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.414+0000 7f0a7fd96700 1 -- 172.21.15.63:0/1026519447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 msgr2=0x7f0a781022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.414+0000 7f0a7fd96700 1 --2- 172.21.15.63:0/1026519447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 0x7f0a781022a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009910 tx=0x7f0a68009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.414+0000 7f0a7fd96700 1 -- 172.21.15.63:0/1026519447 shutdown_connections 2024-05-09T06:34:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.414+0000 7f0a7fd96700 1 --2- 172.21.15.63:0/1026519447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 0x7f0a781022a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.414+0000 7f0a7fd96700 1 --2- 172.21.15.63:0/1026519447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78100c40 0x7f0a78101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:50.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.414+0000 7f0a7fd96700 1 -- 172.21.15.63:0/1026519447 >> 172.21.15.63:0/1026519447 conn(0x7f0a780fc1e0 msgr2=0x7f0a780fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:50.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.414+0000 7f0a7fd96700 1 -- 172.21.15.63:0/1026519447 shutdown_connections 2024-05-09T06:34:50.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.414+0000 7f0a7fd96700 1 -- 172.21.15.63:0/1026519447 wait complete. 2024-05-09T06:34:50.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7fd96700 1 Processor -- start 2024-05-09T06:34:50.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7fd96700 1 -- start start 2024-05-09T06:34:50.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7fd96700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78100c40 0x7f0a78196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:50.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7fd96700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 0x7f0a78196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:50.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7fd96700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781971b0 con 0x7f0a78101e40 2024-05-09T06:34:50.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7fd96700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781972f0 con 0x7f0a78100c40 2024-05-09T06:34:50.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7d331700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 0x7f0a78196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7d331700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 0x7f0a78196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39548/0 (socket says 172.21.15.63:39548) 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7d331700 1 -- 172.21.15.63:0/3676225126 learned_addr learned my addr 172.21.15.63:0/3676225126 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.415+0000 7f0a7db32700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78100c40 0x7f0a78196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.416+0000 7f0a7d331700 1 -- 172.21.15.63:0/3676225126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78100c40 msgr2=0x7f0a78196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.416+0000 7f0a7d331700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78100c40 0x7f0a78196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.416+0000 7f0a7d331700 1 -- 172.21.15.63:0/3676225126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a680095f0 con 0x7f0a78101e40 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.416+0000 7f0a7d331700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 0x7f0a78196b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0a680092e0 tx=0x7f0a6800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.416+0000 7f0a6effd700 1 -- 172.21.15.63:0/3676225126 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a6801d070 con 0x7f0a78101e40 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.416+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7819bca0 con 0x7f0a78101e40 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.416+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7819c210 con 0x7f0a78101e40 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.416+0000 7f0a6effd700 1 -- 172.21.15.63:0/3676225126 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a6800f460 con 0x7f0a78101e40 2024-05-09T06:34:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.416+0000 7f0a6effd700 1 -- 172.21.15.63:0/3676225126 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a68021620 con 0x7f0a78101e40 2024-05-09T06:34:50.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.417+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c005320 con 0x7f0a78101e40 2024-05-09T06:34:50.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.418+0000 7f0a6effd700 1 -- 172.21.15.63:0/3676225126 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0a68003a60 con 0x7f0a78101e40 2024-05-09T06:34:50.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.418+0000 7f0a6effd700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0a6406cce0 0x7f0a6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:50.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.419+0000 7f0a7db32700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0a6406cce0 0x7f0a6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:50.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.419+0000 7f0a6effd700 1 -- 172.21.15.63:0/3676225126 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0a6808cfe0 con 0x7f0a78101e40 2024-05-09T06:34:50.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.419+0000 7f0a7db32700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0a6406cce0 0x7f0a6406f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a78101ca0 tx=0x7f0a7400b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:50.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.423+0000 7f0a6effd700 1 -- 172.21.15.63:0/3676225126 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0a6805b200 con 0x7f0a78101e40 2024-05-09T06:34:50.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.638+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a5c000bf0 con 0x7f0a6406cce0 2024-05-09T06:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.647+0000 7f0a6effd700 1 -- 172.21.15.63:0/3676225126 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3100 (secure 0 0 0) 0x7f0a5c000bf0 con 0x7f0a6406cce0 2024-05-09T06:34:50.628 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:34:50.628 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (3m) 2m ago 5m 26.6M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:34:50.628 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (5m) 2m ago 5m 7013k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:34:50.628 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (4m) 67s ago 4m 6962k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (5m) 2m ago 5m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (4m) 67s ago 4m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (3m) 2m ago 4m 80.7M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:9283,8765,8443 running (6m) 2m ago 6m 493M - 18.2.2-1297-g1758e8ea 196422f1b641 a66e4acc828f 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (4m) 67s ago 4m 449M - 18.2.2-1297-g1758e8ea 196422f1b641 b87b5901e715 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (6m) 2m ago 6m 45.3M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 7d88727e4bbb 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (4m) 67s ago 4m 42.5M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (4m) 67s ago 4m 15.7M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (3m) 2m ago 3m 40.2M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (2m) 2m ago 2m 42.0M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (2m) 2m ago 2m 40.8M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (2m) 67s ago 2m 40.8M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (98s) 67s ago 98s 47.4M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (72s) 67s ago 72s 11.5M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:34:50.629 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (3m) 2m ago 4m 38.2M - 2.43.0 a07b618ecd1d 9f4deb8cc8b4 2024-05-09T06:34:50.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.649+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0a6406cce0 msgr2=0x7f0a6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:50.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.649+0000 7f0a7fd96700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0a6406cce0 0x7f0a6406f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a78101ca0 tx=0x7f0a7400b410 comp rx=0 tx=0).stop 2024-05-09T06:34:50.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.649+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 msgr2=0x7f0a78196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:50.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.649+0000 7f0a7fd96700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 0x7f0a78196b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0a680092e0 tx=0x7f0a6800bd70 comp rx=0 tx=0).stop 2024-05-09T06:34:50.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.650+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 shutdown_connections 2024-05-09T06:34:50.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.650+0000 7f0a7fd96700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0a6406cce0 0x7f0a6406f1a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:50.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.650+0000 7f0a7fd96700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a78100c40 0x7f0a78196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:50.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.650+0000 7f0a7fd96700 1 --2- 172.21.15.63:0/3676225126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a78101e40 0x7f0a78196b90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:50.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.650+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 >> 172.21.15.63:0/3676225126 conn(0x7f0a780fc1e0 msgr2=0x7f0a781053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:50.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.650+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 shutdown_connections 2024-05-09T06:34:50.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:50.650+0000 7f0a7fd96700 1 -- 172.21.15.63:0/3676225126 wait complete. 2024-05-09T06:34:51.371 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T06:34:51.588 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:51.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:51 smithi063 ceph-mon[27171]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:51.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:51 smithi193 ceph-mon[35141]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:52.654 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:52 smithi063 ceph-mon[27171]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:34:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:52 smithi193 ceph-mon[35141]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:34:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.183+0000 7f5619a50700 1 -- 172.21.15.63:0/3640270933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56141016e0 msgr2=0x7f5614103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.183+0000 7f5619a50700 1 --2- 172.21.15.63:0/3640270933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56141016e0 0x7f5614103ad0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f560400b0a0 tx=0x7f560400b3b0 comp rx=0 tx=0).stop 2024-05-09T06:34:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.183+0000 7f5619a50700 1 -- 172.21.15.63:0/3640270933 shutdown_connections 2024-05-09T06:34:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.183+0000 7f5619a50700 1 --2- 172.21.15.63:0/3640270933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5614104010 0x7f5614106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.183+0000 7f5619a50700 1 --2- 172.21.15.63:0/3640270933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56141016e0 0x7f5614103ad0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.183+0000 7f5619a50700 1 -- 172.21.15.63:0/3640270933 >> 172.21.15.63:0/3640270933 conn(0x7f56140fb0f0 msgr2=0x7f56140fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.183+0000 7f5619a50700 1 -- 172.21.15.63:0/3640270933 shutdown_connections 2024-05-09T06:34:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.183+0000 7f5619a50700 1 -- 172.21.15.63:0/3640270933 wait complete. 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f5619a50700 1 Processor -- start 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f5619a50700 1 -- start start 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f5619a50700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56141016e0 0x7f56141966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f5619a50700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5614104010 0x7f5614196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f5619a50700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5614197230 con 0x7f5614104010 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f5619a50700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5614197370 con 0x7f56141016e0 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f56127fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5614104010 0x7f5614196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f56127fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5614104010 0x7f5614196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39578/0 (socket says 172.21.15.63:39578) 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f56127fc700 1 -- 172.21.15.63:0/1922042255 learned_addr learned my addr 172.21.15.63:0/1922042255 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f5612ffd700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56141016e0 0x7f56141966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f56127fc700 1 -- 172.21.15.63:0/1922042255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56141016e0 msgr2=0x7f56141966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f56127fc700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56141016e0 0x7f56141966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f56127fc700 1 -- 172.21.15.63:0/1922042255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5604009d00 con 0x7f5614104010 2024-05-09T06:34:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.184+0000 7f5612ffd700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56141016e0 0x7f56141966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:34:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.185+0000 7f56127fc700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5614104010 0x7f5614196c10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b4b0 tx=0x7f55fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.185+0000 7f5618a4e700 1 -- 172.21.15.63:0/1922042255 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55fc011800 con 0x7f5614104010 2024-05-09T06:34:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.185+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561419bdd0 con 0x7f5614104010 2024-05-09T06:34:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.185+0000 7f5618a4e700 1 -- 172.21.15.63:0/1922042255 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55fc011e20 con 0x7f5614104010 2024-05-09T06:34:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.185+0000 7f5618a4e700 1 -- 172.21.15.63:0/1922042255 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55fc00f2e0 con 0x7f5614104010 2024-05-09T06:34:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.185+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561419c370 con 0x7f5614104010 2024-05-09T06:34:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.186+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561404ebb0 con 0x7f5614104010 2024-05-09T06:34:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.186+0000 7f5618a4e700 1 -- 172.21.15.63:0/1922042255 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f55fc011960 con 0x7f5614104010 2024-05-09T06:34:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.187+0000 7f5618a4e700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f560006cce0 0x7f560006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.187+0000 7f5618a4e700 1 -- 172.21.15.63:0/1922042255 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f55fc08bb10 con 0x7f5614104010 2024-05-09T06:34:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.187+0000 7f5612ffd700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f560006cce0 0x7f560006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.187+0000 7f5612ffd700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f560006cce0 0x7f560006f1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5604009480 tx=0x7f560400b7d0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.191+0000 7f5618a4e700 1 -- 172.21.15.63:0/1922042255 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f55fc059cb0 con 0x7f5614104010 2024-05-09T06:34:53.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.403+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f56141049f0 con 0x7f560006cce0 2024-05-09T06:34:53.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.407+0000 7f5618a4e700 1 -- 172.21.15.63:0/1922042255 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1300 (secure 0 0 0) 0x7f56141049f0 con 0x7f560006cce0 2024-05-09T06:34:53.388 INFO:teuthology.orchestra.run.smithi063.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T06:34:53.388 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-05-09T06:34:53.388 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter 2/2 2m ago 6m * 2024-05-09T06:34:53.388 INFO:teuthology.orchestra.run.smithi063.stdout:crash 2/2 2m ago 6m * 2024-05-09T06:34:53.388 INFO:teuthology.orchestra.run.smithi063.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-05-09T06:34:53.388 INFO:teuthology.orchestra.run.smithi063.stdout:mgr 2/2 2m ago 6m count:2 2024-05-09T06:34:53.388 INFO:teuthology.orchestra.run.smithi063.stdout:mon 2/2 2m ago 5m smithi063:172.21.15.63=smithi063;smithi193:172.21.15.193=smithi193;count:2 2024-05-09T06:34:53.389 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-05-09T06:34:53.389 INFO:teuthology.orchestra.run.smithi063.stdout:osd 6 2m ago - 2024-05-09T06:34:53.389 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-05-09T06:34:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.409+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f560006cce0 msgr2=0x7f560006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.409+0000 7f5619a50700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f560006cce0 0x7f560006f1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5604009480 tx=0x7f560400b7d0 comp rx=0 tx=0).stop 2024-05-09T06:34:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.409+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5614104010 msgr2=0x7f5614196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.409+0000 7f5619a50700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5614104010 0x7f5614196c10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b4b0 tx=0x7f55fc00b870 comp rx=0 tx=0).stop 2024-05-09T06:34:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.409+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 shutdown_connections 2024-05-09T06:34:53.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.409+0000 7f5619a50700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f560006cce0 0x7f560006f1a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:53.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.409+0000 7f5619a50700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56141016e0 0x7f56141966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:53.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.409+0000 7f5619a50700 1 --2- 172.21.15.63:0/1922042255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5614104010 0x7f5614196c10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:53.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.409+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 >> 172.21.15.63:0/1922042255 conn(0x7f56140fb0f0 msgr2=0x7f56140fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:53.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.410+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 shutdown_connections 2024-05-09T06:34:53.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:53.410+0000 7f5619a50700 1 -- 172.21.15.63:0/1922042255 wait complete. 2024-05-09T06:34:53.838 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:53 smithi063 ceph-mon[27171]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:53.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:53 smithi193 ceph-mon[35141]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:54.084 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T06:34:54.300 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:54.663 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:54 smithi063 ceph-mon[27171]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:34:54.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:54 smithi193 ceph-mon[35141]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:34:55.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:55 smithi063 ceph-mon[27171]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:55.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:55 smithi193 ceph-mon[35141]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:55.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.922+0000 7f8920e66700 1 -- 172.21.15.63:0/1001216844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c100c50 msgr2=0x7f891c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:55.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.922+0000 7f8920e66700 1 --2- 172.21.15.63:0/1001216844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c100c50 0x7f891c101070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f890c009910 tx=0x7f890c009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:55.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.922+0000 7f8920e66700 1 -- 172.21.15.63:0/1001216844 shutdown_connections 2024-05-09T06:34:55.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.922+0000 7f8920e66700 1 --2- 172.21.15.63:0/1001216844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f891c101e50 0x7f891c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:55.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.922+0000 7f8920e66700 1 --2- 172.21.15.63:0/1001216844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c100c50 0x7f891c101070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.922+0000 7f8920e66700 1 -- 172.21.15.63:0/1001216844 >> 172.21.15.63:0/1001216844 conn(0x7f891c0fc210 msgr2=0x7f891c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.922+0000 7f8920e66700 1 -- 172.21.15.63:0/1001216844 shutdown_connections 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.922+0000 7f8920e66700 1 -- 172.21.15.63:0/1001216844 wait complete. 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8920e66700 1 Processor -- start 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8920e66700 1 -- start start 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8920e66700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f891c100c50 0x7f891c074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8920e66700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c101e50 0x7f891c0732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8920e66700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c073820 con 0x7f891c101e50 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8920e66700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c073960 con 0x7f891c100c50 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8919d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c101e50 0x7f891c0732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8919d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c101e50 0x7f891c0732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44126/0 (socket says 172.21.15.63:44126) 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8919d9b700 1 -- 172.21.15.63:0/900310642 learned_addr learned my addr 172.21.15.63:0/900310642 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:55.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f891a59c700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f891c100c50 0x7f891c074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8919d9b700 1 -- 172.21.15.63:0/900310642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f891c100c50 msgr2=0x7f891c074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8919d9b700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f891c100c50 0x7f891c074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8919d9b700 1 -- 172.21.15.63:0/900310642 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c0095f0 con 0x7f891c101e50 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.923+0000 7f8919d9b700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c101e50 0x7f891c0732e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f890400d8d0 tx=0x7f890400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.924+0000 7f89137fe700 1 -- 172.21.15.63:0/900310642 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89040098a0 con 0x7f891c101e50 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.924+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c073c40 con 0x7f891c101e50 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.924+0000 7f89137fe700 1 -- 172.21.15.63:0/900310642 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8904010460 con 0x7f891c101e50 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.924+0000 7f89137fe700 1 -- 172.21.15.63:0/900310642 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890400f5f0 con 0x7f891c101e50 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.924+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c19e6c0 con 0x7f891c101e50 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.925+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c1907d0 con 0x7f891c101e50 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.925+0000 7f89137fe700 1 -- 172.21.15.63:0/900310642 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8904009a00 con 0x7f891c101e50 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.926+0000 7f89137fe700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f890806cce0 0x7f890806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.926+0000 7f89137fe700 1 -- 172.21.15.63:0/900310642 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f890408be00 con 0x7f891c101e50 2024-05-09T06:34:55.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.926+0000 7f891a59c700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f890806cce0 0x7f890806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:55.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.926+0000 7f891a59c700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f890806cce0 0x7f890806f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f890c000c00 tx=0x7f890c005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:55.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:55.930+0000 7f89137fe700 1 -- 172.21.15.63:0/900310642 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8904059fa0 con 0x7f891c101e50 2024-05-09T06:34:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.141+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f891c061330 con 0x7f890806cce0 2024-05-09T06:34:56.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.143+0000 7f89137fe700 1 -- 172.21.15.63:0/900310642 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f891c061330 con 0x7f890806cce0 2024-05-09T06:34:56.124 INFO:teuthology.orchestra.run.smithi063.stdout:HOST ADDR LABELS STATUS 2024-05-09T06:34:56.124 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 172.21.15.63 2024-05-09T06:34:56.124 INFO:teuthology.orchestra.run.smithi063.stdout:smithi193 172.21.15.193 2024-05-09T06:34:56.124 INFO:teuthology.orchestra.run.smithi063.stdout:2 hosts in cluster 2024-05-09T06:34:56.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f890806cce0 msgr2=0x7f890806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:56.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f890806cce0 0x7f890806f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f890c000c00 tx=0x7f890c005960 comp rx=0 tx=0).stop 2024-05-09T06:34:56.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c101e50 msgr2=0x7f891c0732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:56.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c101e50 0x7f891c0732e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f890400d8d0 tx=0x7f890400dbe0 comp rx=0 tx=0).stop 2024-05-09T06:34:56.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 shutdown_connections 2024-05-09T06:34:56.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f890806cce0 0x7f890806f1a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:56.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f891c100c50 0x7f891c074bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:56.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 --2- 172.21.15.63:0/900310642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f891c101e50 0x7f891c0732e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:56.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 >> 172.21.15.63:0/900310642 conn(0x7f891c0fc210 msgr2=0x7f891c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:56.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 shutdown_connections 2024-05-09T06:34:56.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:56.145+0000 7f8920e66700 1 -- 172.21.15.63:0/900310642 wait complete. 2024-05-09T06:34:56.762 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T06:34:56.979 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:57.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:57 smithi063 ceph-mon[27171]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:57.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:57 smithi063 ceph-mon[27171]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:34:57.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:34:57.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:57 smithi193 ceph-mon[35141]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:57.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:57 smithi193 ceph-mon[35141]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:34:57.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:34:58.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.584+0000 7f176d114700 1 -- 172.21.15.63:0/3257610912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17680feef0 msgr2=0x7f17680ff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.584+0000 7f176d114700 1 --2- 172.21.15.63:0/3257610912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17680feef0 0x7f17680ff310 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1750009910 tx=0x7f1750009c20 comp rx=0 tx=0).stop 2024-05-09T06:34:58.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.584+0000 7f176d114700 1 -- 172.21.15.63:0/3257610912 shutdown_connections 2024-05-09T06:34:58.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.584+0000 7f176d114700 1 --2- 172.21.15.63:0/3257610912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1768101ed0 0x7f1768102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.584+0000 7f176d114700 1 --2- 172.21.15.63:0/3257610912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17680feef0 0x7f17680ff310 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.584+0000 7f176d114700 1 -- 172.21.15.63:0/3257610912 >> 172.21.15.63:0/3257610912 conn(0x7f17680fa850 msgr2=0x7f17680fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.584+0000 7f176d114700 1 -- 172.21.15.63:0/3257610912 shutdown_connections 2024-05-09T06:34:58.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.584+0000 7f176d114700 1 -- 172.21.15.63:0/3257610912 wait complete. 2024-05-09T06:34:58.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.585+0000 7f176d114700 1 Processor -- start 2024-05-09T06:34:58.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.585+0000 7f176d114700 1 -- start start 2024-05-09T06:34:58.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.585+0000 7f176d114700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17680feef0 0x7f1768196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.585+0000 7f176d114700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1768101ed0 0x7f1768196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.585+0000 7f176d114700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17681971d0 con 0x7f1768101ed0 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.585+0000 7f176d114700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1768197310 con 0x7f17680feef0 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.585+0000 7f176659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1768101ed0 0x7f1768196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.585+0000 7f176659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1768101ed0 0x7f1768196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44156/0 (socket says 172.21.15.63:44156) 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f176659c700 1 -- 172.21.15.63:0/888491738 learned_addr learned my addr 172.21.15.63:0/888491738 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f1766d9d700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17680feef0 0x7f1768196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f176659c700 1 -- 172.21.15.63:0/888491738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17680feef0 msgr2=0x7f1768196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f176659c700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17680feef0 0x7f1768196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f176659c700 1 -- 172.21.15.63:0/888491738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17500095f0 con 0x7f1768101ed0 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f176659c700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1768101ed0 0x7f1768196bb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f175800c8a0 tx=0x7f175800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f175ffff700 1 -- 172.21.15.63:0/888491738 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f175800cea0 con 0x7f1768101ed0 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176819bd20 con 0x7f1768101ed0 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f175ffff700 1 -- 172.21.15.63:0/888491738 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1758004830 con 0x7f1768101ed0 2024-05-09T06:34:58.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f175ffff700 1 -- 172.21.15.63:0/888491738 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17580055b0 con 0x7f1768101ed0 2024-05-09T06:34:58.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.586+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176819c2f0 con 0x7f1768101ed0 2024-05-09T06:34:58.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.587+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17681907b0 con 0x7f1768101ed0 2024-05-09T06:34:58.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.588+0000 7f175ffff700 1 -- 172.21.15.63:0/888491738 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f175800f450 con 0x7f1768101ed0 2024-05-09T06:34:58.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.588+0000 7f175ffff700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f175406cce0 0x7f175406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:34:58.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.588+0000 7f175ffff700 1 -- 172.21.15.63:0/888491738 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f175808bbc0 con 0x7f1768101ed0 2024-05-09T06:34:58.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.588+0000 7f1766d9d700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f175406cce0 0x7f175406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:34:58.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.589+0000 7f1766d9d700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f175406cce0 0x7f175406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1750006010 tx=0x7f175000b580 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:34:58.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.593+0000 7f175ffff700 1 -- 172.21.15.63:0/888491738 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1758059eb0 con 0x7f1768101ed0 2024-05-09T06:34:58.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.806+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f176802cea0 con 0x7f175406cce0 2024-05-09T06:34:58.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.807+0000 7f175ffff700 1 -- 172.21.15.63:0/888491738 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f176802cea0 con 0x7f175406cce0 2024-05-09T06:34:58.788 INFO:teuthology.orchestra.run.smithi063.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T06:34:58.788 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5344005B400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-09T06:34:58.788 INFO:teuthology.orchestra.run.smithi063.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 71s ago Has a FileSystem, LVM detected 2024-05-09T06:34:58.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.809+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f175406cce0 msgr2=0x7f175406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.809+0000 7f176d114700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f175406cce0 0x7f175406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1750006010 tx=0x7f175000b580 comp rx=0 tx=0).stop 2024-05-09T06:34:58.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.809+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1768101ed0 msgr2=0x7f1768196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:34:58.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.809+0000 7f176d114700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1768101ed0 0x7f1768196bb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f175800c8a0 tx=0x7f175800cc60 comp rx=0 tx=0).stop 2024-05-09T06:34:58.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.809+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 shutdown_connections 2024-05-09T06:34:58.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.809+0000 7f176d114700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f175406cce0 0x7f175406f1a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.809+0000 7f176d114700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17680feef0 0x7f1768196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.809+0000 7f176d114700 1 --2- 172.21.15.63:0/888491738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1768101ed0 0x7f1768196bb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:34:58.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.809+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 >> 172.21.15.63:0/888491738 conn(0x7f17680fa850 msgr2=0x7f17681054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:34:58.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.810+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 shutdown_connections 2024-05-09T06:34:58.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:34:58.810+0000 7f176d114700 1 -- 172.21.15.63:0/888491738 wait complete. 2024-05-09T06:34:59.609 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T06:34:59.618 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:34:59.618 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-05-09T06:34:59.832 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:34:59.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:34:59 smithi063 ceph-mon[27171]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:34:59.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:34:59 smithi193 ceph-mon[35141]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:00 smithi063 ceph-mon[27171]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:35:00.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:00 smithi193 ceph-mon[35141]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.512+0000 7f720b902700 1 -- 172.21.15.63:0/3803094369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204100c40 msgr2=0x7f7204101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.512+0000 7f720b902700 1 --2- 172.21.15.63:0/3803094369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204100c40 0x7f7204101060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f720000b0a0 tx=0x7f720000b3b0 comp rx=0 tx=0).stop 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 -- 172.21.15.63:0/3803094369 shutdown_connections 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 --2- 172.21.15.63:0/3803094369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204101e40 0x7f72041022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 --2- 172.21.15.63:0/3803094369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204100c40 0x7f7204101060 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 -- 172.21.15.63:0/3803094369 >> 172.21.15.63:0/3803094369 conn(0x7f72040fc1e0 msgr2=0x7f72040fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 -- 172.21.15.63:0/3803094369 shutdown_connections 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 -- 172.21.15.63:0/3803094369 wait complete. 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 Processor -- start 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 -- start start 2024-05-09T06:35:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204100c40 0x7f7204074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204101e40 0x7f72040732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204073820 con 0x7f7204101e40 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.513+0000 7f720b902700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204073960 con 0x7f7204100c40 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f7208e9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204101e40 0x7f72040732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f7208e9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204101e40 0x7f72040732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44180/0 (socket says 172.21.15.63:44180) 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f7208e9d700 1 -- 172.21.15.63:0/8461730 learned_addr learned my addr 172.21.15.63:0/8461730 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f720969e700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204100c40 0x7f7204074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f7208e9d700 1 -- 172.21.15.63:0/8461730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204100c40 msgr2=0x7f7204074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f7208e9d700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204100c40 0x7f7204074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f7208e9d700 1 -- 172.21.15.63:0/8461730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7200009d00 con 0x7f7204101e40 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f720969e700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204100c40 0x7f7204074bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f7208e9d700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204101e40 0x7f72040732e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d900 tx=0x7f71f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f71fa7fc700 1 -- 172.21.15.63:0/8461730 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71f4004a00 con 0x7f7204101e40 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7204073c40 con 0x7f7204101e40 2024-05-09T06:35:01.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f71fa7fc700 1 -- 172.21.15.63:0/8461730 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71f40054e0 con 0x7f7204101e40 2024-05-09T06:35:01.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f71fa7fc700 1 -- 172.21.15.63:0/8461730 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71f4009d60 con 0x7f7204101e40 2024-05-09T06:35:01.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.514+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720419e730 con 0x7f7204101e40 2024-05-09T06:35:01.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.515+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72041907e0 con 0x7f7204101e40 2024-05-09T06:35:01.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.516+0000 7f71fa7fc700 1 -- 172.21.15.63:0/8461730 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f71f4005020 con 0x7f7204101e40 2024-05-09T06:35:01.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.516+0000 7f71fa7fc700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f71f006cce0 0x7f71f006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:01.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.516+0000 7f71fa7fc700 1 -- 172.21.15.63:0/8461730 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f71f408c0e0 con 0x7f7204101e40 2024-05-09T06:35:01.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.516+0000 7f720969e700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f71f006cce0 0x7f71f006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:01.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.517+0000 7f720969e700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f71f006cce0 0x7f71f006f1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f720000b830 tx=0x7f7200007910 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:01.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.521+0000 7f71fa7fc700 1 -- 172.21.15.63:0/8461730 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f71f405a280 con 0x7f7204101e40 2024-05-09T06:35:01.502 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:01 smithi063 ceph-mon[27171]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:01.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:01.752+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7204061330 con 0x7f71f006cce0 2024-05-09T06:35:01.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:01 smithi193 ceph-mon[35141]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:02.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:02 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-09T06:35:02.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:02 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-09T06:35:03.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.476+0000 7f71fa7fc700 1 -- 172.21.15.63:0/8461730 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7204061330 con 0x7f71f006cce0 2024-05-09T06:35:03.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f71f006cce0 msgr2=0x7f71f006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:03.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f71f006cce0 0x7f71f006f1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f720000b830 tx=0x7f7200007910 comp rx=0 tx=0).stop 2024-05-09T06:35:03.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204101e40 msgr2=0x7f72040732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:03.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204101e40 0x7f72040732e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d900 tx=0x7f71f400dcc0 comp rx=0 tx=0).stop 2024-05-09T06:35:03.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 shutdown_connections 2024-05-09T06:35:03.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f71f006cce0 0x7f71f006f1a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:03.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204100c40 0x7f7204074bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:03.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 --2- 172.21.15.63:0/8461730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204101e40 0x7f72040732e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:03.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 >> 172.21.15.63:0/8461730 conn(0x7f72040fc1e0 msgr2=0x7f7204105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:03.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 shutdown_connections 2024-05-09T06:35:03.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:03.479+0000 7f720b902700 1 -- 172.21.15.63:0/8461730 wait complete. 2024-05-09T06:35:03.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:03 smithi063 ceph-mon[27171]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:03.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:03 smithi063 ceph-mon[27171]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:35:03.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:03 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-09T06:35:03.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:03 smithi063 ceph-mon[27171]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T06:35:03.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:03 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-09T06:35:03.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:03 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063[27167]: 2024-05-09T06:35:03.455+0000 7f6d9e0ef700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:35:03.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:03 smithi193 ceph-mon[35141]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:03.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:03 smithi193 ceph-mon[35141]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:35:03.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:03 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-09T06:35:03.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:03 smithi193 ceph-mon[35141]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T06:35:03.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:03 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-09T06:35:04.265 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T06:35:04.497 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: osdmap e43: 6 total, 6 up, 6 in 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: osdmap e44: 6 total, 6 up, 6 in 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: fsmap cephfs:0 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: Saving service mds.cephfs spec with placement count:4 2024-05-09T06:35:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:04.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:04 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:04.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-09T06:35:04.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: osdmap e43: 6 total, 6 up, 6 in 2024-05-09T06:35:04.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-09T06:35:04.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:35:04.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T06:35:04.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: osdmap e44: 6 total, 6 up, 6 in 2024-05-09T06:35:04.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-09T06:35:04.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: fsmap cephfs:0 2024-05-09T06:35:04.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: Saving service mds.cephfs spec with placement count:4 2024-05-09T06:35:04.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:04.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:04 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:05.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:05 smithi063 ceph-mon[27171]: pgmap v178: 65 pgs: 7 creating+peering, 1 active+clean, 57 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:05.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:05 smithi063 ceph-mon[27171]: osdmap e45: 6 total, 6 up, 6 in 2024-05-09T06:35:05.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:05 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:05.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:05 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:05.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:05 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:05.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:05 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.aoiibw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:35:05.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:05 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.aoiibw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T06:35:05.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:05 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:05.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:05 smithi193 ceph-mon[35141]: pgmap v178: 65 pgs: 7 creating+peering, 1 active+clean, 57 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:05.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:05 smithi193 ceph-mon[35141]: osdmap e45: 6 total, 6 up, 6 in 2024-05-09T06:35:05.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:05 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:05.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:05 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:05.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:05 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:05.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:05 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.aoiibw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:35:05.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:05 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.aoiibw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T06:35:05.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:05 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 -- 172.21.15.63:0/544450480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c100c40 msgr2=0x7f6d5c101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 --2- 172.21.15.63:0/544450480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c100c40 0x7f6d5c101060 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009910 tx=0x7f6d4c009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 -- 172.21.15.63:0/544450480 shutdown_connections 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 --2- 172.21.15.63:0/544450480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d5c101e40 0x7f6d5c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 --2- 172.21.15.63:0/544450480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c100c40 0x7f6d5c101060 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 -- 172.21.15.63:0/544450480 >> 172.21.15.63:0/544450480 conn(0x7f6d5c0fc1e0 msgr2=0x7f6d5c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 -- 172.21.15.63:0/544450480 shutdown_connections 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 -- 172.21.15.63:0/544450480 wait complete. 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 Processor -- start 2024-05-09T06:35:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.362+0000 7f6d62940700 1 -- start start 2024-05-09T06:35:06.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d62940700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d5c100c40 0x7f6d5c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:06.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d62940700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c101e40 0x7f6d5c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:06.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d62940700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c199360 con 0x7f6d5c101e40 2024-05-09T06:35:06.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d62940700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c1994a0 con 0x7f6d5c100c40 2024-05-09T06:35:06.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d5b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c101e40 0x7f6d5c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:06.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d5b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c101e40 0x7f6d5c198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56584/0 (socket says 172.21.15.63:56584) 2024-05-09T06:35:06.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d5b7fe700 1 -- 172.21.15.63:0/1725690540 learned_addr learned my addr 172.21.15.63:0/1725690540 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:06.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d5bfff700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d5c100c40 0x7f6d5c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:06.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d5b7fe700 1 -- 172.21.15.63:0/1725690540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d5c100c40 msgr2=0x7f6d5c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d5b7fe700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d5c100c40 0x7f6d5c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d5b7fe700 1 -- 172.21.15.63:0/1725690540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c0095f0 con 0x7f6d5c101e40 2024-05-09T06:35:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d5b7fe700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c101e40 0x7f6d5c198dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c930 tx=0x7f6d5000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d597fa700 1 -- 172.21.15.63:0/1725690540 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d50007ad0 con 0x7f6d5c101e40 2024-05-09T06:35:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d597fa700 1 -- 172.21.15.63:0/1725690540 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d5000ce80 con 0x7f6d5c101e40 2024-05-09T06:35:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.363+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c19dec0 con 0x7f6d5c101e40 2024-05-09T06:35:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.364+0000 7f6d597fa700 1 -- 172.21.15.63:0/1725690540 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d50018630 con 0x7f6d5c101e40 2024-05-09T06:35:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.364+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c19e460 con 0x7f6d5c101e40 2024-05-09T06:35:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.364+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c106810 con 0x7f6d5c101e40 2024-05-09T06:35:06.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.365+0000 7f6d597fa700 1 -- 172.21.15.63:0/1725690540 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6d50007c30 con 0x7f6d5c101e40 2024-05-09T06:35:06.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.365+0000 7f6d597fa700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d4406ccf0 0x7f6d4406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:06.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.366+0000 7f6d597fa700 1 -- 172.21.15.63:0/1725690540 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6d5008c220 con 0x7f6d5c101e40 2024-05-09T06:35:06.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.366+0000 7f6d5bfff700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d4406ccf0 0x7f6d4406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:06.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.366+0000 7f6d5bfff700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d4406ccf0 0x7f6d4406f1b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c005090 tx=0x7f6d4c005180 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:06.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.370+0000 7f6d597fa700 1 -- 172.21.15.63:0/1725690540 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6d5005a080 con 0x7f6d5c101e40 2024-05-09T06:35:06.505 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:06 smithi063 ceph-mon[27171]: Deploying daemon mds.cephfs.smithi063.aoiibw on smithi063 2024-05-09T06:35:06.505 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:06 smithi063 ceph-mon[27171]: osdmap e46: 6 total, 6 up, 6 in 2024-05-09T06:35:06.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.626+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6d5c1097a0 con 0x7f6d5c101e40 2024-05-09T06:35:06.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.626+0000 7f6d597fa700 1 -- 172.21.15.63:0/1725690540 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1114 (secure 0 0 0) 0x7f6d50059c10 con 0x7f6d5c101e40 2024-05-09T06:35:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:e2 2024-05-09T06:35:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:35:06.607 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 2 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:03.455919+0000 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:35:06.608 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 0 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:in 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:up {} 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:35:06.609 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 0 2024-05-09T06:35:06.610 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:35:06.610 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:35:06.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d4406ccf0 msgr2=0x7f6d4406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d4406ccf0 0x7f6d4406f1b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c005090 tx=0x7f6d4c005180 comp rx=0 tx=0).stop 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c101e40 msgr2=0x7f6d5c198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c101e40 0x7f6d5c198dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c930 tx=0x7f6d5000cc40 comp rx=0 tx=0).stop 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 shutdown_connections 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d4406ccf0 0x7f6d4406f1b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d5c100c40 0x7f6d5c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 --2- 172.21.15.63:0/1725690540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d5c101e40 0x7f6d5c198dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 >> 172.21.15.63:0/1725690540 conn(0x7f6d5c0fc1e0 msgr2=0x7f6d5c0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 shutdown_connections 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:06.629+0000 7f6d62940700 1 -- 172.21.15.63:0/1725690540 wait complete. 2024-05-09T06:35:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 2 2024-05-09T06:35:06.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:06 smithi193 ceph-mon[35141]: Deploying daemon mds.cephfs.smithi063.aoiibw on smithi063 2024-05-09T06:35:06.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:06 smithi193 ceph-mon[35141]: osdmap e46: 6 total, 6 up, 6 in 2024-05-09T06:35:07.424 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T06:35:07.434 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:35:07.434 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-05-09T06:35:07.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:07 smithi063 ceph-mon[27171]: pgmap v181: 65 pgs: 7 creating+peering, 31 active+clean, 27 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:07.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:07 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1725690540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:35:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:07 smithi193 ceph-mon[35141]: pgmap v181: 65 pgs: 7 creating+peering, 31 active+clean, 27 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:07 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1725690540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:35:08.005 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:09.523 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:09 smithi063 ceph-mon[27171]: pgmap v182: 65 pgs: 7 creating+peering, 53 active+clean, 5 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.785+0000 7fbbc49bf700 1 -- 172.21.15.63:0/2818024145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 msgr2=0x7fbbc0101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.785+0000 7fbbc49bf700 1 --2- 172.21.15.63:0/2818024145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 0x7fbbc0101180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009910 tx=0x7fbbb4009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.785+0000 7fbbc49bf700 1 -- 172.21.15.63:0/2818024145 shutdown_connections 2024-05-09T06:35:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.785+0000 7fbbc49bf700 1 --2- 172.21.15.63:0/2818024145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 0x7fbbc0101180 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.785+0000 7fbbc49bf700 1 --2- 172.21.15.63:0/2818024145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc00ffb20 0x7fbbc00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:09.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.785+0000 7fbbc49bf700 1 -- 172.21.15.63:0/2818024145 >> 172.21.15.63:0/2818024145 conn(0x7fbbc00fb0c0 msgr2=0x7fbbc00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:09.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.785+0000 7fbbc49bf700 1 -- 172.21.15.63:0/2818024145 shutdown_connections 2024-05-09T06:35:09.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.785+0000 7fbbc49bf700 1 -- 172.21.15.63:0/2818024145 wait complete. 2024-05-09T06:35:09.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.786+0000 7fbbc49bf700 1 Processor -- start 2024-05-09T06:35:09.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.786+0000 7fbbc49bf700 1 -- start start 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.786+0000 7fbbc49bf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc00ffb20 0x7fbbc0196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.786+0000 7fbbc49bf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 0x7fbbc0196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.786+0000 7fbbc49bf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0197120 con 0x7fbbc0100d20 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.786+0000 7fbbc49bf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0197260 con 0x7fbbc00ffb20 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbbdd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 0x7fbbc0196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbbdd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 0x7fbbc0196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56606/0 (socket says 172.21.15.63:56606) 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbbdd9b700 1 -- 172.21.15.63:0/3799525787 learned_addr learned my addr 172.21.15.63:0/3799525787 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbbe59c700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc00ffb20 0x7fbbc0196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbbdd9b700 1 -- 172.21.15.63:0/3799525787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc00ffb20 msgr2=0x7fbbc0196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbbdd9b700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc00ffb20 0x7fbbc0196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbbdd9b700 1 -- 172.21.15.63:0/3799525787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb40095f0 con 0x7fbbc0100d20 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbbdd9b700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 0x7fbbc0196b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400bd70 tx=0x7fbbb400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbaf7fe700 1 -- 172.21.15.63:0/3799525787 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbb401d070 con 0x7fbbc0100d20 2024-05-09T06:35:09.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc019bc20 con 0x7fbbc0100d20 2024-05-09T06:35:09.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbaf7fe700 1 -- 172.21.15.63:0/3799525787 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbb400f460 con 0x7fbbc0100d20 2024-05-09T06:35:09.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbaf7fe700 1 -- 172.21.15.63:0/3799525787 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbb4021640 con 0x7fbbc0100d20 2024-05-09T06:35:09.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.787+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc019c160 con 0x7fbbc0100d20 2024-05-09T06:35:09.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.788+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc004ebb0 con 0x7fbbc0100d20 2024-05-09T06:35:09.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.789+0000 7fbbaf7fe700 1 -- 172.21.15.63:0/3799525787 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbbb400fad0 con 0x7fbbc0100d20 2024-05-09T06:35:09.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.789+0000 7fbbaf7fe700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbba806ccf0 0x7fbba806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:09.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.790+0000 7fbbaf7fe700 1 -- 172.21.15.63:0/3799525787 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbbb408d570 con 0x7fbbc0100d20 2024-05-09T06:35:09.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.790+0000 7fbbbe59c700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbba806ccf0 0x7fbba806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:09.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.790+0000 7fbbbe59c700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbba806ccf0 0x7fbba806f1b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0100b80 tx=0x7fbbb000a3b0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:09.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:09.795+0000 7fbbaf7fe700 1 -- 172.21.15.63:0/3799525787 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbbb405b3d0 con 0x7fbbc0100d20 2024-05-09T06:35:09.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:09 smithi193 ceph-mon[35141]: pgmap v182: 65 pgs: 7 creating+peering, 53 active+clean, 5 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:10.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.046+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7fbbc0108680 con 0x7fbbc0100d20 2024-05-09T06:35:10.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.491+0000 7fbbaf7fe700 1 -- 172.21.15.63:0/3799525787 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v3) v1 ==== 105+0+0 (secure 0 0 0) 0x7fbbc0108680 con 0x7fbbc0100d20 2024-05-09T06:35:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbba806ccf0 msgr2=0x7fbba806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbba806ccf0 0x7fbba806f1b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0100b80 tx=0x7fbbb000a3b0 comp rx=0 tx=0).stop 2024-05-09T06:35:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 msgr2=0x7fbbc0196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 0x7fbbc0196b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400bd70 tx=0x7fbbb400bda0 comp rx=0 tx=0).stop 2024-05-09T06:35:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 shutdown_connections 2024-05-09T06:35:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbba806ccf0 0x7fbba806f1b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc00ffb20 0x7fbbc0196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 --2- 172.21.15.63:0/3799525787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc0100d20 0x7fbbc0196b90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 >> 172.21.15.63:0/3799525787 conn(0x7fbbc00fb0c0 msgr2=0x7fbbc0068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:10.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 shutdown_connections 2024-05-09T06:35:10.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:10.493+0000 7fbbc49bf700 1 -- 172.21.15.63:0/3799525787 wait complete. 2024-05-09T06:35:10.787 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:10 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3799525787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-09T06:35:10.787 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:10.787 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:10.788 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:10.788 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ggtmyb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:35:10.788 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ggtmyb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T06:35:10.788 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:10.801 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:10 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3799525787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-09T06:35:10.801 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:10.801 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:10.802 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:10.802 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ggtmyb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:35:10.802 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ggtmyb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T06:35:10.802 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:11.087 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T06:35:11.097 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:35:11.098 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-05-09T06:35:11.320 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:11.707 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: pgmap v183: 65 pgs: 65 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: Deploying daemon mds.cephfs.smithi193.ggtmyb on smithi193 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3799525787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.63:6826/1318147125,v1:172.21.15.63:6827/1318147125] up:boot 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: daemon mds.cephfs.smithi063.aoiibw assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: Cluster is now healthy 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: fsmap cephfs:0 1 up:standby 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:creating} 2024-05-09T06:35:11.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:11 smithi063 ceph-mon[27171]: daemon mds.cephfs.smithi063.aoiibw is now active in filesystem cephfs as rank 0 2024-05-09T06:35:11.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: pgmap v183: 65 pgs: 65 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:35:11.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: Deploying daemon mds.cephfs.smithi193.ggtmyb on smithi193 2024-05-09T06:35:11.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3799525787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-09T06:35:11.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.63:6826/1318147125,v1:172.21.15.63:6827/1318147125] up:boot 2024-05-09T06:35:11.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: daemon mds.cephfs.smithi063.aoiibw assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-09T06:35:11.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T06:35:11.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T06:35:11.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: Cluster is now healthy 2024-05-09T06:35:11.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: fsmap cephfs:0 1 up:standby 2024-05-09T06:35:11.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:35:11.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:creating} 2024-05-09T06:35:11.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:11 smithi193 ceph-mon[35141]: daemon mds.cephfs.smithi063.aoiibw is now active in filesystem cephfs as rank 0 2024-05-09T06:35:12.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:12 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.63:6826/1318147125,v1:172.21.15.63:6827/1318147125] up:active 2024-05-09T06:35:12.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:12 smithi063 ceph-mon[27171]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 2024-05-09T06:35:12.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:12 smithi063 ceph-mon[27171]: pgmap v184: 65 pgs: 65 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 1 op/s 2024-05-09T06:35:12.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:12 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:35:12.751 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:12 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.63:6826/1318147125,v1:172.21.15.63:6827/1318147125] up:active 2024-05-09T06:35:12.751 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:12 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 2024-05-09T06:35:12.751 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:12 smithi193 ceph-mon[35141]: pgmap v184: 65 pgs: 65 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 1 op/s 2024-05-09T06:35:12.751 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:12 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:35:12.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/844335253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 msgr2=0x7f6aa8103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:12.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 --2- 172.21.15.63:0/844335253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 0x7f6aa8103ab0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009910 tx=0x7f6a9c009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:12.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/844335253 shutdown_connections 2024-05-09T06:35:12.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 --2- 172.21.15.63:0/844335253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8103ff0 0x7f6aa81063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:12.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 --2- 172.21.15.63:0/844335253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 0x7f6aa8103ab0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:12.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/844335253 >> 172.21.15.63:0/844335253 conn(0x7f6aa80fb0f0 msgr2=0x7f6aa80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/844335253 shutdown_connections 2024-05-09T06:35:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/844335253 wait complete. 2024-05-09T06:35:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 Processor -- start 2024-05-09T06:35:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.918+0000 7f6aaf1ac700 1 -- start start 2024-05-09T06:35:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aaf1ac700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 0x7f6aa8100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aaf1ac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8103ff0 0x7f6aa80ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aaf1ac700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80ffbb0 con 0x7f6aa81016c0 2024-05-09T06:35:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aaf1ac700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80ffcf0 con 0x7f6aa8103ff0 2024-05-09T06:35:12.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aacf48700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 0x7f6aa8100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aacf48700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 0x7f6aa8100fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56644/0 (socket says 172.21.15.63:56644) 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aacf48700 1 -- 172.21.15.63:0/1276577334 learned_addr learned my addr 172.21.15.63:0/1276577334 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aa7fff700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8103ff0 0x7f6aa80ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aacf48700 1 -- 172.21.15.63:0/1276577334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8103ff0 msgr2=0x7f6aa80ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aacf48700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8103ff0 0x7f6aa80ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aacf48700 1 -- 172.21.15.63:0/1276577334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0095f0 con 0x7f6aa81016c0 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aacf48700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 0x7f6aa8100fd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0092e0 tx=0x7f6a9c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aa5ffb700 1 -- 172.21.15.63:0/1276577334 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9c01d070 con 0x7f6aa81016c0 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa80fff70 con 0x7f6aa81016c0 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aa5ffb700 1 -- 172.21.15.63:0/1276577334 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a9c00f460 con 0x7f6aa81016c0 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.919+0000 7f6aa5ffb700 1 -- 172.21.15.63:0/1276577334 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9c021640 con 0x7f6aa81016c0 2024-05-09T06:35:12.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.920+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa81004e0 con 0x7f6aa81016c0 2024-05-09T06:35:12.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.921+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa804ebb0 con 0x7f6aa81016c0 2024-05-09T06:35:12.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.921+0000 7f6aa5ffb700 1 -- 172.21.15.63:0/1276577334 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6a9c00f5d0 con 0x7f6aa81016c0 2024-05-09T06:35:12.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.922+0000 7f6aa5ffb700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6a9806cce0 0x7f6a9806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.922+0000 7f6aa7fff700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6a9806cce0 0x7f6a9806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.922+0000 7f6aa5ffb700 1 -- 172.21.15.63:0/1276577334 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6a9c08d430 con 0x7f6aa81016c0 2024-05-09T06:35:12.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.922+0000 7f6aa7fff700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6a9806cce0 0x7f6a9806f1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006fd0 tx=0x7f6a94008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:12.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:12.926+0000 7f6aa5ffb700 1 -- 172.21.15.63:0/1276577334 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6a9c05b290 con 0x7f6aa81016c0 2024-05-09T06:35:13.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.180+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f6aa8109810 con 0x7f6aa81016c0 2024-05-09T06:35:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.505+0000 7f6aa5ffb700 1 -- 172.21.15.63:0/1276577334 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v6) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6aa8109810 con 0x7f6aa81016c0 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6a9806cce0 msgr2=0x7f6a9806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6a9806cce0 0x7f6a9806f1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006fd0 tx=0x7f6a94008040 comp rx=0 tx=0).stop 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 msgr2=0x7f6aa8100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 0x7f6aa8100fd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0092e0 tx=0x7f6a9c00be20 comp rx=0 tx=0).stop 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 shutdown_connections 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa81016c0 0x7f6aa8100fd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6a9806cce0 0x7f6a9806f1a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 --2- 172.21.15.63:0/1276577334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8103ff0 0x7f6aa80ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 >> 172.21.15.63:0/1276577334 conn(0x7f6aa80fb0f0 msgr2=0x7f6aa80fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 shutdown_connections 2024-05-09T06:35:13.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:13.508+0000 7f6aaf1ac700 1 -- 172.21.15.63:0/1276577334 wait complete. 2024-05-09T06:35:14.237 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:14.237 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.fbzxbf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.fbzxbf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: Deploying daemon mds.cephfs.smithi063.fbzxbf on smithi063 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1276577334' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:boot 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1276577334' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:35:14.238 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:14 smithi063 ceph-mon[27171]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 2024-05-09T06:35:14.246 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:14.246 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:14.246 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:14.246 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.fbzxbf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:35:14.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.fbzxbf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T06:35:14.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:14.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: Deploying daemon mds.cephfs.smithi063.fbzxbf on smithi063 2024-05-09T06:35:14.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1276577334' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T06:35:14.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:boot 2024-05-09T06:35:14.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1276577334' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T06:35:14.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby 2024-05-09T06:35:14.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:35:14.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:14 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 2024-05-09T06:35:14.263 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T06:35:14.276 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:35:14.276 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-05-09T06:35:15.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:15 smithi063 ceph-mon[27171]: pgmap v185: 65 pgs: 65 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 224 B/s wr, 2 op/s 2024-05-09T06:35:15.166 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:15.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:15 smithi193 ceph-mon[35141]: pgmap v185: 65 pgs: 65 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 224 B/s wr, 2 op/s 2024-05-09T06:35:16.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.829+0000 7f6d2742e700 1 -- 172.21.15.63:0/2321690113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d20100da0 msgr2=0x7f6d20101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:16.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.829+0000 7f6d2742e700 1 --2- 172.21.15.63:0/2321690113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d20100da0 0x7f6d20101200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800b0a0 tx=0x7f6d1800b3b0 comp rx=0 tx=0).stop 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.829+0000 7f6d2742e700 1 -- 172.21.15.63:0/2321690113 shutdown_connections 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.829+0000 7f6d2742e700 1 --2- 172.21.15.63:0/2321690113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d20100da0 0x7f6d20101200 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.829+0000 7f6d2742e700 1 --2- 172.21.15.63:0/2321690113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d200ffba0 0x7f6d200fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.829+0000 7f6d2742e700 1 -- 172.21.15.63:0/2321690113 >> 172.21.15.63:0/2321690113 conn(0x7f6d200fb120 msgr2=0x7f6d200fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.829+0000 7f6d2742e700 1 -- 172.21.15.63:0/2321690113 shutdown_connections 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.829+0000 7f6d2742e700 1 -- 172.21.15.63:0/2321690113 wait complete. 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.830+0000 7f6d2742e700 1 Processor -- start 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.830+0000 7f6d2742e700 1 -- start start 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.830+0000 7f6d2742e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d200ffba0 0x7f6d200690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.830+0000 7f6d2742e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d20100da0 0x7f6d2019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.830+0000 7f6d2742e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2019b9e0 con 0x7f6d200ffba0 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.830+0000 7f6d2742e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2019bb20 con 0x7f6d20100da0 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d251ca700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d200ffba0 0x7f6d200690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:16.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d251ca700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d200ffba0 0x7f6d200690b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39594/0 (socket says 172.21.15.63:39594) 2024-05-09T06:35:16.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d251ca700 1 -- 172.21.15.63:0/3347346238 learned_addr learned my addr 172.21.15.63:0/3347346238 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:16.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d249c9700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d20100da0 0x7f6d2019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:16.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d251ca700 1 -- 172.21.15.63:0/3347346238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d20100da0 msgr2=0x7f6d2019b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:16.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d251ca700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d20100da0 0x7f6d2019b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:16.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d251ca700 1 -- 172.21.15.63:0/3347346238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d18009d00 con 0x7f6d200ffba0 2024-05-09T06:35:16.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d251ca700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d200ffba0 0x7f6d200690b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b480 tx=0x7f6d1c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:16.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d167fc700 1 -- 172.21.15.63:0/3347346238 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d1c011800 con 0x7f6d200ffba0 2024-05-09T06:35:16.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d167fc700 1 -- 172.21.15.63:0/3347346238 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d1c011e20 con 0x7f6d200ffba0 2024-05-09T06:35:16.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2019be00 con 0x7f6d200ffba0 2024-05-09T06:35:16.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d167fc700 1 -- 172.21.15.63:0/3347346238 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d1c010ea0 con 0x7f6d200ffba0 2024-05-09T06:35:16.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.831+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2019c350 con 0x7f6d200ffba0 2024-05-09T06:35:16.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.833+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2018f7c0 con 0x7f6d200ffba0 2024-05-09T06:35:16.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.833+0000 7f6d167fc700 1 -- 172.21.15.63:0/3347346238 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6d1c010380 con 0x7f6d200ffba0 2024-05-09T06:35:16.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.834+0000 7f6d167fc700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d0c06cce0 0x7f6d0c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:16.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.834+0000 7f6d249c9700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d0c06cce0 0x7f6d0c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:16.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.834+0000 7f6d167fc700 1 -- 172.21.15.63:0/3347346238 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6d1c08be20 con 0x7f6d200ffba0 2024-05-09T06:35:16.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.834+0000 7f6d249c9700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d0c06cce0 0x7f6d0c06f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6d18007820 tx=0x7f6d18006000 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:16.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:16.840+0000 7f6d167fc700 1 -- 172.21.15.63:0/3347346238 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6d1c059bd0 con 0x7f6d200ffba0 2024-05-09T06:35:17.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:17.095+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f6d2004ebb0 con 0x7f6d200ffba0 2024-05-09T06:35:17.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:17 smithi063 ceph-mon[27171]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 810 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-09T06:35:17.187 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:17 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:17.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:17 smithi193 ceph-mon[35141]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 810 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-09T06:35:17.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:17 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:17.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.015+0000 7f6d167fc700 1 -- 172.21.15.63:0/3347346238 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v8) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6d1c059760 con 0x7f6d200ffba0 2024-05-09T06:35:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.017+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d0c06cce0 msgr2=0x7f6d0c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.017+0000 7f6d2742e700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d0c06cce0 0x7f6d0c06f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6d18007820 tx=0x7f6d18006000 comp rx=0 tx=0).stop 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.017+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d200ffba0 msgr2=0x7f6d200690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.017+0000 7f6d2742e700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d200ffba0 0x7f6d200690b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b480 tx=0x7f6d1c00b790 comp rx=0 tx=0).stop 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.018+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 shutdown_connections 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.018+0000 7f6d2742e700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d200ffba0 0x7f6d200690b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.018+0000 7f6d2742e700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f6d0c06cce0 0x7f6d0c06f1a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.018+0000 7f6d2742e700 1 --2- 172.21.15.63:0/3347346238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d20100da0 0x7f6d2019b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.018+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 >> 172.21.15.63:0/3347346238 conn(0x7f6d200fb120 msgr2=0x7f6d201043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.018+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 shutdown_connections 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:18.018+0000 7f6d2742e700 1 -- 172.21.15.63:0/3347346238 wait complete. 2024-05-09T06:35:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:inline data disabled 2024-05-09T06:35:18.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:18 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3347346238' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-05-09T06:35:18.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:18.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:18.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:18.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sygxnf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:35:18.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sygxnf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T06:35:18.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:18 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:18.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:18 smithi193 ceph-mon[35141]: Deploying daemon mds.cephfs.smithi193.sygxnf on smithi193 2024-05-09T06:35:18.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:18 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3347346238' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-05-09T06:35:18.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:18.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:18.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:18.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sygxnf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:35:18.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sygxnf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T06:35:18.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:18 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:18.284 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:18 smithi063 ceph-mon[27171]: Deploying daemon mds.cephfs.smithi193.sygxnf on smithi193 2024-05-09T06:35:18.588 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T06:35:18.598 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:35:18.598 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T06:35:18.820 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:19.202 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:19 smithi063 ceph-mon[27171]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 682 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-09T06:35:19.203 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:19 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3347346238' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-05-09T06:35:19.203 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:19 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] up:boot 2024-05-09T06:35:19.203 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:19 smithi063 ceph-mon[27171]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 1 up:standby 2024-05-09T06:35:19.203 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:19 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:35:19.298 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:19 smithi193 ceph-mon[35141]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 682 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-09T06:35:19.298 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:19 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3347346238' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-05-09T06:35:19.298 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:19 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] up:boot 2024-05-09T06:35:19.298 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:19 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 1 up:standby 2024-05-09T06:35:19.298 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:19 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.462+0000 7f2a43894700 1 -- 172.21.15.63:0/1389328698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c101e20 msgr2=0x7f2a3c102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.462+0000 7f2a43894700 1 --2- 172.21.15.63:0/1389328698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c101e20 0x7f2a3c102280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009910 tx=0x7f2a2c009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.462+0000 7f2a43894700 1 -- 172.21.15.63:0/1389328698 shutdown_connections 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.462+0000 7f2a43894700 1 --2- 172.21.15.63:0/1389328698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c101e20 0x7f2a3c102280 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.462+0000 7f2a43894700 1 --2- 172.21.15.63:0/1389328698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3c100c20 0x7f2a3c101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.462+0000 7f2a43894700 1 -- 172.21.15.63:0/1389328698 >> 172.21.15.63:0/1389328698 conn(0x7f2a3c0fc200 msgr2=0x7f2a3c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.463+0000 7f2a43894700 1 -- 172.21.15.63:0/1389328698 shutdown_connections 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.463+0000 7f2a43894700 1 -- 172.21.15.63:0/1389328698 wait complete. 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.463+0000 7f2a43894700 1 Processor -- start 2024-05-09T06:35:20.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.463+0000 7f2a43894700 1 -- start start 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.463+0000 7f2a43894700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c100c20 0x7f2a3c196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.463+0000 7f2a43894700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3c101e20 0x7f2a3c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.463+0000 7f2a43894700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c197170 con 0x7f2a3c100c20 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.463+0000 7f2a43894700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1972b0 con 0x7f2a3c101e20 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a41630700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c100c20 0x7f2a3c196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a41630700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c100c20 0x7f2a3c196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39630/0 (socket says 172.21.15.63:39630) 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a41630700 1 -- 172.21.15.63:0/4098733485 learned_addr learned my addr 172.21.15.63:0/4098733485 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a40e2f700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3c101e20 0x7f2a3c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a41630700 1 -- 172.21.15.63:0/4098733485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3c101e20 msgr2=0x7f2a3c196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a41630700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3c101e20 0x7f2a3c196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:20.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a41630700 1 -- 172.21.15.63:0/4098733485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0095f0 con 0x7f2a3c100c20 2024-05-09T06:35:20.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a41630700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c100c20 0x7f2a3c196610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800d900 tx=0x7f2a3800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:20.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a327fc700 1 -- 172.21.15.63:0/4098733485 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3800b920 con 0x7f2a3c100c20 2024-05-09T06:35:20.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a327fc700 1 -- 172.21.15.63:0/4098733485 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a38010460 con 0x7f2a3c100c20 2024-05-09T06:35:20.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a327fc700 1 -- 172.21.15.63:0/4098733485 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3800f590 con 0x7f2a3c100c20 2024-05-09T06:35:20.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c19bcc0 con 0x7f2a3c100c20 2024-05-09T06:35:20.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.464+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c19c1c0 con 0x7f2a3c100c20 2024-05-09T06:35:20.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.465+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c1907a0 con 0x7f2a3c100c20 2024-05-09T06:35:20.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.466+0000 7f2a327fc700 1 -- 172.21.15.63:0/4098733485 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2a380105d0 con 0x7f2a3c100c20 2024-05-09T06:35:20.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.467+0000 7f2a327fc700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a2806cce0 0x7f2a2806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:20.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.467+0000 7f2a327fc700 1 -- 172.21.15.63:0/4098733485 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2a3808bf00 con 0x7f2a3c100c20 2024-05-09T06:35:20.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.467+0000 7f2a40e2f700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a2806cce0 0x7f2a2806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:20.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.467+0000 7f2a40e2f700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a2806cce0 0x7f2a2806f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009fd0 tx=0x7f2a2c005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:20.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.472+0000 7f2a327fc700 1 -- 172.21.15.63:0/4098733485 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2a38059d60 con 0x7f2a3c100c20 2024-05-09T06:35:20.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.729+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2a3c109850 con 0x7f2a3c100c20 2024-05-09T06:35:20.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.730+0000 7f2a327fc700 1 -- 172.21.15.63:0/4098733485 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 9 v9) v1 ==== 75+0+1818 (secure 0 0 0) 0x7f2a38020730 con 0x7f2a3c100c20 2024-05-09T06:35:20.711 INFO:teuthology.orchestra.run.smithi063.stdout:e9 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 8 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:18.010515+0000 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:35:20.712 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 0 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14476} 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:35:20.713 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{0:14476} state up:active seq 2 addr [v2:172.21.15.63:6826/1318147125,v1:172.21.15.63:6827/1318147125] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:standby-replay seq 1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{-1:14492} state up:standby seq 1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:35:20.714 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:35:20.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a2806cce0 msgr2=0x7f2a2806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:20.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a2806cce0 0x7f2a2806f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009fd0 tx=0x7f2a2c005960 comp rx=0 tx=0).stop 2024-05-09T06:35:20.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c100c20 msgr2=0x7f2a3c196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:20.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c100c20 0x7f2a3c196610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800d900 tx=0x7f2a3800dc10 comp rx=0 tx=0).stop 2024-05-09T06:35:20.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 shutdown_connections 2024-05-09T06:35:20.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a3c100c20 0x7f2a3c196610 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:20.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a2806cce0 0x7f2a2806f1a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:20.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 --2- 172.21.15.63:0/4098733485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a3c101e20 0x7f2a3c196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:20.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 >> 172.21.15.63:0/4098733485 conn(0x7f2a3c0fc200 msgr2=0x7f2a3c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:20.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 shutdown_connections 2024-05-09T06:35:20.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:20.732+0000 7f2a43894700 1 -- 172.21.15.63:0/4098733485 wait complete. 2024-05-09T06:35:20.716 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 9 2024-05-09T06:35:21.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-09T06:35:21.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:boot 2024-05-09T06:35:21.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:35:21.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:35:21.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.340 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.340 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.340 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.340 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:21.340 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:21 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/4098733485' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:35:21.361 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:boot 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:21 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/4098733485' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:35:21.987 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:22.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:22 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:standby-replay 2024-05-09T06:35:22.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:22 smithi063 ceph-mon[27171]: Dropping low affinity active daemon mds.cephfs.smithi063.aoiibw in favor of higher affinity standby. 2024-05-09T06:35:22.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:22 smithi063 ceph-mon[27171]: Replacing daemon mds.cephfs.smithi063.aoiibw as rank 0 with standby daemon mds.cephfs.smithi193.ggtmyb 2024-05-09T06:35:22.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:22 smithi063 ceph-mon[27171]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T06:35:22.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:22 smithi063 ceph-mon[27171]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:35:22.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:22 smithi063 ceph-mon[27171]: osdmap e47: 6 total, 6 up, 6 in 2024-05-09T06:35:22.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:22 smithi063 ceph-mon[27171]: fsmap cephfs:1/1 {0=cephfs.smithi193.ggtmyb=up:replay} 2 up:standby 2024-05-09T06:35:22.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:22 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:22 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:standby-replay 2024-05-09T06:35:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:22 smithi193 ceph-mon[35141]: Dropping low affinity active daemon mds.cephfs.smithi063.aoiibw in favor of higher affinity standby. 2024-05-09T06:35:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:22 smithi193 ceph-mon[35141]: Replacing daemon mds.cephfs.smithi063.aoiibw as rank 0 with standby daemon mds.cephfs.smithi193.ggtmyb 2024-05-09T06:35:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:22 smithi193 ceph-mon[35141]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T06:35:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:22 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:35:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:22 smithi193 ceph-mon[35141]: osdmap e47: 6 total, 6 up, 6 in 2024-05-09T06:35:22.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:22 smithi193 ceph-mon[35141]: fsmap cephfs:1/1 {0=cephfs.smithi193.ggtmyb=up:replay} 2 up:standby 2024-05-09T06:35:22.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:22 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:23.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:23 smithi063 ceph-mon[27171]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-09T06:35:23.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:23 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] up:boot 2024-05-09T06:35:23.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:23 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] up:standby 2024-05-09T06:35:23.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:23 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:reconnect 2024-05-09T06:35:23.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:23 smithi063 ceph-mon[27171]: fsmap cephfs:1/1 {0=cephfs.smithi193.ggtmyb=up:reconnect} 3 up:standby 2024-05-09T06:35:23.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:35:23.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:23.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:23 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.635+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3349738638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad0103ff0 msgr2=0x7f0ad01063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.635+0000 7f0ad4b58700 1 --2- 172.21.15.63:0/3349738638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad0103ff0 0x7f0ad01063e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009910 tx=0x7f0ab8009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.635+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3349738638 shutdown_connections 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.635+0000 7f0ad4b58700 1 --2- 172.21.15.63:0/3349738638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad0103ff0 0x7f0ad01063e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.635+0000 7f0ad4b58700 1 --2- 172.21.15.63:0/3349738638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad01016c0 0x7f0ad0103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.635+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3349738638 >> 172.21.15.63:0/3349738638 conn(0x7f0ad00fb0f0 msgr2=0x7f0ad00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.635+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3349738638 shutdown_connections 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.635+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3349738638 wait complete. 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ad4b58700 1 Processor -- start 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ad4b58700 1 -- start start 2024-05-09T06:35:23.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ad4b58700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad01016c0 0x7f0ad0196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ad4b58700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad0103ff0 0x7f0ad0196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ad4b58700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0197180 con 0x7f0ad01016c0 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ad4b58700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad01972c0 con 0x7f0ad0103ff0 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ace59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad01016c0 0x7f0ad0196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ace59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad01016c0 0x7f0ad0196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39680/0 (socket says 172.21.15.63:39680) 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ace59c700 1 -- 172.21.15.63:0/3319804674 learned_addr learned my addr 172.21.15.63:0/3319804674 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0acdd9b700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad0103ff0 0x7f0ad0196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ace59c700 1 -- 172.21.15.63:0/3319804674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad0103ff0 msgr2=0x7f0ad0196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ace59c700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad0103ff0 0x7f0ad0196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0ace59c700 1 -- 172.21.15.63:0/3319804674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80095f0 con 0x7f0ad01016c0 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.636+0000 7f0acdd9b700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad0103ff0 0x7f0ad0196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.637+0000 7f0ace59c700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad01016c0 0x7f0ad0196620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000c8a0 tx=0x7f0ac000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.637+0000 7f0ac77fe700 1 -- 172.21.15.63:0/3319804674 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac000cea0 con 0x7f0ad01016c0 2024-05-09T06:35:23.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.637+0000 7f0ac77fe700 1 -- 172.21.15.63:0/3319804674 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ac0004830 con 0x7f0ad01016c0 2024-05-09T06:35:23.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.637+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad019bd20 con 0x7f0ad01016c0 2024-05-09T06:35:23.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.637+0000 7f0ac77fe700 1 -- 172.21.15.63:0/3319804674 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac00055b0 con 0x7f0ad01016c0 2024-05-09T06:35:23.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.637+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad019c290 con 0x7f0ad01016c0 2024-05-09T06:35:23.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.638+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad004ec20 con 0x7f0ad01016c0 2024-05-09T06:35:23.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.638+0000 7f0ac77fe700 1 -- 172.21.15.63:0/3319804674 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0ac00049a0 con 0x7f0ad01016c0 2024-05-09T06:35:23.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.639+0000 7f0ac77fe700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0abc06cce0 0x7f0abc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:23.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.639+0000 7f0ac77fe700 1 -- 172.21.15.63:0/3319804674 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0ac008bf50 con 0x7f0ad01016c0 2024-05-09T06:35:23.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.639+0000 7f0acdd9b700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0abc06cce0 0x7f0abc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:23.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.639+0000 7f0acdd9b700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0abc06cce0 0x7f0abc06f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80059d0 tx=0x7f0ab8005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:23.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:23 smithi193 ceph-mon[35141]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-09T06:35:23.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:23 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] up:boot 2024-05-09T06:35:23.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:23 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] up:standby 2024-05-09T06:35:23.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:23 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:reconnect 2024-05-09T06:35:23.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:23 smithi193 ceph-mon[35141]: fsmap cephfs:1/1 {0=cephfs.smithi193.ggtmyb=up:reconnect} 3 up:standby 2024-05-09T06:35:23.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:35:23.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:23.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:23 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:23.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.644+0000 7f0ac77fe700 1 -- 172.21.15.63:0/3319804674 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0ac00bfc50 con 0x7f0ad01016c0 2024-05-09T06:35:23.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.897+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ad0109a20 con 0x7f0ad01016c0 2024-05-09T06:35:23.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.897+0000 7f0ac77fe700 1 -- 172.21.15.63:0/3319804674 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4826 (secure 0 0 0) 0x7f0ad004ec20 con 0x7f0ad01016c0 2024-05-09T06:35:23.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0abc06cce0 msgr2=0x7f0abc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:23.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0abc06cce0 0x7f0abc06f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80059d0 tx=0x7f0ab8005960 comp rx=0 tx=0).stop 2024-05-09T06:35:23.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad01016c0 msgr2=0x7f0ad0196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:23.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad01016c0 0x7f0ad0196620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000c8a0 tx=0x7f0ac000cc60 comp rx=0 tx=0).stop 2024-05-09T06:35:23.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 shutdown_connections 2024-05-09T06:35:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ad01016c0 0x7f0ad0196620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f0abc06cce0 0x7f0abc06f1a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 --2- 172.21.15.63:0/3319804674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ad0103ff0 0x7f0ad0196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 >> 172.21.15.63:0/3319804674 conn(0x7f0ad00fb0f0 msgr2=0x7f0ad00fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 shutdown_connections 2024-05-09T06:35:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:23.900+0000 7f0ad4b58700 1 -- 172.21.15.63:0/3319804674 wait complete. 2024-05-09T06:35:23.884 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 13 2024-05-09T06:35:23.892 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:35:24.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:24 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:rejoin 2024-05-09T06:35:24.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:24 smithi063 ceph-mon[27171]: fsmap cephfs:1/1 {0=cephfs.smithi193.ggtmyb=up:rejoin} 3 up:standby 2024-05-09T06:35:24.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:24 smithi063 ceph-mon[27171]: daemon mds.cephfs.smithi193.ggtmyb is now active in filesystem cephfs as rank 0 2024-05-09T06:35:24.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:24 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3319804674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T06:35:24.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:24 smithi063 ceph-mon[27171]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T06:35:24.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:24 smithi063 ceph-mon[27171]: Cluster is now healthy 2024-05-09T06:35:24.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:24 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:active 2024-05-09T06:35:24.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:24 smithi063 ceph-mon[27171]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:standby 2024-05-09T06:35:24.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:24 smithi063 ceph-mon[27171]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 3 up:standby 2024-05-09T06:35:24.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:24 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:rejoin 2024-05-09T06:35:24.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:24 smithi193 ceph-mon[35141]: fsmap cephfs:1/1 {0=cephfs.smithi193.ggtmyb=up:rejoin} 3 up:standby 2024-05-09T06:35:24.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:24 smithi193 ceph-mon[35141]: daemon mds.cephfs.smithi193.ggtmyb is now active in filesystem cephfs as rank 0 2024-05-09T06:35:24.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:24 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3319804674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T06:35:24.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:24 smithi193 ceph-mon[35141]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T06:35:24.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:24 smithi193 ceph-mon[35141]: Cluster is now healthy 2024-05-09T06:35:24.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:24 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] up:active 2024-05-09T06:35:24.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:24 smithi193 ceph-mon[35141]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:standby 2024-05-09T06:35:24.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:24 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 3 up:standby 2024-05-09T06:35:24.916 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-05-09T06:35:25.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:25 smithi063 ceph-mon[27171]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-09T06:35:25.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:25 smithi063 ceph-mon[27171]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:35:25.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:25 smithi193 ceph-mon[35141]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-09T06:35:25.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:25 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:35:25.626 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:27.287 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:27 smithi063 ceph-mon[27171]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-09T06:35:27.287 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:35:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:27 smithi193 ceph-mon[35141]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-09T06:35:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:35:28.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.442+0000 7f030d721700 1 -- 172.21.15.63:0/4040876708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03080ffba0 msgr2=0x7f03080fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.442+0000 7f030d721700 1 --2- 172.21.15.63:0/4040876708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03080ffba0 0x7f03080fffc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f02f800b0a0 tx=0x7f02f800b3b0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.442+0000 7f030d721700 1 -- 172.21.15.63:0/4040876708 shutdown_connections 2024-05-09T06:35:28.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.442+0000 7f030d721700 1 --2- 172.21.15.63:0/4040876708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0308100da0 0x7f0308101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.442+0000 7f030d721700 1 --2- 172.21.15.63:0/4040876708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03080ffba0 0x7f03080fffc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.442+0000 7f030d721700 1 -- 172.21.15.63:0/4040876708 >> 172.21.15.63:0/4040876708 conn(0x7f03080fb120 msgr2=0x7f03080fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.442+0000 7f030d721700 1 -- 172.21.15.63:0/4040876708 shutdown_connections 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.442+0000 7f030d721700 1 -- 172.21.15.63:0/4040876708 wait complete. 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f030d721700 1 Processor -- start 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f030d721700 1 -- start start 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f030d721700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03080ffba0 0x7f03080690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f030d721700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308100da0 0x7f030819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f030d721700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030819b990 con 0x7f0308100da0 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f030d721700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030819bad0 con 0x7f03080ffba0 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f03067fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308100da0 0x7f030819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f03067fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308100da0 0x7f030819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38066/0 (socket says 172.21.15.63:38066) 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f03067fc700 1 -- 172.21.15.63:0/1357057711 learned_addr learned my addr 172.21.15.63:0/1357057711 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f0306ffd700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03080ffba0 0x7f03080690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f03067fc700 1 -- 172.21.15.63:0/1357057711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03080ffba0 msgr2=0x7f03080690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f03067fc700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03080ffba0 0x7f03080690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.443+0000 7f03067fc700 1 -- 172.21.15.63:0/1357057711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f8009d00 con 0x7f0308100da0 2024-05-09T06:35:28.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.444+0000 7f03067fc700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308100da0 0x7f030819b450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f02f000d8d0 tx=0x7f02f000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.444+0000 7f02fffff700 1 -- 172.21.15.63:0/1357057711 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f00098a0 con 0x7f0308100da0 2024-05-09T06:35:28.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.444+0000 7f02fffff700 1 -- 172.21.15.63:0/1357057711 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02f0010460 con 0x7f0308100da0 2024-05-09T06:35:28.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.444+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030819bdb0 con 0x7f0308100da0 2024-05-09T06:35:28.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.444+0000 7f02fffff700 1 -- 172.21.15.63:0/1357057711 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f000f5f0 con 0x7f0308100da0 2024-05-09T06:35:28.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.444+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030819c2d0 con 0x7f0308100da0 2024-05-09T06:35:28.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.445+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030818f720 con 0x7f0308100da0 2024-05-09T06:35:28.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.445+0000 7f02fffff700 1 -- 172.21.15.63:0/1357057711 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f02f0009a00 con 0x7f0308100da0 2024-05-09T06:35:28.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.446+0000 7f02fffff700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f02f406cce0 0x7f02f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:28.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.446+0000 7f02fffff700 1 -- 172.21.15.63:0/1357057711 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f02f008c130 con 0x7f0308100da0 2024-05-09T06:35:28.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.446+0000 7f0306ffd700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f02f406cce0 0x7f02f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:28.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.446+0000 7f0306ffd700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f02f406cce0 0x7f02f406f1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009fd0 tx=0x7f02f8006000 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:28.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.451+0000 7f02fffff700 1 -- 172.21.15.63:0/1357057711 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f02f0059f20 con 0x7f0308100da0 2024-05-09T06:35:28.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.763+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f030804ebb0 con 0x7f0308100da0 2024-05-09T06:35:28.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.763+0000 7f02fffff700 1 -- 172.21.15.63:0/1357057711 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+98 (secure 0 0 0) 0x7f02f0059ab0 con 0x7f0308100da0 2024-05-09T06:35:28.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f02f406cce0 msgr2=0x7f02f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f02f406cce0 0x7f02f406f1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009fd0 tx=0x7f02f8006000 comp rx=0 tx=0).stop 2024-05-09T06:35:28.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308100da0 msgr2=0x7f030819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:28.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308100da0 0x7f030819b450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f02f000d8d0 tx=0x7f02f000dbe0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 shutdown_connections 2024-05-09T06:35:28.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f02f406cce0 0x7f02f406f1a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03080ffba0 0x7f03080690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 --2- 172.21.15.63:0/1357057711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0308100da0 0x7f030819b450 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:28.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 >> 172.21.15.63:0/1357057711 conn(0x7f03080fb120 msgr2=0x7f03081043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:28.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 shutdown_connections 2024-05-09T06:35:28.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:28.766+0000 7f030d721700 1 -- 172.21.15.63:0/1357057711 wait complete. 2024-05-09T06:35:28.759 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:35:29.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:29 smithi063 ceph-mon[27171]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T06:35:29.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:29 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:29.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:29 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:29.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:29 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:29.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:29 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:29.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:29 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:29.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:29 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1357057711' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-09T06:35:29.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:29 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:29.352 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-05-09T06:35:29.370 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:35:29.407 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:29 smithi193 ceph-mon[35141]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T06:35:29.407 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:29 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:29.407 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:29 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:29.407 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:29 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:29.407 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:29 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:29.407 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:29 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:29.407 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:29 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1357057711' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-09T06:35:29.407 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:29 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:29.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.509+0000 7f95bf646640 1 -- 172.21.15.63:0/695110491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 msgr2=0x7f95b80ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.509+0000 7f95bf646640 1 --2- 172.21.15.63:0/695110491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 0x7f95b80ff430 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009880 tx=0x7f95b402bce0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.509+0000 7f95bf646640 1 -- 172.21.15.63:0/695110491 shutdown_connections 2024-05-09T06:35:29.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.509+0000 7f95bf646640 1 --2- 172.21.15.63:0/695110491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95b8100200 0x7f95b8100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.509+0000 7f95bf646640 1 --2- 172.21.15.63:0/695110491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 0x7f95b80ff430 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.509+0000 7f95bf646640 1 -- 172.21.15.63:0/695110491 >> 172.21.15.63:0/695110491 conn(0x7f95b80fa5a0 msgr2=0x7f95b80fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.509+0000 7f95bf646640 1 -- 172.21.15.63:0/695110491 shutdown_connections 2024-05-09T06:35:29.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.509+0000 7f95bf646640 1 -- 172.21.15.63:0/695110491 wait complete. 2024-05-09T06:35:29.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bf646640 1 Processor -- start 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bf646640 1 -- start start 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bf646640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 0x7f95b8195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bf646640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95b8100200 0x7f95b81961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bf646640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b8196790 con 0x7f95b80ff010 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bf646640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b8196900 con 0x7f95b8100200 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bd3e2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 0x7f95b8195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bd3e2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 0x7f95b8195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38080/0 (socket says 172.21.15.63:38080) 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bd3e2640 1 -- 172.21.15.63:0/2007313599 learned_addr learned my addr 172.21.15.63:0/2007313599 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.510+0000 7f95bcbe1640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95b8100200 0x7f95b81961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.511+0000 7f95bd3e2640 1 -- 172.21.15.63:0/2007313599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95b8100200 msgr2=0x7f95b81961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.511+0000 7f95bd3e2640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95b8100200 0x7f95b81961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.511+0000 7f95bd3e2640 1 -- 172.21.15.63:0/2007313599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b4009560 con 0x7f95b80ff010 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.511+0000 7f95bd3e2640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 0x7f95b8195c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95b402c960 tx=0x7f95b402c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.511+0000 7f95ae7fc640 1 -- 172.21.15.63:0/2007313599 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b4004520 con 0x7f95b80ff010 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.511+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b819b2d0 con 0x7f95b80ff010 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.511+0000 7f95ae7fc640 1 -- 172.21.15.63:0/2007313599 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95b4003ab0 con 0x7f95b80ff010 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.511+0000 7f95ae7fc640 1 -- 172.21.15.63:0/2007313599 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b4005660 con 0x7f95b80ff010 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.511+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b819b850 con 0x7f95b80ff010 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.512+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b8107c10 con 0x7f95b80ff010 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.513+0000 7f95ae7fc640 1 -- 172.21.15.63:0/2007313599 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f95b4004ed0 con 0x7f95b80ff010 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.513+0000 7f95ae7fc640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f95a406cbf0 0x7f95a406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.513+0000 7f95ae7fc640 1 -- 172.21.15.63:0/2007313599 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f95b40ae5b0 con 0x7f95b80ff010 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.513+0000 7f95bcbe1640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f95a406cbf0 0x7f95a406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.514+0000 7f95bcbe1640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f95a406cbf0 0x7f95a406f0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009bb0 tx=0x7f95a8005f90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.518+0000 7f95ae7fc640 1 -- 172.21.15.63:0/2007313599 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f95b40789d0 con 0x7f95b80ff010 2024-05-09T06:35:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.709+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95b8107da0 con 0x7f95b80ff010 2024-05-09T06:35:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.710+0000 7f95ae7fc640 1 -- 172.21.15.63:0/2007313599 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15127 (secure 0 0 0) 0x7f95b407bff0 con 0x7f95b80ff010 2024-05-09T06:35:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:35:29.691 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":47,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","created":"2024-05-09T06:27:53.398729+0000","modified":"2024-05-09T06:35:21.124811+0000","last_up_change":"2024-05-09T06:33:45.888973+0000","last_in_change":"2024-05-09T06:33:25.853191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:32:32.042216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-09T06:35:02.184376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-09T06:35:02.871327+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eea045d9-973d-4960-a34e-689f9403d55f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6803","nonce":1596324997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6805","nonce":1596324997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6809","nonce":1596324997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1596324997},{"type":"v1","addr":"172.21.15.63:6807","nonce":1596324997}]},"public_addr":"172.21.15.63:6803/1596324997","cluster_addr":"172.21.15.63:6805/1596324997","heartbeat_back_addr":"172.21.15.63:6809/1596324997","heartbeat_front_addr":"172.21.15.63:6807/1596324997","state":["exists","up"]},{"osd":1,"uuid":"da187f97-762d-40d3-94d0-fef449ce1dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6811","nonce":2457092004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6813","nonce":2457092004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6817","nonce":2457092004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2457092004},{"type":"v1","addr":"172.21.15.63:6815","nonce":2457092004}]},"public_addr":"172.21.15.63:6811/2457092004","cluster_addr":"172.21.15.63:6813/2457092004","heartbeat_back_addr":"172.21.15.63:6817/2457092004","heartbeat_front_addr":"172.21.15.63:6815/2457092004","state":["exists","up"]},{"osd":2,"uuid":"6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6819","nonce":3467870918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6821","nonce":3467870918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6825","nonce":3467870918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3467870918},{"type":"v1","addr":"172.21.15.63:6823","nonce":3467870918}]},"public_addr":"172.21.15.63:6819/3467870918","cluster_addr":"172.21.15.63:6821/3467870918","heartbeat_back_addr":"172.21.15.63:6825/3467870918","heartbeat_front_addr":"172.21.15.63:6823/3467870918","state":["exists","up"]},{"osd":3,"uuid":"ff1a9d2a-4481-4e61-aaa5-e5046f3c0055","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6801","nonce":974699773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6803","nonce":974699773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6807","nonce":974699773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":974699773},{"type":"v1","addr":"172.21.15.193:6805","nonce":974699773}]},"public_addr":"172.21.15.193:6801/974699773","cluster_addr":"172.21.15.193:6803/974699773","heartbeat_back_addr":"172.21.15.193:6807/974699773","heartbeat_front_addr":"172.21.15.193:6805/974699773","state":["exists","up"]},{"osd":4,"uuid":"23f0fabc-4f38-438c-a691-307209bb0f31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6809","nonce":1396318320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6811","nonce":1396318320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6815","nonce":1396318320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1396318320},{"type":"v1","addr":"172.21.15.193:6813","nonce":1396318320}]},"public_addr":"172.21.15.193:6809/1396318320","cluster_addr":"172.21.15.193:6811/1396318320","heartbeat_back_addr":"172.21.15.193:6815/1396318320","heartbeat_front_addr":"172.21.15.193:6813/1396318320","state":["exists","up"]},{"osd":5,"uuid":"eb6226de-63c7-4200-8978-4ee9491c1427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6817","nonce":449909828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6819","nonce":449909828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6823","nonce":449909828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":449909828},{"type":"v1","addr":"172.21.15.193:6821","nonce":449909828}]},"public_addr":"172.21.15.193:6817/449909828","cluster_addr":"172.21.15.193:6819/449909828","heartbeat_back_addr":"172.21.15.193:6823/449909828","heartbeat_front_addr":"172.21.15.193:6821/449909828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:32.202943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:59.919211+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:28.132162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:52.038182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:17.334991+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:43.406303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6827/1318147125":"2024-05-10T06:35:21.124789+0000","172.21.15.63:0/3728039632":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/494791920":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/4061231873":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/1971919126":"2024-05-10T06:28:29.800161+0000","172.21.15.63:6801/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6801/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/750127211":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/576235286":"2024-05-10T06:28:29.800161+0000","172.21.15.63:6800/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6801/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/3349953626":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/1507850411":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/1465547870":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6800/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6826/1318147125":"2024-05-10T06:35:21.124789+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:35:29.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.712+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f95a406cbf0 msgr2=0x7f95a406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.712+0000 7f95bf646640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f95a406cbf0 0x7f95a406f0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009bb0 tx=0x7f95a8005f90 comp rx=0 tx=0).stop 2024-05-09T06:35:29.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.712+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 msgr2=0x7f95b8195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.712+0000 7f95bf646640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 0x7f95b8195c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95b402c960 tx=0x7f95b402c750 comp rx=0 tx=0).stop 2024-05-09T06:35:29.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.712+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 shutdown_connections 2024-05-09T06:35:29.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.712+0000 7f95bf646640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f95a406cbf0 0x7f95a406f0b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.713+0000 7f95bf646640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95b8100200 0x7f95b81961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.713+0000 7f95bf646640 1 --2- 172.21.15.63:0/2007313599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95b80ff010 0x7f95b8195c80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.713+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 >> 172.21.15.63:0/2007313599 conn(0x7f95b80fa5a0 msgr2=0x7f95b80673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.713+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 shutdown_connections 2024-05-09T06:35:29.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.713+0000 7f95bf646640 1 -- 172.21.15.63:0/2007313599 wait complete. 2024-05-09T06:35:29.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:32:32.042216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-09T06:35:02.184376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-09T06:35:02.871327+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:35:29.705 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:35:29.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.831+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/946867466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb00ff030 msgr2=0x7f7eb00ff450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.831+0000 7f7eb4b5f640 1 --2- 172.21.15.63:0/946867466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb00ff030 0x7f7eb00ff450 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009880 tx=0x7f7ea002bce0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.831+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/946867466 shutdown_connections 2024-05-09T06:35:29.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.831+0000 7f7eb4b5f640 1 --2- 172.21.15.63:0/946867466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7eb0100220 0x7f7eb01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.831+0000 7f7eb4b5f640 1 --2- 172.21.15.63:0/946867466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb00ff030 0x7f7eb00ff450 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.831+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/946867466 >> 172.21.15.63:0/946867466 conn(0x7f7eb00fa5a0 msgr2=0x7f7eb00fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:29.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.831+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/946867466 shutdown_connections 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.831+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/946867466 wait complete. 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7eb4b5f640 1 Processor -- start 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7eb4b5f640 1 -- start start 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7eb4b5f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7eb00ff030 0x7f7eb0069010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7eb4b5f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb0100220 0x7f7eb019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7eb4b5f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019af30 con 0x7f7eb0100220 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7eb4b5f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019b070 con 0x7f7eb00ff030 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7ea7fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb0100220 0x7f7eb019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7ea7fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb0100220 0x7f7eb019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38098/0 (socket says 172.21.15.63:38098) 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7ea7fff640 1 -- 172.21.15.63:0/2553986710 learned_addr learned my addr 172.21.15.63:0/2553986710 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.832+0000 7f7eae59c640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7eb00ff030 0x7f7eb0069010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.833+0000 7f7ea7fff640 1 -- 172.21.15.63:0/2553986710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7eb00ff030 msgr2=0x7f7eb0069010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.833+0000 7f7ea7fff640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7eb00ff030 0x7f7eb0069010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.833+0000 7f7ea7fff640 1 -- 172.21.15.63:0/2553986710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0009560 con 0x7f7eb0100220 2024-05-09T06:35:29.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.833+0000 7f7ea7fff640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb0100220 0x7f7eb019a9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d900 tx=0x7f7e9800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.833+0000 7f7ea77fe640 1 -- 172.21.15.63:0/2553986710 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e98009c00 con 0x7f7eb0100220 2024-05-09T06:35:29.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.833+0000 7f7ea77fe640 1 -- 172.21.15.63:0/2553986710 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e9800b660 con 0x7f7eb0100220 2024-05-09T06:35:29.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.833+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb019b320 con 0x7f7eb0100220 2024-05-09T06:35:29.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.833+0000 7f7ea77fe640 1 -- 172.21.15.63:0/2553986710 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e9800f680 con 0x7f7eb0100220 2024-05-09T06:35:29.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.833+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb019b900 con 0x7f7eb0100220 2024-05-09T06:35:29.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.834+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb018eba0 con 0x7f7eb0100220 2024-05-09T06:35:29.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.835+0000 7f7ea77fe640 1 -- 172.21.15.63:0/2553986710 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7e98010ab0 con 0x7f7eb0100220 2024-05-09T06:35:29.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.835+0000 7f7ea77fe640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7e9c06cc60 0x7f7e9c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:29.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.835+0000 7f7ea77fe640 1 -- 172.21.15.63:0/2553986710 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7e9808c080 con 0x7f7eb0100220 2024-05-09T06:35:29.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.835+0000 7f7eae59c640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7e9c06cc60 0x7f7e9c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:29.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.836+0000 7f7eae59c640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7e9c06cc60 0x7f7e9c06f120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006010 tx=0x7f7ea002c920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:29.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:29.840+0000 7f7ea77fe640 1 -- 172.21.15.63:0/2553986710 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7e980563f0 con 0x7f7eb0100220 2024-05-09T06:35:30.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.022+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7eb0107e70 con 0x7f7eb0100220 2024-05-09T06:35:30.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.022+0000 7f7ea77fe640 1 -- 172.21.15.63:0/2553986710 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e98059a10 con 0x7f7eb0100220 2024-05-09T06:35:30.003 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-09T06:35:30.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.024+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7e9c06cc60 msgr2=0x7f7e9c06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.024+0000 7f7eb4b5f640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7e9c06cc60 0x7f7e9c06f120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006010 tx=0x7f7ea002c920 comp rx=0 tx=0).stop 2024-05-09T06:35:30.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.024+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb0100220 msgr2=0x7f7eb019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.024+0000 7f7eb4b5f640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb0100220 0x7f7eb019a9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d900 tx=0x7f7e9800ddc0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.025+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 shutdown_connections 2024-05-09T06:35:30.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.025+0000 7f7eb4b5f640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f7e9c06cc60 0x7f7e9c06f120 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.025+0000 7f7eb4b5f640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eb0100220 0x7f7eb019a9f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.025+0000 7f7eb4b5f640 1 --2- 172.21.15.63:0/2553986710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7eb00ff030 0x7f7eb0069010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.025+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 >> 172.21.15.63:0/2553986710 conn(0x7f7eb00fa5a0 msgr2=0x7f7eb0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.025+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 shutdown_connections 2024-05-09T06:35:30.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.025+0000 7f7eb4b5f640 1 -- 172.21.15.63:0/2553986710 wait complete. 2024-05-09T06:35:30.018 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-09T06:35:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.150+0000 7ff93480d640 1 -- 172.21.15.63:0/1677253432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 msgr2=0x7ff930102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.150+0000 7ff93480d640 1 --2- 172.21.15.63:0/1677253432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 0x7ff930102f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff9200098b0 tx=0x7ff92002bcd0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.150+0000 7ff93480d640 1 -- 172.21.15.63:0/1677253432 shutdown_connections 2024-05-09T06:35:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.150+0000 7ff93480d640 1 --2- 172.21.15.63:0/1677253432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9301034b0 0x7ff9301058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.150+0000 7ff93480d640 1 --2- 172.21.15.63:0/1677253432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 0x7ff930102f70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.150+0000 7ff93480d640 1 -- 172.21.15.63:0/1677253432 >> 172.21.15.63:0/1677253432 conn(0x7ff9300fa570 msgr2=0x7ff9300fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.150+0000 7ff93480d640 1 -- 172.21.15.63:0/1677253432 shutdown_connections 2024-05-09T06:35:30.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.150+0000 7ff93480d640 1 -- 172.21.15.63:0/1677253432 wait complete. 2024-05-09T06:35:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.151+0000 7ff93480d640 1 Processor -- start 2024-05-09T06:35:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.151+0000 7ff93480d640 1 -- start start 2024-05-09T06:35:30.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.151+0000 7ff93480d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 0x7ff930195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.151+0000 7ff93480d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9301034b0 0x7ff9301961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.151+0000 7ff93480d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff930196790 con 0x7ff930100b80 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.151+0000 7ff93480d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff930196900 con 0x7ff9301034b0 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.151+0000 7ff92e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 0x7ff930195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.151+0000 7ff92e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 0x7ff930195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38122/0 (socket says 172.21.15.63:38122) 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.151+0000 7ff92e59c640 1 -- 172.21.15.63:0/2035767903 learned_addr learned my addr 172.21.15.63:0/2035767903 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff92dd9b640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9301034b0 0x7ff9301961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff92e59c640 1 -- 172.21.15.63:0/2035767903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9301034b0 msgr2=0x7ff9301961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff92e59c640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9301034b0 0x7ff9301961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff92e59c640 1 -- 172.21.15.63:0/2035767903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920009560 con 0x7ff930100b80 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff92e59c640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 0x7ff930195c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff920009880 tx=0x7ff920004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff91f7fe640 1 -- 172.21.15.63:0/2035767903 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9200035f0 con 0x7ff930100b80 2024-05-09T06:35:30.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff91f7fe640 1 -- 172.21.15.63:0/2035767903 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff920003c10 con 0x7ff930100b80 2024-05-09T06:35:30.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93019b2d0 con 0x7ff930100b80 2024-05-09T06:35:30.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff91f7fe640 1 -- 172.21.15.63:0/2035767903 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff92003e450 con 0x7ff930100b80 2024-05-09T06:35:30.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.152+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93019b8b0 con 0x7ff930100b80 2024-05-09T06:35:30.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.153+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff930108d00 con 0x7ff930100b80 2024-05-09T06:35:30.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.154+0000 7ff91f7fe640 1 -- 172.21.15.63:0/2035767903 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff920004ff0 con 0x7ff930100b80 2024-05-09T06:35:30.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.155+0000 7ff91f7fe640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff91806cc50 0x7ff91806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.155+0000 7ff91f7fe640 1 -- 172.21.15.63:0/2035767903 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff9200aee80 con 0x7ff930100b80 2024-05-09T06:35:30.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.155+0000 7ff92dd9b640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff91806cc50 0x7ff91806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.155+0000 7ff92dd9b640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff91806cc50 0x7ff91806f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff930067f00 tx=0x7ff92400b390 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.160+0000 7ff91f7fe640 1 -- 172.21.15.63:0/2035767903 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff9200791f0 con 0x7ff930100b80 2024-05-09T06:35:30.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.347+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff93019b460 con 0x7ff930100b80 2024-05-09T06:35:30.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.347+0000 7ff91f7fe640 1 -- 172.21.15.63:0/2035767903 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff93019b460 con 0x7ff930100b80 2024-05-09T06:35:30.328 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff91806cc50 msgr2=0x7ff91806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff91806cc50 0x7ff91806f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff930067f00 tx=0x7ff92400b390 comp rx=0 tx=0).stop 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 msgr2=0x7ff930195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 0x7ff930195c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff920009880 tx=0x7ff920004040 comp rx=0 tx=0).stop 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 shutdown_connections 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff91806cc50 0x7ff91806f110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9301034b0 0x7ff9301961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 --2- 172.21.15.63:0/2035767903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff930100b80 0x7ff930195c80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 >> 172.21.15.63:0/2035767903 conn(0x7ff9300fa570 msgr2=0x7ff9300fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 shutdown_connections 2024-05-09T06:35:30.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.349+0000 7ff93480d640 1 -- 172.21.15.63:0/2035767903 wait complete. 2024-05-09T06:35:30.340 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:30 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2007313599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:35:30.340 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:30 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2553986710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:35:30.341 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-09T06:35:30.436 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:30 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2007313599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:35:30.436 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:30 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2553986710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:35:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.479+0000 7f795f1ca640 1 -- 172.21.15.63:0/370722621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79580ff000 msgr2=0x7f79580ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.479+0000 7f795f1ca640 1 --2- 172.21.15.63:0/370722621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79580ff000 0x7f79580ff420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f794c009880 tx=0x7f794c02bce0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.479+0000 7f795f1ca640 1 -- 172.21.15.63:0/370722621 shutdown_connections 2024-05-09T06:35:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.479+0000 7f795f1ca640 1 --2- 172.21.15.63:0/370722621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79581001f0 0x7f7958100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.479+0000 7f795f1ca640 1 --2- 172.21.15.63:0/370722621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79580ff000 0x7f79580ff420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.479+0000 7f795f1ca640 1 -- 172.21.15.63:0/370722621 >> 172.21.15.63:0/370722621 conn(0x7f79580fa570 msgr2=0x7f79580fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.479+0000 7f795f1ca640 1 -- 172.21.15.63:0/370722621 shutdown_connections 2024-05-09T06:35:30.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.479+0000 7f795f1ca640 1 -- 172.21.15.63:0/370722621 wait complete. 2024-05-09T06:35:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f795f1ca640 1 Processor -- start 2024-05-09T06:35:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f795f1ca640 1 -- start start 2024-05-09T06:35:30.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f795f1ca640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79580ff000 0x7f7958102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f795f1ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79581001f0 0x7f79581015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f795f1ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79581034c0 con 0x7f79581001f0 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f795f1ca640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7958101b40 con 0x7f79580ff000 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f7957fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79581001f0 0x7f79581015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f7957fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79581001f0 0x7f79581015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38140/0 (socket says 172.21.15.63:38140) 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f7957fff640 1 -- 172.21.15.63:0/1225857024 learned_addr learned my addr 172.21.15.63:0/1225857024 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.480+0000 7f795cf66640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79580ff000 0x7f7958102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.481+0000 7f7957fff640 1 -- 172.21.15.63:0/1225857024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79580ff000 msgr2=0x7f7958102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.481+0000 7f7957fff640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79580ff000 0x7f7958102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.481+0000 7f7957fff640 1 -- 172.21.15.63:0/1225857024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c009560 con 0x7f79581001f0 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.481+0000 7f7957fff640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79581001f0 0x7f79581015d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f794400e970 tx=0x7f794400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.481+0000 7f7955ffb640 1 -- 172.21.15.63:0/1225857024 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f794400cc90 con 0x7f79581001f0 2024-05-09T06:35:30.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.481+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7958101e20 con 0x7f79581001f0 2024-05-09T06:35:30.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.481+0000 7f7955ffb640 1 -- 172.21.15.63:0/1225857024 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f794400cdf0 con 0x7f79581001f0 2024-05-09T06:35:30.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.481+0000 7f7955ffb640 1 -- 172.21.15.63:0/1225857024 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7944010430 con 0x7f79581001f0 2024-05-09T06:35:30.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.481+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7958102400 con 0x7f79581001f0 2024-05-09T06:35:30.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.482+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7958107c00 con 0x7f79581001f0 2024-05-09T06:35:30.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.483+0000 7f7955ffb640 1 -- 172.21.15.63:0/1225857024 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f79440046c0 con 0x7f79581001f0 2024-05-09T06:35:30.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.483+0000 7f7955ffb640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f794806cc50 0x7f794806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.483+0000 7f7955ffb640 1 -- 172.21.15.63:0/1225857024 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7944014070 con 0x7f79581001f0 2024-05-09T06:35:30.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.483+0000 7f795cf66640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f794806cc50 0x7f794806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.484+0000 7f795cf66640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f794806cc50 0x7f794806f110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f794c02c960 tx=0x7f794c005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.488+0000 7f7955ffb640 1 -- 172.21.15.63:0/1225857024 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7944056b90 con 0x7f79581001f0 2024-05-09T06:35:30.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.689+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f7958107d90 con 0x7f79581001f0 2024-05-09T06:35:30.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.690+0000 7f7955ffb640 1 -- 172.21.15.63:0/1225857024 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f794405a1b0 con 0x7f79581001f0 2024-05-09T06:35:30.671 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f794806cc50 msgr2=0x7f794806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f794806cc50 0x7f794806f110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f794c02c960 tx=0x7f794c005fd0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79581001f0 msgr2=0x7f79581015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79581001f0 0x7f79581015d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f794400e970 tx=0x7f794400ed70 comp rx=0 tx=0).stop 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 shutdown_connections 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f794806cc50 0x7f794806f110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79581001f0 0x7f79581015d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 --2- 172.21.15.63:0/1225857024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79580ff000 0x7f7958102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 >> 172.21.15.63:0/1225857024 conn(0x7f79580fa570 msgr2=0x7f7958067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 shutdown_connections 2024-05-09T06:35:30.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.692+0000 7f795f1ca640 1 -- 172.21.15.63:0/1225857024 wait complete. 2024-05-09T06:35:30.684 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:35:30.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.807+0000 7fecf5b31640 1 -- 172.21.15.63:0/2722736880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf01034e0 msgr2=0x7fecf01058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.807+0000 7fecf5b31640 1 --2- 172.21.15.63:0/2722736880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf01034e0 0x7fecf01058d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fece40098b0 tx=0x7fece402bcd0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.807+0000 7fecf5b31640 1 -- 172.21.15.63:0/2722736880 shutdown_connections 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.807+0000 7fecf5b31640 1 --2- 172.21.15.63:0/2722736880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf01034e0 0x7fecf01058d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.807+0000 7fecf5b31640 1 --2- 172.21.15.63:0/2722736880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecf0100bb0 0x7fecf0102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.807+0000 7fecf5b31640 1 -- 172.21.15.63:0/2722736880 >> 172.21.15.63:0/2722736880 conn(0x7fecf00fa5a0 msgr2=0x7fecf00fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.807+0000 7fecf5b31640 1 -- 172.21.15.63:0/2722736880 shutdown_connections 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.807+0000 7fecf5b31640 1 -- 172.21.15.63:0/2722736880 wait complete. 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.808+0000 7fecf5b31640 1 Processor -- start 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.808+0000 7fecf5b31640 1 -- start start 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.808+0000 7fecf5b31640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf0100bb0 0x7fecf01004d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.808+0000 7fecf5b31640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecf01034e0 0x7fecf00feb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.808+0000 7fecf5b31640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf0199900 con 0x7fecf0100bb0 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.808+0000 7fecf5b31640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf0199a70 con 0x7fecf01034e0 2024-05-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.808+0000 7fecef7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf0100bb0 0x7fecf01004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.808+0000 7fecef7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf0100bb0 0x7fecf01004d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38172/0 (socket says 172.21.15.63:38172) 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.808+0000 7fecef7fe640 1 -- 172.21.15.63:0/923754115 learned_addr learned my addr 172.21.15.63:0/923754115 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7feceeffd640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecf01034e0 0x7fecf00feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7fecef7fe640 1 -- 172.21.15.63:0/923754115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecf01034e0 msgr2=0x7fecf00feb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7fecef7fe640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecf01034e0 0x7fecf00feb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7fecef7fe640 1 -- 172.21.15.63:0/923754115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece4009560 con 0x7fecf0100bb0 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7fecef7fe640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf0100bb0 0x7fecf01004d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fece000d8d0 tx=0x7fece000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7fececff9640 1 -- 172.21.15.63:0/923754115 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fece0009c00 con 0x7fecf0100bb0 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7fececff9640 1 -- 172.21.15.63:0/923754115 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fece0009d60 con 0x7fecf0100bb0 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7fececff9640 1 -- 172.21.15.63:0/923754115 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fece00049d0 con 0x7fecf0100bb0 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf00ff0a0 con 0x7fecf0100bb0 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.809+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf00ff5d0 con 0x7fecf0100bb0 2024-05-09T06:35:30.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.810+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf01096e0 con 0x7fecf0100bb0 2024-05-09T06:35:30.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.811+0000 7fececff9640 1 -- 172.21.15.63:0/923754115 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fece0004c40 con 0x7fecf0100bb0 2024-05-09T06:35:30.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.811+0000 7fececff9640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fecd806cc50 0x7fecd806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:30.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.811+0000 7fececff9640 1 -- 172.21.15.63:0/923754115 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fece008bf80 con 0x7fecf0100bb0 2024-05-09T06:35:30.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.811+0000 7feceeffd640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fecd806cc50 0x7fecd806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:30.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.812+0000 7feceeffd640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fecd806cc50 0x7fecd806f110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fece4000c00 tx=0x7fece4005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:30.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:30.816+0000 7fececff9640 1 -- 172.21.15.63:0/923754115 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fecf01096e0 con 0x7fecf0100bb0 2024-05-09T06:35:31.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.048+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecf0108dc0 con 0x7fecf0100bb0 2024-05-09T06:35:31.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.049+0000 7fececff9640 1 -- 172.21.15.63:0/923754115 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4833 (secure 0 0 0) 0x7fece0056370 con 0x7fecf0100bb0 2024-05-09T06:35:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:35:31.030 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14504,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1277475441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1277475441},{"type":"v1","addr":"172.21.15.63:6827","nonce":1277475441}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:35:24.143344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14492":{"gid":14492,"name":"cephfs.smithi063.fbzxbf","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.63:6829/1815819553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1815819553},{"type":"v1","addr":"172.21.15.63:6829","nonce":1815819553}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi193.ggtmyb","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6825/296432349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":296432349},{"type":"v1","addr":"172.21.15.193:6825","nonce":296432349}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-09T06:35:31.030 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:35:31.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fecd806cc50 msgr2=0x7fecd806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fecd806cc50 0x7fecd806f110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fece4000c00 tx=0x7fece4005fd0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf0100bb0 msgr2=0x7fecf01004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:31.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf0100bb0 0x7fecf01004d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fece000d8d0 tx=0x7fece000dd90 comp rx=0 tx=0).stop 2024-05-09T06:35:31.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 shutdown_connections 2024-05-09T06:35:31.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fecd806cc50 0x7fecd806f110 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecf01034e0 0x7fecf00feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 --2- 172.21.15.63:0/923754115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fecf0100bb0 0x7fecf01004d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:31.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 >> 172.21.15.63:0/923754115 conn(0x7fecf00fa5a0 msgr2=0x7fecf00fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:31.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 shutdown_connections 2024-05-09T06:35:31.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:31.051+0000 7fecf5b31640 1 -- 172.21.15.63:0/923754115 wait complete. 2024-05-09T06:35:31.043 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-05-09T06:35:31.043 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-09T06:35:31.073 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-09T06:35:31.073 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-05-09T06:35:31.073 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-09T06:35:31.073 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:35:31.073 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:31.074 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-09T06:35:31.074 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:35:31.074 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:31.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:31.074 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-09T06:35:31.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:31.091 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-09T06:35:31.159 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-05-09T06:35:31.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:35:31.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:31.161 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-05-09T06:35:31.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:31.177 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link delete ceph-brx 2024-05-09T06:35:31.246 INFO:teuthology.orchestra.run.smithi193.stderr:Cannot find device "ceph-brx" 2024-05-09T06:35:31.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:35:31.248 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-09T06:35:31.248 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T06:35:31.270 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:31 smithi063 ceph-mon[27171]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 9.1 KiB/s rd, 9 op/s 2024-05-09T06:35:31.270 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:31 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2035767903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-09T06:35:31.270 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:31 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1225857024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-09T06:35:31.270 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:31 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:31.270 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:31 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:31.270 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:31 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/923754115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T06:35:31.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:31 smithi193 ceph-mon[35141]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 9.1 KiB/s rd, 9 op/s 2024-05-09T06:35:31.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:31 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2035767903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-09T06:35:31.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:31 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1225857024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-09T06:35:31.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:31 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:31.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:31 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:31.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:31 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/923754115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T06:35:32.114 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:33.541 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:33 smithi063 ceph-mon[27171]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 9 op/s 2024-05-09T06:35:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:33 smithi193 ceph-mon[35141]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 9 op/s 2024-05-09T06:35:34.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.255+0000 7fd752b57700 1 -- 172.21.15.63:0/1921425564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 msgr2=0x7fd74c10abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:34.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.255+0000 7fd752b57700 1 --2- 172.21.15.63:0/1921425564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 0x7fd74c10abd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd738009910 tx=0x7fd738009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:34.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.255+0000 7fd752b57700 1 -- 172.21.15.63:0/1921425564 shutdown_connections 2024-05-09T06:35:34.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.255+0000 7fd752b57700 1 --2- 172.21.15.63:0/1921425564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 0x7fd74c10abd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:34.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.255+0000 7fd752b57700 1 --2- 172.21.15.63:0/1921425564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd74c101dc0 0x7fd74c1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:34.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.255+0000 7fd752b57700 1 -- 172.21.15.63:0/1921425564 >> 172.21.15.63:0/1921425564 conn(0x7fd74c0fb650 msgr2=0x7fd74c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:34.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.255+0000 7fd752b57700 1 -- 172.21.15.63:0/1921425564 shutdown_connections 2024-05-09T06:35:34.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.255+0000 7fd752b57700 1 -- 172.21.15.63:0/1921425564 wait complete. 2024-05-09T06:35:34.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.255+0000 7fd752b57700 1 Processor -- start 2024-05-09T06:35:34.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd752b57700 1 -- start start 2024-05-09T06:35:34.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd752b57700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd74c101dc0 0x7fd74c0ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:34.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd752b57700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 0x7fd74c0fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:34.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd752b57700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c1004a0 con 0x7fd74c1026e0 2024-05-09T06:35:34.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd752b57700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c1005e0 con 0x7fd74c101dc0 2024-05-09T06:35:34.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd74bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 0x7fd74c0fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:34.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd74bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 0x7fd74c0fff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38198/0 (socket says 172.21.15.63:38198) 2024-05-09T06:35:34.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd74bfff700 1 -- 172.21.15.63:0/1774597821 learned_addr learned my addr 172.21.15.63:0/1774597821 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:34.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd7508f3700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd74c101dc0 0x7fd74c0ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:34.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd74bfff700 1 -- 172.21.15.63:0/1774597821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd74c101dc0 msgr2=0x7fd74c0ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:34.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd74bfff700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd74c101dc0 0x7fd74c0ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:34.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd74bfff700 1 -- 172.21.15.63:0/1774597821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380095f0 con 0x7fd74c1026e0 2024-05-09T06:35:34.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd74bfff700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 0x7fd74c0fff60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd7380092e0 tx=0x7fd73800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:34.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.256+0000 7fd749ffb700 1 -- 172.21.15.63:0/1774597821 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd73801d070 con 0x7fd74c1026e0 2024-05-09T06:35:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.257+0000 7fd749ffb700 1 -- 172.21.15.63:0/1774597821 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd738003d50 con 0x7fd74c1026e0 2024-05-09T06:35:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.257+0000 7fd749ffb700 1 -- 172.21.15.63:0/1774597821 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd738017400 con 0x7fd74c1026e0 2024-05-09T06:35:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.257+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74c100860 con 0x7fd74c1026e0 2024-05-09T06:35:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.257+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74c1a30a0 con 0x7fd74c1026e0 2024-05-09T06:35:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.258+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74c069a00 con 0x7fd74c1026e0 2024-05-09T06:35:34.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.259+0000 7fd749ffb700 1 -- 172.21.15.63:0/1774597821 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd73800f6f0 con 0x7fd74c1026e0 2024-05-09T06:35:34.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.260+0000 7fd749ffb700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd73c06ccf0 0x7fd73c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:34.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.260+0000 7fd749ffb700 1 -- 172.21.15.63:0/1774597821 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd73808d5a0 con 0x7fd74c1026e0 2024-05-09T06:35:34.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.260+0000 7fd7508f3700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd73c06ccf0 0x7fd73c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:34.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.260+0000 7fd7508f3700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd73c06ccf0 0x7fd73c06f1b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd740009a10 tx=0x7fd740009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:34.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.264+0000 7fd749ffb700 1 -- 172.21.15.63:0/1774597821 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd738057990 con 0x7fd74c1026e0 2024-05-09T06:35:34.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.528+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd74c04ebb0 con 0x7fd74c1026e0 2024-05-09T06:35:34.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.529+0000 7fd749ffb700 1 -- 172.21.15.63:0/1774597821 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7fd73805afb0 con 0x7fd74c1026e0 2024-05-09T06:35:34.510 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-09T06:35:34.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd73c06ccf0 msgr2=0x7fd73c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:34.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd73c06ccf0 0x7fd73c06f1b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd740009a10 tx=0x7fd740009190 comp rx=0 tx=0).stop 2024-05-09T06:35:34.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 msgr2=0x7fd74c0fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:34.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 0x7fd74c0fff60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd7380092e0 tx=0x7fd73800bd70 comp rx=0 tx=0).stop 2024-05-09T06:35:34.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 shutdown_connections 2024-05-09T06:35:34.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd73c06ccf0 0x7fd73c06f1b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:34.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd74c101dc0 0x7fd74c0ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:34.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 --2- 172.21.15.63:0/1774597821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd74c1026e0 0x7fd74c0fff60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:34.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 >> 172.21.15.63:0/1774597821 conn(0x7fd74c0fb650 msgr2=0x7fd74c1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:34.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 shutdown_connections 2024-05-09T06:35:34.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:34.531+0000 7fd752b57700 1 -- 172.21.15.63:0/1774597821 wait complete. 2024-05-09T06:35:35.251 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:35 smithi063 ceph-mon[27171]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T06:35:35.252 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:35 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1774597821' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T06:35:35.278 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:35:35.278 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:35:35.278 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-09T06:35:35.278 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:35:35.278 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:35.278 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:35:35.278 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:35:35.278 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:35:35.279 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:35:35.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:35.279 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:35:35.338 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:35.339 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-09T06:35:35.339 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:35:35.339 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:35.339 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:35:35.339 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:35.339 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T06:35:35.343 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 42266sec preferred_lft 42266sec 2024-05-09T06:35:35.343 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-09T06:35:35.343 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:35.343 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T06:35:35.344 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:35.344 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-09T06:35:35.344 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-05-09T06:35:35.344 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-05-09T06:35:35.344 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-05-09T06:35:35.344 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T06:35:35.344 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-09T06:35:35.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:35.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:35.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:35.600 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:35:35.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:35 smithi193 ceph-mon[35141]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T06:35:35.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:35 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1774597821' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T06:35:35.628 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-09T06:35:35.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:35.630 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-09T06:35:35.707 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-09T06:35:35.707 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:35:35.707 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T06:35:35.707 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T06:35:35.707 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:35:35.708 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:35.708 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-09T06:35:35.708 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T06:35:35.708 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T06:35:35.708 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T06:35:35.708 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-09T06:35:35.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:36.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:36.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:36.015 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-09T06:35:36.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:36.032 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-09T06:35:36.088 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:36.088 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-09T06:35:36.088 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:36.088 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T06:35:36.089 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-09T06:35:36.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:36.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:36.211 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T06:35:36.211 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:36.211 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-09T06:35:36.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T06:35:36.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T06:35:36.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T06:35:36.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T06:35:36.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T06:35:36.211 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-09T06:35:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:36.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:36.344 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:36.344 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-09T06:35:36.345 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-05-09T06:35:36.345 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T06:35:36.345 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-09T06:35:36.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:36.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:36.424 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T06:35:36.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:35:36.424 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:36.440 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:35:36.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:35:36.440 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:36.495 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-09T06:35:36.562 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-09T06:35:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:35:36.621 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/c12176f2745af3d7697ed947d785fcadd540f6f38adafd9f9c3e26d5b780dad0/merged 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/70720f3d37f476ae214fc24aa28c87c1287615a5f6d75f018e0ab92b6c4f8b95/merged 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/98a2d345cfba912d3b169c9df3e616eeb3aa29d6df07dc542b7136ab36110c87/merged 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/b9852e770b7e82b971074bb16b1fb53068bb0d8e112dd62dbf6f06ab393c6b63/merged 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/81beb2ad953f350579e1a1c20705750a139bb39c16b7d420a8c5308c6a2e741b/merged 2024-05-09T06:35:36.622 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/eb9ada62f3ae979c4e0d57cbdbd5aa9f8147505aa4f01f574af4230cab505467/merged 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2a08e230695728db4c8ad9f9c73315164a984bda46ca80c8d9be7b49b3460c1d/merged 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/f97d71aceefeef2b006c29b40c9b74a2e3710adf2c464603dd999a4c90fb5585/merged 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/6ade3bdfde844048711e484d3fb4f97d9c1d28ef68d5fe73a2fba1631a7d4965/merged 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/5f74a7cf7bca7be2592be46eaa2feb2f05418d900f541a9c5c084e3a9f6903d3/merged 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1009406c37f1ef2503db90ef7bfbed7cb50a39562a66057d25b08c4987d76eea/merged 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/7c565e03048df2d90b7fa596966cffa2585d1c057cf016b4510104f58f14b4dd/merged 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/587aa8c7e995a9771cd2e2dc665a2e3482aca20d2fa30b6ec8125cd930734a29/merged 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:36.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:36.623 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-09T06:35:36.678 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:35:36.678 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T06:35:36.721 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-09T06:35:36.747 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:35:36.805 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-09T06:35:36.805 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-09T06:35:36.805 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-09T06:35:36.806 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-09T06:35:36.806 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-09T06:35:36.806 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-09T06:35:36.806 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-09T06:35:36.806 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-09T06:35:36.806 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-09T06:35:36.807 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-09T06:35:36.807 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-09T06:35:36.807 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-09T06:35:36.807 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:36.807 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-09T06:35:36.807 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-09T06:35:36.808 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-09T06:35:36.808 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-09T06:35:36.808 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-09T06:35:36.808 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:35:36.808 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:35:36.808 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-09T06:35:36.809 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:35:36.809 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:35:36.809 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-09T06:35:36.809 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-09T06:35:36.809 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-09T06:35:36.809 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/c12176f2745af3d7697ed947d785fcadd540f6f38adafd9f9c3e26d5b780dad0/merged 2024-05-09T06:35:36.810 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/70720f3d37f476ae214fc24aa28c87c1287615a5f6d75f018e0ab92b6c4f8b95/merged 2024-05-09T06:35:36.810 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-09T06:35:36.810 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:36.810 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/98a2d345cfba912d3b169c9df3e616eeb3aa29d6df07dc542b7136ab36110c87/merged 2024-05-09T06:35:36.810 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/b9852e770b7e82b971074bb16b1fb53068bb0d8e112dd62dbf6f06ab393c6b63/merged 2024-05-09T06:35:36.810 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/81beb2ad953f350579e1a1c20705750a139bb39c16b7d420a8c5308c6a2e741b/merged 2024-05-09T06:35:36.811 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/eb9ada62f3ae979c4e0d57cbdbd5aa9f8147505aa4f01f574af4230cab505467/merged 2024-05-09T06:35:36.811 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2a08e230695728db4c8ad9f9c73315164a984bda46ca80c8d9be7b49b3460c1d/merged 2024-05-09T06:35:36.811 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/f97d71aceefeef2b006c29b40c9b74a2e3710adf2c464603dd999a4c90fb5585/merged 2024-05-09T06:35:36.811 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/6ade3bdfde844048711e484d3fb4f97d9c1d28ef68d5fe73a2fba1631a7d4965/merged 2024-05-09T06:35:36.811 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/5f74a7cf7bca7be2592be46eaa2feb2f05418d900f541a9c5c084e3a9f6903d3/merged 2024-05-09T06:35:36.811 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1009406c37f1ef2503db90ef7bfbed7cb50a39562a66057d25b08c4987d76eea/merged 2024-05-09T06:35:36.812 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/7c565e03048df2d90b7fa596966cffa2585d1c057cf016b4510104f58f14b4dd/merged 2024-05-09T06:35:36.812 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/587aa8c7e995a9771cd2e2dc665a2e3482aca20d2fa30b6ec8125cd930734a29/merged 2024-05-09T06:35:36.812 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-09T06:35:36.812 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:36.812 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:35:36.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:36.813 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-09T06:35:36.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:2024-05-09T06:35:36.833+0000 7f062f83e480 -1 init, newargv = 0x55db98204300 newargc=15 2024-05-09T06:35:36.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[75595]: starting ceph client 2024-05-09T06:35:36.820 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[75595]: starting fuse 2024-05-09T06:35:36.831 INFO:teuthology.orchestra.run.smithi063.stdout:126 2024-05-09T06:35:36.831 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [126] 2024-05-09T06:35:36.831 DEBUG:teuthology.orchestra.run.smithi063:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:35:36.831 DEBUG:teuthology.orchestra.run.smithi063:> import glob 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> import re 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> import os 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> import subprocess 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> def _find_admin_socket(client_name): 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> files = glob.glob(asok_path) 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> # Given a non-glob path, it better be there 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> if "*" not in asok_path: 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> assert(len(files) == 1) 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> return files[0] 2024-05-09T06:35:36.832 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> for f in files: 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> contents = proc_f.read() 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> if mountpoint in contents: 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> return f 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> print(_find_admin_socket("client.0")) 2024-05-09T06:35:36.833 DEBUG:teuthology.orchestra.run.smithi063:> ' 2024-05-09T06:35:36.954 INFO:teuthology.orchestra.run.smithi063.stdout:/var/run/ceph/ceph-client.0.75595.asok 2024-05-09T06:35:36.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:36.965 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.75595.asok 2024-05-09T06:35:36.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:36.965 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.75595.asok status 2024-05-09T06:35:37.104 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "metadata": { 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_sha1": "bf64272e3d7edf5c7f8706082e690401a7a11750", 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 18.2.2-1306-gbf64272e (bf64272e3d7edf5c7f8706082e690401a7a11750) reef (stable)", 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "entity_id": "0", 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": "smithi063", 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "pid": "75595", 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "root": "/" 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_count": 0, 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_pinned_count": 0, 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 14540, 2024-05-09T06:35:37.105 INFO:teuthology.orchestra.run.smithi063.stdout: "inst": { 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: "name": { 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "client", 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: "num": 14540 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2082070818 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-09T06:35:37.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2082070818 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "inst_str": "client.14540 192.168.0.1:0/2082070818", 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "addr_str": "192.168.0.1:0/2082070818", 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "inode_count": 1, 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_epoch": 15, 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch": 47, 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklisted": false, 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs" 2024-05-09T06:35:37.107 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:35:37.113 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T06:35:37.326 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:36 smithi063 ceph-mon[27171]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T06:35:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:36 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:36 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:36 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:36 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:36 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:37.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:36 smithi193 ceph-mon[35141]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T06:35:37.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:36 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:37.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:36 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:37.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:36 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:37.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:36 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:37.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:36 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:38.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.956+0000 7f96b3177700 1 -- 172.21.15.63:0/1433944349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac0ffae0 msgr2=0x7f96ac0fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:38.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 --2- 172.21.15.63:0/1433944349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac0ffae0 0x7f96ac0fff60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9698009910 tx=0x7f9698009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:38.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 -- 172.21.15.63:0/1433944349 shutdown_connections 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 --2- 172.21.15.63:0/1433944349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac0ffae0 0x7f96ac0fff60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 --2- 172.21.15.63:0/1433944349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96ac105b70 0x7f96ac105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 -- 172.21.15.63:0/1433944349 >> 172.21.15.63:0/1433944349 conn(0x7f96ac0fb5f0 msgr2=0x7f96ac0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 -- 172.21.15.63:0/1433944349 shutdown_connections 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 -- 172.21.15.63:0/1433944349 wait complete. 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 Processor -- start 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 -- start start 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96ac0ffae0 0x7f96ac19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac105b70 0x7f96ac19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac19d460 con 0x7f96ac105b70 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.957+0000 7f96b3177700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac196a40 con 0x7f96ac0ffae0 2024-05-09T06:35:38.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96abfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac105b70 0x7f96ac19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96abfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac105b70 0x7f96ac19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33090/0 (socket says 172.21.15.63:33090) 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96abfff700 1 -- 172.21.15.63:0/1960677331 learned_addr learned my addr 172.21.15.63:0/1960677331 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96b0f13700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96ac0ffae0 0x7f96ac19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96abfff700 1 -- 172.21.15.63:0/1960677331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96ac0ffae0 msgr2=0x7f96ac19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96abfff700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96ac0ffae0 0x7f96ac19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96abfff700 1 -- 172.21.15.63:0/1960677331 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980095f0 con 0x7f96ac105b70 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96abfff700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac105b70 0x7f96ac19ce40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f969800bd70 tx=0x7f969800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96a9ffb700 1 -- 172.21.15.63:0/1960677331 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f969801d070 con 0x7f96ac105b70 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96a9ffb700 1 -- 172.21.15.63:0/1960677331 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f969800f460 con 0x7f96ac105b70 2024-05-09T06:35:38.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96a9ffb700 1 -- 172.21.15.63:0/1960677331 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9698017630 con 0x7f96ac105b70 2024-05-09T06:35:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.958+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac196cc0 con 0x7f96ac105b70 2024-05-09T06:35:38.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.959+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ac1971b0 con 0x7f96ac105b70 2024-05-09T06:35:38.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.960+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ac109110 con 0x7f96ac105b70 2024-05-09T06:35:38.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.960+0000 7f96a9ffb700 1 -- 172.21.15.63:0/1960677331 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f969800f5d0 con 0x7f96ac105b70 2024-05-09T06:35:38.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.961+0000 7f96a9ffb700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f969c06cce0 0x7f969c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:38.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.961+0000 7f96a9ffb700 1 -- 172.21.15.63:0/1960677331 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f969808d730 con 0x7f96ac105b70 2024-05-09T06:35:38.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.961+0000 7f96b0f13700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f969c06cce0 0x7f969c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:38.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.961+0000 7f96b0f13700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f969c06cce0 0x7f969c06f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f96a0006fd0 tx=0x7f96a0008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:38.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:38.965+0000 7f96a9ffb700 1 -- 172.21.15.63:0/1960677331 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f969805b5b0 con 0x7f96ac105b70 2024-05-09T06:35:38.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:38 smithi063 ceph-mon[27171]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-09T06:35:39.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.217+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96ac1011e0 con 0x7f96ac105b70 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.217+0000 7f96a9ffb700 1 -- 172.21.15.63:0/1960677331 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f969805b140 con 0x7f96ac105b70 2024-05-09T06:35:39.198 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f969c06cce0 msgr2=0x7f969c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f969c06cce0 0x7f969c06f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f96a0006fd0 tx=0x7f96a0008040 comp rx=0 tx=0).stop 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac105b70 msgr2=0x7f96ac19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac105b70 0x7f96ac19ce40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f969800bd70 tx=0x7f969800be50 comp rx=0 tx=0).stop 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 shutdown_connections 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f969c06cce0 0x7f969c06f1a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96ac0ffae0 0x7f96ac19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 --2- 172.21.15.63:0/1960677331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96ac105b70 0x7f96ac19ce40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 >> 172.21.15.63:0/1960677331 conn(0x7f96ac0fb5f0 msgr2=0x7f96ac104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 shutdown_connections 2024-05-09T06:35:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:39.219+0000 7f96b3177700 1 -- 172.21.15.63:0/1960677331 wait complete. 2024-05-09T06:35:39.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:38 smithi193 ceph-mon[35141]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-09T06:35:39.867 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:35:39.868 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:35:39.868 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi193.front.sepia.ceph.com 2024-05-09T06:35:39.868 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-09T06:35:39.868 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:39.868 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:35:39.868 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:35:39.868 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:35:39.868 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T06:35:39.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:39.868 DEBUG:teuthology.orchestra.run.smithi193:> ip addr 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 ::1/128 scope host 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b6 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f0 2024-05-09T06:35:39.888 INFO:teuthology.orchestra.run.smithi193.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b7 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f1 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:52 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:53 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout: inet 172.21.15.193/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft 42227sec preferred_lft 42227sec 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 fe80::ec4:7aff:fe88:7253/64 scope link 2024-05-09T06:35:39.889 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:35:39.889 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T06:35:39.890 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:39.890 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-05-09T06:35:39.890 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link add name ceph-brx type bridge 2024-05-09T06:35:39.890 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip addr flush dev ceph-brx 2024-05-09T06:35:39.890 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set ceph-brx up 2024-05-09T06:35:39.890 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T06:35:39.890 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-05-09T06:35:39.984 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:40.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:40.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:40.103 DEBUG:teuthology.orchestra.run.smithi193:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:35:40.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:39 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1960677331' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T06:35:40.129 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-05-09T06:35:40.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:40.131 DEBUG:teuthology.orchestra.run.smithi193:> route 2024-05-09T06:35:40.196 INFO:teuthology.orchestra.run.smithi193.stdout:Kernel IP routing table 2024-05-09T06:35:40.196 INFO:teuthology.orchestra.run.smithi193.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:35:40.196 INFO:teuthology.orchestra.run.smithi193.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi193.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi193.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:35:40.197 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:40.197 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-05-09T06:35:40.197 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T06:35:40.197 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T06:35:40.197 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T06:35:40.197 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-05-09T06:35:40.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:40.293 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:39 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1960677331' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T06:35:40.446 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:40.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:40.452 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-05-09T06:35:40.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:40.468 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list-id 2024-05-09T06:35:40.523 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:40.523 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-05-09T06:35:40.523 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:35:40.523 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-09T06:35:40.523 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-05-09T06:35:40.610 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:40.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:40.640 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-09T06:35:40.640 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:40.640 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-05-09T06:35:40.640 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-09T06:35:40.641 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T06:35:40.641 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-09T06:35:40.641 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-09T06:35:40.641 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-09T06:35:40.641 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-05-09T06:35:40.689 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:40.761 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:40.768 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:35:40.768 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-05-09T06:35:40.768 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set brx.0 up 2024-05-09T06:35:40.768 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T06:35:40.768 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-05-09T06:35:40.817 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:35:40.839 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:40.845 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-09T06:35:40.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:35:40.846 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:40.862 INFO:teuthology.orchestra.run.smithi193.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-09T06:35:40.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:35:40.863 DEBUG:teuthology.orchestra.run.smithi193:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:40.915 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-05-09T06:35:40.981 DEBUG:teuthology.orchestra.run.smithi193:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:35:41.038 INFO:teuthology.orchestra.run.smithi193.stdout:/proc 2024-05-09T06:35:41.038 INFO:teuthology.orchestra.run.smithi193.stdout:/sys 2024-05-09T06:35:41.038 INFO:teuthology.orchestra.run.smithi193.stdout:/dev 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/security 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/shm 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/pts 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/run 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/cgroup 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/pstore 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/bpf 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/ 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/selinux 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/debug 2024-05-09T06:35:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/tracing 2024-05-09T06:35:41.041 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/mqueue 2024-05-09T06:35:41.041 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/hugepages 2024-05-09T06:35:41.041 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:35:41.041 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/fuse/connections 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/config 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/1000 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/0 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/fe1400a31c94a22a33c617bb9f3d5d1cdf3d2322eef25568a73f88020b54b40b/merged 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/52302770118a94bed7372759820af8552a0ad62b462f744165e5c40cd9c472f2/merged 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/fa941022d07d69c8dc97499089ee64b2b1c0a1a3f3e7072d2ed5e461be7c2cd8/merged 2024-05-09T06:35:41.042 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/496771ba521a4010e744ff2c191560df815860128090d875146b58c00f11e89b/merged 2024-05-09T06:35:41.043 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/adbfc7a87ced115e2e3cb5cef09ff38be037c13bb843b60022424c8fb344d41d/merged 2024-05-09T06:35:41.043 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/467418e281da208a9fe50861e10b9144619b94a4a16dba531aa1b5a2d24dff63/merged 2024-05-09T06:35:41.043 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/eeb585d9a8d440f510ee7c209e73d1b20d363873baf265c3827a26c41a8e7d0a/merged 2024-05-09T06:35:41.043 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/a6df4d62d6d91ffe016ef59e3ff0b37d501f3e67e510f2fc77fc0c4e9bfdb10c/merged 2024-05-09T06:35:41.043 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/c7d5b04366e2b82e43ac9a88a6be066be3dccae3ca42c69dfbcc287bc8e99d20/merged 2024-05-09T06:35:41.043 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/fdfa9b7a5a2d979e67f682433fcae48982ee4daf8be9e7e4026ae0b402381493/merged 2024-05-09T06:35:41.043 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns 2024-05-09T06:35:41.043 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:35:41.043 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:35:41.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:41.044 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-05-09T06:35:41.093 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T06:35:41.094 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-09T06:35:41.136 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-05-09T06:35:41.160 DEBUG:teuthology.orchestra.run.smithi193:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:35:41.172 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:40 smithi193 ceph-mon[35141]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-09T06:35:41.187 INFO:teuthology.orchestra.run.smithi193.stdout:/proc 2024-05-09T06:35:41.187 INFO:teuthology.orchestra.run.smithi193.stdout:/sys 2024-05-09T06:35:41.187 INFO:teuthology.orchestra.run.smithi193.stdout:/dev 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/security 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/shm 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/pts 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/run 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/cgroup 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/pstore 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/bpf 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/ 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/selinux 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/debug 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/tracing 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/mqueue 2024-05-09T06:35:41.188 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/hugepages 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/fuse/connections 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/config 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/1000 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/0 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/fe1400a31c94a22a33c617bb9f3d5d1cdf3d2322eef25568a73f88020b54b40b/merged 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/52302770118a94bed7372759820af8552a0ad62b462f744165e5c40cd9c472f2/merged 2024-05-09T06:35:41.189 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/fa941022d07d69c8dc97499089ee64b2b1c0a1a3f3e7072d2ed5e461be7c2cd8/merged 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/496771ba521a4010e744ff2c191560df815860128090d875146b58c00f11e89b/merged 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/adbfc7a87ced115e2e3cb5cef09ff38be037c13bb843b60022424c8fb344d41d/merged 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/467418e281da208a9fe50861e10b9144619b94a4a16dba531aa1b5a2d24dff63/merged 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/eeb585d9a8d440f510ee7c209e73d1b20d363873baf265c3827a26c41a8e7d0a/merged 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/a6df4d62d6d91ffe016ef59e3ff0b37d501f3e67e510f2fc77fc0c4e9bfdb10c/merged 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/c7d5b04366e2b82e43ac9a88a6be066be3dccae3ca42c69dfbcc287bc8e99d20/merged 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/fdfa9b7a5a2d979e67f682433fcae48982ee4daf8be9e7e4026ae0b402381493/merged 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:35:41.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:41.191 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-05-09T06:35:41.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:2024-05-09T06:35:41.290+0000 7f3673f60480 -1 init, newargv = 0x55f16d1f0330 newargc=15 2024-05-09T06:35:41.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:ceph-fuse[61209]: starting ceph client 2024-05-09T06:35:41.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:ceph-fuse[61209]: starting fuse 2024-05-09T06:35:41.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:40 smithi063 ceph-mon[27171]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-09T06:35:42.344 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-05-09T06:35:42.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:41 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:35:42.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:41 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:35:42.378 DEBUG:teuthology.orchestra.run.smithi193:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T06:35:42.437 INFO:teuthology.orchestra.run.smithi193.stdout:/proc 2024-05-09T06:35:42.437 INFO:teuthology.orchestra.run.smithi193.stdout:/sys 2024-05-09T06:35:42.437 INFO:teuthology.orchestra.run.smithi193.stdout:/dev 2024-05-09T06:35:42.437 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/security 2024-05-09T06:35:42.437 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/shm 2024-05-09T06:35:42.437 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/pts 2024-05-09T06:35:42.437 INFO:teuthology.orchestra.run.smithi193.stdout:/run 2024-05-09T06:35:42.437 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/cgroup 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/pstore 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/bpf 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/ 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/selinux 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/debug 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/tracing 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/mqueue 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/hugepages 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T06:35:42.438 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/fuse/connections 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/config 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/1000 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/0 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/fe1400a31c94a22a33c617bb9f3d5d1cdf3d2322eef25568a73f88020b54b40b/merged 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/52302770118a94bed7372759820af8552a0ad62b462f744165e5c40cd9c472f2/merged 2024-05-09T06:35:42.439 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/fa941022d07d69c8dc97499089ee64b2b1c0a1a3f3e7072d2ed5e461be7c2cd8/merged 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/496771ba521a4010e744ff2c191560df815860128090d875146b58c00f11e89b/merged 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/adbfc7a87ced115e2e3cb5cef09ff38be037c13bb843b60022424c8fb344d41d/merged 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/467418e281da208a9fe50861e10b9144619b94a4a16dba531aa1b5a2d24dff63/merged 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/eeb585d9a8d440f510ee7c209e73d1b20d363873baf265c3827a26c41a8e7d0a/merged 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/a6df4d62d6d91ffe016ef59e3ff0b37d501f3e67e510f2fc77fc0c4e9bfdb10c/merged 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/c7d5b04366e2b82e43ac9a88a6be066be3dccae3ca42c69dfbcc287bc8e99d20/merged 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/fdfa9b7a5a2d979e67f682433fcae48982ee4daf8be9e7e4026ae0b402381493/merged 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:35:42.440 INFO:teuthology.orchestra.run.smithi193.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:42.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:42.441 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-05-09T06:35:42.494 INFO:teuthology.orchestra.run.smithi193.stdout:122 2024-05-09T06:35:42.494 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [122] 2024-05-09T06:35:42.495 DEBUG:teuthology.orchestra.run.smithi193:> sudo stdin-killer -- python3 -c ' 2024-05-09T06:35:42.495 DEBUG:teuthology.orchestra.run.smithi193:> import glob 2024-05-09T06:35:42.495 DEBUG:teuthology.orchestra.run.smithi193:> import re 2024-05-09T06:35:42.495 DEBUG:teuthology.orchestra.run.smithi193:> import os 2024-05-09T06:35:42.495 DEBUG:teuthology.orchestra.run.smithi193:> import subprocess 2024-05-09T06:35:42.495 DEBUG:teuthology.orchestra.run.smithi193:> 2024-05-09T06:35:42.495 DEBUG:teuthology.orchestra.run.smithi193:> def _find_admin_socket(client_name): 2024-05-09T06:35:42.495 DEBUG:teuthology.orchestra.run.smithi193:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-09T06:35:42.496 DEBUG:teuthology.orchestra.run.smithi193:> files = glob.glob(asok_path) 2024-05-09T06:35:42.496 DEBUG:teuthology.orchestra.run.smithi193:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-09T06:35:42.496 DEBUG:teuthology.orchestra.run.smithi193:> 2024-05-09T06:35:42.496 DEBUG:teuthology.orchestra.run.smithi193:> # Given a non-glob path, it better be there 2024-05-09T06:35:42.496 DEBUG:teuthology.orchestra.run.smithi193:> if "*" not in asok_path: 2024-05-09T06:35:42.496 DEBUG:teuthology.orchestra.run.smithi193:> assert(len(files) == 1) 2024-05-09T06:35:42.496 DEBUG:teuthology.orchestra.run.smithi193:> return files[0] 2024-05-09T06:35:42.496 DEBUG:teuthology.orchestra.run.smithi193:> 2024-05-09T06:35:42.496 DEBUG:teuthology.orchestra.run.smithi193:> for f in files: 2024-05-09T06:35:42.497 DEBUG:teuthology.orchestra.run.smithi193:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T06:35:42.497 DEBUG:teuthology.orchestra.run.smithi193:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T06:35:42.497 DEBUG:teuthology.orchestra.run.smithi193:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T06:35:42.497 DEBUG:teuthology.orchestra.run.smithi193:> contents = proc_f.read() 2024-05-09T06:35:42.497 DEBUG:teuthology.orchestra.run.smithi193:> if mountpoint in contents: 2024-05-09T06:35:42.497 DEBUG:teuthology.orchestra.run.smithi193:> return f 2024-05-09T06:35:42.498 DEBUG:teuthology.orchestra.run.smithi193:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T06:35:42.498 DEBUG:teuthology.orchestra.run.smithi193:> 2024-05-09T06:35:42.498 DEBUG:teuthology.orchestra.run.smithi193:> print(_find_admin_socket("client.1")) 2024-05-09T06:35:42.498 DEBUG:teuthology.orchestra.run.smithi193:> ' 2024-05-09T06:35:42.613 INFO:teuthology.orchestra.run.smithi193.stdout:/var/run/ceph/ceph-client.1.61209.asok 2024-05-09T06:35:42.617 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-09T06:35:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:35:42.625 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61209.asok 2024-05-09T06:35:42.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:42.625 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61209.asok status 2024-05-09T06:35:42.763 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-09T06:35:42.763 INFO:teuthology.orchestra.run.smithi193.stdout: "metadata": { 2024-05-09T06:35:42.763 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_sha1": "bf64272e3d7edf5c7f8706082e690401a7a11750", 2024-05-09T06:35:42.763 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 18.2.2-1306-gbf64272e (bf64272e3d7edf5c7f8706082e690401a7a11750) reef (stable)", 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "entity_id": "1", 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "hostname": "smithi193", 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "pid": "61209", 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "root": "/" 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_count": 0, 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_pinned_count": 0, 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "id": 24327, 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "inst": { 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "name": { 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "client", 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: "num": 24327 2024-05-09T06:35:42.764 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 1228019762 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.1:0", 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 1228019762 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "inst_str": "client.24327 192.168.0.1:0/1228019762", 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "addr_str": "192.168.0.1:0/1228019762", 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "inode_count": 1, 2024-05-09T06:35:42.765 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_epoch": 15, 2024-05-09T06:35:42.766 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch": 47, 2024-05-09T06:35:42.766 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch_barrier": 0, 2024-05-09T06:35:42.766 INFO:teuthology.orchestra.run.smithi193.stdout: "blocklisted": false, 2024-05-09T06:35:42.766 INFO:teuthology.orchestra.run.smithi193.stdout: "fs_name": "cephfs" 2024-05-09T06:35:42.766 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-09T06:35:42.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:42.774 DEBUG:teuthology.orchestra.run.smithi063:> stat --file-system '--printf=%T 2024-05-09T06:35:42.774 DEBUG:teuthology.orchestra.run.smithi063:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:42.796 INFO:teuthology.orchestra.run.smithi063.stdout:fuseblk 2024-05-09T06:35:42.797 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:42.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:42.797 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:42.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:42.871 DEBUG:teuthology.orchestra.run.smithi193:> stat --file-system '--printf=%T 2024-05-09T06:35:42.872 DEBUG:teuthology.orchestra.run.smithi193:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:42.887 INFO:teuthology.orchestra.run.smithi193.stdout:fuseblk 2024-05-09T06:35:42.887 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:42.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:35:42.887 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:42.953 INFO:teuthology.run_tasks:Running task print... 2024-05-09T06:35:42.964 INFO:teuthology.task.print:**** done client 2024-05-09T06:35:42.964 INFO:teuthology.run_tasks:Running task parallel... 2024-05-09T06:35:42.985 INFO:teuthology.task.parallel:starting parallel... 2024-05-09T06:35:42.985 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-09T06:35:42.986 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-09T06:35:42.986 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:35:42.986 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-05-09T06:35:42.987 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-09T06:35:42.987 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-09T06:35:42.991 INFO:tasks.workunit:Pulling workunits from ref be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T06:35:42.991 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T06:35:42.991 INFO:tasks.workunit:timeout=3h 2024-05-09T06:35:42.991 INFO:tasks.workunit:cleanup=True 2024-05-09T06:35:42.991 DEBUG:teuthology.orchestra.run.smithi063:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:43.012 INFO:teuthology.orchestra.run.smithi063.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-09T06:35:43.013 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 7eh/126d Inode: 1 Links: 2 2024-05-09T06:35:43.013 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:43.013 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-09T06:35:43.013 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-09T06:35:43.013 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-09 06:35:10.496640003 +0000 2024-05-09T06:35:43.013 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-09 06:35:42.888680579 +0000 2024-05-09T06:35:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-05-09T06:35:43.013 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T06:35:43.013 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T06:35:43.086 DEBUG:teuthology.orchestra.run.smithi193:> stat -- /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-09T06:35:43.103 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 7ah/122d Inode: 1 Links: 3 2024-05-09T06:35:43.103 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:43.103 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-09T06:35:43.103 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-09T06:35:43.103 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-09 06:35:43.100391928 +0000 2024-05-09T06:35:43.103 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-09 06:35:43.100391928 +0000 2024-05-09T06:35:43.103 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2024-05-09T06:35:43.104 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-05-09T06:35:43.104 DEBUG:teuthology.orchestra.run.smithi193:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-05-09T06:35:43.173 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T06:35:43.174 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T06:35:43.190 INFO:tasks.workunit.client.0.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T06:35:43.221 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:43.228 INFO:tasks.workunit.client.1.smithi193.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-05-09T06:35:43.328 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:42 smithi063 ceph-mon[27171]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-09T06:35:43.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:42 smithi193 ceph-mon[35141]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-09T06:35:44.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 -- 172.21.15.63:0/695843713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364104060 msgr2=0x7f6364106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 --2- 172.21.15.63:0/695843713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364104060 0x7f6364106450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6354009870 tx=0x7f6354009b80 comp rx=0 tx=0).stop 2024-05-09T06:35:44.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 -- 172.21.15.63:0/695843713 shutdown_connections 2024-05-09T06:35:44.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 --2- 172.21.15.63:0/695843713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364104060 0x7f6364106450 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 --2- 172.21.15.63:0/695843713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6364101730 0x7f6364103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 -- 172.21.15.63:0/695843713 >> 172.21.15.63:0/695843713 conn(0x7f63640fb120 msgr2=0x7f63640fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 -- 172.21.15.63:0/695843713 shutdown_connections 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 -- 172.21.15.63:0/695843713 wait complete. 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 Processor -- start 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.791+0000 7f636c7c3700 1 -- start start 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636c7c3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364101730 0x7f6364101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636c7c3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6364104060 0x7f63640ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636c7c3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63640ffc20 con 0x7f6364101730 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636c7c3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63640ffd60 con 0x7f6364104060 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636a55f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364101730 0x7f6364101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636a55f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364101730 0x7f6364101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44124/0 (socket says 172.21.15.63:44124) 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636a55f700 1 -- 172.21.15.63:0/3585258258 learned_addr learned my addr 172.21.15.63:0/3585258258 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f6369d5e700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6364104060 0x7f63640ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636a55f700 1 -- 172.21.15.63:0/3585258258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6364104060 msgr2=0x7f63640ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636a55f700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6364104060 0x7f63640ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636a55f700 1 -- 172.21.15.63:0/3585258258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354009520 con 0x7f6364101730 2024-05-09T06:35:44.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f6369d5e700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6364104060 0x7f63640ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:44.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636a55f700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364101730 0x7f6364101040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f636000e940 tx=0x7f636000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f635b7fe700 1 -- 172.21.15.63:0/3585258258 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f636000eee0 con 0x7f6364101730 2024-05-09T06:35:44.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f635b7fe700 1 -- 172.21.15.63:0/3585258258 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6360004cf0 con 0x7f6364101730 2024-05-09T06:35:44.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6364100040 con 0x7f6364101730 2024-05-09T06:35:44.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.792+0000 7f635b7fe700 1 -- 172.21.15.63:0/3585258258 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6360005650 con 0x7f6364101730 2024-05-09T06:35:44.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.793+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636419e670 con 0x7f6364101730 2024-05-09T06:35:44.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.793+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636404ebb0 con 0x7f6364101730 2024-05-09T06:35:44.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.794+0000 7f635b7fe700 1 -- 172.21.15.63:0/3585258258 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6360004750 con 0x7f6364101730 2024-05-09T06:35:44.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.795+0000 7f635b7fe700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f635006cce0 0x7f635006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.795+0000 7f635b7fe700 1 -- 172.21.15.63:0/3585258258 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6360014070 con 0x7f6364101730 2024-05-09T06:35:44.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.795+0000 7f6369d5e700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f635006cce0 0x7f635006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.795+0000 7f6369d5e700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f635006cce0 0x7f635006f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f635400b820 tx=0x7f6354019040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:44.799+0000 7f635b7fe700 1 -- 172.21.15.63:0/3585258258 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f636005a4b0 con 0x7f6364101730 2024-05-09T06:35:44.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.008+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f6364109800 con 0x7f6364101730 2024-05-09T06:35:44.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.013+0000 7f635b7fe700 1 -- 172.21.15.63:0/3585258258 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v24)=0 v24) v1 ==== 126+0+0 (secure 0 0 0) 0x7f636005a040 con 0x7f6364101730 2024-05-09T06:35:44.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.015+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f635006cce0 msgr2=0x7f635006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.015+0000 7f636c7c3700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f635006cce0 0x7f635006f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f635400b820 tx=0x7f6354019040 comp rx=0 tx=0).stop 2024-05-09T06:35:44.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.015+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364101730 msgr2=0x7f6364101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.015+0000 7f636c7c3700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364101730 0x7f6364101040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f636000e940 tx=0x7f636000ed00 comp rx=0 tx=0).stop 2024-05-09T06:35:44.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.015+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 shutdown_connections 2024-05-09T06:35:44.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.015+0000 7f636c7c3700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6364101730 0x7f6364101040 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.015+0000 7f636c7c3700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f635006cce0 0x7f635006f1a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.015+0000 7f636c7c3700 1 --2- 172.21.15.63:0/3585258258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6364104060 0x7f63640ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.015+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 >> 172.21.15.63:0/3585258258 conn(0x7f63640fb120 msgr2=0x7f63640fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.016+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 shutdown_connections 2024-05-09T06:35:44.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:45.016+0000 7f636c7c3700 1 -- 172.21.15.63:0/3585258258 wait complete. 2024-05-09T06:35:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:44 smithi063 ceph-mon[27171]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-09T06:35:45.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:44 smithi193 ceph-mon[35141]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-09T06:35:45.712 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-09T06:35:45.713 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:35:45.713 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-09T06:35:45.974 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:46.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:46 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3585258258' entity='client.admin' 2024-05-09T06:35:46.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:46.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:46.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:46.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:46 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:46.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:46 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3585258258' entity='client.admin' 2024-05-09T06:35:46.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:46.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:46.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:46.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:46 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:47.320 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:47 smithi063 ceph-mon[27171]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-09T06:35:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:47 smithi193 ceph-mon[35141]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-09T06:35:47.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.575+0000 7f33c3e43700 1 -- 172.21.15.63:0/182894564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc100c40 msgr2=0x7f33bc101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:47.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.575+0000 7f33c3e43700 1 --2- 172.21.15.63:0/182894564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc100c40 0x7f33bc101060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009910 tx=0x7f33b8009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:47.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.575+0000 7f33c3e43700 1 -- 172.21.15.63:0/182894564 shutdown_connections 2024-05-09T06:35:47.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.575+0000 7f33c3e43700 1 --2- 172.21.15.63:0/182894564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc101e40 0x7f33bc1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:47.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.575+0000 7f33c3e43700 1 --2- 172.21.15.63:0/182894564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc100c40 0x7f33bc101060 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:47.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.575+0000 7f33c3e43700 1 -- 172.21.15.63:0/182894564 >> 172.21.15.63:0/182894564 conn(0x7f33bc0fc1e0 msgr2=0x7f33bc0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:47.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.575+0000 7f33c3e43700 1 -- 172.21.15.63:0/182894564 shutdown_connections 2024-05-09T06:35:47.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.575+0000 7f33c3e43700 1 -- 172.21.15.63:0/182894564 wait complete. 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c3e43700 1 Processor -- start 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c3e43700 1 -- start start 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c3e43700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc100c40 0x7f33bc1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c3e43700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc101e40 0x7f33bc198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c3e43700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc199400 con 0x7f33bc101e40 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c3e43700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc199540 con 0x7f33bc100c40 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c13de700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc101e40 0x7f33bc198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c13de700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc101e40 0x7f33bc198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44148/0 (socket says 172.21.15.63:44148) 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c13de700 1 -- 172.21.15.63:0/388508818 learned_addr learned my addr 172.21.15.63:0/388508818 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c1bdf700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc100c40 0x7f33bc1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c13de700 1 -- 172.21.15.63:0/388508818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc100c40 msgr2=0x7f33bc1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c13de700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc100c40 0x7f33bc1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:47.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c13de700 1 -- 172.21.15.63:0/388508818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b80095f0 con 0x7f33bc101e40 2024-05-09T06:35:47.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.576+0000 7f33c13de700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc101e40 0x7f33bc198de0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00d8d0 tx=0x7f33ac00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:47.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.577+0000 7f33b2ffd700 1 -- 172.21.15.63:0/388508818 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33ac0098a0 con 0x7f33bc101e40 2024-05-09T06:35:47.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.577+0000 7f33b2ffd700 1 -- 172.21.15.63:0/388508818 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33ac010460 con 0x7f33bc101e40 2024-05-09T06:35:47.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.577+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33bc19df50 con 0x7f33bc101e40 2024-05-09T06:35:47.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.577+0000 7f33b2ffd700 1 -- 172.21.15.63:0/388508818 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33ac00f5f0 con 0x7f33bc101e40 2024-05-09T06:35:47.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.577+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33bc19e4f0 con 0x7f33bc101e40 2024-05-09T06:35:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.578+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33bc04ebb0 con 0x7f33bc101e40 2024-05-09T06:35:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.578+0000 7f33b2ffd700 1 -- 172.21.15.63:0/388508818 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f33ac009a00 con 0x7f33bc101e40 2024-05-09T06:35:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.579+0000 7f33b2ffd700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f33a806cce0 0x7f33a806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.579+0000 7f33b2ffd700 1 -- 172.21.15.63:0/388508818 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f33ac08c170 con 0x7f33bc101e40 2024-05-09T06:35:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.579+0000 7f33c1bdf700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f33a806cce0 0x7f33a806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.579+0000 7f33c1bdf700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f33a806cce0 0x7f33a806f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006010 tx=0x7f33b800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:47.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.583+0000 7f33b2ffd700 1 -- 172.21.15.63:0/388508818 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f33ac059f70 con 0x7f33bc101e40 2024-05-09T06:35:47.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.810+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f33bc073970 con 0x7f33bc101e40 2024-05-09T06:35:47.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.810+0000 7f33b2ffd700 1 -- 172.21.15.63:0/388508818 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v24)=0 v24) v1 ==== 155+0+0 (secure 0 0 0) 0x7f33ac059b00 con 0x7f33bc101e40 2024-05-09T06:35:47.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.812+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f33a806cce0 msgr2=0x7f33a806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:47.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.812+0000 7f33c3e43700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f33a806cce0 0x7f33a806f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006010 tx=0x7f33b800b540 comp rx=0 tx=0).stop 2024-05-09T06:35:47.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.812+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc101e40 msgr2=0x7f33bc198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:47.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.812+0000 7f33c3e43700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc101e40 0x7f33bc198de0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00d8d0 tx=0x7f33ac00dbe0 comp rx=0 tx=0).stop 2024-05-09T06:35:47.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.813+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 shutdown_connections 2024-05-09T06:35:47.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.813+0000 7f33c3e43700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f33a806cce0 0x7f33a806f1a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:47.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.813+0000 7f33c3e43700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc100c40 0x7f33bc1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:47.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.813+0000 7f33c3e43700 1 --2- 172.21.15.63:0/388508818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33bc101e40 0x7f33bc198de0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:47.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.813+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 >> 172.21.15.63:0/388508818 conn(0x7f33bc0fc1e0 msgr2=0x7f33bc0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:47.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.813+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 shutdown_connections 2024-05-09T06:35:47.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:47.813+0000 7f33c3e43700 1 -- 172.21.15.63:0/388508818 wait complete. 2024-05-09T06:35:48.408 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-09T06:35:48.634 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:49.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:49 smithi063 ceph-mon[27171]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-09T06:35:49.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:49 smithi193 ceph-mon[35141]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-09T06:35:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.302+0000 7f95418ed700 1 -- 172.21.15.63:0/979343051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c101780 msgr2=0x7f953c103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.302+0000 7f95418ed700 1 --2- 172.21.15.63:0/979343051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c101780 0x7f953c103b70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f952c00b0a0 tx=0x7f952c00b3b0 comp rx=0 tx=0).stop 2024-05-09T06:35:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.302+0000 7f95418ed700 1 -- 172.21.15.63:0/979343051 shutdown_connections 2024-05-09T06:35:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.302+0000 7f95418ed700 1 --2- 172.21.15.63:0/979343051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f953c1040b0 0x7f953c1064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.302+0000 7f95418ed700 1 --2- 172.21.15.63:0/979343051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c101780 0x7f953c103b70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.302+0000 7f95418ed700 1 -- 172.21.15.63:0/979343051 >> 172.21.15.63:0/979343051 conn(0x7f953c0fb0d0 msgr2=0x7f953c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.302+0000 7f95418ed700 1 -- 172.21.15.63:0/979343051 shutdown_connections 2024-05-09T06:35:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.302+0000 7f95418ed700 1 -- 172.21.15.63:0/979343051 wait complete. 2024-05-09T06:35:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.303+0000 7f95418ed700 1 Processor -- start 2024-05-09T06:35:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.303+0000 7f95418ed700 1 -- start start 2024-05-09T06:35:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.303+0000 7f95418ed700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f953c101780 0x7f953c101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.303+0000 7f95418ed700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c1040b0 0x7f953c0ff730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.303+0000 7f95418ed700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c0ffc70 con 0x7f953c1040b0 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.303+0000 7f95418ed700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c0ffdb0 con 0x7f953c101780 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.303+0000 7f953a7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c1040b0 0x7f953c0ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f953a7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c1040b0 0x7f953c0ff730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44166/0 (socket says 172.21.15.63:44166) 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f953a7fc700 1 -- 172.21.15.63:0/2221074993 learned_addr learned my addr 172.21.15.63:0/2221074993 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f953affd700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f953c101780 0x7f953c101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f953a7fc700 1 -- 172.21.15.63:0/2221074993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f953c101780 msgr2=0x7f953c101040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f953a7fc700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f953c101780 0x7f953c101040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f953a7fc700 1 -- 172.21.15.63:0/2221074993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c009d00 con 0x7f953c1040b0 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f953a7fc700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c1040b0 0x7f953c0ff730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f952400b4b0 tx=0x7f952400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f95408eb700 1 -- 172.21.15.63:0/2221074993 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9524011800 con 0x7f953c1040b0 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f95408eb700 1 -- 172.21.15.63:0/2221074993 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9524011e20 con 0x7f953c1040b0 2024-05-09T06:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f95408eb700 1 -- 172.21.15.63:0/2221074993 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f952400f2e0 con 0x7f953c1040b0 2024-05-09T06:35:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953c100090 con 0x7f953c1040b0 2024-05-09T06:35:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.304+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c19e800 con 0x7f953c1040b0 2024-05-09T06:35:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.305+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c04ebb0 con 0x7f953c1040b0 2024-05-09T06:35:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.306+0000 7f95408eb700 1 -- 172.21.15.63:0/2221074993 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9524010380 con 0x7f953c1040b0 2024-05-09T06:35:50.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.306+0000 7f95408eb700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f952806ccf0 0x7f952806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:50.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.306+0000 7f95408eb700 1 -- 172.21.15.63:0/2221074993 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f952408be90 con 0x7f953c1040b0 2024-05-09T06:35:50.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.306+0000 7f953affd700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f952806ccf0 0x7f952806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:50.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.307+0000 7f953affd700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f952806ccf0 0x7f952806f1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f952c000f80 tx=0x7f952c00b7b0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.311+0000 7f95408eb700 1 -- 172.21.15.63:0/2221074993 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9524059c90 con 0x7f953c1040b0 2024-05-09T06:35:50.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.515+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f953c109810 con 0x7f953c1040b0 2024-05-09T06:35:50.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.516+0000 7f95408eb700 1 -- 172.21.15.63:0/2221074993 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v24)=0 v24) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9524059820 con 0x7f953c1040b0 2024-05-09T06:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f952806ccf0 msgr2=0x7f952806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f952806ccf0 0x7f952806f1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f952c000f80 tx=0x7f952c00b7b0 comp rx=0 tx=0).stop 2024-05-09T06:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c1040b0 msgr2=0x7f953c0ff730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c1040b0 0x7f953c0ff730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f952400b4b0 tx=0x7f952400b870 comp rx=0 tx=0).stop 2024-05-09T06:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 shutdown_connections 2024-05-09T06:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f952806ccf0 0x7f952806f1b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f953c101780 0x7f953c101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 --2- 172.21.15.63:0/2221074993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953c1040b0 0x7f953c0ff730 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 >> 172.21.15.63:0/2221074993 conn(0x7f953c0fb0d0 msgr2=0x7f953c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:50.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 shutdown_connections 2024-05-09T06:35:50.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:50.518+0000 7f95418ed700 1 -- 172.21.15.63:0/2221074993 wait complete. 2024-05-09T06:35:51.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:51 smithi063 ceph-mon[27171]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-09T06:35:51.207 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-09T06:35:51.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:51 smithi193 ceph-mon[35141]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-09T06:35:51.425 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.195+0000 7f8692f6b700 1 -- 172.21.15.63:0/1036174216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c100da0 msgr2=0x7f868c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.195+0000 7f8692f6b700 1 --2- 172.21.15.63:0/1036174216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c100da0 0x7f868c101200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8678009910 tx=0x7f8678009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.195+0000 7f8692f6b700 1 -- 172.21.15.63:0/1036174216 shutdown_connections 2024-05-09T06:35:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.195+0000 7f8692f6b700 1 --2- 172.21.15.63:0/1036174216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c100da0 0x7f868c101200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.195+0000 7f8692f6b700 1 --2- 172.21.15.63:0/1036174216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c0ffba0 0x7f868c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.195+0000 7f8692f6b700 1 -- 172.21.15.63:0/1036174216 >> 172.21.15.63:0/1036174216 conn(0x7f868c0fb120 msgr2=0x7f868c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.195+0000 7f8692f6b700 1 -- 172.21.15.63:0/1036174216 shutdown_connections 2024-05-09T06:35:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.195+0000 7f8692f6b700 1 -- 172.21.15.63:0/1036174216 wait complete. 2024-05-09T06:35:53.177 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:53 smithi063 ceph-mon[27171]: pgmap v205: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-09T06:35:53.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.195+0000 7f8692f6b700 1 Processor -- start 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8692f6b700 1 -- start start 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8692f6b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c0ffba0 0x7f868c0690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8692f6b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c100da0 0x7f868c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8692f6b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c19b960 con 0x7f868c0ffba0 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8692f6b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c19baa0 con 0x7f868c100da0 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8690d07700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c0ffba0 0x7f868c0690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8690d07700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c0ffba0 0x7f868c0690b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44188/0 (socket says 172.21.15.63:44188) 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8690d07700 1 -- 172.21.15.63:0/2740235168 learned_addr learned my addr 172.21.15.63:0/2740235168 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f868bfff700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c100da0 0x7f868c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8690d07700 1 -- 172.21.15.63:0/2740235168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c100da0 msgr2=0x7f868c19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8690d07700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c100da0 0x7f868c19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.196+0000 7f8690d07700 1 -- 172.21.15.63:0/2740235168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780095f0 con 0x7f868c0ffba0 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.197+0000 7f8690d07700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c0ffba0 0x7f868c0690b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f868000d900 tx=0x7f868000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.197+0000 7f8689ffb700 1 -- 172.21.15.63:0/2740235168 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f868000b920 con 0x7f868c0ffba0 2024-05-09T06:35:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.197+0000 7f8689ffb700 1 -- 172.21.15.63:0/2740235168 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8680010460 con 0x7f868c0ffba0 2024-05-09T06:35:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.197+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c19bd80 con 0x7f868c0ffba0 2024-05-09T06:35:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.197+0000 7f8689ffb700 1 -- 172.21.15.63:0/2740235168 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f868000f590 con 0x7f868c0ffba0 2024-05-09T06:35:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.197+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c19c270 con 0x7f868c0ffba0 2024-05-09T06:35:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.198+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c18f720 con 0x7f868c0ffba0 2024-05-09T06:35:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.199+0000 7f8689ffb700 1 -- 172.21.15.63:0/2740235168 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f86800105d0 con 0x7f868c0ffba0 2024-05-09T06:35:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.199+0000 7f8689ffb700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f867c06ccf0 0x7f867c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.199+0000 7f8689ffb700 1 -- 172.21.15.63:0/2740235168 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f868008bee0 con 0x7f868c0ffba0 2024-05-09T06:35:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.199+0000 7f868bfff700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f867c06ccf0 0x7f867c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.200+0000 7f868bfff700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f867c06ccf0 0x7f867c06f1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8678006010 tx=0x7f8678005180 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:53.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.204+0000 7f8689ffb700 1 -- 172.21.15.63:0/2740235168 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8680059cd0 con 0x7f868c0ffba0 2024-05-09T06:35:53.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:53 smithi193 ceph-mon[35141]: pgmap v205: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-09T06:35:53.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.406+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f868c108980 con 0x7f868c0ffba0 2024-05-09T06:35:53.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.406+0000 7f8689ffb700 1 -- 172.21.15.63:0/2740235168 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v24)=0 v24) v1 ==== 135+0+0 (secure 0 0 0) 0x7f8680059860 con 0x7f868c0ffba0 2024-05-09T06:35:53.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f867c06ccf0 msgr2=0x7f867c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:53.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f867c06ccf0 0x7f867c06f1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8678006010 tx=0x7f8678005180 comp rx=0 tx=0).stop 2024-05-09T06:35:53.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c0ffba0 msgr2=0x7f868c0690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:53.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c0ffba0 0x7f868c0690b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f868000d900 tx=0x7f868000dc10 comp rx=0 tx=0).stop 2024-05-09T06:35:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 shutdown_connections 2024-05-09T06:35:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c0ffba0 0x7f868c0690b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f867c06ccf0 0x7f867c06f1b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 --2- 172.21.15.63:0/2740235168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f868c100da0 0x7f868c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 >> 172.21.15.63:0/2740235168 conn(0x7f868c0fb120 msgr2=0x7f868c1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 shutdown_connections 2024-05-09T06:35:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:53.408+0000 7f8692f6b700 1 -- 172.21.15.63:0/2740235168 wait complete. 2024-05-09T06:35:53.957 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-05-09T06:35:54.178 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:55.265 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:55 smithi063 ceph-mon[27171]: pgmap v206: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-09T06:35:55.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:55 smithi193 ceph-mon[35141]: pgmap v206: 65 pgs: 65 active+clean; 585 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-09T06:35:55.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.772+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4274038613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c8101130 msgr2=0x7f11c8101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:55.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.772+0000 7f11cfcfe700 1 --2- 172.21.15.63:0/4274038613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c8101130 0x7f11c8101550 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009910 tx=0x7f11c4009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:55.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.772+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4274038613 shutdown_connections 2024-05-09T06:35:55.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.772+0000 7f11cfcfe700 1 --2- 172.21.15.63:0/4274038613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11c80ffde0 0x7f11c8100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:55.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.772+0000 7f11cfcfe700 1 --2- 172.21.15.63:0/4274038613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c8101130 0x7f11c8101550 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.772+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4274038613 >> 172.21.15.63:0/4274038613 conn(0x7f11c80fa870 msgr2=0x7f11c80fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.773+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4274038613 shutdown_connections 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.773+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4274038613 wait complete. 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.773+0000 7f11cfcfe700 1 Processor -- start 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.773+0000 7f11cfcfe700 1 -- start start 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.773+0000 7f11cfcfe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c80ffde0 0x7f11c8103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.773+0000 7f11cfcfe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11c8101130 0x7f11c81021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.773+0000 7f11cfcfe700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8102710 con 0x7f11c80ffde0 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.773+0000 7f11cfcfe700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8102850 con 0x7f11c8101130 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cda9a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c80ffde0 0x7f11c8103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cda9a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c80ffde0 0x7f11c8103b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36200/0 (socket says 172.21.15.63:36200) 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cda9a700 1 -- 172.21.15.63:0/4210443790 learned_addr learned my addr 172.21.15.63:0/4210443790 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cd299700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11c8101130 0x7f11c81021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:55.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cda9a700 1 -- 172.21.15.63:0/4210443790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11c8101130 msgr2=0x7f11c81021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:55.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cda9a700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11c8101130 0x7f11c81021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:55.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cda9a700 1 -- 172.21.15.63:0/4210443790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40095f0 con 0x7f11c80ffde0 2024-05-09T06:35:55.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cd299700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11c8101130 0x7f11c81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:55.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cda9a700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c80ffde0 0x7f11c8103b30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11c400bd70 tx=0x7f11c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:55.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11baffd700 1 -- 172.21.15.63:0/4210443790 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11c401d070 con 0x7f11c80ffde0 2024-05-09T06:35:55.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c8102ad0 con 0x7f11c80ffde0 2024-05-09T06:35:55.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11baffd700 1 -- 172.21.15.63:0/4210443790 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11c400f460 con 0x7f11c80ffde0 2024-05-09T06:35:55.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c8103040 con 0x7f11c80ffde0 2024-05-09T06:35:55.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.774+0000 7f11baffd700 1 -- 172.21.15.63:0/4210443790 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11c4021600 con 0x7f11c80ffde0 2024-05-09T06:35:55.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.775+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c804ebb0 con 0x7f11c80ffde0 2024-05-09T06:35:55.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.776+0000 7f11baffd700 1 -- 172.21.15.63:0/4210443790 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f11c400fa50 con 0x7f11c80ffde0 2024-05-09T06:35:55.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.776+0000 7f11baffd700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f11b406ccf0 0x7f11b406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:55.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.776+0000 7f11baffd700 1 -- 172.21.15.63:0/4210443790 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f11c408d450 con 0x7f11c80ffde0 2024-05-09T06:35:55.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.776+0000 7f11cd299700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f11b406ccf0 0x7f11b406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:55.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.777+0000 7f11cd299700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f11b406ccf0 0x7f11b406f1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005fd0 tx=0x7f11bc005dc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:55.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:55.781+0000 7f11baffd700 1 -- 172.21.15.63:0/4210443790 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f11c405b250 con 0x7f11c80ffde0 2024-05-09T06:35:55.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.007+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", "target": ["mon-mgr", ""]}) v1 -- 0x7f11c8103040 con 0x7f11b406ccf0 2024-05-09T06:35:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.013+0000 7f11baffd700 1 -- 172.21.15.63:0/4210443790 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f11c8103040 con 0x7f11b406ccf0 2024-05-09T06:35:55.993 INFO:teuthology.orchestra.run.smithi063.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:35:55.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f11b406ccf0 msgr2=0x7f11b406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:55.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f11b406ccf0 0x7f11b406f1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005fd0 tx=0x7f11bc005dc0 comp rx=0 tx=0).stop 2024-05-09T06:35:55.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c80ffde0 msgr2=0x7f11c8103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:55.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c80ffde0 0x7f11c8103b30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11c400bd70 tx=0x7f11c400be50 comp rx=0 tx=0).stop 2024-05-09T06:35:55.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 shutdown_connections 2024-05-09T06:35:55.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11c80ffde0 0x7f11c8103b30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:55.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f11b406ccf0 0x7f11b406f1b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:55.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 --2- 172.21.15.63:0/4210443790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11c8101130 0x7f11c81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:55.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 >> 172.21.15.63:0/4210443790 conn(0x7f11c80fa870 msgr2=0x7f11c80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:55.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 shutdown_connections 2024-05-09T06:35:55.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:56.015+0000 7f11cfcfe700 1 -- 172.21.15.63:0/4210443790 wait complete. 2024-05-09T06:35:56.802 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-09T06:35:56.803 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:35:56.803 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-05-09T06:35:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:35:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:57 smithi063 ceph-mon[27171]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-09T06:35:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:57 smithi063 ceph-mon[27171]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:35:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:57 smithi063 ceph-mon[27171]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:35:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:57.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:35:57.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:57 smithi193 ceph-mon[35141]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-09T06:35:57.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:57 smithi193 ceph-mon[35141]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:35:57.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:57 smithi193 ceph-mon[35141]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:35:57.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:57.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:35:57.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:35:57.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:35:57.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:35:57.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:35:58.493 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:58 smithi063 ceph-mon[27171]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:35:58.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:58 smithi193 ceph-mon[35141]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:35:58.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.903+0000 7f3b4e136700 1 -- 172.21.15.63:0/152336778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b48100d60 msgr2=0x7f3b481011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.903+0000 7f3b4e136700 1 --2- 172.21.15.63:0/152336778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b48100d60 0x7f3b481011c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009910 tx=0x7f3b3c009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.903+0000 7f3b4e136700 1 -- 172.21.15.63:0/152336778 shutdown_connections 2024-05-09T06:35:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.903+0000 7f3b4e136700 1 --2- 172.21.15.63:0/152336778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b48100d60 0x7f3b481011c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.903+0000 7f3b4e136700 1 --2- 172.21.15.63:0/152336778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480ffb60 0x7f3b480fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.903+0000 7f3b4e136700 1 -- 172.21.15.63:0/152336778 >> 172.21.15.63:0/152336778 conn(0x7f3b480fb0c0 msgr2=0x7f3b480fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.903+0000 7f3b4e136700 1 -- 172.21.15.63:0/152336778 shutdown_connections 2024-05-09T06:35:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.903+0000 7f3b4e136700 1 -- 172.21.15.63:0/152336778 wait complete. 2024-05-09T06:35:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.904+0000 7f3b4e136700 1 Processor -- start 2024-05-09T06:35:58.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.904+0000 7f3b4e136700 1 -- start start 2024-05-09T06:35:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.904+0000 7f3b4e136700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b480ffb60 0x7f3b48069bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.904+0000 7f3b4e136700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48100d60 0x7f3b4819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.904+0000 7f3b4e136700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4819b9c0 con 0x7f3b480ffb60 2024-05-09T06:35:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.904+0000 7f3b4e136700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4819bb30 con 0x7f3b48100d60 2024-05-09T06:35:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b477fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b480ffb60 0x7f3b48069bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b477fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b480ffb60 0x7f3b48069bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36220/0 (socket says 172.21.15.63:36220) 2024-05-09T06:35:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b477fe700 1 -- 172.21.15.63:0/1431920690 learned_addr learned my addr 172.21.15.63:0/1431920690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b46ffd700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48100d60 0x7f3b4819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b477fe700 1 -- 172.21.15.63:0/1431920690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48100d60 msgr2=0x7f3b4819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b477fe700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48100d60 0x7f3b4819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b477fe700 1 -- 172.21.15.63:0/1431920690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c0095f0 con 0x7f3b480ffb60 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b46ffd700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48100d60 0x7f3b4819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b477fe700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b480ffb60 0x7f3b48069bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800b480 tx=0x7f3b3800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b44ff9700 1 -- 172.21.15.63:0/1431920690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b380118c0 con 0x7f3b480ffb60 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b44ff9700 1 -- 172.21.15.63:0/1431920690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b38010380 con 0x7f3b480ffb60 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.905+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4819be10 con 0x7f3b480ffb60 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.906+0000 7f3b44ff9700 1 -- 172.21.15.63:0/1431920690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b3800f4f0 con 0x7f3b480ffb60 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.906+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4819c350 con 0x7f3b480ffb60 2024-05-09T06:35:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.907+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4818f780 con 0x7f3b480ffb60 2024-05-09T06:35:58.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.908+0000 7f3b44ff9700 1 -- 172.21.15.63:0/1431920690 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3b380104f0 con 0x7f3b480ffb60 2024-05-09T06:35:58.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.908+0000 7f3b44ff9700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3b3006cce0 0x7f3b3006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:58.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.908+0000 7f3b44ff9700 1 -- 172.21.15.63:0/1431920690 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3b3808bea0 con 0x7f3b480ffb60 2024-05-09T06:35:58.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.908+0000 7f3b46ffd700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3b3006cce0 0x7f3b3006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:58.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.909+0000 7f3b46ffd700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3b3006cce0 0x7f3b3006f1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0051c0 tx=0x7f3b3c005920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:58.914+0000 7f3b44ff9700 1 -- 172.21.15.63:0/1431920690 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3b38059c90 con 0x7f3b480ffb60 2024-05-09T06:35:59.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.153+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b4802ce50 con 0x7f3b3006cce0 2024-05-09T06:35:59.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.154+0000 7f3b44ff9700 1 -- 172.21.15.63:0/1431920690 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f3b4802ce50 con 0x7f3b3006cce0 2024-05-09T06:35:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.156+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3b3006cce0 msgr2=0x7f3b3006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.156+0000 7f3b4e136700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3b3006cce0 0x7f3b3006f1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0051c0 tx=0x7f3b3c005920 comp rx=0 tx=0).stop 2024-05-09T06:35:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.156+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b480ffb60 msgr2=0x7f3b48069bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.156+0000 7f3b4e136700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b480ffb60 0x7f3b48069bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800b480 tx=0x7f3b3800b840 comp rx=0 tx=0).stop 2024-05-09T06:35:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.157+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 shutdown_connections 2024-05-09T06:35:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.157+0000 7f3b4e136700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b480ffb60 0x7f3b48069bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.157+0000 7f3b4e136700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f3b3006cce0 0x7f3b3006f1a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.157+0000 7f3b4e136700 1 --2- 172.21.15.63:0/1431920690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48100d60 0x7f3b4819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.157+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 >> 172.21.15.63:0/1431920690 conn(0x7f3b480fb0c0 msgr2=0x7f3b481043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.157+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 shutdown_connections 2024-05-09T06:35:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.157+0000 7f3b4e136700 1 -- 172.21.15.63:0/1431920690 wait complete. 2024-05-09T06:35:59.149 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:35:59.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.282+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/4112058404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 msgr2=0x7fc5b8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.282+0000 7fc5bdeb4700 1 --2- 172.21.15.63:0/4112058404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 0x7fc5b8101200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009910 tx=0x7fc5a0009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:59.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.282+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/4112058404 shutdown_connections 2024-05-09T06:35:59.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.282+0000 7fc5bdeb4700 1 --2- 172.21.15.63:0/4112058404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 0x7fc5b8101200 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.282+0000 7fc5bdeb4700 1 --2- 172.21.15.63:0/4112058404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b80ffba0 0x7fc5b80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.282+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/4112058404 >> 172.21.15.63:0/4112058404 conn(0x7fc5b80fb120 msgr2=0x7fc5b80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:59.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.282+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/4112058404 shutdown_connections 2024-05-09T06:35:59.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.282+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/4112058404 wait complete. 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5bdeb4700 1 Processor -- start 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5bdeb4700 1 -- start start 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5bdeb4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b80ffba0 0x7fc5b8103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5bdeb4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 0x7fc5b8102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5bdeb4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b8104000 con 0x7fc5b8100da0 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5bdeb4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b81026c0 con 0x7fc5b80ffba0 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5aeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 0x7fc5b8102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5aeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 0x7fc5b8102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36226/0 (socket says 172.21.15.63:36226) 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5aeffd700 1 -- 172.21.15.63:0/3614560492 learned_addr learned my addr 172.21.15.63:0/3614560492 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.283+0000 7fc5b77fe700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b80ffba0 0x7fc5b8103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.284+0000 7fc5aeffd700 1 -- 172.21.15.63:0/3614560492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b80ffba0 msgr2=0x7fc5b8103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.284+0000 7fc5aeffd700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b80ffba0 0x7fc5b8103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.284+0000 7fc5aeffd700 1 -- 172.21.15.63:0/3614560492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a00095f0 con 0x7fc5b8100da0 2024-05-09T06:35:59.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.284+0000 7fc5aeffd700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 0x7fc5b8102180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000bd70 tx=0x7fc5a000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:59.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.284+0000 7fc5b57fa700 1 -- 172.21.15.63:0/3614560492 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a001d070 con 0x7fc5b8100da0 2024-05-09T06:35:59.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.284+0000 7fc5b57fa700 1 -- 172.21.15.63:0/3614560492 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5a000f460 con 0x7fc5b8100da0 2024-05-09T06:35:59.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.284+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b8102940 con 0x7fc5b8100da0 2024-05-09T06:35:59.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.284+0000 7fc5b57fa700 1 -- 172.21.15.63:0/3614560492 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a0021640 con 0x7fc5b8100da0 2024-05-09T06:35:59.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.284+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b8102e30 con 0x7fc5b8100da0 2024-05-09T06:35:59.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.285+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b804ebb0 con 0x7fc5b8100da0 2024-05-09T06:35:59.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.286+0000 7fc5b57fa700 1 -- 172.21.15.63:0/3614560492 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc5a000fad0 con 0x7fc5b8100da0 2024-05-09T06:35:59.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.286+0000 7fc5b57fa700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fc5a406ccf0 0x7fc5a406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:59.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.287+0000 7fc5b57fa700 1 -- 172.21.15.63:0/3614560492 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc5a008d590 con 0x7fc5b8100da0 2024-05-09T06:35:59.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.287+0000 7fc5b77fe700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fc5a406ccf0 0x7fc5a406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:59.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.287+0000 7fc5b77fe700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fc5a406ccf0 0x7fc5a406f1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8100c00 tx=0x7fc5a800a3b0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:59.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.291+0000 7fc5b57fa700 1 -- 172.21.15.63:0/3614560492 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc5a005b410 con 0x7fc5b8100da0 2024-05-09T06:35:59.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.566+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5b8068d50 con 0x7fc5a406ccf0 2024-05-09T06:35:59.547 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:35:59 smithi063 ceph-mon[27171]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-09T06:35:59.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.567+0000 7fc5b57fa700 1 -- 172.21.15.63:0/3614560492 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fc5b8068d50 con 0x7fc5a406ccf0 2024-05-09T06:35:59.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fc5a406ccf0 msgr2=0x7fc5a406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fc5a406ccf0 0x7fc5a406f1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8100c00 tx=0x7fc5a800a3b0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 msgr2=0x7fc5b8102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 0x7fc5b8102180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000bd70 tx=0x7fc5a000bda0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 shutdown_connections 2024-05-09T06:35:59.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fc5a406ccf0 0x7fc5a406f1b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b80ffba0 0x7fc5b8103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 --2- 172.21.15.63:0/3614560492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b8100da0 0x7fc5b8102180 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 >> 172.21.15.63:0/3614560492 conn(0x7fc5b80fb120 msgr2=0x7fc5b8068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:59.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 shutdown_connections 2024-05-09T06:35:59.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.570+0000 7fc5bdeb4700 1 -- 172.21.15.63:0/3614560492 wait complete. 2024-05-09T06:35:59.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:35:59 smithi193 ceph-mon[35141]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-09T06:35:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.694+0000 7fe6678d2700 1 -- 172.21.15.63:0/2630627307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 msgr2=0x7fe6600fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.694+0000 7fe6678d2700 1 --2- 172.21.15.63:0/2630627307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 0x7fe6600fff50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009910 tx=0x7fe65c009c20 comp rx=0 tx=0).stop 2024-05-09T06:35:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.694+0000 7fe6678d2700 1 -- 172.21.15.63:0/2630627307 shutdown_connections 2024-05-09T06:35:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.694+0000 7fe6678d2700 1 --2- 172.21.15.63:0/2630627307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe660100d30 0x7fe660101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.694+0000 7fe6678d2700 1 --2- 172.21.15.63:0/2630627307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 0x7fe6600fff50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.694+0000 7fe6678d2700 1 -- 172.21.15.63:0/2630627307 >> 172.21.15.63:0/2630627307 conn(0x7fe6600fb0d0 msgr2=0x7fe6600fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.695+0000 7fe6678d2700 1 -- 172.21.15.63:0/2630627307 shutdown_connections 2024-05-09T06:35:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.695+0000 7fe6678d2700 1 -- 172.21.15.63:0/2630627307 wait complete. 2024-05-09T06:35:59.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.695+0000 7fe6678d2700 1 Processor -- start 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.695+0000 7fe6678d2700 1 -- start start 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe6678d2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 0x7fe6601039b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe6678d2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe660100d30 0x7fe660102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe6678d2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660103fd0 con 0x7fe6600ffb30 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe6678d2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660102650 con 0x7fe660100d30 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe66566e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 0x7fe6601039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe66566e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 0x7fe6601039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36248/0 (socket says 172.21.15.63:36248) 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe66566e700 1 -- 172.21.15.63:0/1318494016 learned_addr learned my addr 172.21.15.63:0/1318494016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe664e6d700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe660100d30 0x7fe660102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe66566e700 1 -- 172.21.15.63:0/1318494016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe660100d30 msgr2=0x7fe660102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe66566e700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe660100d30 0x7fe660102110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe66566e700 1 -- 172.21.15.63:0/1318494016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0095f0 con 0x7fe6600ffb30 2024-05-09T06:35:59.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.696+0000 7fe66566e700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 0x7fe6601039b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00bd70 tx=0x7fe65c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:59.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.697+0000 7fe6567fc700 1 -- 172.21.15.63:0/1318494016 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe65c01d070 con 0x7fe6600ffb30 2024-05-09T06:35:59.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.697+0000 7fe6567fc700 1 -- 172.21.15.63:0/1318494016 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe65c00f460 con 0x7fe6600ffb30 2024-05-09T06:35:59.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.697+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe660102880 con 0x7fe6600ffb30 2024-05-09T06:35:59.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.697+0000 7fe6567fc700 1 -- 172.21.15.63:0/1318494016 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe65c021620 con 0x7fe6600ffb30 2024-05-09T06:35:59.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.697+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe660102df0 con 0x7fe6600ffb30 2024-05-09T06:35:59.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.698+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66004ebb0 con 0x7fe6600ffb30 2024-05-09T06:35:59.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.699+0000 7fe6567fc700 1 -- 172.21.15.63:0/1318494016 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe65c00fa70 con 0x7fe6600ffb30 2024-05-09T06:35:59.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.699+0000 7fe6567fc700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe64c06ccf0 0x7fe64c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:59.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.699+0000 7fe6567fc700 1 -- 172.21.15.63:0/1318494016 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe65c08d5d0 con 0x7fe6600ffb30 2024-05-09T06:35:59.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.699+0000 7fe664e6d700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe64c06ccf0 0x7fe64c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:59.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.700+0000 7fe664e6d700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe64c06ccf0 0x7fe64c06f1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe650006fd0 tx=0x7fe650008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:59.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.708+0000 7fe6567fc700 1 -- 172.21.15.63:0/1318494016 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe65c05b3d0 con 0x7fe6600ffb30 2024-05-09T06:35:59.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.958+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6600fd290 con 0x7fe64c06ccf0 2024-05-09T06:35:59.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.970+0000 7fe6567fc700 1 -- 172.21.15.63:0/1318494016 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe6600fd290 con 0x7fe64c06ccf0 2024-05-09T06:35:59.950 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (4m) 24s ago 6m 25.3M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (6m) 24s ago 6m 7465k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (5m) 28s ago 5m 7382k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (6m) 24s ago 6m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (5m) 28s ago 5m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (4m) 24s ago 6m 87.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (50s) 24s ago 49s 11.7M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (43s) 24s ago 42s 14.7M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (47s) 28s ago 46s 16.5M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (40s) 28s ago 39s 13.2M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:9283,8765,8443 running (7m) 24s ago 7m 500M - 18.2.2-1297-g1758e8ea 196422f1b641 a66e4acc828f 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (5m) 28s ago 5m 450M - 18.2.2-1297-g1758e8ea 196422f1b641 b87b5901e715 2024-05-09T06:35:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (7m) 24s ago 8m 45.4M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 7d88727e4bbb 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (5m) 28s ago 5m 37.0M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (6m) 24s ago 6m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (5m) 28s ago 5m 15.4M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (4m) 24s ago 4m 47.2M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (4m) 24s ago 4m 48.3M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (3m) 24s ago 3m 46.4M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (3m) 28s ago 3m 46.7M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (2m) 28s ago 2m 51.7M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (2m) 28s ago 2m 53.1M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:35:59.952 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (4m) 24s ago 5m 58.2M - 2.43.0 a07b618ecd1d 9f4deb8cc8b4 2024-05-09T06:35:59.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe64c06ccf0 msgr2=0x7fe64c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe64c06ccf0 0x7fe64c06f1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe650006fd0 tx=0x7fe650008040 comp rx=0 tx=0).stop 2024-05-09T06:35:59.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 msgr2=0x7fe6601039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:59.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 0x7fe6601039b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00bd70 tx=0x7fe65c00be50 comp rx=0 tx=0).stop 2024-05-09T06:35:59.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 shutdown_connections 2024-05-09T06:35:59.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6600ffb30 0x7fe6601039b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe64c06ccf0 0x7fe64c06f1b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 --2- 172.21.15.63:0/1318494016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe660100d30 0x7fe660102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:59.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 >> 172.21.15.63:0/1318494016 conn(0x7fe6600fb0d0 msgr2=0x7fe6600fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:59.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 shutdown_connections 2024-05-09T06:35:59.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:35:59.972+0000 7fe6678d2700 1 -- 172.21.15.63:0/1318494016 wait complete. 2024-05-09T06:36:00.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.098+0000 7fbfe010a700 1 -- 172.21.15.63:0/2329812232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 msgr2=0x7fbfd81064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.098+0000 7fbfe010a700 1 --2- 172.21.15.63:0/2329812232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 0x7fbfd81064b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009910 tx=0x7fbfc8009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:00.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.098+0000 7fbfe010a700 1 -- 172.21.15.63:0/2329812232 shutdown_connections 2024-05-09T06:36:00.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.098+0000 7fbfe010a700 1 --2- 172.21.15.63:0/2329812232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 0x7fbfd81064b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.098+0000 7fbfe010a700 1 --2- 172.21.15.63:0/2329812232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd8101790 0x7fbfd8103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.098+0000 7fbfe010a700 1 -- 172.21.15.63:0/2329812232 >> 172.21.15.63:0/2329812232 conn(0x7fbfd80fb120 msgr2=0x7fbfd80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.098+0000 7fbfe010a700 1 -- 172.21.15.63:0/2329812232 shutdown_connections 2024-05-09T06:36:00.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.098+0000 7fbfe010a700 1 -- 172.21.15.63:0/2329812232 wait complete. 2024-05-09T06:36:00.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.099+0000 7fbfe010a700 1 Processor -- start 2024-05-09T06:36:00.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.099+0000 7fbfe010a700 1 -- start start 2024-05-09T06:36:00.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.099+0000 7fbfe010a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd8101790 0x7fbfd81966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.099+0000 7fbfe010a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 0x7fbfd8196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.099+0000 7fbfe010a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd8197240 con 0x7fbfd81040c0 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.099+0000 7fbfe010a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd8197380 con 0x7fbfd8101790 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.099+0000 7fbfdd6a5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 0x7fbfd8196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.099+0000 7fbfdd6a5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 0x7fbfd8196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36266/0 (socket says 172.21.15.63:36266) 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.099+0000 7fbfdd6a5700 1 -- 172.21.15.63:0/4037719522 learned_addr learned my addr 172.21.15.63:0/4037719522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfddea6700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd8101790 0x7fbfd81966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfdd6a5700 1 -- 172.21.15.63:0/4037719522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd8101790 msgr2=0x7fbfd81966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfdd6a5700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd8101790 0x7fbfd81966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfdd6a5700 1 -- 172.21.15.63:0/4037719522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80095f0 con 0x7fbfd81040c0 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfddea6700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd8101790 0x7fbfd81966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfdd6a5700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 0x7fbfd8196c20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8006010 tx=0x7fbfc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfceffd700 1 -- 172.21.15.63:0/4037719522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc801d070 con 0x7fbfd81040c0 2024-05-09T06:36:00.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfceffd700 1 -- 172.21.15.63:0/4037719522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfc800f460 con 0x7fbfd81040c0 2024-05-09T06:36:00.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfceffd700 1 -- 172.21.15.63:0/4037719522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc8021640 con 0x7fbfd81040c0 2024-05-09T06:36:00.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd819bd80 con 0x7fbfd81040c0 2024-05-09T06:36:00.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.100+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd819c230 con 0x7fbfd81040c0 2024-05-09T06:36:00.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.101+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd804ebb0 con 0x7fbfd81040c0 2024-05-09T06:36:00.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.102+0000 7fbfceffd700 1 -- 172.21.15.63:0/4037719522 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbfc800fab0 con 0x7fbfd81040c0 2024-05-09T06:36:00.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.102+0000 7fbfceffd700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbfc406cce0 0x7fbfc406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.102+0000 7fbfceffd700 1 -- 172.21.15.63:0/4037719522 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbfc808d580 con 0x7fbfd81040c0 2024-05-09T06:36:00.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.102+0000 7fbfddea6700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbfc406cce0 0x7fbfc406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.103+0000 7fbfddea6700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbfc406cce0 0x7fbfc406f1a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4007900 tx=0x7fbfd4008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.109+0000 7fbfceffd700 1 -- 172.21.15.63:0/4037719522 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbfc805b400 con 0x7fbfd81040c0 2024-05-09T06:36:00.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:00 smithi063 ceph-mon[27171]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:00.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.430+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbfd8109a30 con 0x7fbfd81040c0 2024-05-09T06:36:00.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.431+0000 7fbfceffd700 1 -- 172.21.15.63:0/4037719522 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fbfc805af90 con 0x7fbfd81040c0 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:36:00.412 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:36:00.413 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 14 2024-05-09T06:36:00.413 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:36:00.413 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbfc406cce0 msgr2=0x7fbfc406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbfc406cce0 0x7fbfc406f1a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4007900 tx=0x7fbfd4008040 comp rx=0 tx=0).stop 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 msgr2=0x7fbfd8196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 0x7fbfd8196c20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8006010 tx=0x7fbfc800be20 comp rx=0 tx=0).stop 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 shutdown_connections 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fbfc406cce0 0x7fbfc406f1a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd8101790 0x7fbfd81966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 --2- 172.21.15.63:0/4037719522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd81040c0 0x7fbfd8196c20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 >> 172.21.15.63:0/4037719522 conn(0x7fbfd80fb120 msgr2=0x7fbfd80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 shutdown_connections 2024-05-09T06:36:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.433+0000 7fbfe010a700 1 -- 172.21.15.63:0/4037719522 wait complete. 2024-05-09T06:36:00.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.549+0000 7f4314d7f700 1 -- 172.21.15.63:0/727655316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 msgr2=0x7f43100ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.549+0000 7f4314d7f700 1 --2- 172.21.15.63:0/727655316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 0x7f43100ff2c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4300009910 tx=0x7f4300009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:00.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.549+0000 7f4314d7f700 1 -- 172.21.15.63:0/727655316 shutdown_connections 2024-05-09T06:36:00.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.549+0000 7f4314d7f700 1 --2- 172.21.15.63:0/727655316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4310101f20 0x7f4310102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.549+0000 7f4314d7f700 1 --2- 172.21.15.63:0/727655316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 0x7f43100ff2c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.549+0000 7f4314d7f700 1 -- 172.21.15.63:0/727655316 >> 172.21.15.63:0/727655316 conn(0x7f43100fa800 msgr2=0x7f43100fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.549+0000 7f4314d7f700 1 -- 172.21.15.63:0/727655316 shutdown_connections 2024-05-09T06:36:00.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.549+0000 7f4314d7f700 1 -- 172.21.15.63:0/727655316 wait complete. 2024-05-09T06:36:00.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.550+0000 7f4314d7f700 1 Processor -- start 2024-05-09T06:36:00.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.550+0000 7f4314d7f700 1 -- start start 2024-05-09T06:36:00.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.550+0000 7f4314d7f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 0x7f43101017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.550+0000 7f4314d7f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4310101f20 0x7f43100ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.550+0000 7f4314d7f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4310100410 con 0x7f43100feea0 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.550+0000 7f4314d7f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4310100550 con 0x7f4310101f20 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f430e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 0x7f43101017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f430e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 0x7f43101017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36296/0 (socket says 172.21.15.63:36296) 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f430e59c700 1 -- 172.21.15.63:0/3731260142 learned_addr learned my addr 172.21.15.63:0/3731260142 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f430dd9b700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4310101f20 0x7f43100ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f430e59c700 1 -- 172.21.15.63:0/3731260142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4310101f20 msgr2=0x7f43100ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f430e59c700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4310101f20 0x7f43100ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f430e59c700 1 -- 172.21.15.63:0/3731260142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43000095f0 con 0x7f43100feea0 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f430e59c700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 0x7f43101017e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4300005850 tx=0x7f430000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f42ff7fe700 1 -- 172.21.15.63:0/3731260142 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f430001d070 con 0x7f43100feea0 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43101007d0 con 0x7f43100feea0 2024-05-09T06:36:00.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f42ff7fe700 1 -- 172.21.15.63:0/3731260142 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4300005dc0 con 0x7f43100feea0 2024-05-09T06:36:00.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f42ff7fe700 1 -- 172.21.15.63:0/3731260142 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4300017400 con 0x7f43100feea0 2024-05-09T06:36:00.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.551+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4310100d40 con 0x7f43100feea0 2024-05-09T06:36:00.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.553+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4310190740 con 0x7f43100feea0 2024-05-09T06:36:00.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.553+0000 7f42ff7fe700 1 -- 172.21.15.63:0/3731260142 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f430000f6b0 con 0x7f43100feea0 2024-05-09T06:36:00.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.554+0000 7f42ff7fe700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f42f806cce0 0x7f42f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.554+0000 7f42ff7fe700 1 -- 172.21.15.63:0/3731260142 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f430008d560 con 0x7f43100feea0 2024-05-09T06:36:00.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.554+0000 7f430dd9b700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f42f806cce0 0x7f42f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.555+0000 7f430dd9b700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f42f806cce0 0x7f42f806f1a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4304005950 tx=0x7f430400b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.559+0000 7f42ff7fe700 1 -- 172.21.15.63:0/3731260142 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f430005b360 con 0x7f43100feea0 2024-05-09T06:36:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:00 smithi193 ceph-mon[35141]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:00.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.839+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4310100d40 con 0x7f43100feea0 2024-05-09T06:36:00.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.839+0000 7f42ff7fe700 1 -- 172.21.15.63:0/3731260142 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f4310100d40 con 0x7f43100feea0 2024-05-09T06:36:00.820 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:36:00.820 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:36:00.820 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:36:00.820 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:36:00.820 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:36:00.820 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:36:00.820 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:36:00.820 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:36:00.820 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:36:00.821 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:36:00.822 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:36:00.823 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:36:00.823 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:36:00.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.842+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f42f806cce0 msgr2=0x7f42f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.842+0000 7f4314d7f700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f42f806cce0 0x7f42f806f1a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4304005950 tx=0x7f430400b410 comp rx=0 tx=0).stop 2024-05-09T06:36:00.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.842+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 msgr2=0x7f43101017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.842+0000 7f4314d7f700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 0x7f43101017e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4300005850 tx=0x7f430000bd70 comp rx=0 tx=0).stop 2024-05-09T06:36:00.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.843+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 shutdown_connections 2024-05-09T06:36:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.843+0000 7f4314d7f700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43100feea0 0x7f43101017e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.843+0000 7f4314d7f700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f42f806cce0 0x7f42f806f1a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.843+0000 7f4314d7f700 1 --2- 172.21.15.63:0/3731260142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4310101f20 0x7f43100ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.843+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 >> 172.21.15.63:0/3731260142 conn(0x7f43100fa800 msgr2=0x7f4310105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.843+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 shutdown_connections 2024-05-09T06:36:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.843+0000 7f4314d7f700 1 -- 172.21.15.63:0/3731260142 wait complete. 2024-05-09T06:36:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:36:00.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.960+0000 7f639c08a700 1 -- 172.21.15.63:0/3449736053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394100c20 msgr2=0x7f6394101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.960+0000 7f639c08a700 1 --2- 172.21.15.63:0/3449736053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394100c20 0x7f6394101040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6390009910 tx=0x7f6390009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:00.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.960+0000 7f639c08a700 1 -- 172.21.15.63:0/3449736053 shutdown_connections 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.960+0000 7f639c08a700 1 --2- 172.21.15.63:0/3449736053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6394101e20 0x7f6394102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.960+0000 7f639c08a700 1 --2- 172.21.15.63:0/3449736053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394100c20 0x7f6394101040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.960+0000 7f639c08a700 1 -- 172.21.15.63:0/3449736053 >> 172.21.15.63:0/3449736053 conn(0x7f63940fc200 msgr2=0x7f63940fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.960+0000 7f639c08a700 1 -- 172.21.15.63:0/3449736053 shutdown_connections 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.960+0000 7f639c08a700 1 -- 172.21.15.63:0/3449736053 wait complete. 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.961+0000 7f639c08a700 1 Processor -- start 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.961+0000 7f639c08a700 1 -- start start 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.961+0000 7f639c08a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6394100c20 0x7f6394196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.961+0000 7f639c08a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394101e20 0x7f6394196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.961+0000 7f639c08a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63941971a0 con 0x7f6394101e20 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.961+0000 7f639c08a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63941972e0 con 0x7f6394100c20 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.961+0000 7f6399625700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394101e20 0x7f6394196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.961+0000 7f6399625700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394101e20 0x7f6394196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36314/0 (socket says 172.21.15.63:36314) 2024-05-09T06:36:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.961+0000 7f6399625700 1 -- 172.21.15.63:0/652140484 learned_addr learned my addr 172.21.15.63:0/652140484 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f6399e26700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6394100c20 0x7f6394196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f6399625700 1 -- 172.21.15.63:0/652140484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6394100c20 msgr2=0x7f6394196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f6399625700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6394100c20 0x7f6394196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f6399625700 1 -- 172.21.15.63:0/652140484 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63900095f0 con 0x7f6394101e20 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f6399e26700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6394100c20 0x7f6394196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f6399625700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394101e20 0x7f6394196b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f638400d8d0 tx=0x7f638400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f638affd700 1 -- 172.21.15.63:0/652140484 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6384009940 con 0x7f6394101e20 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f638affd700 1 -- 172.21.15.63:0/652140484 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6384010460 con 0x7f6394101e20 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639419bd40 con 0x7f6394101e20 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f638affd700 1 -- 172.21.15.63:0/652140484 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f638400f5f0 con 0x7f6394101e20 2024-05-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.962+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639419c310 con 0x7f6394101e20 2024-05-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.963+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63941907a0 con 0x7f6394101e20 2024-05-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.964+0000 7f638affd700 1 -- 172.21.15.63:0/652140484 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6384009aa0 con 0x7f6394101e20 2024-05-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.964+0000 7f638affd700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f638006cce0 0x7f638006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.964+0000 7f638affd700 1 -- 172.21.15.63:0/652140484 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f638408c2a0 con 0x7f6394101e20 2024-05-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.964+0000 7f6399e26700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f638006cce0 0x7f638006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.965+0000 7f6399e26700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f638006cce0 0x7f638006f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63900051c0 tx=0x7f6390005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:00.969+0000 7f638affd700 1 -- 172.21.15.63:0/652140484 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f638405a0a0 con 0x7f6394101e20 2024-05-09T06:36:01.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.221+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f639402ce50 con 0x7f638006cce0 2024-05-09T06:36:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.222+0000 7f638affd700 1 -- 172.21.15.63:0/652140484 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f639402ce50 con 0x7f638006cce0 2024-05-09T06:36:01.202 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:36:01.202 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:36:01.203 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:36:01.203 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:36:01.203 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-05-09T06:36:01.203 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "", 2024-05-09T06:36:01.203 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc image", 2024-05-09T06:36:01.203 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:36:01.203 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:36:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f638006cce0 msgr2=0x7f638006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f638006cce0 0x7f638006f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63900051c0 tx=0x7f6390005960 comp rx=0 tx=0).stop 2024-05-09T06:36:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394101e20 msgr2=0x7f6394196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394101e20 0x7f6394196b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f638400d8d0 tx=0x7f638400dc90 comp rx=0 tx=0).stop 2024-05-09T06:36:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 shutdown_connections 2024-05-09T06:36:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f638006cce0 0x7f638006f1a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6394100c20 0x7f6394196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 --2- 172.21.15.63:0/652140484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6394101e20 0x7f6394196b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 >> 172.21.15.63:0/652140484 conn(0x7f63940fc200 msgr2=0x7f6394105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:01.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 shutdown_connections 2024-05-09T06:36:01.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.224+0000 7f639c08a700 1 -- 172.21.15.63:0/652140484 wait complete. 2024-05-09T06:36:01.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.343+0000 7f26ad9ab700 1 -- 172.21.15.63:0/3054455981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 msgr2=0x7f26a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.343+0000 7f26ad9ab700 1 --2- 172.21.15.63:0/3054455981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 0x7f26a8101200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f269c009910 tx=0x7f269c009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:01.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.343+0000 7f26ad9ab700 1 -- 172.21.15.63:0/3054455981 shutdown_connections 2024-05-09T06:36:01.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.343+0000 7f26ad9ab700 1 --2- 172.21.15.63:0/3054455981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 0x7f26a8101200 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.343+0000 7f26ad9ab700 1 --2- 172.21.15.63:0/3054455981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26a80ffba0 0x7f26a80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.343+0000 7f26ad9ab700 1 -- 172.21.15.63:0/3054455981 >> 172.21.15.63:0/3054455981 conn(0x7f26a80fb120 msgr2=0x7f26a80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:01.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.343+0000 7f26ad9ab700 1 -- 172.21.15.63:0/3054455981 shutdown_connections 2024-05-09T06:36:01.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.343+0000 7f26ad9ab700 1 -- 172.21.15.63:0/3054455981 wait complete. 2024-05-09T06:36:01.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26ad9ab700 1 Processor -- start 2024-05-09T06:36:01.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26ad9ab700 1 -- start start 2024-05-09T06:36:01.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26ad9ab700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26a80ffba0 0x7f26a81966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26ad9ab700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 0x7f26a8196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26ad9ab700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a8197210 con 0x7f26a8100da0 2024-05-09T06:36:01.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26ad9ab700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a8197350 con 0x7f26a80ffba0 2024-05-09T06:36:01.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26a67fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 0x7f26a8196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26a67fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 0x7f26a8196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36324/0 (socket says 172.21.15.63:36324) 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26a67fc700 1 -- 172.21.15.63:0/831581533 learned_addr learned my addr 172.21.15.63:0/831581533 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.344+0000 7f26a6ffd700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26a80ffba0 0x7f26a81966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.345+0000 7f26a67fc700 1 -- 172.21.15.63:0/831581533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26a80ffba0 msgr2=0x7f26a81966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.345+0000 7f26a67fc700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26a80ffba0 0x7f26a81966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.345+0000 7f26a67fc700 1 -- 172.21.15.63:0/831581533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c0095f0 con 0x7f26a8100da0 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.345+0000 7f26a67fc700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 0x7f26a8196bf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f269c0092e0 tx=0x7f269c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.345+0000 7f26ac9a9700 1 -- 172.21.15.63:0/831581533 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f269c01d070 con 0x7f26a8100da0 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.345+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a819bd50 con 0x7f26a8100da0 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.345+0000 7f26ac9a9700 1 -- 172.21.15.63:0/831581533 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f269c00f460 con 0x7f26a8100da0 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.345+0000 7f26ac9a9700 1 -- 172.21.15.63:0/831581533 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f269c021620 con 0x7f26a8100da0 2024-05-09T06:36:01.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.345+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a819c240 con 0x7f26a8100da0 2024-05-09T06:36:01.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.346+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a804ebb0 con 0x7f26a8100da0 2024-05-09T06:36:01.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.347+0000 7f26ac9a9700 1 -- 172.21.15.63:0/831581533 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f269c00f5d0 con 0x7f26a8100da0 2024-05-09T06:36:01.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.347+0000 7f26ac9a9700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f269006cce0 0x7f269006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.348+0000 7f26ac9a9700 1 -- 172.21.15.63:0/831581533 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f269c08d4c0 con 0x7f26a8100da0 2024-05-09T06:36:01.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.348+0000 7f26a6ffd700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f269006cce0 0x7f269006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.348+0000 7f26a6ffd700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f269006cce0 0x7f269006f1a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26a8100c00 tx=0x7f269800b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:01.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.353+0000 7f26ac9a9700 1 -- 172.21.15.63:0/831581533 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f269c05b2c0 con 0x7f26a8100da0 2024-05-09T06:36:01.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:01 smithi193 ceph-mon[35141]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:01.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:01 smithi193 ceph-mon[35141]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-09T06:36:01.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:01 smithi193 ceph-mon[35141]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:01.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:01 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/4037719522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:36:01.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:01 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3731260142' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:36:01.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:01 smithi063 ceph-mon[27171]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:01.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:01 smithi063 ceph-mon[27171]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-09T06:36:01.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:01 smithi063 ceph-mon[27171]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:01.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:01 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/4037719522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:36:01.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:01 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3731260142' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:36:01.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.675+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f26a8108700 con 0x7f26a8100da0 2024-05-09T06:36:01.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.675+0000 7f26ac9a9700 1 -- 172.21.15.63:0/831581533 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f269c05ae50 con 0x7f26a8100da0 2024-05-09T06:36:01.656 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f269006cce0 msgr2=0x7f269006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f269006cce0 0x7f269006f1a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26a8100c00 tx=0x7f269800b410 comp rx=0 tx=0).stop 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 msgr2=0x7f26a8196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 0x7f26a8196bf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f269c0092e0 tx=0x7f269c00bd70 comp rx=0 tx=0).stop 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 shutdown_connections 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f269006cce0 0x7f269006f1a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26a80ffba0 0x7f26a81966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 --2- 172.21.15.63:0/831581533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8100da0 0x7f26a8196bf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 >> 172.21.15.63:0/831581533 conn(0x7f26a80fb120 msgr2=0x7f26a8068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 shutdown_connections 2024-05-09T06:36:01.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:01.678+0000 7f26ad9ab700 1 -- 172.21.15.63:0/831581533 wait complete. 2024-05-09T06:36:02.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:02 smithi063 ceph-mon[27171]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:02.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:02 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/831581533' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:36:02.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:02 smithi193 ceph-mon[35141]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:02.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:02 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/831581533' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:36:03.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:03 smithi063 ceph-mon[27171]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-09T06:36:03.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:03 smithi193 ceph-mon[35141]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-09T06:36:05.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:05 smithi063 ceph-mon[27171]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-09T06:36:05.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:05 smithi193 ceph-mon[35141]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-09T06:36:07.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:07 smithi063 ceph-mon[27171]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-05-09T06:36:07.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:07 smithi193 ceph-mon[35141]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-05-09T06:36:09.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:09 smithi063 ceph-mon[27171]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-05-09T06:36:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:09 smithi193 ceph-mon[35141]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-05-09T06:36:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:11 smithi193 ceph-mon[35141]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-09T06:36:11.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:11 smithi063 ceph-mon[27171]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-09T06:36:12.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:12 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:36:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:12 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:36:13.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:13 smithi063 ceph-mon[27171]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:13.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:13 smithi193 ceph-mon[35141]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:15.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:15 smithi063 ceph-mon[27171]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:15.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:15 smithi193 ceph-mon[35141]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:17.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:17 smithi063 ceph-mon[27171]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:17.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:17 smithi193 ceph-mon[35141]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:19.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:19 smithi063 ceph-mon[27171]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:19.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:19 smithi193 ceph-mon[35141]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:21.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:21 smithi063 ceph-mon[27171]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:21.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:21 smithi193 ceph-mon[35141]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:23 smithi063 ceph-mon[27171]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:23.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:23 smithi193 ceph-mon[35141]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:25.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:25 smithi063 ceph-mon[27171]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:25.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:25 smithi193 ceph-mon[35141]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:27.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:27 smithi063 ceph-mon[27171]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:27 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:36:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:27 smithi193 ceph-mon[35141]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:27 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:36:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:29 smithi063 ceph-mon[27171]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:29.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:29 smithi193 ceph-mon[35141]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:31.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:31 smithi063 ceph-mon[27171]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:31.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:31 smithi193 ceph-mon[35141]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:31.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.814+0000 7ff5d5501700 1 -- 172.21.15.63:0/265096327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0104100 msgr2=0x7ff5d01064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.814+0000 7ff5d5501700 1 --2- 172.21.15.63:0/265096327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0104100 0x7ff5d01064f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009910 tx=0x7ff5c4009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.814+0000 7ff5d5501700 1 -- 172.21.15.63:0/265096327 shutdown_connections 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.814+0000 7ff5d5501700 1 --2- 172.21.15.63:0/265096327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0104100 0x7ff5d01064f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.814+0000 7ff5d5501700 1 --2- 172.21.15.63:0/265096327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d01017d0 0x7ff5d0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.814+0000 7ff5d5501700 1 -- 172.21.15.63:0/265096327 >> 172.21.15.63:0/265096327 conn(0x7ff5d00fb120 msgr2=0x7ff5d00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.814+0000 7ff5d5501700 1 -- 172.21.15.63:0/265096327 shutdown_connections 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.814+0000 7ff5d5501700 1 -- 172.21.15.63:0/265096327 wait complete. 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.815+0000 7ff5d5501700 1 Processor -- start 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.815+0000 7ff5d5501700 1 -- start start 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.815+0000 7ff5d5501700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d01017d0 0x7ff5d0196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.815+0000 7ff5d5501700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d0104100 0x7ff5d0196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.815+0000 7ff5d5501700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d01971d0 con 0x7ff5d01017d0 2024-05-09T06:36:31.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.815+0000 7ff5d5501700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d0197310 con 0x7ff5d0104100 2024-05-09T06:36:31.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.816+0000 7ff5ceffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d01017d0 0x7ff5d0196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.816+0000 7ff5ceffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d01017d0 0x7ff5d0196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55930/0 (socket says 172.21.15.63:55930) 2024-05-09T06:36:31.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.816+0000 7ff5ceffd700 1 -- 172.21.15.63:0/2285661857 learned_addr learned my addr 172.21.15.63:0/2285661857 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:31.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.816+0000 7ff5ce7fc700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d0104100 0x7ff5d0196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.816+0000 7ff5ceffd700 1 -- 172.21.15.63:0/2285661857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d0104100 msgr2=0x7ff5d0196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.816+0000 7ff5ceffd700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d0104100 0x7ff5d0196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.816+0000 7ff5ceffd700 1 -- 172.21.15.63:0/2285661857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c40095f0 con 0x7ff5d01017d0 2024-05-09T06:36:31.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.816+0000 7ff5ceffd700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d01017d0 0x7ff5d0196670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e920 tx=0x7ff5c000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.817+0000 7ff5b7fff700 1 -- 172.21.15.63:0/2285661857 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c000c9d0 con 0x7ff5d01017d0 2024-05-09T06:36:31.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.817+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d019bd70 con 0x7ff5d01017d0 2024-05-09T06:36:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.817+0000 7ff5b7fff700 1 -- 172.21.15.63:0/2285661857 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5c0010430 con 0x7ff5d01017d0 2024-05-09T06:36:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.817+0000 7ff5b7fff700 1 -- 172.21.15.63:0/2285661857 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c0018630 con 0x7ff5d01017d0 2024-05-09T06:36:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.817+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d019c260 con 0x7ff5d01017d0 2024-05-09T06:36:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.818+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d004ebb0 con 0x7ff5d01017d0 2024-05-09T06:36:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.819+0000 7ff5b7fff700 1 -- 172.21.15.63:0/2285661857 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff5c00105a0 con 0x7ff5d01017d0 2024-05-09T06:36:31.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.820+0000 7ff5b7fff700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff5b806cce0 0x7ff5b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.820+0000 7ff5b7fff700 1 -- 172.21.15.63:0/2285661857 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff5c0014070 con 0x7ff5d01017d0 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.820+0000 7ff5ce7fc700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff5b806cce0 0x7ff5b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.821+0000 7ff5ce7fc700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff5b806cce0 0x7ff5b806f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400b800 tx=0x7ff5c4009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:31.828+0000 7ff5b7fff700 1 -- 172.21.15.63:0/2285661857 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff5c0052c50 con 0x7ff5d01017d0 2024-05-09T06:36:32.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.159+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5d0104ac0 con 0x7ff5b806cce0 2024-05-09T06:36:32.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.160+0000 7ff5b7fff700 1 -- 172.21.15.63:0/2285661857 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7ff5d0104ac0 con 0x7ff5b806cce0 2024-05-09T06:36:32.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff5b806cce0 msgr2=0x7ff5b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff5b806cce0 0x7ff5b806f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400b800 tx=0x7ff5c4009f70 comp rx=0 tx=0).stop 2024-05-09T06:36:32.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d01017d0 msgr2=0x7ff5d0196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d01017d0 0x7ff5d0196670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e920 tx=0x7ff5c000ec30 comp rx=0 tx=0).stop 2024-05-09T06:36:32.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 shutdown_connections 2024-05-09T06:36:32.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d01017d0 0x7ff5d0196670 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff5b806cce0 0x7ff5b806f1a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 --2- 172.21.15.63:0/2285661857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d0104100 0x7ff5d0196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 >> 172.21.15.63:0/2285661857 conn(0x7ff5d00fb120 msgr2=0x7ff5d00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 shutdown_connections 2024-05-09T06:36:32.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.163+0000 7ff5d5501700 1 -- 172.21.15.63:0/2285661857 wait complete. 2024-05-09T06:36:32.157 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:36:32.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.299+0000 7f6117b11700 1 -- 172.21.15.63:0/3159818166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6110100d20 msgr2=0x7f6110101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.299+0000 7f6117b11700 1 --2- 172.21.15.63:0/3159818166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6110100d20 0x7f6110101180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6100009910 tx=0x7f6100009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.300+0000 7f6117b11700 1 -- 172.21.15.63:0/3159818166 shutdown_connections 2024-05-09T06:36:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.300+0000 7f6117b11700 1 --2- 172.21.15.63:0/3159818166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6110100d20 0x7f6110101180 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.300+0000 7f6117b11700 1 --2- 172.21.15.63:0/3159818166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61100ffb20 0x7f61100fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.300+0000 7f6117b11700 1 -- 172.21.15.63:0/3159818166 >> 172.21.15.63:0/3159818166 conn(0x7f61100fb0c0 msgr2=0x7f61100fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.300+0000 7f6117b11700 1 -- 172.21.15.63:0/3159818166 shutdown_connections 2024-05-09T06:36:32.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.300+0000 7f6117b11700 1 -- 172.21.15.63:0/3159818166 wait complete. 2024-05-09T06:36:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.300+0000 7f6117b11700 1 Processor -- start 2024-05-09T06:36:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f6117b11700 1 -- start start 2024-05-09T06:36:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f6117b11700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100ffb20 0x7f6110196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f6117b11700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110100d20 0x7f6110196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f6117b11700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61101971d0 con 0x7f61100ffb20 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f6117b11700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6110197310 con 0x7f6110100d20 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f61158ad700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100ffb20 0x7f6110196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f61158ad700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100ffb20 0x7f6110196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55958/0 (socket says 172.21.15.63:55958) 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f61158ad700 1 -- 172.21.15.63:0/352547077 learned_addr learned my addr 172.21.15.63:0/352547077 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f61150ac700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110100d20 0x7f6110196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f61158ad700 1 -- 172.21.15.63:0/352547077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110100d20 msgr2=0x7f6110196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f61158ad700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110100d20 0x7f6110196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.301+0000 7f61158ad700 1 -- 172.21.15.63:0/352547077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61000095f0 con 0x7f61100ffb20 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.302+0000 7f61158ad700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100ffb20 0x7f6110196670 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f610c00e8f0 tx=0x7f610c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.302+0000 7f6106ffd700 1 -- 172.21.15.63:0/352547077 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f610c009790 con 0x7f61100ffb20 2024-05-09T06:36:32.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.302+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611019bd70 con 0x7f61100ffb20 2024-05-09T06:36:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.302+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611019c2e0 con 0x7f61100ffb20 2024-05-09T06:36:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.302+0000 7f6106ffd700 1 -- 172.21.15.63:0/352547077 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f610c00c810 con 0x7f61100ffb20 2024-05-09T06:36:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.302+0000 7f6106ffd700 1 -- 172.21.15.63:0/352547077 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f610c003cc0 con 0x7f61100ffb20 2024-05-09T06:36:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.303+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f40052f0 con 0x7f61100ffb20 2024-05-09T06:36:32.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.304+0000 7f6106ffd700 1 -- 172.21.15.63:0/352547077 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f610c00c980 con 0x7f61100ffb20 2024-05-09T06:36:32.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.304+0000 7f6106ffd700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f60fc06ccf0 0x7f60fc06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.304+0000 7f6106ffd700 1 -- 172.21.15.63:0/352547077 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f610c014070 con 0x7f61100ffb20 2024-05-09T06:36:32.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.305+0000 7f61150ac700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f60fc06ccf0 0x7f60fc06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.305+0000 7f61150ac700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f60fc06ccf0 0x7f60fc06f1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f610000b800 tx=0x7f6100009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.309+0000 7f6106ffd700 1 -- 172.21.15.63:0/352547077 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f610c05a3b0 con 0x7f61100ffb20 2024-05-09T06:36:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.627+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60f4000bc0 con 0x7f60fc06ccf0 2024-05-09T06:36:32.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.628+0000 7f6106ffd700 1 -- 172.21.15.63:0/352547077 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f60f4000bc0 con 0x7f60fc06ccf0 2024-05-09T06:36:32.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.631+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f60fc06ccf0 msgr2=0x7f60fc06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.631+0000 7f6117b11700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f60fc06ccf0 0x7f60fc06f1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f610000b800 tx=0x7f6100009f70 comp rx=0 tx=0).stop 2024-05-09T06:36:32.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.631+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100ffb20 msgr2=0x7f6110196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.631+0000 7f6117b11700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100ffb20 0x7f6110196670 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f610c00e8f0 tx=0x7f610c00ec00 comp rx=0 tx=0).stop 2024-05-09T06:36:32.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.631+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 shutdown_connections 2024-05-09T06:36:32.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.631+0000 7f6117b11700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61100ffb20 0x7f6110196670 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.631+0000 7f6117b11700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f60fc06ccf0 0x7f60fc06f1b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.631+0000 7f6117b11700 1 --2- 172.21.15.63:0/352547077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110100d20 0x7f6110196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.631+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 >> 172.21.15.63:0/352547077 conn(0x7f61100fb0c0 msgr2=0x7f6110068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.632+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 shutdown_connections 2024-05-09T06:36:32.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.632+0000 7f6117b11700 1 -- 172.21.15.63:0/352547077 wait complete. 2024-05-09T06:36:32.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.777+0000 7f87bb29f700 1 -- 172.21.15.63:0/3127049617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b4100da0 msgr2=0x7f87b4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.777+0000 7f87bb29f700 1 --2- 172.21.15.63:0/3127049617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b4100da0 0x7f87b4101200 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009910 tx=0x7f87a4009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:32.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.777+0000 7f87bb29f700 1 -- 172.21.15.63:0/3127049617 shutdown_connections 2024-05-09T06:36:32.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.777+0000 7f87bb29f700 1 --2- 172.21.15.63:0/3127049617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b4100da0 0x7f87b4101200 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.777+0000 7f87bb29f700 1 --2- 172.21.15.63:0/3127049617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b40ffba0 0x7f87b40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.777+0000 7f87bb29f700 1 -- 172.21.15.63:0/3127049617 >> 172.21.15.63:0/3127049617 conn(0x7f87b40fb120 msgr2=0x7f87b40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.777+0000 7f87bb29f700 1 -- 172.21.15.63:0/3127049617 shutdown_connections 2024-05-09T06:36:32.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.777+0000 7f87bb29f700 1 -- 172.21.15.63:0/3127049617 wait complete. 2024-05-09T06:36:32.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.778+0000 7f87bb29f700 1 Processor -- start 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.778+0000 7f87bb29f700 1 -- start start 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.778+0000 7f87bb29f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b40ffba0 0x7f87b4071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.778+0000 7f87bb29f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b4100da0 0x7f87b4072460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.778+0000 7f87bb29f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4072aa0 con 0x7f87b40ffba0 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.778+0000 7f87bb29f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4191fa0 con 0x7f87b4100da0 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.778+0000 7f87b903b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b40ffba0 0x7f87b4071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.778+0000 7f87b903b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b40ffba0 0x7f87b4071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55966/0 (socket says 172.21.15.63:55966) 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.778+0000 7f87b903b700 1 -- 172.21.15.63:0/1754681046 learned_addr learned my addr 172.21.15.63:0/1754681046 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.779+0000 7f87b883a700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b4100da0 0x7f87b4072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.779+0000 7f87b903b700 1 -- 172.21.15.63:0/1754681046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b4100da0 msgr2=0x7f87b4072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.779+0000 7f87b903b700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b4100da0 0x7f87b4072460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.779+0000 7f87b903b700 1 -- 172.21.15.63:0/1754681046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a40095f0 con 0x7f87b40ffba0 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.779+0000 7f87b903b700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b40ffba0 0x7f87b4071ee0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e8f0 tx=0x7f87b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.779+0000 7f87aa7fc700 1 -- 172.21.15.63:0/1754681046 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b0009790 con 0x7f87b40ffba0 2024-05-09T06:36:32.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.779+0000 7f87aa7fc700 1 -- 172.21.15.63:0/1754681046 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87b00044e0 con 0x7f87b40ffba0 2024-05-09T06:36:32.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.780+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b4192280 con 0x7f87b40ffba0 2024-05-09T06:36:32.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.780+0000 7f87aa7fc700 1 -- 172.21.15.63:0/1754681046 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b0003e00 con 0x7f87b40ffba0 2024-05-09T06:36:32.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.780+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b4068850 con 0x7f87b40ffba0 2024-05-09T06:36:32.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.781+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b404ec20 con 0x7f87b40ffba0 2024-05-09T06:36:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.781+0000 7f87aa7fc700 1 -- 172.21.15.63:0/1754681046 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f87b000cba0 con 0x7f87b40ffba0 2024-05-09T06:36:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.782+0000 7f87aa7fc700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f87a00710f0 0x7f87a00735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.782+0000 7f87aa7fc700 1 -- 172.21.15.63:0/1754681046 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f87b0014070 con 0x7f87b40ffba0 2024-05-09T06:36:32.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.782+0000 7f87b883a700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f87a00710f0 0x7f87a00735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.783+0000 7f87b883a700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f87a00710f0 0x7f87a00735b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87b4072fc0 tx=0x7f87a4009f90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:32.787+0000 7f87aa7fc700 1 -- 172.21.15.63:0/1754681046 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f87b404ec20 con 0x7f87b40ffba0 2024-05-09T06:36:33.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.100+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f87b404ebb0 con 0x7f87a00710f0 2024-05-09T06:36:33.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.119+0000 7f87aa7fc700 1 -- 172.21.15.63:0/1754681046 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f87b404ebb0 con 0x7f87a00710f0 2024-05-09T06:36:33.100 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:36:33.100 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (5m) 57s ago 6m 25.3M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:36:33.100 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (7m) 57s ago 7m 7465k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:36:33.100 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (6m) 62s ago 6m 7382k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:36:33.100 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (7m) 57s ago 7m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:36:33.100 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (6m) 62s ago 6m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:36:33.100 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 57s ago 6m 87.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (83s) 57s ago 82s 11.7M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (76s) 57s ago 75s 14.7M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (80s) 62s ago 80s 16.5M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (73s) 62s ago 73s 13.2M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:9283,8765,8443 running (8m) 57s ago 8m 500M - 18.2.2-1297-g1758e8ea 196422f1b641 a66e4acc828f 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (5m) 62s ago 5m 450M - 18.2.2-1297-g1758e8ea 196422f1b641 b87b5901e715 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (8m) 57s ago 8m 45.4M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 7d88727e4bbb 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (5m) 62s ago 5m 37.0M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 57s ago 7m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (5m) 62s ago 5m 15.4M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (5m) 57s ago 5m 47.2M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (4m) 57s ago 4m 48.3M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 57s ago 4m 46.4M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (3m) 62s ago 3m 46.7M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (3m) 62s ago 3m 51.7M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:36:33.101 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (2m) 62s ago 2m 53.1M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:36:33.102 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 57s ago 6m 58.2M - 2.43.0 a07b618ecd1d 9f4deb8cc8b4 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f87a00710f0 msgr2=0x7f87a00735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f87a00710f0 0x7f87a00735b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87b4072fc0 tx=0x7f87a4009f90 comp rx=0 tx=0).stop 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b40ffba0 msgr2=0x7f87b4071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b40ffba0 0x7f87b4071ee0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e8f0 tx=0x7f87b000ecb0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 shutdown_connections 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87b40ffba0 0x7f87b4071ee0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f87a00710f0 0x7f87a00735b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 --2- 172.21.15.63:0/1754681046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b4100da0 0x7f87b4072460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 >> 172.21.15.63:0/1754681046 conn(0x7f87b40fb120 msgr2=0x7f87b41043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 shutdown_connections 2024-05-09T06:36:33.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.122+0000 7f87bb29f700 1 -- 172.21.15.63:0/1754681046 wait complete. 2024-05-09T06:36:33.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.276+0000 7f124b450700 1 -- 172.21.15.63:0/3410341059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 msgr2=0x7f1244101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.276+0000 7f124b450700 1 --2- 172.21.15.63:0/3410341059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 0x7f1244101200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1234009910 tx=0x7f1234009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.276+0000 7f124b450700 1 -- 172.21.15.63:0/3410341059 shutdown_connections 2024-05-09T06:36:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.276+0000 7f124b450700 1 --2- 172.21.15.63:0/3410341059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 0x7f1244101200 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.276+0000 7f124b450700 1 --2- 172.21.15.63:0/3410341059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12440ffba0 0x7f12440fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.276+0000 7f124b450700 1 -- 172.21.15.63:0/3410341059 >> 172.21.15.63:0/3410341059 conn(0x7f12440fb120 msgr2=0x7f12440fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.276+0000 7f124b450700 1 -- 172.21.15.63:0/3410341059 shutdown_connections 2024-05-09T06:36:33.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.276+0000 7f124b450700 1 -- 172.21.15.63:0/3410341059 wait complete. 2024-05-09T06:36:33.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f124b450700 1 Processor -- start 2024-05-09T06:36:33.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f124b450700 1 -- start start 2024-05-09T06:36:33.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f124b450700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12440ffba0 0x7f1244196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f124b450700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 0x7f1244196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f124b450700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1244197290 con 0x7f1244100da0 2024-05-09T06:36:33.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f124b450700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12441973d0 con 0x7f12440ffba0 2024-05-09T06:36:33.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f12489eb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 0x7f1244196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f12489eb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 0x7f1244196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55990/0 (socket says 172.21.15.63:55990) 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f12489eb700 1 -- 172.21.15.63:0/2891728440 learned_addr learned my addr 172.21.15.63:0/2891728440 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.277+0000 7f12491ec700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12440ffba0 0x7f1244196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.278+0000 7f12489eb700 1 -- 172.21.15.63:0/2891728440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12440ffba0 msgr2=0x7f1244196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.278+0000 7f12489eb700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12440ffba0 0x7f1244196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.278+0000 7f12489eb700 1 -- 172.21.15.63:0/2891728440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12340095f0 con 0x7f1244100da0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.278+0000 7f12489eb700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 0x7f1244196c70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1234005b40 tx=0x7f123400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.278+0000 7f123a7fc700 1 -- 172.21.15.63:0/2891728440 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123401d070 con 0x7f1244100da0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.278+0000 7f123a7fc700 1 -- 172.21.15.63:0/2891728440 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12340050d0 con 0x7f1244100da0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.278+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124419bd30 con 0x7f1244100da0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.278+0000 7f123a7fc700 1 -- 172.21.15.63:0/2891728440 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1234022470 con 0x7f1244100da0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.278+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124419c220 con 0x7f1244100da0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.280+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124404ebb0 con 0x7f1244100da0 2024-05-09T06:36:33.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.280+0000 7f123a7fc700 1 -- 172.21.15.63:0/2891728440 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f12340037f0 con 0x7f1244100da0 2024-05-09T06:36:33.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.281+0000 7f123a7fc700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f123006ccf0 0x7f123006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.281+0000 7f123a7fc700 1 -- 172.21.15.63:0/2891728440 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f123408d800 con 0x7f1244100da0 2024-05-09T06:36:33.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.281+0000 7f12491ec700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f123006ccf0 0x7f123006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.281+0000 7f12491ec700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f123006ccf0 0x7f123006f1b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1244100c00 tx=0x7f1240009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.286+0000 7f123a7fc700 1 -- 172.21.15.63:0/2891728440 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f123405b600 con 0x7f1244100da0 2024-05-09T06:36:33.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:33 smithi063 ceph-mon[27171]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:33.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:33 smithi063 ceph-mon[27171]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:33 smithi193 ceph-mon[35141]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:33 smithi193 ceph-mon[35141]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:33.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.747+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1244108920 con 0x7f1244100da0 2024-05-09T06:36:33.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.747+0000 7f123a7fc700 1 -- 172.21.15.63:0/2891728440 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f123405b190 con 0x7f1244100da0 2024-05-09T06:36:33.728 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:36:33.728 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:36:33.728 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:36:33.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:36:33.728 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 14 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:36:33.729 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:36:33.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.750+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f123006ccf0 msgr2=0x7f123006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.750+0000 7f124b450700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f123006ccf0 0x7f123006f1b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1244100c00 tx=0x7f1240009260 comp rx=0 tx=0).stop 2024-05-09T06:36:33.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.750+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 msgr2=0x7f1244196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.750+0000 7f124b450700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 0x7f1244196c70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1234005b40 tx=0x7f123400f690 comp rx=0 tx=0).stop 2024-05-09T06:36:33.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.751+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 shutdown_connections 2024-05-09T06:36:33.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.751+0000 7f124b450700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f123006ccf0 0x7f123006f1b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.751+0000 7f124b450700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12440ffba0 0x7f1244196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.751+0000 7f124b450700 1 --2- 172.21.15.63:0/2891728440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1244100da0 0x7f1244196c70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.751+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 >> 172.21.15.63:0/2891728440 conn(0x7f12440fb120 msgr2=0x7f12440685d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.751+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 shutdown_connections 2024-05-09T06:36:33.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.751+0000 7f124b450700 1 -- 172.21.15.63:0/2891728440 wait complete. 2024-05-09T06:36:33.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.876+0000 7fd417aa8700 1 -- 172.21.15.63:0/1833789744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4101040a0 msgr2=0x7fd410106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.876+0000 7fd417aa8700 1 --2- 172.21.15.63:0/1833789744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4101040a0 0x7fd410106490 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd400009910 tx=0x7fd400009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:33.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.877+0000 7fd417aa8700 1 -- 172.21.15.63:0/1833789744 shutdown_connections 2024-05-09T06:36:33.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.877+0000 7fd417aa8700 1 --2- 172.21.15.63:0/1833789744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4101040a0 0x7fd410106490 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.877+0000 7fd417aa8700 1 --2- 172.21.15.63:0/1833789744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd410101770 0x7fd410103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.877+0000 7fd417aa8700 1 -- 172.21.15.63:0/1833789744 >> 172.21.15.63:0/1833789744 conn(0x7fd4100fb120 msgr2=0x7fd4100fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.877+0000 7fd417aa8700 1 -- 172.21.15.63:0/1833789744 shutdown_connections 2024-05-09T06:36:33.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.877+0000 7fd417aa8700 1 -- 172.21.15.63:0/1833789744 wait complete. 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.877+0000 7fd417aa8700 1 Processor -- start 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.877+0000 7fd417aa8700 1 -- start start 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd417aa8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd410101770 0x7fd410196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd417aa8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4101040a0 0x7fd410196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd417aa8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd410197290 con 0x7fd410101770 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd417aa8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4101973d0 con 0x7fd4101040a0 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd415844700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd410101770 0x7fd410196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd415844700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd410101770 0x7fd410196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56014/0 (socket says 172.21.15.63:56014) 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd415844700 1 -- 172.21.15.63:0/2236408704 learned_addr learned my addr 172.21.15.63:0/2236408704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd415043700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4101040a0 0x7fd410196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd415844700 1 -- 172.21.15.63:0/2236408704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4101040a0 msgr2=0x7fd410196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd415844700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4101040a0 0x7fd410196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd415844700 1 -- 172.21.15.63:0/2236408704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4000095f0 con 0x7fd410101770 2024-05-09T06:36:33.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.878+0000 7fd415844700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd410101770 0x7fd410196730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00d900 tx=0x7fd40c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.879+0000 7fd406ffd700 1 -- 172.21.15.63:0/2236408704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40c0041f0 con 0x7fd410101770 2024-05-09T06:36:33.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.879+0000 7fd406ffd700 1 -- 172.21.15.63:0/2236408704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd40c004cf0 con 0x7fd410101770 2024-05-09T06:36:33.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.879+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41019be30 con 0x7fd410101770 2024-05-09T06:36:33.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.880+0000 7fd406ffd700 1 -- 172.21.15.63:0/2236408704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40c00b750 con 0x7fd410101770 2024-05-09T06:36:33.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.880+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41019c2f0 con 0x7fd410101770 2024-05-09T06:36:33.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.881+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41004ebb0 con 0x7fd410101770 2024-05-09T06:36:33.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.881+0000 7fd406ffd700 1 -- 172.21.15.63:0/2236408704 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd40c00b8b0 con 0x7fd410101770 2024-05-09T06:36:33.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.882+0000 7fd406ffd700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd3fc06cd30 0x7fd3fc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.882+0000 7fd406ffd700 1 -- 172.21.15.63:0/2236408704 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd40c08c140 con 0x7fd410101770 2024-05-09T06:36:33.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.882+0000 7fd415043700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd3fc06cd30 0x7fd3fc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.883+0000 7fd415043700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd3fc06cd30 0x7fd3fc06f1f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd400009fd0 tx=0x7fd400005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:33.890+0000 7fd406ffd700 1 -- 172.21.15.63:0/2236408704 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd40c059f20 con 0x7fd410101770 2024-05-09T06:36:34.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.157+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd41019c770 con 0x7fd410101770 2024-05-09T06:36:34.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.158+0000 7fd406ffd700 1 -- 172.21.15.63:0/2236408704 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fd40c059ab0 con 0x7fd410101770 2024-05-09T06:36:34.138 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:36:34.138 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:36:34.138 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:36:34.138 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:36:34.140 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd3fc06cd30 msgr2=0x7fd3fc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd3fc06cd30 0x7fd3fc06f1f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd400009fd0 tx=0x7fd400005960 comp rx=0 tx=0).stop 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd410101770 msgr2=0x7fd410196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd410101770 0x7fd410196730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00d900 tx=0x7fd40c00dc10 comp rx=0 tx=0).stop 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 shutdown_connections 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd410101770 0x7fd410196730 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00d900 tx=0x7fd40c00dc10 comp rx=0 tx=0).stop 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd3fc06cd30 0x7fd3fc06f1f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 --2- 172.21.15.63:0/2236408704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4101040a0 0x7fd410196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 >> 172.21.15.63:0/2236408704 conn(0x7fd4100fb120 msgr2=0x7fd4100fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 shutdown_connections 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.160+0000 7fd417aa8700 1 -- 172.21.15.63:0/2236408704 wait complete. 2024-05-09T06:36:34.143 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:36:34.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.280+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/2972884134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d4104060 msgr2=0x7fd0d4106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.280+0000 7fd0dbbe1700 1 --2- 172.21.15.63:0/2972884134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d4104060 0x7fd0d4106450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009910 tx=0x7fd0c4009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:34.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.280+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/2972884134 shutdown_connections 2024-05-09T06:36:34.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.280+0000 7fd0dbbe1700 1 --2- 172.21.15.63:0/2972884134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d4104060 0x7fd0d4106450 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.280+0000 7fd0dbbe1700 1 --2- 172.21.15.63:0/2972884134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0d4101730 0x7fd0d4103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.280+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/2972884134 >> 172.21.15.63:0/2972884134 conn(0x7fd0d40fb120 msgr2=0x7fd0d40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.280+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/2972884134 shutdown_connections 2024-05-09T06:36:34.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.280+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/2972884134 wait complete. 2024-05-09T06:36:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0dbbe1700 1 Processor -- start 2024-05-09T06:36:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0dbbe1700 1 -- start start 2024-05-09T06:36:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0dbbe1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0d4101730 0x7fd0d4196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0dbbe1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d4104060 0x7fd0d4196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0dbbe1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4197170 con 0x7fd0d4104060 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0dbbe1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d41972b0 con 0x7fd0d4101730 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0d917c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d4104060 0x7fd0d4196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0d997d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0d4101730 0x7fd0d4196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0d997d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0d4101730 0x7fd0d4196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.63:57600/0 (socket says 172.21.15.63:57600) 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.281+0000 7fd0d997d700 1 -- 172.21.15.63:0/3636491468 learned_addr learned my addr 172.21.15.63:0/3636491468 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0d997d700 1 -- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d4104060 msgr2=0x7fd0d4196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0d997d700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d4104060 0x7fd0d4196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0d997d700 1 -- 172.21.15.63:0/3636491468 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c40095f0 con 0x7fd0d4101730 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0d917c700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d4104060 0x7fd0d4196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0d997d700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0d4101730 0x7fd0d4196610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000e920 tx=0x7fd0d000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0caffd700 1 -- 172.21.15.63:0/3636491468 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0d000c9d0 con 0x7fd0d4101730 2024-05-09T06:36:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0caffd700 1 -- 172.21.15.63:0/3636491468 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0d0010430 con 0x7fd0d4101730 2024-05-09T06:36:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d419bd10 con 0x7fd0d4101730 2024-05-09T06:36:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0caffd700 1 -- 172.21.15.63:0/3636491468 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0d0018630 con 0x7fd0d4101730 2024-05-09T06:36:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.282+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d419c200 con 0x7fd0d4101730 2024-05-09T06:36:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.283+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d404ebb0 con 0x7fd0d4101730 2024-05-09T06:36:34.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.284+0000 7fd0caffd700 1 -- 172.21.15.63:0/3636491468 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd0d00105a0 con 0x7fd0d4101730 2024-05-09T06:36:34.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.285+0000 7fd0caffd700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd0c006cd30 0x7fd0c006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.285+0000 7fd0d917c700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd0c006cd30 0x7fd0c006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.286+0000 7fd0d917c700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd0c006cd30 0x7fd0c006f1f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400b800 tx=0x7fd0c4009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.286+0000 7fd0caffd700 1 -- 172.21.15.63:0/3636491468 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd0d0014070 con 0x7fd0d4101730 2024-05-09T06:36:34.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.290+0000 7fd0caffd700 1 -- 172.21.15.63:0/3636491468 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd0d0052c50 con 0x7fd0d4101730 2024-05-09T06:36:34.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.530+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0d4105ef0 con 0x7fd0c006cd30 2024-05-09T06:36:34.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:34 smithi063 ceph-mon[27171]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:34.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:34 smithi063 ceph-mon[27171]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:34.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:34 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2891728440' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:36:34.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:34 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2236408704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:36:34.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.531+0000 7fd0caffd700 1 -- 172.21.15.63:0/3636491468 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fd0d4105ef0 con 0x7fd0c006cd30 2024-05-09T06:36:34.512 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:36:34.512 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T06:36:34.512 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:36:34.512 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:36:34.512 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-05-09T06:36:34.512 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "", 2024-05-09T06:36:34.512 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc image", 2024-05-09T06:36:34.513 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:36:34.513 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:36:34.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd0c006cd30 msgr2=0x7fd0c006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd0c006cd30 0x7fd0c006f1f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400b800 tx=0x7fd0c4009f70 comp rx=0 tx=0).stop 2024-05-09T06:36:34.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0d4101730 msgr2=0x7fd0d4196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0d4101730 0x7fd0d4196610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000e920 tx=0x7fd0d000ec30 comp rx=0 tx=0).stop 2024-05-09T06:36:34.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 shutdown_connections 2024-05-09T06:36:34.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd0c006cd30 0x7fd0c006f1f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0d4101730 0x7fd0d4196610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 --2- 172.21.15.63:0/3636491468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d4104060 0x7fd0d4196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 >> 172.21.15.63:0/3636491468 conn(0x7fd0d40fb120 msgr2=0x7fd0d40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 shutdown_connections 2024-05-09T06:36:34.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.534+0000 7fd0dbbe1700 1 -- 172.21.15.63:0/3636491468 wait complete. 2024-05-09T06:36:34.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:34 smithi193 ceph-mon[35141]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:34.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:34 smithi193 ceph-mon[35141]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:34.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:34 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2891728440' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:36:34.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:34 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2236408704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:36:34.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.680+0000 7fa8a323a700 1 -- 172.21.15.63:0/1765249450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 msgr2=0x7fa89c106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.680+0000 7fa8a323a700 1 --2- 172.21.15.63:0/1765249450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 0x7fa89c106410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa888009910 tx=0x7fa888009c20 comp rx=0 tx=0).stop 2024-05-09T06:36:34.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.680+0000 7fa8a323a700 1 -- 172.21.15.63:0/1765249450 shutdown_connections 2024-05-09T06:36:34.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.680+0000 7fa8a323a700 1 --2- 172.21.15.63:0/1765249450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 0x7fa89c106410 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.680+0000 7fa8a323a700 1 --2- 172.21.15.63:0/1765249450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa89c1016f0 0x7fa89c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.680+0000 7fa8a323a700 1 -- 172.21.15.63:0/1765249450 >> 172.21.15.63:0/1765249450 conn(0x7fa89c0fb0c0 msgr2=0x7fa89c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.680+0000 7fa8a323a700 1 -- 172.21.15.63:0/1765249450 shutdown_connections 2024-05-09T06:36:34.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.680+0000 7fa8a323a700 1 -- 172.21.15.63:0/1765249450 wait complete. 2024-05-09T06:36:34.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.681+0000 7fa8a323a700 1 Processor -- start 2024-05-09T06:36:34.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.681+0000 7fa8a323a700 1 -- start start 2024-05-09T06:36:34.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.681+0000 7fa8a323a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa89c1016f0 0x7fa89c101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.681+0000 7fa8a323a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 0x7fa89c0ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.681+0000 7fa8a323a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c0ffbe0 con 0x7fa89c104020 2024-05-09T06:36:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.681+0000 7fa8a323a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c0ffd20 con 0x7fa89c1016f0 2024-05-09T06:36:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa89bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 0x7fa89c0ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa89bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 0x7fa89c0ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56062/0 (socket says 172.21.15.63:56062) 2024-05-09T06:36:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa89bfff700 1 -- 172.21.15.63:0/1286312668 learned_addr learned my addr 172.21.15.63:0/1286312668 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:36:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa8a0fd6700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa89c1016f0 0x7fa89c101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa89bfff700 1 -- 172.21.15.63:0/1286312668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa89c1016f0 msgr2=0x7fa89c101000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa89bfff700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa89c1016f0 0x7fa89c101000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa89bfff700 1 -- 172.21.15.63:0/1286312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880095f0 con 0x7fa89c104020 2024-05-09T06:36:34.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa89bfff700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 0x7fa89c0ff6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa888005b40 tx=0x7fa88800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa899ffb700 1 -- 172.21.15.63:0/1286312668 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa88801d070 con 0x7fa89c104020 2024-05-09T06:36:34.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.682+0000 7fa899ffb700 1 -- 172.21.15.63:0/1286312668 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8880050d0 con 0x7fa89c104020 2024-05-09T06:36:34.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.683+0000 7fa899ffb700 1 -- 172.21.15.63:0/1286312668 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa888022470 con 0x7fa89c104020 2024-05-09T06:36:34.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.683+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89c0fffa0 con 0x7fa89c104020 2024-05-09T06:36:34.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.683+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89c100510 con 0x7fa89c104020 2024-05-09T06:36:34.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.684+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89c04ebb0 con 0x7fa89c104020 2024-05-09T06:36:34.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.685+0000 7fa899ffb700 1 -- 172.21.15.63:0/1286312668 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa8880037f0 con 0x7fa89c104020 2024-05-09T06:36:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.685+0000 7fa899ffb700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa88c06ccf0 0x7fa88c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.686+0000 7fa899ffb700 1 -- 172.21.15.63:0/1286312668 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa88808d680 con 0x7fa89c104020 2024-05-09T06:36:34.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.686+0000 7fa8a0fd6700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa88c06ccf0 0x7fa88c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.687+0000 7fa8a0fd6700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa88c06ccf0 0x7fa88c06f1b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa890009c60 tx=0x7fa890009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:34.694+0000 7fa899ffb700 1 -- 172.21.15.63:0/1286312668 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa88805b510 con 0x7fa89c104020 2024-05-09T06:36:35.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.081+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa89c100510 con 0x7fa89c104020 2024-05-09T06:36:35.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.081+0000 7fa899ffb700 1 -- 172.21.15.63:0/1286312668 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa89c100510 con 0x7fa89c104020 2024-05-09T06:36:35.062 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:36:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa88c06ccf0 msgr2=0x7fa88c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa88c06ccf0 0x7fa88c06f1b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa890009c60 tx=0x7fa890009260 comp rx=0 tx=0).stop 2024-05-09T06:36:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 msgr2=0x7fa89c0ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 0x7fa89c0ff6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa888005b40 tx=0x7fa88800f690 comp rx=0 tx=0).stop 2024-05-09T06:36:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 shutdown_connections 2024-05-09T06:36:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fa88c06ccf0 0x7fa88c06f1b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa89c1016f0 0x7fa89c101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 --2- 172.21.15.63:0/1286312668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa89c104020 0x7fa89c0ff6a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 >> 172.21.15.63:0/1286312668 conn(0x7fa89c0fb0c0 msgr2=0x7fa89c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 shutdown_connections 2024-05-09T06:36:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:36:35.084+0000 7fa8a323a700 1 -- 172.21.15.63:0/1286312668 wait complete. 2024-05-09T06:36:35.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:35 smithi063 ceph-mon[27171]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:35.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:35 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1286312668' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:36:35.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:35 smithi193 ceph-mon[35141]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:35.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:35 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1286312668' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:36:36.540 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:36 smithi063 ceph-mon[27171]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:36.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:36 smithi193 ceph-mon[35141]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:36:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:37 smithi063 ceph-mon[27171]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:37 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:36:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:37 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:36:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:37 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:36:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:37 smithi193 ceph-mon[35141]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:37 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:36:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:37 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:36:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:37 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:36:38.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:38 smithi063 ceph-mon[27171]: Upgrade: Target is version 19.0.0-2515-g96805dcf (unknown) 2024-05-09T06:36:38.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:38 smithi063 ceph-mon[27171]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, digests ['quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac'] 2024-05-09T06:36:38.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:38 smithi063 ceph-mon[27171]: Upgrade: Need to upgrade myself (mgr.smithi063.egfguy) 2024-05-09T06:36:38.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:38 smithi063 ceph-mon[27171]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac on smithi193 2024-05-09T06:36:38.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:38 smithi193 ceph-mon[35141]: Upgrade: Target is version 19.0.0-2515-g96805dcf (unknown) 2024-05-09T06:36:38.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:38 smithi193 ceph-mon[35141]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, digests ['quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac'] 2024-05-09T06:36:38.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:38 smithi193 ceph-mon[35141]: Upgrade: Need to upgrade myself (mgr.smithi063.egfguy) 2024-05-09T06:36:38.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:38 smithi193 ceph-mon[35141]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac on smithi193 2024-05-09T06:36:39.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:39 smithi063 ceph-mon[27171]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:39.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:39 smithi193 ceph-mon[35141]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:41.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:41 smithi063 ceph-mon[27171]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:41.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:41 smithi193 ceph-mon[35141]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:42.523 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:36:42.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:36:43.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:43 smithi063 ceph-mon[27171]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-09T06:36:43.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:43 smithi193 ceph-mon[35141]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-09T06:36:45.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:45 smithi063 ceph-mon[27171]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-09T06:36:45.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:45 smithi193 ceph-mon[35141]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-09T06:36:47.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:47 smithi063 ceph-mon[27171]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:47.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:47 smithi193 ceph-mon[35141]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:49.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:49 smithi063 ceph-mon[27171]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-09T06:36:49.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:49 smithi193 ceph-mon[35141]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-09T06:36:51.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:51 smithi063 ceph-mon[27171]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:51.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:51 smithi193 ceph-mon[35141]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:53.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:53 smithi063 ceph-mon[27171]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-09T06:36:53.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:53 smithi193 ceph-mon[35141]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-05-09T06:36:55.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:55 smithi063 ceph-mon[27171]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:55.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:55 smithi193 ceph-mon[35141]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:57 smithi063 ceph-mon[27171]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:57.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:36:57.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:57 smithi193 ceph-mon[35141]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:36:57.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:36:58.441 INFO:tasks.workunit.client.0.smithi063.stderr:Updating files: 95% (12002/12535) Updating files: 96% (12034/12535) Updating files: 97% (12159/12535) Updating files: 98% (12285/12535) Updating files: 99% (12410/12535) Updating files: 100% (12535/12535) Updating files: 100% (12535/12535), done. 2024-05-09T06:36:59.047 INFO:tasks.workunit.client.0.smithi063.stderr:Note: switching to 'be85c202969d25af6e130678e7d73a20e9c4ca11'. 2024-05-09T06:36:59.047 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T06:36:59.047 INFO:tasks.workunit.client.0.smithi063.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T06:36:59.047 INFO:tasks.workunit.client.0.smithi063.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T06:36:59.047 INFO:tasks.workunit.client.0.smithi063.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T06:36:59.047 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr: git switch -c 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr:Or undo this operation with: 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr: git switch - 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T06:36:59.048 INFO:tasks.workunit.client.0.smithi063.stderr:HEAD is now at be85c202969 Merge pull request #57242 from cbodley/wip-65767-squid 2024-05-09T06:36:59.051 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T06:36:59.083 INFO:tasks.workunit.client.0.smithi063.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T06:36:59.085 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T06:36:59.086 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T06:36:59.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:36:59 smithi063 ceph-mon[27171]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:59.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:36:59 smithi193 ceph-mon[35141]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:36:59.807 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T06:36:59.878 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T06:36:59.922 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T06:36:59.923 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T06:36:59.924 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T06:36:59.966 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T06:36:59.969 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:36:59.969 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T06:37:00.026 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-05-09T06:37:00.028 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-09T06:37:00.028 DEBUG:teuthology.orchestra.run.smithi063:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=be85c202969d25af6e130678e7d73a20e9c4ca11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-05-09T06:37:00.092 INFO:tasks.workunit.client.0.smithi063.stderr:+ mkdir -p fsstress 2024-05-09T06:37:00.095 INFO:tasks.workunit.client.0.smithi063.stderr:+ pushd fsstress 2024-05-09T06:37:00.096 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-09T06:37:00.096 INFO:tasks.workunit.client.0.smithi063.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-09T06:37:01.436 INFO:tasks.workunit.client.1.smithi193.stderr:Updating files: 96% (12037/12535) Updating files: 97% (12159/12535) Updating files: 98% (12285/12535) Updating files: 99% (12410/12535) Updating files: 100% (12535/12535) Updating files: 100% (12535/12535), done. 2024-05-09T06:37:01.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:01 smithi063 ceph-mon[27171]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:37:01.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:01 smithi193 ceph-mon[35141]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:37:02.121 INFO:tasks.workunit.client.1.smithi193.stderr:Note: switching to 'be85c202969d25af6e130678e7d73a20e9c4ca11'. 2024-05-09T06:37:02.122 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-05-09T06:37:02.122 INFO:tasks.workunit.client.1.smithi193.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T06:37:02.122 INFO:tasks.workunit.client.1.smithi193.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T06:37:02.122 INFO:tasks.workunit.client.1.smithi193.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T06:37:02.122 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-05-09T06:37:02.122 INFO:tasks.workunit.client.1.smithi193.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T06:37:02.122 INFO:tasks.workunit.client.1.smithi193.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T06:37:02.122 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-05-09T06:37:02.123 INFO:tasks.workunit.client.1.smithi193.stderr: git switch -c 2024-05-09T06:37:02.123 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-05-09T06:37:02.123 INFO:tasks.workunit.client.1.smithi193.stderr:Or undo this operation with: 2024-05-09T06:37:02.123 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-05-09T06:37:02.123 INFO:tasks.workunit.client.1.smithi193.stderr: git switch - 2024-05-09T06:37:02.123 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-05-09T06:37:02.123 INFO:tasks.workunit.client.1.smithi193.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T06:37:02.123 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-05-09T06:37:02.123 INFO:tasks.workunit.client.1.smithi193.stderr:HEAD is now at be85c202969 Merge pull request #57242 from cbodley/wip-65767-squid 2024-05-09T06:37:02.127 DEBUG:teuthology.orchestra.run.smithi193:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-05-09T06:37:02.198 INFO:tasks.workunit.client.1.smithi193.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T06:37:02.200 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-09T06:37:02.200 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T06:37:02.300 INFO:tasks.workunit.client.0.smithi063.stderr:+ tar xzf ltp-full.tgz 2024-05-09T06:37:02.899 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T06:37:02.969 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T06:37:03.019 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-09T06:37:03.021 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-09T06:37:03.021 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T06:37:03.066 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-09T06:37:03.070 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-09T06:37:03.071 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-05-09T06:37:03.127 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-05-09T06:37:03.128 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-09T06:37:03.128 DEBUG:teuthology.orchestra.run.smithi193:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=be85c202969d25af6e130678e7d73a20e9c4ca11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-05-09T06:37:03.197 INFO:tasks.workunit.client.1.smithi193.stderr:+ mkdir -p fsstress 2024-05-09T06:37:03.201 INFO:tasks.workunit.client.1.smithi193.stderr:+ pushd fsstress 2024-05-09T06:37:03.202 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-09T06:37:03.202 INFO:tasks.workunit.client.1.smithi193.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-09T06:37:03.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:03 smithi063 ceph-mon[27171]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:37:03.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:03 smithi193 ceph-mon[35141]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:37:03.889 INFO:tasks.workunit.client.1.smithi193.stderr:+ tar xzf ltp-full.tgz 2024-05-09T06:37:05.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.223+0000 7fe83ed34700 1 -- 172.21.15.63:0/1477260298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 msgr2=0x7fe838101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.223+0000 7fe83ed34700 1 --2- 172.21.15.63:0/1477260298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 0x7fe838101200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe824009910 tx=0x7fe824009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:05.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.223+0000 7fe83ed34700 1 -- 172.21.15.63:0/1477260298 shutdown_connections 2024-05-09T06:37:05.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.223+0000 7fe83ed34700 1 --2- 172.21.15.63:0/1477260298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 0x7fe838101200 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.223+0000 7fe83ed34700 1 --2- 172.21.15.63:0/1477260298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8380ffba0 0x7fe8380fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.223+0000 7fe83ed34700 1 -- 172.21.15.63:0/1477260298 >> 172.21.15.63:0/1477260298 conn(0x7fe8380fb120 msgr2=0x7fe8380fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.223+0000 7fe83ed34700 1 -- 172.21.15.63:0/1477260298 shutdown_connections 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.223+0000 7fe83ed34700 1 -- 172.21.15.63:0/1477260298 wait complete. 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe83ed34700 1 Processor -- start 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe83ed34700 1 -- start start 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe83ed34700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8380ffba0 0x7fe838195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe83ed34700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 0x7fe838195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe83ed34700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8381960f0 con 0x7fe838100da0 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe837fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 0x7fe838195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe837fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 0x7fe838195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40312/0 (socket says 172.21.15.63:40312) 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe837fff700 1 -- 172.21.15.63:0/1841996143 learned_addr learned my addr 172.21.15.63:0/1841996143 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe83ed34700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe838196230 con 0x7fe8380ffba0 2024-05-09T06:37:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.224+0000 7fe83cad0700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8380ffba0 0x7fe838195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.225+0000 7fe837fff700 1 -- 172.21.15.63:0/1841996143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8380ffba0 msgr2=0x7fe838195590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.225+0000 7fe837fff700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8380ffba0 0x7fe838195590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.225+0000 7fe837fff700 1 -- 172.21.15.63:0/1841996143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8240095f0 con 0x7fe838100da0 2024-05-09T06:37:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.225+0000 7fe837fff700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 0x7fe838195ad0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe82400b800 tx=0x7fe824005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.225+0000 7fe835ffb700 1 -- 172.21.15.63:0/1841996143 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82401d070 con 0x7fe838100da0 2024-05-09T06:37:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.225+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83819ac30 con 0x7fe838100da0 2024-05-09T06:37:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.225+0000 7fe835ffb700 1 -- 172.21.15.63:0/1841996143 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8240052a0 con 0x7fe838100da0 2024-05-09T06:37:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.225+0000 7fe835ffb700 1 -- 172.21.15.63:0/1841996143 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe824017400 con 0x7fe838100da0 2024-05-09T06:37:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.226+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83819b120 con 0x7fe838100da0 2024-05-09T06:37:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.226+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83818f720 con 0x7fe838100da0 2024-05-09T06:37:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.227+0000 7fe835ffb700 1 -- 172.21.15.63:0/1841996143 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe824003780 con 0x7fe838100da0 2024-05-09T06:37:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.228+0000 7fe835ffb700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe82806cce0 0x7fe82806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.229+0000 7fe835ffb700 1 -- 172.21.15.63:0/1841996143 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe82408d710 con 0x7fe838100da0 2024-05-09T06:37:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.229+0000 7fe83cad0700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe82806cce0 0x7fe82806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.229+0000 7fe83cad0700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe82806cce0 0x7fe82806f1a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe838100c00 tx=0x7fe82c009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.234+0000 7fe835ffb700 1 -- 172.21.15.63:0/1841996143 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe82405b510 con 0x7fe838100da0 2024-05-09T06:37:05.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.517+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe83802cdb0 con 0x7fe82806cce0 2024-05-09T06:37:05.499 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:05 smithi063 ceph-mon[27171]: pgmap v241: 65 pgs: 65 active+clean; 745 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 15 KiB/s wr, 3 op/s 2024-05-09T06:37:05.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.519+0000 7fe835ffb700 1 -- 172.21.15.63:0/1841996143 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fe83802cdb0 con 0x7fe82806cce0 2024-05-09T06:37:05.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe82806cce0 msgr2=0x7fe82806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe82806cce0 0x7fe82806f1a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe838100c00 tx=0x7fe82c009260 comp rx=0 tx=0).stop 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 msgr2=0x7fe838195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 0x7fe838195ad0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe82400b800 tx=0x7fe824005f00 comp rx=0 tx=0).stop 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 shutdown_connections 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fe82806cce0 0x7fe82806f1a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8380ffba0 0x7fe838195590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 --2- 172.21.15.63:0/1841996143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe838100da0 0x7fe838195ad0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 >> 172.21.15.63:0/1841996143 conn(0x7fe8380fb120 msgr2=0x7fe8381043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 shutdown_connections 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.521+0000 7fe83ed34700 1 -- 172.21.15.63:0/1841996143 wait complete. 2024-05-09T06:37:05.515 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:37:05.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:05 smithi193 ceph-mon[35141]: pgmap v241: 65 pgs: 65 active+clean; 745 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 15 KiB/s wr, 3 op/s 2024-05-09T06:37:05.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.652+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/1805987223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a40ffba0 msgr2=0x7fd4a40fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.652+0000 7fd4a9f6d700 1 --2- 172.21.15.63:0/1805987223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a40ffba0 0x7fd4a40fffc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd494009910 tx=0x7fd494009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:05.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.652+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/1805987223 shutdown_connections 2024-05-09T06:37:05.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.652+0000 7fd4a9f6d700 1 --2- 172.21.15.63:0/1805987223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4100da0 0x7fd4a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.652+0000 7fd4a9f6d700 1 --2- 172.21.15.63:0/1805987223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a40ffba0 0x7fd4a40fffc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.653+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/1805987223 >> 172.21.15.63:0/1805987223 conn(0x7fd4a40fb120 msgr2=0x7fd4a40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:05.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.653+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/1805987223 shutdown_connections 2024-05-09T06:37:05.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.653+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/1805987223 wait complete. 2024-05-09T06:37:05.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.653+0000 7fd4a9f6d700 1 Processor -- start 2024-05-09T06:37:05.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.653+0000 7fd4a9f6d700 1 -- start start 2024-05-09T06:37:05.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a9f6d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a40ffba0 0x7fd4a40690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a9f6d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4100da0 0x7fd4a419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a9f6d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a419b9e0 con 0x7fd4a40ffba0 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a9f6d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a419bb20 con 0x7fd4a4100da0 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a2ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4100da0 0x7fd4a419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a2ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4100da0 0x7fd4a419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.63:33128/0 (socket says 172.21.15.63:33128) 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a2ffd700 1 -- 172.21.15.63:0/2011234001 learned_addr learned my addr 172.21.15.63:0/2011234001 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a37fe700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a40ffba0 0x7fd4a40690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a2ffd700 1 -- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a40ffba0 msgr2=0x7fd4a40690f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a2ffd700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a40ffba0 0x7fd4a40690f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.654+0000 7fd4a2ffd700 1 -- 172.21.15.63:0/2011234001 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4940095f0 con 0x7fd4a4100da0 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.655+0000 7fd4a37fe700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a40ffba0 0x7fd4a40690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.655+0000 7fd4a2ffd700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4100da0 0x7fd4a419b410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e920 tx=0x7fd49800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.655+0000 7fd4a0ff9700 1 -- 172.21.15.63:0/2011234001 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd498009790 con 0x7fd4a4100da0 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.655+0000 7fd4a0ff9700 1 -- 172.21.15.63:0/2011234001 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd49800cba0 con 0x7fd4a4100da0 2024-05-09T06:37:05.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.655+0000 7fd4a0ff9700 1 -- 172.21.15.63:0/2011234001 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd498018650 con 0x7fd4a4100da0 2024-05-09T06:37:05.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.655+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a4195240 con 0x7fd4a4100da0 2024-05-09T06:37:05.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.655+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a4195740 con 0x7fd4a4100da0 2024-05-09T06:37:05.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.657+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a418f720 con 0x7fd4a4100da0 2024-05-09T06:37:05.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.657+0000 7fd4a0ff9700 1 -- 172.21.15.63:0/2011234001 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd49800cd10 con 0x7fd4a4100da0 2024-05-09T06:37:05.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.658+0000 7fd4a0ff9700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd48c0710f0 0x7fd48c0735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.658+0000 7fd4a37fe700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd48c0710f0 0x7fd48c0735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.658+0000 7fd4a0ff9700 1 -- 172.21.15.63:0/2011234001 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd498014070 con 0x7fd4a4100da0 2024-05-09T06:37:05.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.658+0000 7fd4a37fe700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd48c0710f0 0x7fd48c0735b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd494000c00 tx=0x7fd494005c00 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:05.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.664+0000 7fd4a0ff9700 1 -- 172.21.15.63:0/2011234001 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd498052c50 con 0x7fd4a4100da0 2024-05-09T06:37:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.946+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4a402cea0 con 0x7fd48c0710f0 2024-05-09T06:37:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.947+0000 7fd4a0ff9700 1 -- 172.21.15.63:0/2011234001 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fd4a402cea0 con 0x7fd48c0710f0 2024-05-09T06:37:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.949+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd48c0710f0 msgr2=0x7fd48c0735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.949+0000 7fd4a9f6d700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd48c0710f0 0x7fd48c0735b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd494000c00 tx=0x7fd494005c00 comp rx=0 tx=0).stop 2024-05-09T06:37:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.949+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4100da0 msgr2=0x7fd4a419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.949+0000 7fd4a9f6d700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4100da0 0x7fd4a419b410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e920 tx=0x7fd49800ece0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.950+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 shutdown_connections 2024-05-09T06:37:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.950+0000 7fd4a9f6d700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4a40ffba0 0x7fd4a40690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.950+0000 7fd4a9f6d700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd48c0710f0 0x7fd48c0735b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.950+0000 7fd4a9f6d700 1 --2- 172.21.15.63:0/2011234001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4100da0 0x7fd4a419b410 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.950+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 >> 172.21.15.63:0/2011234001 conn(0x7fd4a40fb120 msgr2=0x7fd4a41043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.950+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 shutdown_connections 2024-05-09T06:37:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:05.950+0000 7fd4a9f6d700 1 -- 172.21.15.63:0/2011234001 wait complete. 2024-05-09T06:37:06.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.085+0000 7f90722a1700 1 -- 172.21.15.63:0/1766185216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 msgr2=0x7f906c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.085+0000 7f90722a1700 1 --2- 172.21.15.63:0/1766185216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 0x7f906c0fffc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f905c009910 tx=0x7f905c009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:06.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.085+0000 7f90722a1700 1 -- 172.21.15.63:0/1766185216 shutdown_connections 2024-05-09T06:37:06.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.085+0000 7f90722a1700 1 --2- 172.21.15.63:0/1766185216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f906c100da0 0x7f906c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.085+0000 7f90722a1700 1 --2- 172.21.15.63:0/1766185216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 0x7f906c0fffc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.085+0000 7f90722a1700 1 -- 172.21.15.63:0/1766185216 >> 172.21.15.63:0/1766185216 conn(0x7f906c0fb120 msgr2=0x7f906c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:06.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.086+0000 7f90722a1700 1 -- 172.21.15.63:0/1766185216 shutdown_connections 2024-05-09T06:37:06.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.086+0000 7f90722a1700 1 -- 172.21.15.63:0/1766185216 wait complete. 2024-05-09T06:37:06.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.086+0000 7f90722a1700 1 Processor -- start 2024-05-09T06:37:06.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.086+0000 7f90722a1700 1 -- start start 2024-05-09T06:37:06.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f90722a1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 0x7f906c103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f90722a1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f906c100da0 0x7f906c102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f90722a1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c1026c0 con 0x7f906c0ffba0 2024-05-09T06:37:06.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f90722a1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c102800 con 0x7f906c100da0 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f906bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 0x7f906c103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f906bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 0x7f906c103a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40366/0 (socket says 172.21.15.63:40366) 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f906bfff700 1 -- 172.21.15.63:0/1461520928 learned_addr learned my addr 172.21.15.63:0/1461520928 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f9063fff700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f906c100da0 0x7f906c102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f906bfff700 1 -- 172.21.15.63:0/1461520928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f906c100da0 msgr2=0x7f906c102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f906bfff700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f906c100da0 0x7f906c102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f906bfff700 1 -- 172.21.15.63:0/1461520928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c0095f0 con 0x7f906c0ffba0 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f9063fff700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f906c100da0 0x7f906c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.087+0000 7f906bfff700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 0x7f906c103a90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f905c00bd70 tx=0x7f905c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.088+0000 7f9069ffb700 1 -- 172.21.15.63:0/1461520928 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f905c01d070 con 0x7f906c0ffba0 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.088+0000 7f9069ffb700 1 -- 172.21.15.63:0/1461520928 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f905c005dc0 con 0x7f906c0ffba0 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.088+0000 7f9069ffb700 1 -- 172.21.15.63:0/1461520928 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f905c021410 con 0x7f906c0ffba0 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.088+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c102a80 con 0x7f906c0ffba0 2024-05-09T06:37:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.088+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c102f70 con 0x7f906c0ffba0 2024-05-09T06:37:06.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.089+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c04ebb0 con 0x7f906c0ffba0 2024-05-09T06:37:06.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.090+0000 7f9069ffb700 1 -- 172.21.15.63:0/1461520928 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f905c00f820 con 0x7f906c0ffba0 2024-05-09T06:37:06.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.090+0000 7f9069ffb700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f905806ccf0 0x7f905806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.090+0000 7f9069ffb700 1 -- 172.21.15.63:0/1461520928 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f905c08d420 con 0x7f906c0ffba0 2024-05-09T06:37:06.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.090+0000 7f9063fff700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f905806ccf0 0x7f905806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.091+0000 7f9063fff700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f905806ccf0 0x7f905806f1b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9054005fd0 tx=0x7f9054005f60 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:06.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.096+0000 7f9069ffb700 1 -- 172.21.15.63:0/1461520928 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f905c05b2a0 con 0x7f906c0ffba0 2024-05-09T06:37:06.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.384+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f906c068d90 con 0x7f905806ccf0 2024-05-09T06:37:06.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.406+0000 7f9069ffb700 1 -- 172.21.15.63:0/1461520928 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3864 (secure 0 0 0) 0x7f906c068d90 con 0x7f905806ccf0 2024-05-09T06:37:06.388 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:37:06.388 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (6m) 90s ago 7m 25.3M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:37:06.388 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (7m) 90s ago 7m 7465k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:37:06.388 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (6m) 95s ago 6m 7382k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:37:06.388 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (7m) 90s ago 7m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:37:06.388 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (6m) 95s ago 6m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:37:06.389 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 90s ago 7m 87.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:37:06.389 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (116s) 90s ago 116s 11.7M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:37:06.389 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (109s) 90s ago 109s 14.7M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:37:06.389 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (113s) 95s ago 113s 16.5M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:37:06.389 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (106s) 95s ago 106s 13.2M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:37:06.389 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:9283,8765,8443 running (8m) 90s ago 8m 500M - 18.2.2-1297-g1758e8ea 196422f1b641 a66e4acc828f 2024-05-09T06:37:06.389 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (6m) 95s ago 6m 450M - 18.2.2-1297-g1758e8ea 196422f1b641 b87b5901e715 2024-05-09T06:37:06.389 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (8m) 90s ago 9m 45.4M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 7d88727e4bbb 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (6m) 95s ago 6m 37.0M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 90s ago 7m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (6m) 95s ago 6m 15.4M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (5m) 90s ago 5m 47.2M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 90s ago 5m 48.3M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 90s ago 4m 46.4M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (4m) 95s ago 4m 46.7M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (3m) 95s ago 3m 51.7M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (3m) 95s ago 3m 53.1M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:37:06.390 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 90s ago 7m 58.2M - 2.43.0 a07b618ecd1d 9f4deb8cc8b4 2024-05-09T06:37:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.409+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f905806ccf0 msgr2=0x7f905806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.409+0000 7f90722a1700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f905806ccf0 0x7f905806f1b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9054005fd0 tx=0x7f9054005f60 comp rx=0 tx=0).stop 2024-05-09T06:37:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.409+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 msgr2=0x7f906c103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.409+0000 7f90722a1700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 0x7f906c103a90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f905c00bd70 tx=0x7f905c00be50 comp rx=0 tx=0).stop 2024-05-09T06:37:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.409+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 shutdown_connections 2024-05-09T06:37:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.409+0000 7f90722a1700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f906c0ffba0 0x7f906c103a90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.409+0000 7f90722a1700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f905806ccf0 0x7f905806f1b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.409+0000 7f90722a1700 1 --2- 172.21.15.63:0/1461520928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f906c100da0 0x7f906c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.409+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 >> 172.21.15.63:0/1461520928 conn(0x7f906c0fb120 msgr2=0x7f906c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.410+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 shutdown_connections 2024-05-09T06:37:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.410+0000 7f90722a1700 1 -- 172.21.15.63:0/1461520928 wait complete. 2024-05-09T06:37:06.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.562+0000 7f1de48e8700 1 -- 172.21.15.63:0/2260571236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0104050 msgr2=0x7f1de0106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.562+0000 7f1de48e8700 1 --2- 172.21.15.63:0/2260571236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0104050 0x7f1de0106440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009870 tx=0x7f1dd4009b80 comp rx=0 tx=0).stop 2024-05-09T06:37:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.563+0000 7f1de48e8700 1 -- 172.21.15.63:0/2260571236 shutdown_connections 2024-05-09T06:37:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.563+0000 7f1de48e8700 1 --2- 172.21.15.63:0/2260571236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0104050 0x7f1de0106440 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.563+0000 7f1de48e8700 1 --2- 172.21.15.63:0/2260571236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de0101720 0x7f1de0103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.563+0000 7f1de48e8700 1 -- 172.21.15.63:0/2260571236 >> 172.21.15.63:0/2260571236 conn(0x7f1de00fb0d0 msgr2=0x7f1de00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.563+0000 7f1de48e8700 1 -- 172.21.15.63:0/2260571236 shutdown_connections 2024-05-09T06:37:06.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.563+0000 7f1de48e8700 1 -- 172.21.15.63:0/2260571236 wait complete. 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1de48e8700 1 Processor -- start 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1de48e8700 1 -- start start 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1de48e8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0101720 0x7f1de0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1de48e8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de0104050 0x7f1de0198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1de48e8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de01993b0 con 0x7f1de0101720 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1de48e8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de01994f0 con 0x7f1de0104050 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1dde59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0101720 0x7f1de0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1dde59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0101720 0x7f1de0198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40380/0 (socket says 172.21.15.63:40380) 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1dde59c700 1 -- 172.21.15.63:0/1168510249 learned_addr learned my addr 172.21.15.63:0/1168510249 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:06.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.564+0000 7f1dddd9b700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de0104050 0x7f1de0198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.565+0000 7f1dde59c700 1 -- 172.21.15.63:0/1168510249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de0104050 msgr2=0x7f1de0198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.565+0000 7f1dde59c700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de0104050 0x7f1de0198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.565+0000 7f1dde59c700 1 -- 172.21.15.63:0/1168510249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4009520 con 0x7f1de0101720 2024-05-09T06:37:06.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.565+0000 7f1dde59c700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0101720 0x7f1de0198850 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000e940 tx=0x7f1dd000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:06.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.565+0000 7f1dcf7fe700 1 -- 172.21.15.63:0/1168510249 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd0004a00 con 0x7f1de0101720 2024-05-09T06:37:06.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.565+0000 7f1dcf7fe700 1 -- 172.21.15.63:0/1168510249 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dd000cc80 con 0x7f1de0101720 2024-05-09T06:37:06.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.565+0000 7f1dcf7fe700 1 -- 172.21.15.63:0/1168510249 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd0005590 con 0x7f1de0101720 2024-05-09T06:37:06.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.565+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de019df50 con 0x7f1de0101720 2024-05-09T06:37:06.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.565+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de019e490 con 0x7f1de0101720 2024-05-09T06:37:06.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.567+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de004ebb0 con 0x7f1de0101720 2024-05-09T06:37:06.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.567+0000 7f1dcf7fe700 1 -- 172.21.15.63:0/1168510249 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1dd000c7c0 con 0x7f1de0101720 2024-05-09T06:37:06.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.568+0000 7f1dcf7fe700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1dc806cce0 0x7f1dc806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.568+0000 7f1dcf7fe700 1 -- 172.21.15.63:0/1168510249 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1dd0014070 con 0x7f1de0101720 2024-05-09T06:37:06.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.568+0000 7f1dddd9b700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1dc806cce0 0x7f1dc806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.569+0000 7f1dddd9b700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1dc806cce0 0x7f1dc806f1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400b6f0 tx=0x7f1dd4019040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:06.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.574+0000 7f1dcf7fe700 1 -- 172.21.15.63:0/1168510249 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1dd005a2e0 con 0x7f1de0101720 2024-05-09T06:37:06.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.954+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1de0109a00 con 0x7f1de0101720 2024-05-09T06:37:06.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.954+0000 7f1dcf7fe700 1 -- 172.21.15.63:0/1168510249 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1dd0059e70 con 0x7f1de0101720 2024-05-09T06:37:06.936 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:37:06.936 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:37:06.936 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:37:06.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:37:06.936 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:37:06.936 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:37:06.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:37:06.936 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:37:06.936 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:37:06.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:37:06.937 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:37:06.937 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:37:06.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:37:06.937 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:37:06.937 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 14 2024-05-09T06:37:06.937 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:37:06.937 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:37:06.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.956+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1dc806cce0 msgr2=0x7f1dc806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.956+0000 7f1de48e8700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1dc806cce0 0x7f1dc806f1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400b6f0 tx=0x7f1dd4019040 comp rx=0 tx=0).stop 2024-05-09T06:37:06.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.956+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0101720 msgr2=0x7f1de0198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.956+0000 7f1de48e8700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0101720 0x7f1de0198850 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000e940 tx=0x7f1dd000ec50 comp rx=0 tx=0).stop 2024-05-09T06:37:06.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.957+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 shutdown_connections 2024-05-09T06:37:06.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.957+0000 7f1de48e8700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1de0101720 0x7f1de0198850 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.957+0000 7f1de48e8700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f1dc806cce0 0x7f1dc806f1a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.957+0000 7f1de48e8700 1 --2- 172.21.15.63:0/1168510249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de0104050 0x7f1de0198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.957+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 >> 172.21.15.63:0/1168510249 conn(0x7f1de00fb0d0 msgr2=0x7f1de00ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:06.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.957+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 shutdown_connections 2024-05-09T06:37:06.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:06.957+0000 7f1de48e8700 1 -- 172.21.15.63:0/1168510249 wait complete. 2024-05-09T06:37:07.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.095+0000 7f31d99ec700 1 -- 172.21.15.63:0/4186118483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41017b0 msgr2=0x7f31d4103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.095+0000 7f31d99ec700 1 --2- 172.21.15.63:0/4186118483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41017b0 0x7f31d4103ba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009910 tx=0x7f31c4009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:07.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.095+0000 7f31d99ec700 1 -- 172.21.15.63:0/4186118483 shutdown_connections 2024-05-09T06:37:07.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.095+0000 7f31d99ec700 1 --2- 172.21.15.63:0/4186118483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d41040e0 0x7f31d41064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.095+0000 7f31d99ec700 1 --2- 172.21.15.63:0/4186118483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41017b0 0x7f31d4103ba0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.095+0000 7f31d99ec700 1 -- 172.21.15.63:0/4186118483 >> 172.21.15.63:0/4186118483 conn(0x7f31d40fb120 msgr2=0x7f31d40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:07.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.095+0000 7f31d99ec700 1 -- 172.21.15.63:0/4186118483 shutdown_connections 2024-05-09T06:37:07.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.095+0000 7f31d99ec700 1 -- 172.21.15.63:0/4186118483 wait complete. 2024-05-09T06:37:07.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.096+0000 7f31d99ec700 1 Processor -- start 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.096+0000 7f31d99ec700 1 -- start start 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d99ec700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d41017b0 0x7f31d41010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d99ec700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41040e0 0x7f31d40ff760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d99ec700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d40ffca0 con 0x7f31d41040e0 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d99ec700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d40ffde0 con 0x7f31d41017b0 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d27fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41040e0 0x7f31d40ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d27fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41040e0 0x7f31d40ff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40386/0 (socket says 172.21.15.63:40386) 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d27fc700 1 -- 172.21.15.63:0/1521251662 learned_addr learned my addr 172.21.15.63:0/1521251662 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d2ffd700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d41017b0 0x7f31d41010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d27fc700 1 -- 172.21.15.63:0/1521251662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d41017b0 msgr2=0x7f31d41010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d27fc700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d41017b0 0x7f31d41010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d27fc700 1 -- 172.21.15.63:0/1521251662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c40095f0 con 0x7f31d41040e0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.097+0000 7f31d27fc700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41040e0 0x7f31d40ff760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f31c800d8d0 tx=0x7f31c800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.098+0000 7f31d89ea700 1 -- 172.21.15.63:0/1521251662 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c80098a0 con 0x7f31d41040e0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.098+0000 7f31d89ea700 1 -- 172.21.15.63:0/1521251662 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31c8010460 con 0x7f31d41040e0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.098+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d41000c0 con 0x7f31d41040e0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.098+0000 7f31d89ea700 1 -- 172.21.15.63:0/1521251662 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c800f5f0 con 0x7f31d41040e0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.098+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d419e6a0 con 0x7f31d41040e0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.099+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d404ebb0 con 0x7f31d41040e0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.100+0000 7f31d89ea700 1 -- 172.21.15.63:0/1521251662 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f31c8009a00 con 0x7f31d41040e0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.101+0000 7f31d89ea700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f31bc06cce0 0x7f31bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.101+0000 7f31d89ea700 1 -- 172.21.15.63:0/1521251662 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f31c808bdf0 con 0x7f31d41040e0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.101+0000 7f31d2ffd700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f31bc06cce0 0x7f31bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.101+0000 7f31d2ffd700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f31bc06cce0 0x7f31bc06f1a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31c4005090 tx=0x7f31c4005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:07.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.107+0000 7f31d89ea700 1 -- 172.21.15.63:0/1521251662 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f31c8059c70 con 0x7f31d41040e0 2024-05-09T06:37:07.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.436+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f31d4109800 con 0x7f31d41040e0 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.437+0000 7f31d89ea700 1 -- 172.21.15.63:0/1521251662 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f31c8016770 con 0x7f31d41040e0 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:37:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:37:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:37:07.421 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:37:07.422 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:37:07.422 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.439+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f31bc06cce0 msgr2=0x7f31bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f31bc06cce0 0x7f31bc06f1a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31c4005090 tx=0x7f31c4005960 comp rx=0 tx=0).stop 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41040e0 msgr2=0x7f31d40ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41040e0 0x7f31d40ff760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f31c800d8d0 tx=0x7f31c800dbe0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 shutdown_connections 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f31bc06cce0 0x7f31bc06f1a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31d41017b0 0x7f31d41010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 --2- 172.21.15.63:0/1521251662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31d41040e0 0x7f31d40ff760 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f31c800d8d0 tx=0x7f31c800dbe0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 >> 172.21.15.63:0/1521251662 conn(0x7f31d40fb120 msgr2=0x7f31d40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 shutdown_connections 2024-05-09T06:37:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.440+0000 7f31d99ec700 1 -- 172.21.15.63:0/1521251662 wait complete. 2024-05-09T06:37:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:37:07.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.577+0000 7ff74b236700 1 -- 172.21.15.63:0/133909477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7440ff2a0 msgr2=0x7ff7440ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.577+0000 7ff74b236700 1 --2- 172.21.15.63:0/133909477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7440ff2a0 0x7ff7440ff6c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff738009910 tx=0x7ff738009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:07.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.578+0000 7ff74b236700 1 -- 172.21.15.63:0/133909477 shutdown_connections 2024-05-09T06:37:07.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.578+0000 7ff74b236700 1 --2- 172.21.15.63:0/133909477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7441004a0 0x7ff744100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.578+0000 7ff74b236700 1 --2- 172.21.15.63:0/133909477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7440ff2a0 0x7ff7440ff6c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.578+0000 7ff74b236700 1 -- 172.21.15.63:0/133909477 >> 172.21.15.63:0/133909477 conn(0x7ff7440fa840 msgr2=0x7ff7440fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:07.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:07 smithi063 ceph-mon[27171]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:07.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:07 smithi063 ceph-mon[27171]: pgmap v242: 65 pgs: 65 active+clean; 10 MiB data, 205 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s rd, 842 KiB/s wr, 44 op/s 2024-05-09T06:37:07.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:07 smithi063 ceph-mon[27171]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:07.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:07 smithi063 ceph-mon[27171]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:07.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:07 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1168510249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.578+0000 7ff74b236700 1 -- 172.21.15.63:0/133909477 shutdown_connections 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.578+0000 7ff74b236700 1 -- 172.21.15.63:0/133909477 wait complete. 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.578+0000 7ff74b236700 1 Processor -- start 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.578+0000 7ff74b236700 1 -- start start 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff74b236700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7440ff2a0 0x7ff744196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff74b236700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7441004a0 0x7ff744196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff74b236700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7441971e0 con 0x7ff7441004a0 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff74b236700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff744197320 con 0x7ff7440ff2a0 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff743fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7441004a0 0x7ff744196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff743fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7441004a0 0x7ff744196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40410/0 (socket says 172.21.15.63:40410) 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff743fff700 1 -- 172.21.15.63:0/2152514668 learned_addr learned my addr 172.21.15.63:0/2152514668 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff748fd2700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7440ff2a0 0x7ff744196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff743fff700 1 -- 172.21.15.63:0/2152514668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7440ff2a0 msgr2=0x7ff744196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff743fff700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7440ff2a0 0x7ff744196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.579+0000 7ff743fff700 1 -- 172.21.15.63:0/2152514668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380095f0 con 0x7ff7441004a0 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.580+0000 7ff743fff700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7441004a0 0x7ff744196bc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff73000ea30 tx=0x7ff73000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.580+0000 7ff741ffb700 1 -- 172.21.15.63:0/2152514668 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff73000cca0 con 0x7ff7441004a0 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.580+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74419bd30 con 0x7ff7441004a0 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.580+0000 7ff741ffb700 1 -- 172.21.15.63:0/2152514668 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff73000ce00 con 0x7ff7441004a0 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.580+0000 7ff741ffb700 1 -- 172.21.15.63:0/2152514668 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff730015760 con 0x7ff7441004a0 2024-05-09T06:37:07.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.580+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74419c2a0 con 0x7ff7441004a0 2024-05-09T06:37:07.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.582+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7441905a0 con 0x7ff7441004a0 2024-05-09T06:37:07.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.582+0000 7ff741ffb700 1 -- 172.21.15.63:0/2152514668 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff730015a30 con 0x7ff7441004a0 2024-05-09T06:37:07.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.583+0000 7ff741ffb700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff73406cce0 0x7ff73406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.583+0000 7ff741ffb700 1 -- 172.21.15.63:0/2152514668 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff73008c680 con 0x7ff7441004a0 2024-05-09T06:37:07.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.583+0000 7ff748fd2700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff73406cce0 0x7ff73406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.584+0000 7ff748fd2700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff73406cce0 0x7ff73406f1a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff73800b800 tx=0x7ff738009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:07.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.589+0000 7ff741ffb700 1 -- 172.21.15.63:0/2152514668 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff73005a390 con 0x7ff7441004a0 2024-05-09T06:37:07.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:07 smithi193 ceph-mon[35141]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:07.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:07 smithi193 ceph-mon[35141]: pgmap v242: 65 pgs: 65 active+clean; 10 MiB data, 205 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s rd, 842 KiB/s wr, 44 op/s 2024-05-09T06:37:07.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:07 smithi193 ceph-mon[35141]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:07.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:07 smithi193 ceph-mon[35141]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:07.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:07 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1168510249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:37:07.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.887+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff74402cea0 con 0x7ff73406cce0 2024-05-09T06:37:07.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.889+0000 7ff741ffb700 1 -- 172.21.15.63:0/2152514668 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7ff74402cea0 con 0x7ff73406cce0 2024-05-09T06:37:07.870 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:37:07.870 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:37:07.870 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:37:07.870 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:37:07.871 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-05-09T06:37:07.871 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "0/23 daemons upgraded", 2024-05-09T06:37:07.871 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac image on host smithi193", 2024-05-09T06:37:07.871 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:37:07.871 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff73406cce0 msgr2=0x7ff73406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff73406cce0 0x7ff73406f1a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff73800b800 tx=0x7ff738009f70 comp rx=0 tx=0).stop 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7441004a0 msgr2=0x7ff744196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7441004a0 0x7ff744196bc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff73000ea30 tx=0x7ff73000ed40 comp rx=0 tx=0).stop 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 shutdown_connections 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff73406cce0 0x7ff73406f1a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7440ff2a0 0x7ff744196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 --2- 172.21.15.63:0/2152514668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7441004a0 0x7ff744196bc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 >> 172.21.15.63:0/2152514668 conn(0x7ff7440fa840 msgr2=0x7ff744103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 shutdown_connections 2024-05-09T06:37:07.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:07.891+0000 7ff74b236700 1 -- 172.21.15.63:0/2152514668 wait complete. 2024-05-09T06:37:07.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.009+0000 7f5b10d58700 1 -- 172.21.15.63:0/2098730319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 msgr2=0x7f5b0c1063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.009+0000 7f5b10d58700 1 --2- 172.21.15.63:0/2098730319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 0x7f5b0c1063e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009910 tx=0x7f5b00009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:07.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.009+0000 7f5b10d58700 1 -- 172.21.15.63:0/2098730319 shutdown_connections 2024-05-09T06:37:07.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.009+0000 7f5b10d58700 1 --2- 172.21.15.63:0/2098730319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 0x7f5b0c1063e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.009+0000 7f5b10d58700 1 --2- 172.21.15.63:0/2098730319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b0c1016c0 0x7f5b0c103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.009+0000 7f5b10d58700 1 -- 172.21.15.63:0/2098730319 >> 172.21.15.63:0/2098730319 conn(0x7f5b0c0fb0d0 msgr2=0x7f5b0c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:07.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.009+0000 7f5b10d58700 1 -- 172.21.15.63:0/2098730319 shutdown_connections 2024-05-09T06:37:07.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.009+0000 7f5b10d58700 1 -- 172.21.15.63:0/2098730319 wait complete. 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.010+0000 7f5b10d58700 1 Processor -- start 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.010+0000 7f5b10d58700 1 -- start start 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.010+0000 7f5b10d58700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b0c1016c0 0x7f5b0c100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b10d58700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 0x7f5b0c0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b10d58700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0ffbb0 con 0x7f5b0c103ff0 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b10d58700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0ffcf0 con 0x7f5b0c1016c0 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b09d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 0x7f5b0c0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b09d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 0x7f5b0c0ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40432/0 (socket says 172.21.15.63:40432) 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b09d9b700 1 -- 172.21.15.63:0/3338048662 learned_addr learned my addr 172.21.15.63:0/3338048662 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:07.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b0a59c700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b0c1016c0 0x7f5b0c100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b09d9b700 1 -- 172.21.15.63:0/3338048662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b0c1016c0 msgr2=0x7f5b0c100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b09d9b700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b0c1016c0 0x7f5b0c100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b09d9b700 1 -- 172.21.15.63:0/3338048662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b000095f0 con 0x7f5b0c103ff0 2024-05-09T06:37:07.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b09d9b700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 0x7f5b0c0ff670 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5b000092e0 tx=0x7f5b0000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:07.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5afb7fe700 1 -- 172.21.15.63:0/3338048662 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b0001d070 con 0x7f5b0c103ff0 2024-05-09T06:37:07.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5afb7fe700 1 -- 172.21.15.63:0/3338048662 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b0000f460 con 0x7f5b0c103ff0 2024-05-09T06:37:07.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c0fff70 con 0x7f5b0c103ff0 2024-05-09T06:37:07.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.011+0000 7f5afb7fe700 1 -- 172.21.15.63:0/3338048662 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b00021620 con 0x7f5b0c103ff0 2024-05-09T06:37:07.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.012+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c1004e0 con 0x7f5b0c103ff0 2024-05-09T06:37:07.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.012+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c04ebb0 con 0x7f5b0c103ff0 2024-05-09T06:37:07.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.013+0000 7f5afb7fe700 1 -- 172.21.15.63:0/3338048662 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5b0000f5d0 con 0x7f5b0c103ff0 2024-05-09T06:37:07.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.014+0000 7f5afb7fe700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5af406cce0 0x7f5af406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.014+0000 7f5afb7fe700 1 -- 172.21.15.63:0/3338048662 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5b0008d350 con 0x7f5b0c103ff0 2024-05-09T06:37:07.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.014+0000 7f5b0a59c700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5af406cce0 0x7f5af406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.014+0000 7f5b0a59c700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5af406cce0 0x7f5af406f1a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5afc005950 tx=0x7f5afc00b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.018+0000 7f5afb7fe700 1 -- 172.21.15.63:0/3338048662 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5b0005b150 con 0x7f5b0c103ff0 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.383+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5b0c1004e0 con 0x7f5b0c103ff0 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.384+0000 7f5afb7fe700 1 -- 172.21.15.63:0/3338048662 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5b0005ace0 con 0x7f5b0c103ff0 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.386+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5af406cce0 msgr2=0x7f5af406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.386+0000 7f5b10d58700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5af406cce0 0x7f5af406f1a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5afc005950 tx=0x7f5afc00b410 comp rx=0 tx=0).stop 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.386+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 msgr2=0x7f5b0c0ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.386+0000 7f5b10d58700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 0x7f5b0c0ff670 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5b000092e0 tx=0x7f5b0000bd70 comp rx=0 tx=0).stop 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.386+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 shutdown_connections 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.386+0000 7f5b10d58700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f5af406cce0 0x7f5af406f1a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.386+0000 7f5b10d58700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b0c1016c0 0x7f5b0c100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.386+0000 7f5b10d58700 1 --2- 172.21.15.63:0/3338048662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b0c103ff0 0x7f5b0c0ff670 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.386+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 >> 172.21.15.63:0/3338048662 conn(0x7f5b0c0fb0d0 msgr2=0x7f5b0c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:08.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.387+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 shutdown_connections 2024-05-09T06:37:08.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:08.387+0000 7f5b10d58700 1 -- 172.21.15.63:0/3338048662 wait complete. 2024-05-09T06:37:08.593 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:08 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1521251662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:37:08.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:08 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1521251662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:37:09.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:09 smithi063 ceph-mon[27171]: pgmap v243: 65 pgs: 65 active+clean; 20 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-05-09T06:37:09.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:09 smithi063 ceph-mon[27171]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:09.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:09 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3338048662' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:37:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:09 smithi193 ceph-mon[35141]: pgmap v243: 65 pgs: 65 active+clean; 20 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-05-09T06:37:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:09 smithi193 ceph-mon[35141]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:09 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3338048662' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:37:11.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:10 smithi193 ceph-mon[35141]: pgmap v244: 65 pgs: 65 active+clean; 33 MiB data, 313 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-05-09T06:37:11.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:10 smithi193 ceph-mon[35141]: Upgrade: Updating mgr.smithi193.mmmqzh 2024-05-09T06:37:11.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:11.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:37:11.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:37:11.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:10 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:37:11.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:10 smithi193 ceph-mon[35141]: Deploying daemon mgr.smithi193.mmmqzh on smithi193 2024-05-09T06:37:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:10 smithi063 ceph-mon[27171]: pgmap v244: 65 pgs: 65 active+clean; 33 MiB data, 313 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-05-09T06:37:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:10 smithi063 ceph-mon[27171]: Upgrade: Updating mgr.smithi193.mmmqzh 2024-05-09T06:37:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:37:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:37:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:10 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:37:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:10 smithi063 ceph-mon[27171]: Deploying daemon mgr.smithi193.mmmqzh on smithi193 2024-05-09T06:37:12.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:11 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:37:12.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:11 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:37:13.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:12 smithi063 ceph-mon[27171]: pgmap v245: 65 pgs: 65 active+clean; 37 MiB data, 345 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 3.2 MiB/s wr, 295 op/s 2024-05-09T06:37:13.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:12 smithi193 ceph-mon[35141]: pgmap v245: 65 pgs: 65 active+clean; 37 MiB data, 345 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 3.2 MiB/s wr, 295 op/s 2024-05-09T06:37:15.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:14 smithi063 ceph-mon[27171]: pgmap v246: 65 pgs: 65 active+clean; 43 MiB data, 360 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 3.7 MiB/s wr, 325 op/s 2024-05-09T06:37:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:14 smithi193 ceph-mon[35141]: pgmap v246: 65 pgs: 65 active+clean; 43 MiB data, 360 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 3.7 MiB/s wr, 325 op/s 2024-05-09T06:37:17.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:16 smithi063 ceph-mon[27171]: pgmap v247: 65 pgs: 65 active+clean; 56 MiB data, 393 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 402 op/s 2024-05-09T06:37:17.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:16 smithi193 ceph-mon[35141]: pgmap v247: 65 pgs: 65 active+clean; 56 MiB data, 393 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 402 op/s 2024-05-09T06:37:19.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:18 smithi063 ceph-mon[27171]: pgmap v248: 65 pgs: 65 active+clean; 60 MiB data, 412 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 408 op/s 2024-05-09T06:37:19.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:18 smithi193 ceph-mon[35141]: pgmap v248: 65 pgs: 65 active+clean; 60 MiB data, 412 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 408 op/s 2024-05-09T06:37:21.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:20 smithi063 ceph-mon[27171]: pgmap v249: 65 pgs: 65 active+clean; 68 MiB data, 457 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 423 op/s 2024-05-09T06:37:21.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:20 smithi193 ceph-mon[35141]: pgmap v249: 65 pgs: 65 active+clean; 68 MiB data, 457 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 423 op/s 2024-05-09T06:37:23.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:22 smithi063 ceph-mon[27171]: pgmap v250: 65 pgs: 65 active+clean; 82 MiB data, 494 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 424 op/s 2024-05-09T06:37:23.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:22 smithi193 ceph-mon[35141]: pgmap v250: 65 pgs: 65 active+clean; 82 MiB data, 494 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 424 op/s 2024-05-09T06:37:25.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:24 smithi063 ceph-mon[27171]: pgmap v251: 65 pgs: 65 active+clean; 84 MiB data, 494 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-05-09T06:37:25.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:24 smithi193 ceph-mon[35141]: pgmap v251: 65 pgs: 65 active+clean; 84 MiB data, 494 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-05-09T06:37:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:26 smithi063 ceph-mon[27171]: pgmap v252: 65 pgs: 65 active+clean; 97 MiB data, 516 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 398 op/s 2024-05-09T06:37:27.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:26 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:37:27.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:26 smithi193 ceph-mon[35141]: pgmap v252: 65 pgs: 65 active+clean; 97 MiB data, 516 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 398 op/s 2024-05-09T06:37:27.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:26 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:37:29.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:28 smithi063 ceph-mon[27171]: pgmap v253: 65 pgs: 65 active+clean; 100 MiB data, 529 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 384 op/s 2024-05-09T06:37:29.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:28 smithi193 ceph-mon[35141]: pgmap v253: 65 pgs: 65 active+clean; 100 MiB data, 529 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 384 op/s 2024-05-09T06:37:31.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:30 smithi063 ceph-mon[27171]: pgmap v254: 65 pgs: 65 active+clean; 104 MiB data, 571 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 410 op/s 2024-05-09T06:37:31.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:30 smithi193 ceph-mon[35141]: pgmap v254: 65 pgs: 65 active+clean; 104 MiB data, 571 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 410 op/s 2024-05-09T06:37:33.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:32 smithi063 ceph-mon[27171]: pgmap v255: 65 pgs: 65 active+clean; 111 MiB data, 607 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-05-09T06:37:33.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:32 smithi193 ceph-mon[35141]: pgmap v255: 65 pgs: 65 active+clean; 111 MiB data, 607 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-05-09T06:37:35.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:34 smithi063 ceph-mon[27171]: pgmap v256: 65 pgs: 65 active+clean; 112 MiB data, 620 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 361 op/s 2024-05-09T06:37:35.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:34 smithi193 ceph-mon[35141]: pgmap v256: 65 pgs: 65 active+clean; 112 MiB data, 620 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 361 op/s 2024-05-09T06:37:37.257 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:36 smithi193 ceph-mon[35141]: pgmap v257: 65 pgs: 65 active+clean; 118 MiB data, 641 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2024-05-09T06:37:37.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:36 smithi063 ceph-mon[27171]: pgmap v257: 65 pgs: 65 active+clean; 118 MiB data, 641 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2024-05-09T06:37:38.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.503+0000 7fd8fe047700 1 -- 172.21.15.63:0/3665870498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8101e50 msgr2=0x7fd8f81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:38.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.503+0000 7fd8fe047700 1 --2- 172.21.15.63:0/3665870498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8101e50 0x7fd8f81022b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009910 tx=0x7fd8ec009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:38.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.503+0000 7fd8fe047700 1 -- 172.21.15.63:0/3665870498 shutdown_connections 2024-05-09T06:37:38.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.503+0000 7fd8fe047700 1 --2- 172.21.15.63:0/3665870498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8101e50 0x7fd8f81022b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:38.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.503+0000 7fd8fe047700 1 --2- 172.21.15.63:0/3665870498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8f8100c50 0x7fd8f8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:38.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.503+0000 7fd8fe047700 1 -- 172.21.15.63:0/3665870498 >> 172.21.15.63:0/3665870498 conn(0x7fd8f80fc210 msgr2=0x7fd8f80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:38.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.503+0000 7fd8fe047700 1 -- 172.21.15.63:0/3665870498 shutdown_connections 2024-05-09T06:37:38.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.503+0000 7fd8fe047700 1 -- 172.21.15.63:0/3665870498 wait complete. 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8fe047700 1 Processor -- start 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8fe047700 1 -- start start 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8fe047700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8100c50 0x7fd8f8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8fe047700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8f8101e50 0x7fd8f8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8fe047700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f81993c0 con 0x7fd8f8100c50 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8fe047700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f8199500 con 0x7fd8f8101e50 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8f77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8100c50 0x7fd8f8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8f77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8100c50 0x7fd8f8198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57326/0 (socket says 172.21.15.63:57326) 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8f77fe700 1 -- 172.21.15.63:0/149880188 learned_addr learned my addr 172.21.15.63:0/149880188 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8f6ffd700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8f8101e50 0x7fd8f8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:38.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8f77fe700 1 -- 172.21.15.63:0/149880188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8f8101e50 msgr2=0x7fd8f8198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:38.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8f77fe700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8f8101e50 0x7fd8f8198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:38.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.504+0000 7fd8f77fe700 1 -- 172.21.15.63:0/149880188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec0095f0 con 0x7fd8f8100c50 2024-05-09T06:37:38.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.505+0000 7fd8f77fe700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8100c50 0x7fd8f8198860 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d900 tx=0x7fd8e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:38.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.505+0000 7fd8f4ff9700 1 -- 172.21.15.63:0/149880188 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8e80041f0 con 0x7fd8f8100c50 2024-05-09T06:37:38.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.505+0000 7fd8f4ff9700 1 -- 172.21.15.63:0/149880188 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8e8004350 con 0x7fd8f8100c50 2024-05-09T06:37:38.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.505+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f819df10 con 0x7fd8f8100c50 2024-05-09T06:37:38.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.505+0000 7fd8f4ff9700 1 -- 172.21.15.63:0/149880188 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8e8003cf0 con 0x7fd8f8100c50 2024-05-09T06:37:38.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.505+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f819e450 con 0x7fd8f8100c50 2024-05-09T06:37:38.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.506+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f8192a00 con 0x7fd8f8100c50 2024-05-09T06:37:38.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.507+0000 7fd8f4ff9700 1 -- 172.21.15.63:0/149880188 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd8e8009730 con 0x7fd8f8100c50 2024-05-09T06:37:38.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.507+0000 7fd8f4ff9700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd8e006ccf0 0x7fd8e006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:38.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.507+0000 7fd8f4ff9700 1 -- 172.21.15.63:0/149880188 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd8e808c070 con 0x7fd8f8100c50 2024-05-09T06:37:38.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.507+0000 7fd8f6ffd700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd8e006ccf0 0x7fd8e006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:38.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.508+0000 7fd8f6ffd700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd8e006ccf0 0x7fd8e006f1b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec006010 tx=0x7fd8ec005180 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:38.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.512+0000 7fd8f4ff9700 1 -- 172.21.15.63:0/149880188 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd8e8059ef0 con 0x7fd8f8100c50 2024-05-09T06:37:38.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.769+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8f802ceb0 con 0x7fd8e006ccf0 2024-05-09T06:37:38.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.771+0000 7fd8f4ff9700 1 -- 172.21.15.63:0/149880188 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fd8f802ceb0 con 0x7fd8e006ccf0 2024-05-09T06:37:38.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd8e006ccf0 msgr2=0x7fd8e006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:38.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd8e006ccf0 0x7fd8e006f1b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec006010 tx=0x7fd8ec005180 comp rx=0 tx=0).stop 2024-05-09T06:37:38.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8100c50 msgr2=0x7fd8f8198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:38.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8100c50 0x7fd8f8198860 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d900 tx=0x7fd8e800dc10 comp rx=0 tx=0).stop 2024-05-09T06:37:38.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 shutdown_connections 2024-05-09T06:37:38.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8f8100c50 0x7fd8f8198860 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:38.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7fd8e006ccf0 0x7fd8e006f1b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:38.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 --2- 172.21.15.63:0/149880188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8f8101e50 0x7fd8f8198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:38.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 >> 172.21.15.63:0/149880188 conn(0x7fd8f80fc210 msgr2=0x7fd8f81053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:38.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 shutdown_connections 2024-05-09T06:37:38.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.773+0000 7fd8fe047700 1 -- 172.21.15.63:0/149880188 wait complete. 2024-05-09T06:37:38.763 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:37:38.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.902+0000 7f209f505700 1 -- 172.21.15.63:0/3969963938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 msgr2=0x7f20981011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:38.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.902+0000 7f209f505700 1 --2- 172.21.15.63:0/3969963938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 0x7f20981011b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2094009910 tx=0x7f2094009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:38.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.902+0000 7f209f505700 1 -- 172.21.15.63:0/3969963938 shutdown_connections 2024-05-09T06:37:38.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.902+0000 7f209f505700 1 --2- 172.21.15.63:0/3969963938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 0x7f20981011b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:38.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.902+0000 7f209f505700 1 --2- 172.21.15.63:0/3969963938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20980ffb50 0x7f20980fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:38.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.902+0000 7f209f505700 1 -- 172.21.15.63:0/3969963938 >> 172.21.15.63:0/3969963938 conn(0x7f20980fb0f0 msgr2=0x7f20980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:38.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.902+0000 7f209f505700 1 -- 172.21.15.63:0/3969963938 shutdown_connections 2024-05-09T06:37:38.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.902+0000 7f209f505700 1 -- 172.21.15.63:0/3969963938 wait complete. 2024-05-09T06:37:38.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209f505700 1 Processor -- start 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209f505700 1 -- start start 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209f505700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20980ffb50 0x7f2098198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209f505700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 0x7f2098198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209f505700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20981993a0 con 0x7f2098100d50 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209f505700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20981994e0 con 0x7f20980ffb50 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209caa0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 0x7f2098198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209d2a1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20980ffb50 0x7f2098198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209caa0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 0x7f2098198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57340/0 (socket says 172.21.15.63:57340) 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209d2a1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20980ffb50 0x7f2098198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.63:60764/0 (socket says 172.21.15.63:60764) 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.903+0000 7f209caa0700 1 -- 172.21.15.63:0/2450435798 learned_addr learned my addr 172.21.15.63:0/2450435798 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f209d2a1700 1 -- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 msgr2=0x7f2098198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f209d2a1700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 0x7f2098198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f209d2a1700 1 -- 172.21.15.63:0/2450435798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940095f0 con 0x7f20980ffb50 2024-05-09T06:37:38.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f209caa0700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 0x7f2098198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:38.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f209d2a1700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20980ffb50 0x7f2098198840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f208c00d6a0 tx=0x7f208c00d9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:38.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f208a7fc700 1 -- 172.21.15.63:0/2450435798 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f208c00f860 con 0x7f20980ffb50 2024-05-09T06:37:38.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209819df40 con 0x7f20980ffb50 2024-05-09T06:37:38.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f208a7fc700 1 -- 172.21.15.63:0/2450435798 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f208c00fe80 con 0x7f20980ffb50 2024-05-09T06:37:38.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209819e400 con 0x7f20980ffb50 2024-05-09T06:37:38.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.904+0000 7f208a7fc700 1 -- 172.21.15.63:0/2450435798 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f208c0185f0 con 0x7f20980ffb50 2024-05-09T06:37:38.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.906+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207c005320 con 0x7f20980ffb50 2024-05-09T06:37:38.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.907+0000 7f208a7fc700 1 -- 172.21.15.63:0/2450435798 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f208c0108d0 con 0x7f20980ffb50 2024-05-09T06:37:38.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.907+0000 7f208a7fc700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f208406cd50 0x7f208406f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:38.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.907+0000 7f208a7fc700 1 -- 172.21.15.63:0/2450435798 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f208c08c300 con 0x7f20980ffb50 2024-05-09T06:37:38.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.907+0000 7f209caa0700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f208406cd50 0x7f208406f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:38.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.908+0000 7f209caa0700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f208406cd50 0x7f208406f210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f20940051c0 tx=0x7f209400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:38.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:38.914+0000 7f208a7fc700 1 -- 172.21.15.63:0/2450435798 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f208c05a120 con 0x7f20980ffb50 2024-05-09T06:37:39.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:38 smithi063 ceph-mon[27171]: pgmap v258: 65 pgs: 65 active+clean; 124 MiB data, 661 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 391 op/s 2024-05-09T06:37:39.171 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:38 smithi193 ceph-mon[35141]: pgmap v258: 65 pgs: 65 active+clean; 124 MiB data, 661 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 391 op/s 2024-05-09T06:37:39.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.215+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f207c000bf0 con 0x7f208406cd50 2024-05-09T06:37:39.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.216+0000 7f208a7fc700 1 -- 172.21.15.63:0/2450435798 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f207c000bf0 con 0x7f208406cd50 2024-05-09T06:37:39.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f208406cd50 msgr2=0x7f208406f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:39.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f208406cd50 0x7f208406f210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f20940051c0 tx=0x7f209400b540 comp rx=0 tx=0).stop 2024-05-09T06:37:39.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20980ffb50 msgr2=0x7f2098198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:39.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20980ffb50 0x7f2098198840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f208c00d6a0 tx=0x7f208c00d9b0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 shutdown_connections 2024-05-09T06:37:39.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f208406cd50 0x7f208406f210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20980ffb50 0x7f2098198840 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 --2- 172.21.15.63:0/2450435798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2098100d50 0x7f2098198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 >> 172.21.15.63:0/2450435798 conn(0x7f20980fb0f0 msgr2=0x7f2098069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 shutdown_connections 2024-05-09T06:37:39.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.219+0000 7f209f505700 1 -- 172.21.15.63:0/2450435798 wait complete. 2024-05-09T06:37:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.360+0000 7f2a6658a700 1 -- 172.21.15.63:0/1745522547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a600ffdd0 msgr2=0x7f2a60100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.360+0000 7f2a6658a700 1 --2- 172.21.15.63:0/1745522547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a600ffdd0 0x7f2a60100250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009910 tx=0x7f2a54009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.360+0000 7f2a6658a700 1 -- 172.21.15.63:0/1745522547 shutdown_connections 2024-05-09T06:37:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.360+0000 7f2a6658a700 1 --2- 172.21.15.63:0/1745522547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a600ffdd0 0x7f2a60100250 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.360+0000 7f2a6658a700 1 --2- 172.21.15.63:0/1745522547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a60101170 0x7f2a60101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.360+0000 7f2a6658a700 1 -- 172.21.15.63:0/1745522547 >> 172.21.15.63:0/1745522547 conn(0x7f2a600fa8a0 msgr2=0x7f2a600fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.361+0000 7f2a6658a700 1 -- 172.21.15.63:0/1745522547 shutdown_connections 2024-05-09T06:37:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.361+0000 7f2a6658a700 1 -- 172.21.15.63:0/1745522547 wait complete. 2024-05-09T06:37:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.361+0000 7f2a6658a700 1 Processor -- start 2024-05-09T06:37:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.361+0000 7f2a6658a700 1 -- start start 2024-05-09T06:37:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a6658a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a600ffdd0 0x7f2a60196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a6658a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a60101170 0x7f2a60196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a6658a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a601972a0 con 0x7f2a60101170 2024-05-09T06:37:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a6658a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a601973e0 con 0x7f2a600ffdd0 2024-05-09T06:37:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a5f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a60101170 0x7f2a60196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a5f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a60101170 0x7f2a60196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57354/0 (socket says 172.21.15.63:57354) 2024-05-09T06:37:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a5f7fe700 1 -- 172.21.15.63:0/2889232437 learned_addr learned my addr 172.21.15.63:0/2889232437 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a5ffff700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a600ffdd0 0x7f2a60196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a5f7fe700 1 -- 172.21.15.63:0/2889232437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a600ffdd0 msgr2=0x7f2a60196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a5f7fe700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a600ffdd0 0x7f2a60196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a5f7fe700 1 -- 172.21.15.63:0/2889232437 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540095f0 con 0x7f2a60101170 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a5f7fe700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a60101170 0x7f2a60196c80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2a540092e0 tx=0x7f2a5400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.362+0000 7f2a5d7fa700 1 -- 172.21.15.63:0/2889232437 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a5401d070 con 0x7f2a60101170 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.363+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6019bd90 con 0x7f2a60101170 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.363+0000 7f2a5d7fa700 1 -- 172.21.15.63:0/2889232437 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a5400f460 con 0x7f2a60101170 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.363+0000 7f2a5d7fa700 1 -- 172.21.15.63:0/2889232437 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a54021640 con 0x7f2a60101170 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.363+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a60102000 con 0x7f2a60101170 2024-05-09T06:37:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.364+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6004ebb0 con 0x7f2a60101170 2024-05-09T06:37:39.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.365+0000 7f2a5d7fa700 1 -- 172.21.15.63:0/2889232437 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2a5400f5d0 con 0x7f2a60101170 2024-05-09T06:37:39.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.366+0000 7f2a5d7fa700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a4806cce0 0x7f2a4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:39.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.366+0000 7f2a5d7fa700 1 -- 172.21.15.63:0/2889232437 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2a5408da70 con 0x7f2a60101170 2024-05-09T06:37:39.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.366+0000 7f2a5ffff700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a4806cce0 0x7f2a4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:39.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.367+0000 7f2a5ffff700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a4806cce0 0x7f2a4806f1a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a50006fd0 tx=0x7f2a50008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:39.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.370+0000 7f2a5d7fa700 1 -- 172.21.15.63:0/2889232437 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2a5405b8f0 con 0x7f2a60101170 2024-05-09T06:37:39.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.602+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a600ff400 con 0x7f2a4806cce0 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.617+0000 7f2a5d7fa700 1 -- 172.21.15.63:0/2889232437 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f2a600ff400 con 0x7f2a4806cce0 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (6m) 2m ago 8m 25.3M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (8m) 2m ago 8m 7465k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (7m) 2m ago 7m 7382k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (8m) 2m ago 8m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (7m) 2m ago 7m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6m) 2m ago 7m 87.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (2m) 2m ago 2m 11.7M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:37:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (2m) 2m ago 2m 14.7M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (2m) 2m ago 2m 16.5M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (2m) 2m ago 2m 13.2M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:9283,8765,8443 running (9m) 2m ago 9m 500M - 18.2.2-1297-g1758e8ea 196422f1b641 a66e4acc828f 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 starting - - - - 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (9m) 2m ago 9m 45.4M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 7d88727e4bbb 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (6m) 2m ago 6m 37.0M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (8m) 2m ago 8m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (7m) 2m ago 7m 15.4M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (6m) 2m ago 6m 47.2M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 2m ago 5m 48.3M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (5m) 2m ago 5m 46.4M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (4m) 2m ago 4m 46.7M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (4m) 2m ago 4m 51.7M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (4m) 2m ago 4m 53.1M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:37:39.599 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 2m ago 7m 58.2M - 2.43.0 a07b618ecd1d 9f4deb8cc8b4 2024-05-09T06:37:39.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.620+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a4806cce0 msgr2=0x7f2a4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:39.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.620+0000 7f2a6658a700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a4806cce0 0x7f2a4806f1a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a50006fd0 tx=0x7f2a50008040 comp rx=0 tx=0).stop 2024-05-09T06:37:39.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.620+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a60101170 msgr2=0x7f2a60196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:39.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.620+0000 7f2a6658a700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a60101170 0x7f2a60196c80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2a540092e0 tx=0x7f2a5400be20 comp rx=0 tx=0).stop 2024-05-09T06:37:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.620+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 shutdown_connections 2024-05-09T06:37:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.620+0000 7f2a6658a700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f2a4806cce0 0x7f2a4806f1a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.620+0000 7f2a6658a700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a600ffdd0 0x7f2a60196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.620+0000 7f2a6658a700 1 --2- 172.21.15.63:0/2889232437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a60101170 0x7f2a60196c80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.620+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 >> 172.21.15.63:0/2889232437 conn(0x7f2a600fa8a0 msgr2=0x7f2a600fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.621+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 shutdown_connections 2024-05-09T06:37:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.621+0000 7f2a6658a700 1 -- 172.21.15.63:0/2889232437 wait complete. 2024-05-09T06:37:39.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.761+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/1649207682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 msgr2=0x7f3a08106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:39.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.761+0000 7f3a0f0c5700 1 --2- 172.21.15.63:0/1649207682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 0x7f3a08106400 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009910 tx=0x7f39f4009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:39.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.761+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/1649207682 shutdown_connections 2024-05-09T06:37:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.761+0000 7f3a0f0c5700 1 --2- 172.21.15.63:0/1649207682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 0x7f3a08106400 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.761+0000 7f3a0f0c5700 1 --2- 172.21.15.63:0/1649207682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a081016e0 0x7f3a08103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.761+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/1649207682 >> 172.21.15.63:0/1649207682 conn(0x7f3a080fb0f0 msgr2=0x7f3a080fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.762+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/1649207682 shutdown_connections 2024-05-09T06:37:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.762+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/1649207682 wait complete. 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.763+0000 7f3a0f0c5700 1 Processor -- start 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.763+0000 7f3a0f0c5700 1 -- start start 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.763+0000 7f3a0f0c5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a081016e0 0x7f3a08196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.763+0000 7f3a0f0c5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 0x7f3a08196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.763+0000 7f3a0f0c5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08197180 con 0x7f3a08104010 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.763+0000 7f3a0f0c5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a081972c0 con 0x7f3a081016e0 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.763+0000 7f3a07fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 0x7f3a08196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.763+0000 7f3a07fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 0x7f3a08196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57374/0 (socket says 172.21.15.63:57374) 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.763+0000 7f3a07fff700 1 -- 172.21.15.63:0/3578907601 learned_addr learned my addr 172.21.15.63:0/3578907601 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a07fff700 1 -- 172.21.15.63:0/3578907601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a081016e0 msgr2=0x7f3a08196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a0ce61700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a081016e0 0x7f3a08196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:39.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a07fff700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a081016e0 0x7f3a08196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:39.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a07fff700 1 -- 172.21.15.63:0/3578907601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f40095f0 con 0x7f3a08104010 2024-05-09T06:37:39.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a0ce61700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a081016e0 0x7f3a08196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:39.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a07fff700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 0x7f3a08196b60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f39f4006010 tx=0x7f39f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:39.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a05ffb700 1 -- 172.21.15.63:0/3578907601 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39f401d070 con 0x7f3a08104010 2024-05-09T06:37:39.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a05ffb700 1 -- 172.21.15.63:0/3578907601 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39f400f460 con 0x7f3a08104010 2024-05-09T06:37:39.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0819bcc0 con 0x7f3a08104010 2024-05-09T06:37:39.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.764+0000 7f3a05ffb700 1 -- 172.21.15.63:0/3578907601 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39f4021640 con 0x7f3a08104010 2024-05-09T06:37:39.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.765+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0819c230 con 0x7f3a08104010 2024-05-09T06:37:39.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.766+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0804ebb0 con 0x7f3a08104010 2024-05-09T06:37:39.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.767+0000 7f3a05ffb700 1 -- 172.21.15.63:0/3578907601 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f39f400fab0 con 0x7f3a08104010 2024-05-09T06:37:39.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.767+0000 7f3a05ffb700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806cce0 0x7f39f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:39.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.768+0000 7f3a05ffb700 1 -- 172.21.15.63:0/3578907601 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f39f408d4b0 con 0x7f3a08104010 2024-05-09T06:37:39.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.768+0000 7f3a0ce61700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806cce0 0x7f39f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:39.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.768+0000 7f3a0ce61700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806cce0 0x7f39f806f1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39fc005fd0 tx=0x7f39fc005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:39.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:39.774+0000 7f3a05ffb700 1 -- 172.21.15.63:0/3578907601 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f39f405b330 con 0x7f3a08104010 2024-05-09T06:37:40.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.121+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a081099b0 con 0x7f3a08104010 2024-05-09T06:37:40.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.121+0000 7f3a05ffb700 1 -- 172.21.15.63:0/3578907601 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f39f4057030 con 0x7f3a08104010 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:37:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:37:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 14 2024-05-09T06:37:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:37:40.103 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806cce0 msgr2=0x7f39f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806cce0 0x7f39f806f1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39fc005fd0 tx=0x7f39fc005ee0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 msgr2=0x7f3a08196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 0x7f3a08196b60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f39f4006010 tx=0x7f39f400be20 comp rx=0 tx=0).stop 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 shutdown_connections 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f39f806cce0 0x7f39f806f1a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a081016e0 0x7f3a08196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 --2- 172.21.15.63:0/3578907601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a08104010 0x7f3a08196b60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 >> 172.21.15.63:0/3578907601 conn(0x7f3a080fb0f0 msgr2=0x7f3a080fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 shutdown_connections 2024-05-09T06:37:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.123+0000 7f3a0f0c5700 1 -- 172.21.15.63:0/3578907601 wait complete. 2024-05-09T06:37:40.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.238+0000 7f09af3eb700 1 -- 172.21.15.63:0/1513849669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8101730 msgr2=0x7f09a8103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.238+0000 7f09af3eb700 1 --2- 172.21.15.63:0/1513849669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8101730 0x7f09a8103b20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009910 tx=0x7f09a4009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:40.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.238+0000 7f09af3eb700 1 -- 172.21.15.63:0/1513849669 shutdown_connections 2024-05-09T06:37:40.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.238+0000 7f09af3eb700 1 --2- 172.21.15.63:0/1513849669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a8104060 0x7f09a8106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.238+0000 7f09af3eb700 1 --2- 172.21.15.63:0/1513849669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8101730 0x7f09a8103b20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.238+0000 7f09af3eb700 1 -- 172.21.15.63:0/1513849669 >> 172.21.15.63:0/1513849669 conn(0x7f09a80fb120 msgr2=0x7f09a80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:40.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.238+0000 7f09af3eb700 1 -- 172.21.15.63:0/1513849669 shutdown_connections 2024-05-09T06:37:40.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.238+0000 7f09af3eb700 1 -- 172.21.15.63:0/1513849669 wait complete. 2024-05-09T06:37:40.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09af3eb700 1 Processor -- start 2024-05-09T06:37:40.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09af3eb700 1 -- start start 2024-05-09T06:37:40.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09af3eb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a8101730 0x7f09a8101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:40.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09af3eb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8104060 0x7f09a80ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:40.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09af3eb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80ffc20 con 0x7f09a8104060 2024-05-09T06:37:40.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09af3eb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80ffd60 con 0x7f09a8101730 2024-05-09T06:37:40.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09ac986700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8104060 0x7f09a80ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:40.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09ac986700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8104060 0x7f09a80ff6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57388/0 (socket says 172.21.15.63:57388) 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09ac986700 1 -- 172.21.15.63:0/785887695 learned_addr learned my addr 172.21.15.63:0/785887695 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.239+0000 7f09ad187700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a8101730 0x7f09a8101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.240+0000 7f09ac986700 1 -- 172.21.15.63:0/785887695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a8101730 msgr2=0x7f09a8101040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.240+0000 7f09ac986700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a8101730 0x7f09a8101040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.240+0000 7f09ac986700 1 -- 172.21.15.63:0/785887695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40095f0 con 0x7f09a8104060 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.240+0000 7f09ac986700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8104060 0x7f09a80ff6e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f099800e920 tx=0x7f099800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.240+0000 7f099e7fc700 1 -- 172.21.15.63:0/785887695 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099800c9d0 con 0x7f09a8104060 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.240+0000 7f099e7fc700 1 -- 172.21.15.63:0/785887695 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0998010430 con 0x7f09a8104060 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.240+0000 7f099e7fc700 1 -- 172.21.15.63:0/785887695 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0998018630 con 0x7f09a8104060 2024-05-09T06:37:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.241+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a8100040 con 0x7f09a8104060 2024-05-09T06:37:40.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.241+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a81004d0 con 0x7f09a8104060 2024-05-09T06:37:40.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.242+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a804ebb0 con 0x7f09a8104060 2024-05-09T06:37:40.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.243+0000 7f099e7fc700 1 -- 172.21.15.63:0/785887695 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f09980105a0 con 0x7f09a8104060 2024-05-09T06:37:40.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.243+0000 7f099e7fc700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f099406cce0 0x7f099406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:40.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.243+0000 7f099e7fc700 1 -- 172.21.15.63:0/785887695 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0998014070 con 0x7f09a8104060 2024-05-09T06:37:40.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.243+0000 7f09ad187700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f099406cce0 0x7f099406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:40.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.244+0000 7f09ad187700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f099406cce0 0x7f099406f1a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f09a4006010 tx=0x7f09a4009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:40.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.249+0000 7f099e7fc700 1 -- 172.21.15.63:0/785887695 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0998052c50 con 0x7f09a8104060 2024-05-09T06:37:40.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:40 smithi063 ceph-mon[27171]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:40.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:40 smithi063 ceph-mon[27171]: from='client.24423 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:40.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:40 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:40.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:40 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:40.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:40 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:37:40.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:40 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3578907601' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:37:40.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.553+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f09a8109800 con 0x7f09a8104060 2024-05-09T06:37:40.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.553+0000 7f099e7fc700 1 -- 172.21.15.63:0/785887695 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f099805a340 con 0x7f09a8104060 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:37:40.534 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:37:40.535 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:37:40.536 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:37:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.555+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f099406cce0 msgr2=0x7f099406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.555+0000 7f09af3eb700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f099406cce0 0x7f099406f1a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f09a4006010 tx=0x7f09a4009f70 comp rx=0 tx=0).stop 2024-05-09T06:37:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.555+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8104060 msgr2=0x7f09a80ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.555+0000 7f09af3eb700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8104060 0x7f09a80ff6e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f099800e920 tx=0x7f099800ec30 comp rx=0 tx=0).stop 2024-05-09T06:37:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.556+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 shutdown_connections 2024-05-09T06:37:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.556+0000 7f09af3eb700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f099406cce0 0x7f099406f1a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.556+0000 7f09af3eb700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a8101730 0x7f09a8101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.556+0000 7f09af3eb700 1 --2- 172.21.15.63:0/785887695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a8104060 0x7f09a80ff6e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.556+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 >> 172.21.15.63:0/785887695 conn(0x7f09a80fb120 msgr2=0x7f09a80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:40.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.556+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 shutdown_connections 2024-05-09T06:37:40.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.556+0000 7f09af3eb700 1 -- 172.21.15.63:0/785887695 wait complete. 2024-05-09T06:37:40.538 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:37:40.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:40 smithi193 ceph-mon[35141]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:40.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:40 smithi193 ceph-mon[35141]: from='client.24423 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:40.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:40 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:40.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:40 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:40.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:40 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:37:40.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:40 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3578907601' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:37:40.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.666+0000 7f04e58f5700 1 -- 172.21.15.63:0/3046547132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e00ffb50 msgr2=0x7f04e00fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.666+0000 7f04e58f5700 1 --2- 172.21.15.63:0/3046547132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e00ffb50 0x7f04e00fff70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009910 tx=0x7f04d0009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:40.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.666+0000 7f04e58f5700 1 -- 172.21.15.63:0/3046547132 shutdown_connections 2024-05-09T06:37:40.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.666+0000 7f04e58f5700 1 --2- 172.21.15.63:0/3046547132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e0100d50 0x7f04e01011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.666+0000 7f04e58f5700 1 --2- 172.21.15.63:0/3046547132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e00ffb50 0x7f04e00fff70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.666+0000 7f04e58f5700 1 -- 172.21.15.63:0/3046547132 >> 172.21.15.63:0/3046547132 conn(0x7f04e00fb0f0 msgr2=0x7f04e00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:40.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.666+0000 7f04e58f5700 1 -- 172.21.15.63:0/3046547132 shutdown_connections 2024-05-09T06:37:40.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.666+0000 7f04e58f5700 1 -- 172.21.15.63:0/3046547132 wait complete. 2024-05-09T06:37:40.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04e58f5700 1 Processor -- start 2024-05-09T06:37:40.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04e58f5700 1 -- start start 2024-05-09T06:37:40.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04e58f5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e00ffb50 0x7f04e01039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:40.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04e58f5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e0100d50 0x7f04e0102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04e58f5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0104010 con 0x7f04e0100d50 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04e58f5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0102670 con 0x7f04e00ffb50 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e0100d50 0x7f04e0102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e0100d50 0x7f04e0102130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57394/0 (socket says 172.21.15.63:57394) 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04d7fff700 1 -- 172.21.15.63:0/4129900094 learned_addr learned my addr 172.21.15.63:0/4129900094 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.667+0000 7f04deffd700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e00ffb50 0x7f04e01039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.668+0000 7f04d7fff700 1 -- 172.21.15.63:0/4129900094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e00ffb50 msgr2=0x7f04e01039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.668+0000 7f04d7fff700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e00ffb50 0x7f04e01039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.668+0000 7f04d7fff700 1 -- 172.21.15.63:0/4129900094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d00095f0 con 0x7f04e0100d50 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.668+0000 7f04d7fff700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e0100d50 0x7f04e0102130 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f04c800d900 tx=0x7f04c800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:40.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.668+0000 7f04e48f3700 1 -- 172.21.15.63:0/4129900094 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c80041f0 con 0x7f04e0100d50 2024-05-09T06:37:40.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.668+0000 7f04e48f3700 1 -- 172.21.15.63:0/4129900094 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04c8004cf0 con 0x7f04e0100d50 2024-05-09T06:37:40.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.668+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e0102920 con 0x7f04e0100d50 2024-05-09T06:37:40.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.668+0000 7f04e48f3700 1 -- 172.21.15.63:0/4129900094 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c800b750 con 0x7f04e0100d50 2024-05-09T06:37:40.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.668+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e0102e70 con 0x7f04e0100d50 2024-05-09T06:37:40.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.669+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e004ebb0 con 0x7f04e0100d50 2024-05-09T06:37:40.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.670+0000 7f04e48f3700 1 -- 172.21.15.63:0/4129900094 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f04c8004750 con 0x7f04e0100d50 2024-05-09T06:37:40.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.670+0000 7f04e48f3700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f04cc06cce0 0x7f04cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:40.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.670+0000 7f04e48f3700 1 -- 172.21.15.63:0/4129900094 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f04c808c230 con 0x7f04e0100d50 2024-05-09T06:37:40.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.670+0000 7f04deffd700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f04cc06cce0 0x7f04cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:40.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.671+0000 7f04deffd700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f04cc06cce0 0x7f04cc06f1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f04d000b5c0 tx=0x7f04d0005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.675+0000 7f04e48f3700 1 -- 172.21.15.63:0/4129900094 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f04c805a030 con 0x7f04e0100d50 2024-05-09T06:37:40.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.928+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 --> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04e0069880 con 0x7f04cc06cce0 2024-05-09T06:37:40.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.929+0000 7f04e48f3700 1 -- 172.21.15.63:0/4129900094 <== mgr.14215 v2:172.21.15.63:6800/4130975635 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f04e0069880 con 0x7f04cc06cce0 2024-05-09T06:37:40.909 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:37:40.910 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:37:40.910 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:37:40.910 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:37:40.910 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-05-09T06:37:40.910 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "", 2024-05-09T06:37:40.910 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mgr daemons", 2024-05-09T06:37:40.910 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:37:40.910 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:37:40.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f04cc06cce0 msgr2=0x7f04cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f04cc06cce0 0x7f04cc06f1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f04d000b5c0 tx=0x7f04d0005960 comp rx=0 tx=0).stop 2024-05-09T06:37:40.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e0100d50 msgr2=0x7f04e0102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:40.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e0100d50 0x7f04e0102130 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f04c800d900 tx=0x7f04c800dc10 comp rx=0 tx=0).stop 2024-05-09T06:37:40.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 shutdown_connections 2024-05-09T06:37:40.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7f04cc06cce0 0x7f04cc06f1a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04e00ffb50 0x7f04e01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 --2- 172.21.15.63:0/4129900094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e0100d50 0x7f04e0102130 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:40.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 >> 172.21.15.63:0/4129900094 conn(0x7f04e00fb0f0 msgr2=0x7f04e0069170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:40.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 shutdown_connections 2024-05-09T06:37:40.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:40.932+0000 7f04e58f5700 1 -- 172.21.15.63:0/4129900094 wait complete. 2024-05-09T06:37:41.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.045+0000 7ff652d55700 1 -- 172.21.15.63:0/2684764489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 msgr2=0x7ff64c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:41.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.045+0000 7ff652d55700 1 --2- 172.21.15.63:0/2684764489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 0x7ff64c0fff70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff640009910 tx=0x7ff640009c20 comp rx=0 tx=0).stop 2024-05-09T06:37:41.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.045+0000 7ff652d55700 1 -- 172.21.15.63:0/2684764489 shutdown_connections 2024-05-09T06:37:41.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.045+0000 7ff652d55700 1 --2- 172.21.15.63:0/2684764489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff64c100d50 0x7ff64c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.045+0000 7ff652d55700 1 --2- 172.21.15.63:0/2684764489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 0x7ff64c0fff70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.045+0000 7ff652d55700 1 -- 172.21.15.63:0/2684764489 >> 172.21.15.63:0/2684764489 conn(0x7ff64c0fb0f0 msgr2=0x7ff64c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.045+0000 7ff652d55700 1 -- 172.21.15.63:0/2684764489 shutdown_connections 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.045+0000 7ff652d55700 1 -- 172.21.15.63:0/2684764489 wait complete. 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff652d55700 1 Processor -- start 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff652d55700 1 -- start start 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff652d55700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 0x7ff64c069be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff652d55700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff64c100d50 0x7ff64c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff652d55700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c19b900 con 0x7ff64c0ffb50 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff652d55700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c19ba40 con 0x7ff64c100d50 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff650af1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 0x7ff64c069be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff650af1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 0x7ff64c069be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57422/0 (socket says 172.21.15.63:57422) 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff650af1700 1 -- 172.21.15.63:0/2566236113 learned_addr learned my addr 172.21.15.63:0/2566236113 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:37:41.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff64bfff700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff64c100d50 0x7ff64c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:41.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff650af1700 1 -- 172.21.15.63:0/2566236113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff64c100d50 msgr2=0x7ff64c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:41.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff650af1700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff64c100d50 0x7ff64c19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:41.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.046+0000 7ff650af1700 1 -- 172.21.15.63:0/2566236113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400095f0 con 0x7ff64c0ffb50 2024-05-09T06:37:41.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.047+0000 7ff650af1700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 0x7ff64c069be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff640005f00 tx=0x7ff64000b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:41.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.047+0000 7ff649ffb700 1 -- 172.21.15.63:0/2566236113 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64001d070 con 0x7ff64c0ffb50 2024-05-09T06:37:41.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.047+0000 7ff649ffb700 1 -- 172.21.15.63:0/2566236113 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6400052a0 con 0x7ff64c0ffb50 2024-05-09T06:37:41.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.047+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64c19bc90 con 0x7ff64c0ffb50 2024-05-09T06:37:41.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.047+0000 7ff649ffb700 1 -- 172.21.15.63:0/2566236113 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff640017400 con 0x7ff64c0ffb50 2024-05-09T06:37:41.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.047+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64c19c180 con 0x7ff64c0ffb50 2024-05-09T06:37:41.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.048+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64c18f6d0 con 0x7ff64c0ffb50 2024-05-09T06:37:41.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.049+0000 7ff649ffb700 1 -- 172.21.15.63:0/2566236113 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff640003680 con 0x7ff64c0ffb50 2024-05-09T06:37:41.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.049+0000 7ff649ffb700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff63c06cce0 0x7ff63c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:41.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.049+0000 7ff649ffb700 1 -- 172.21.15.63:0/2566236113 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff64008d640 con 0x7ff64c0ffb50 2024-05-09T06:37:41.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.049+0000 7ff64bfff700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff63c06cce0 0x7ff63c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:41.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.050+0000 7ff64bfff700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff63c06cce0 0x7ff63c06f1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff638009c60 tx=0x7ff638009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:41.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.054+0000 7ff649ffb700 1 -- 172.21.15.63:0/2566236113 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff64005b440 con 0x7ff64c0ffb50 2024-05-09T06:37:41.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.354+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff64c04ebb0 con 0x7ff64c0ffb50 2024-05-09T06:37:41.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.354+0000 7ff649ffb700 1 -- 172.21.15.63:0/2566236113 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff64005afd0 con 0x7ff64c0ffb50 2024-05-09T06:37:41.334 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff63c06cce0 msgr2=0x7ff63c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff63c06cce0 0x7ff63c06f1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff638009c60 tx=0x7ff638009260 comp rx=0 tx=0).stop 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 msgr2=0x7ff64c069be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 0x7ff64c069be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff640005f00 tx=0x7ff64000b5e0 comp rx=0 tx=0).stop 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 shutdown_connections 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff64c0ffb50 0x7ff64c069be0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.63:6800/4130975635,v1:172.21.15.63:6801/4130975635] conn(0x7ff63c06cce0 0x7ff63c06f1a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 --2- 172.21.15.63:0/2566236113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff64c100d50 0x7ff64c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 >> 172.21.15.63:0/2566236113 conn(0x7ff64c0fb0f0 msgr2=0x7ff64c104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 shutdown_connections 2024-05-09T06:37:41.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:37:41.356+0000 7ff652d55700 1 -- 172.21.15.63:0/2566236113 wait complete. 2024-05-09T06:37:41.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:41 smithi063 ceph-mon[27171]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:41.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:41 smithi063 ceph-mon[27171]: pgmap v259: 65 pgs: 65 active+clean; 129 MiB data, 695 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 397 op/s 2024-05-09T06:37:41.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:41 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/785887695' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:37:41.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:41 smithi193 ceph-mon[35141]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:41.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:41 smithi193 ceph-mon[35141]: pgmap v259: 65 pgs: 65 active+clean; 129 MiB data, 695 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 397 op/s 2024-05-09T06:37:41.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:41 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/785887695' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:37:42.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:42 smithi193 ceph-mon[35141]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:42.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:42 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2566236113' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:37:42.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:42 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:37:42.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:42 smithi063 ceph-mon[27171]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:37:42.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:42 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2566236113' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:37:42.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:42 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:37:43.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:43 smithi193 ceph-mon[35141]: pgmap v260: 65 pgs: 65 active+clean; 142 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 403 op/s 2024-05-09T06:37:43.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:43 smithi063 ceph-mon[27171]: pgmap v260: 65 pgs: 65 active+clean; 142 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 403 op/s 2024-05-09T06:37:45.460 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:45 smithi193 ceph-mon[35141]: pgmap v261: 65 pgs: 65 active+clean; 143 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-05-09T06:37:45.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:45 smithi063 ceph-mon[27171]: pgmap v261: 65 pgs: 65 active+clean; 143 MiB data, 726 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 333 op/s 2024-05-09T06:37:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:47 smithi193 ceph-mon[35141]: pgmap v262: 65 pgs: 65 active+clean; 151 MiB data, 758 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-05-09T06:37:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:47 smithi193 ceph-mon[35141]: Standby manager daemon smithi193.mmmqzh restarted 2024-05-09T06:37:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:47 smithi193 ceph-mon[35141]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:37:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:47 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:37:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:47 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:37:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:47 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:37:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:47 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:37:47.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:47 smithi063 ceph-mon[27171]: pgmap v262: 65 pgs: 65 active+clean; 151 MiB data, 758 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-05-09T06:37:47.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:47 smithi063 ceph-mon[27171]: Standby manager daemon smithi193.mmmqzh restarted 2024-05-09T06:37:47.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:47 smithi063 ceph-mon[27171]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:37:47.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:47 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:37:47.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:47 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:37:47.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:47 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:37:47.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:47 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:37:48.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:48 smithi063 ceph-mon[27171]: mgrmap e20: smithi063.egfguy(active, since 7m), standbys: smithi193.mmmqzh 2024-05-09T06:37:48.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:48 smithi193 ceph-mon[35141]: mgrmap e20: smithi063.egfguy(active, since 7m), standbys: smithi193.mmmqzh 2024-05-09T06:37:49.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:49 smithi063 ceph-mon[27171]: pgmap v263: 65 pgs: 65 active+clean; 154 MiB data, 763 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-05-09T06:37:49.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:49 smithi193 ceph-mon[35141]: pgmap v263: 65 pgs: 65 active+clean; 154 MiB data, 763 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-05-09T06:37:51.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:51 smithi063 ceph-mon[27171]: pgmap v264: 65 pgs: 65 active+clean; 161 MiB data, 799 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 317 op/s 2024-05-09T06:37:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:51 smithi193 ceph-mon[35141]: pgmap v264: 65 pgs: 65 active+clean; 161 MiB data, 799 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 317 op/s 2024-05-09T06:37:53.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:53 smithi063 ceph-mon[27171]: pgmap v265: 65 pgs: 65 active+clean; 174 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 341 op/s 2024-05-09T06:37:53.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:53 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:53.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:53 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:53.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:53 smithi193 ceph-mon[35141]: pgmap v265: 65 pgs: 65 active+clean; 174 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 341 op/s 2024-05-09T06:37:53.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:53 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:53.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:53 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:54.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:54 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:54.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:54 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:54.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:54 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:54.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:54 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:55.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:55 smithi193 ceph-mon[35141]: pgmap v266: 65 pgs: 65 active+clean; 175 MiB data, 832 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 287 op/s 2024-05-09T06:37:55.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:55 smithi063 ceph-mon[27171]: pgmap v266: 65 pgs: 65 active+clean; 175 MiB data, 832 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 287 op/s 2024-05-09T06:37:57.135 INFO:tasks.workunit.client.0.smithi063.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-09T06:37:57.140 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-09T06:37:57.141 INFO:tasks.workunit.client.0.smithi063.stderr:+ make 2024-05-09T06:37:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:57 smithi063 ceph-mon[27171]: pgmap v267: 65 pgs: 65 active+clean; 182 MiB data, 875 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 356 op/s 2024-05-09T06:37:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:57 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:37:57.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:57 smithi193 ceph-mon[35141]: pgmap v267: 65 pgs: 65 active+clean; 182 MiB data, 875 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 356 op/s 2024-05-09T06:37:57.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:57 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:37:58.289 INFO:tasks.workunit.client.0.smithi063.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-09T06:37:58.699 INFO:tasks.workunit.client.0.smithi063.stderr:++ readlink -f fsstress 2024-05-09T06:37:58.699 INFO:tasks.workunit.client.0.smithi063.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-09T06:37:58.699 INFO:tasks.workunit.client.0.smithi063.stderr:+ popd 2024-05-09T06:37:58.700 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-09T06:37:58.700 INFO:tasks.workunit.client.0.smithi063.stderr:+ popd 2024-05-09T06:37:58.700 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp 2024-05-09T06:37:58.701 INFO:tasks.workunit.client.0.smithi063.stderr:++ mktemp -d -p . 2024-05-09T06:37:58.947 INFO:tasks.workunit.client.0.smithi063.stderr:+ T=./tmp.iIBGAiE8sD 2024-05-09T06:37:58.947 INFO:tasks.workunit.client.0.smithi063.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.iIBGAiE8sD -l 1 -n 1000 -p 10 -v 2024-05-09T06:37:58.952 INFO:tasks.workunit.client.0.smithi063.stdout:seed = 1714599043 2024-05-09T06:37:58.958 INFO:tasks.workunit.client.0.smithi063.stdout:0/0: stat - no entries 2024-05-09T06:37:58.958 INFO:tasks.workunit.client.0.smithi063.stdout:0/1: write - no filename 2024-05-09T06:37:58.958 INFO:tasks.workunit.client.0.smithi063.stdout:0/2: fdatasync - no filename 2024-05-09T06:37:58.961 INFO:tasks.workunit.client.0.smithi063.stdout:1/0: dwrite - no filename 2024-05-09T06:37:58.964 INFO:tasks.workunit.client.0.smithi063.stdout:0/3: mknod c0 0 2024-05-09T06:37:58.964 INFO:tasks.workunit.client.0.smithi063.stdout:0/4: write - no filename 2024-05-09T06:37:58.964 INFO:tasks.workunit.client.0.smithi063.stdout:0/5: write - no filename 2024-05-09T06:37:58.967 INFO:tasks.workunit.client.0.smithi063.stdout:3/0: write - no filename 2024-05-09T06:37:58.967 INFO:tasks.workunit.client.0.smithi063.stdout:3/1: dread - no filename 2024-05-09T06:37:58.967 INFO:tasks.workunit.client.0.smithi063.stdout:3/2: stat - no entries 2024-05-09T06:37:58.969 INFO:tasks.workunit.client.0.smithi063.stdout:0/6: creat f1 x:0 0 0 2024-05-09T06:37:58.971 INFO:tasks.workunit.client.0.smithi063.stdout:1/1: mknod c0 0 2024-05-09T06:37:58.973 INFO:tasks.workunit.client.0.smithi063.stdout:2/0: creat f0 x:0 0 0 2024-05-09T06:37:58.977 INFO:tasks.workunit.client.0.smithi063.stdout:0/7: unlink f1 0 2024-05-09T06:37:58.977 INFO:tasks.workunit.client.0.smithi063.stdout:0/8: fdatasync - no filename 2024-05-09T06:37:58.978 INFO:tasks.workunit.client.0.smithi063.stdout:1/2: mknod c1 0 2024-05-09T06:37:58.981 INFO:tasks.workunit.client.0.smithi063.stdout:5/0: fsync - no filename 2024-05-09T06:37:58.981 INFO:tasks.workunit.client.0.smithi063.stdout:5/1: chown . 3279 1 2024-05-09T06:37:58.981 INFO:tasks.workunit.client.0.smithi063.stdout:5/2: chown . 79162 1 2024-05-09T06:37:58.981 INFO:tasks.workunit.client.0.smithi063.stdout:5/3: link - no file 2024-05-09T06:37:58.983 INFO:tasks.workunit.client.0.smithi063.stdout:3/3: creat f0 x:0 0 0 2024-05-09T06:37:58.986 INFO:tasks.workunit.client.0.smithi063.stdout:1/3: symlink l2 0 2024-05-09T06:37:58.988 INFO:tasks.workunit.client.0.smithi063.stdout:4/0: symlink l0 0 2024-05-09T06:37:58.990 INFO:tasks.workunit.client.0.smithi063.stdout:2/1: link f0 f1 0 2024-05-09T06:37:58.992 INFO:tasks.workunit.client.0.smithi063.stdout:3/4: creat f1 x:0 0 0 2024-05-09T06:37:58.992 INFO:tasks.workunit.client.0.smithi063.stdout:3/5: stat f0 0 2024-05-09T06:37:58.994 INFO:tasks.workunit.client.0.smithi063.stdout:1/4: creat f3 x:0 0 0 2024-05-09T06:37:58.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/5: dread - f3 zero size 2024-05-09T06:37:58.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/6: rmdir - no directory 2024-05-09T06:37:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/0: rename - no filename 2024-05-09T06:37:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/1: truncate - no filename 2024-05-09T06:37:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/2: chown . 2 1 2024-05-09T06:37:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/3: rename - no filename 2024-05-09T06:37:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/4: write - no filename 2024-05-09T06:37:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/5: write - no filename 2024-05-09T06:37:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/6: truncate - no filename 2024-05-09T06:37:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/7: dread - no filename 2024-05-09T06:37:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/8: rmdir - no directory 2024-05-09T06:37:58.997 INFO:tasks.workunit.client.0.smithi063.stdout:4/1: mknod c1 0 2024-05-09T06:37:58.997 INFO:tasks.workunit.client.0.smithi063.stdout:4/2: read - no filename 2024-05-09T06:37:58.999 INFO:tasks.workunit.client.0.smithi063.stdout:2/2: creat f2 x:0 0 0 2024-05-09T06:37:59.002 INFO:tasks.workunit.client.0.smithi063.stdout:1/7: symlink l4 0 2024-05-09T06:37:59.003 INFO:tasks.workunit.client.0.smithi063.stdout:1/8: stat c1 0 2024-05-09T06:37:59.003 INFO:tasks.workunit.client.0.smithi063.stdout:1/9: read - f3 zero size 2024-05-09T06:37:59.005 INFO:tasks.workunit.client.0.smithi063.stdout:6/0: creat f0 x:0 0 0 2024-05-09T06:37:59.009 INFO:tasks.workunit.client.0.smithi063.stdout:4/3: unlink c1 0 2024-05-09T06:37:59.009 INFO:tasks.workunit.client.0.smithi063.stdout:4/4: truncate - no filename 2024-05-09T06:37:59.009 INFO:tasks.workunit.client.0.smithi063.stdout:2/3: creat f3 x:0 0 0 2024-05-09T06:37:59.011 INFO:tasks.workunit.client.0.smithi063.stdout:2/4: dread - f0 zero size 2024-05-09T06:37:59.011 INFO:tasks.workunit.client.0.smithi063.stdout:3/6: link f0 f2 0 2024-05-09T06:37:59.013 INFO:tasks.workunit.client.0.smithi063.stdout:1/10: rename c1 to c5 0 2024-05-09T06:37:59.014 INFO:tasks.workunit.client.0.smithi063.stdout:6/1: dwrite f0 [0,4194304] 0 2024-05-09T06:37:59.017 INFO:tasks.workunit.client.0.smithi063.stdout:7/9: mkdir d0 0 2024-05-09T06:37:59.017 INFO:tasks.workunit.client.0.smithi063.stdout:7/10: stat d0 0 2024-05-09T06:37:59.017 INFO:tasks.workunit.client.0.smithi063.stdout:7/11: write - no filename 2024-05-09T06:37:59.017 INFO:tasks.workunit.client.0.smithi063.stdout:7/12: write - no filename 2024-05-09T06:37:59.017 INFO:tasks.workunit.client.0.smithi063.stdout:7/13: write - no filename 2024-05-09T06:37:59.017 INFO:tasks.workunit.client.0.smithi063.stdout:7/14: dwrite - no filename 2024-05-09T06:37:59.018 INFO:tasks.workunit.client.0.smithi063.stdout:6/2: dread f0 [0,4194304] 0 2024-05-09T06:37:59.018 INFO:tasks.workunit.client.0.smithi063.stdout:6/3: stat f0 0 2024-05-09T06:37:59.030 INFO:tasks.workunit.client.0.smithi063.stdout:8/0: creat f0 x:0 0 0 2024-05-09T06:37:59.031 INFO:tasks.workunit.client.0.smithi063.stdout:2/5: dwrite f1 [0,4194304] 0 2024-05-09T06:37:59.031 INFO:tasks.workunit.client.0.smithi063.stdout:8/1: truncate f0 1022822 0 2024-05-09T06:37:59.032 INFO:tasks.workunit.client.0.smithi063.stdout:2/6: dread - f3 zero size 2024-05-09T06:37:59.032 INFO:tasks.workunit.client.0.smithi063.stdout:8/2: write f0 [759987,55971] 0 2024-05-09T06:37:59.041 INFO:tasks.workunit.client.0.smithi063.stdout:8/3: dwrite f0 [0,4194304] 0 2024-05-09T06:37:59.042 INFO:tasks.workunit.client.0.smithi063.stdout:8/4: write f0 [789020,67800] 0 2024-05-09T06:37:59.042 INFO:tasks.workunit.client.0.smithi063.stdout:8/5: stat f0 0 2024-05-09T06:37:59.043 INFO:tasks.workunit.client.0.smithi063.stdout:9/0: mkdir d0 0 2024-05-09T06:37:59.043 INFO:tasks.workunit.client.0.smithi063.stdout:9/1: fdatasync - no filename 2024-05-09T06:37:59.043 INFO:tasks.workunit.client.0.smithi063.stdout:9/2: dread - no filename 2024-05-09T06:37:59.044 INFO:tasks.workunit.client.0.smithi063.stdout:7/15: creat d0/f1 x:0 0 0 2024-05-09T06:37:59.045 INFO:tasks.workunit.client.0.smithi063.stdout:0/9: getdents . 0 2024-05-09T06:37:59.045 INFO:tasks.workunit.client.0.smithi063.stdout:0/10: fsync - no filename 2024-05-09T06:37:59.046 INFO:tasks.workunit.client.0.smithi063.stdout:0/11: chown c0 19722 1 2024-05-09T06:37:59.046 INFO:tasks.workunit.client.0.smithi063.stdout:7/16: write d0/f1 [951629,111371] 0 2024-05-09T06:37:59.047 INFO:tasks.workunit.client.0.smithi063.stdout:6/4: link f0 f1 0 2024-05-09T06:37:59.049 INFO:tasks.workunit.client.0.smithi063.stdout:7/17: write d0/f1 [713779,78467] 0 2024-05-09T06:37:59.052 INFO:tasks.workunit.client.0.smithi063.stdout:4/5: link l0 l2 0 2024-05-09T06:37:59.052 INFO:tasks.workunit.client.0.smithi063.stdout:4/6: dread - no filename 2024-05-09T06:37:59.058 INFO:tasks.workunit.client.0.smithi063.stdout:0/12: unlink c0 0 2024-05-09T06:37:59.059 INFO:tasks.workunit.client.0.smithi063.stdout:8/6: fdatasync f0 0 2024-05-09T06:37:59.061 INFO:tasks.workunit.client.0.smithi063.stdout:6/5: mkdir d2 0 2024-05-09T06:37:59.061 INFO:tasks.workunit.client.0.smithi063.stdout:6/6: chown f1 57754 1 2024-05-09T06:37:59.064 INFO:tasks.workunit.client.0.smithi063.stdout:2/7: rename f1 to f4 0 2024-05-09T06:37:59.067 INFO:tasks.workunit.client.0.smithi063.stdout:9/3: creat d0/f1 x:0 0 0 2024-05-09T06:37:59.068 INFO:tasks.workunit.client.0.smithi063.stdout:0/13: creat f2 x:0 0 0 2024-05-09T06:37:59.069 INFO:tasks.workunit.client.0.smithi063.stdout:0/14: write f2 [657144,96207] 0 2024-05-09T06:37:59.069 INFO:tasks.workunit.client.0.smithi063.stdout:8/7: symlink l1 0 2024-05-09T06:37:59.072 INFO:tasks.workunit.client.0.smithi063.stdout:2/8: symlink l5 0 2024-05-09T06:37:59.073 INFO:tasks.workunit.client.0.smithi063.stdout:3/7: getdents . 0 2024-05-09T06:37:59.073 INFO:tasks.workunit.client.0.smithi063.stdout:3/8: write f1 [887977,12308] 0 2024-05-09T06:37:59.074 INFO:tasks.workunit.client.0.smithi063.stdout:7/18: link d0/f1 d0/f2 0 2024-05-09T06:37:59.076 INFO:tasks.workunit.client.0.smithi063.stdout:9/4: mknod d0/c2 0 2024-05-09T06:37:59.079 INFO:tasks.workunit.client.0.smithi063.stdout:8/8: creat f2 x:0 0 0 2024-05-09T06:37:59.080 INFO:tasks.workunit.client.0.smithi063.stdout:8/9: chown l1 838 1 2024-05-09T06:37:59.080 INFO:tasks.workunit.client.0.smithi063.stdout:8/10: write f0 [3827425,84915] 0 2024-05-09T06:37:59.082 INFO:tasks.workunit.client.0.smithi063.stdout:6/7: symlink d2/l3 0 2024-05-09T06:37:59.088 INFO:tasks.workunit.client.0.smithi063.stdout:9/5: creat d0/f3 x:0 0 0 2024-05-09T06:37:59.089 INFO:tasks.workunit.client.0.smithi063.stdout:1/11: rename c5 to c6 0 2024-05-09T06:37:59.090 INFO:tasks.workunit.client.0.smithi063.stdout:8/11: mkdir d3 0 2024-05-09T06:37:59.093 INFO:tasks.workunit.client.0.smithi063.stdout:6/8: mkdir d2/d4 0 2024-05-09T06:37:59.093 INFO:tasks.workunit.client.0.smithi063.stdout:7/19: dwrite d0/f1 [0,4194304] 0 2024-05-09T06:37:59.094 INFO:tasks.workunit.client.0.smithi063.stdout:9/6: mkdir d0/d4 0 2024-05-09T06:37:59.095 INFO:tasks.workunit.client.0.smithi063.stdout:1/12: symlink l7 0 2024-05-09T06:37:59.095 INFO:tasks.workunit.client.0.smithi063.stdout:9/7: truncate d0/f1 1034120 0 2024-05-09T06:37:59.099 INFO:tasks.workunit.client.0.smithi063.stdout:9/8: dread d0/f1 [0,4194304] 0 2024-05-09T06:37:59.099 INFO:tasks.workunit.client.0.smithi063.stdout:6/9: unlink d2/l3 0 2024-05-09T06:37:59.101 INFO:tasks.workunit.client.0.smithi063.stdout:9/9: dread d0/f1 [0,4194304] 0 2024-05-09T06:37:59.104 INFO:tasks.workunit.client.0.smithi063.stdout:1/13: creat f8 x:0 0 0 2024-05-09T06:37:59.114 INFO:tasks.workunit.client.0.smithi063.stdout:1/14: mkdir d9 0 2024-05-09T06:37:59.117 INFO:tasks.workunit.client.0.smithi063.stdout:8/12: rmdir d3 0 2024-05-09T06:37:59.118 INFO:tasks.workunit.client.0.smithi063.stdout:0/15: fsync f2 0 2024-05-09T06:37:59.118 INFO:tasks.workunit.client.0.smithi063.stdout:9/10: getdents d0/d4 0 2024-05-09T06:37:59.119 INFO:tasks.workunit.client.0.smithi063.stdout:0/16: write f2 [251320,14750] 0 2024-05-09T06:37:59.121 INFO:tasks.workunit.client.0.smithi063.stdout:2/9: unlink f4 0 2024-05-09T06:37:59.123 INFO:tasks.workunit.client.0.smithi063.stdout:8/13: creat f4 x:0 0 0 2024-05-09T06:37:59.124 INFO:tasks.workunit.client.0.smithi063.stdout:1/15: creat d9/fa x:0 0 0 2024-05-09T06:37:59.124 INFO:tasks.workunit.client.0.smithi063.stdout:8/14: stat l1 0 2024-05-09T06:37:59.131 INFO:tasks.workunit.client.0.smithi063.stdout:2/10: link f3 f6 0 2024-05-09T06:37:59.133 INFO:tasks.workunit.client.0.smithi063.stdout:9/11: dwrite d0/f3 [0,4194304] 0 2024-05-09T06:37:59.134 INFO:tasks.workunit.client.0.smithi063.stdout:0/17: dwrite f2 [0,4194304] 0 2024-05-09T06:37:59.136 INFO:tasks.workunit.client.0.smithi063.stdout:0/18: chown f2 117072 1 2024-05-09T06:37:59.137 INFO:tasks.workunit.client.0.smithi063.stdout:2/11: fdatasync f6 0 2024-05-09T06:37:59.139 INFO:tasks.workunit.client.0.smithi063.stdout:2/12: dread - f6 zero size 2024-05-09T06:37:59.141 INFO:tasks.workunit.client.0.smithi063.stdout:2/13: readlink l5 0 2024-05-09T06:37:59.141 INFO:tasks.workunit.client.0.smithi063.stdout:1/16: dwrite f3 [0,4194304] 0 2024-05-09T06:37:59.147 INFO:tasks.workunit.client.0.smithi063.stdout:9/12: creat d0/d4/f5 x:0 0 0 2024-05-09T06:37:59.148 INFO:tasks.workunit.client.0.smithi063.stdout:0/19: dread f2 [0,4194304] 0 2024-05-09T06:37:59.149 INFO:tasks.workunit.client.0.smithi063.stdout:7/20: truncate d0/f1 1730192 0 2024-05-09T06:37:59.155 INFO:tasks.workunit.client.0.smithi063.stdout:9/13: creat d0/d4/f6 x:0 0 0 2024-05-09T06:37:59.157 INFO:tasks.workunit.client.0.smithi063.stdout:0/20: fdatasync f2 0 2024-05-09T06:37:59.161 INFO:tasks.workunit.client.0.smithi063.stdout:8/15: getdents . 0 2024-05-09T06:37:59.162 INFO:tasks.workunit.client.0.smithi063.stdout:8/16: write f4 [273729,66045] 0 2024-05-09T06:37:59.165 INFO:tasks.workunit.client.0.smithi063.stdout:0/21: link f2 f3 0 2024-05-09T06:37:59.168 INFO:tasks.workunit.client.0.smithi063.stdout:0/22: creat f4 x:0 0 0 2024-05-09T06:37:59.170 INFO:tasks.workunit.client.0.smithi063.stdout:8/17: dwrite f2 [0,4194304] 0 2024-05-09T06:37:59.171 INFO:tasks.workunit.client.0.smithi063.stdout:0/23: creat f5 x:0 0 0 2024-05-09T06:37:59.172 INFO:tasks.workunit.client.0.smithi063.stdout:8/18: write f4 [1094213,19361] 0 2024-05-09T06:37:59.175 INFO:tasks.workunit.client.0.smithi063.stdout:0/24: creat f6 x:0 0 0 2024-05-09T06:37:59.176 INFO:tasks.workunit.client.0.smithi063.stdout:0/25: stat f3 0 2024-05-09T06:37:59.176 INFO:tasks.workunit.client.0.smithi063.stdout:0/26: readlink - no filename 2024-05-09T06:37:59.177 INFO:tasks.workunit.client.0.smithi063.stdout:8/19: mknod c5 0 2024-05-09T06:37:59.182 INFO:tasks.workunit.client.0.smithi063.stdout:8/20: rename c5 to c6 0 2024-05-09T06:37:59.183 INFO:tasks.workunit.client.0.smithi063.stdout:1/17: truncate f3 1088564 0 2024-05-09T06:37:59.187 INFO:tasks.workunit.client.0.smithi063.stdout:0/27: dwrite f4 [0,4194304] 0 2024-05-09T06:37:59.189 INFO:tasks.workunit.client.0.smithi063.stdout:8/21: creat f7 x:0 0 0 2024-05-09T06:37:59.192 INFO:tasks.workunit.client.0.smithi063.stdout:7/21: dwrite d0/f1 [0,4194304] 0 2024-05-09T06:37:59.192 INFO:tasks.workunit.client.0.smithi063.stdout:0/28: write f6 [929394,29282] 0 2024-05-09T06:37:59.193 INFO:tasks.workunit.client.0.smithi063.stdout:1/18: creat d9/fb x:0 0 0 2024-05-09T06:37:59.202 INFO:tasks.workunit.client.0.smithi063.stdout:1/19: rename l2 to d9/lc 0 2024-05-09T06:37:59.205 INFO:tasks.workunit.client.0.smithi063.stdout:9/14: dwrite d0/f1 [0,4194304] 0 2024-05-09T06:37:59.207 INFO:tasks.workunit.client.0.smithi063.stdout:1/20: write f8 [294869,82893] 0 2024-05-09T06:37:59.207 INFO:tasks.workunit.client.0.smithi063.stdout:0/29: truncate f3 766580 0 2024-05-09T06:37:59.210 INFO:tasks.workunit.client.0.smithi063.stdout:8/22: dread f2 [0,4194304] 0 2024-05-09T06:37:59.211 INFO:tasks.workunit.client.0.smithi063.stdout:8/23: dread - f7 zero size 2024-05-09T06:37:59.212 INFO:tasks.workunit.client.0.smithi063.stdout:0/30: symlink l7 0 2024-05-09T06:37:59.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/22: dwrite d0/f2 [0,4194304] 0 2024-05-09T06:37:59.219 INFO:tasks.workunit.client.0.smithi063.stdout:8/24: mknod c8 0 2024-05-09T06:37:59.224 INFO:tasks.workunit.client.0.smithi063.stdout:7/23: rmdir d0 39 2024-05-09T06:37:59.228 INFO:tasks.workunit.client.0.smithi063.stdout:8/25: mkdir d9 0 2024-05-09T06:37:59.228 INFO:tasks.workunit.client.0.smithi063.stdout:8/26: fdatasync f7 0 2024-05-09T06:37:59.234 INFO:tasks.workunit.client.0.smithi063.stdout:7/24: write d0/f1 [456718,84861] 0 2024-05-09T06:37:59.236 INFO:tasks.workunit.client.0.smithi063.stdout:9/15: getdents d0 0 2024-05-09T06:37:59.238 INFO:tasks.workunit.client.0.smithi063.stdout:1/21: getdents d9 0 2024-05-09T06:37:59.238 INFO:tasks.workunit.client.0.smithi063.stdout:1/22: dread - d9/fa zero size 2024-05-09T06:37:59.240 INFO:tasks.workunit.client.0.smithi063.stdout:8/27: mkdir d9/da 0 2024-05-09T06:37:59.241 INFO:tasks.workunit.client.0.smithi063.stdout:8/28: chown d9 3344 1 2024-05-09T06:37:59.241 INFO:tasks.workunit.client.0.smithi063.stdout:8/29: write f4 [1672446,121100] 0 2024-05-09T06:37:59.242 INFO:tasks.workunit.client.0.smithi063.stdout:7/25: mkdir d0/d3 0 2024-05-09T06:37:59.243 INFO:tasks.workunit.client.0.smithi063.stdout:8/30: truncate f7 603795 0 2024-05-09T06:37:59.243 INFO:tasks.workunit.client.0.smithi063.stdout:7/26: read d0/f1 [1972684,13316] 0 2024-05-09T06:37:59.246 INFO:tasks.workunit.client.0.smithi063.stdout:1/23: mknod d9/cd 0 2024-05-09T06:37:59.255 INFO:tasks.workunit.client.0.smithi063.stdout:0/31: write f3 [555924,66619] 0 2024-05-09T06:37:59.256 INFO:tasks.workunit.client.0.smithi063.stdout:0/32: read f4 [2343179,104571] 0 2024-05-09T06:37:59.256 INFO:tasks.workunit.client.0.smithi063.stdout:0/33: truncate f6 1985602 0 2024-05-09T06:37:59.257 INFO:tasks.workunit.client.0.smithi063.stdout:1/24: creat d9/fe x:0 0 0 2024-05-09T06:37:59.261 INFO:tasks.workunit.client.0.smithi063.stdout:8/31: creat d9/da/fb x:0 0 0 2024-05-09T06:37:59.261 INFO:tasks.workunit.client.0.smithi063.stdout:8/32: chown f0 53512 1 2024-05-09T06:37:59.266 INFO:tasks.workunit.client.0.smithi063.stdout:0/34: mknod c8 0 2024-05-09T06:37:59.267 INFO:tasks.workunit.client.0.smithi063.stdout:0/35: chown c8 515 1 2024-05-09T06:37:59.267 INFO:tasks.workunit.client.0.smithi063.stdout:1/25: fdatasync f3 0 2024-05-09T06:37:59.268 INFO:tasks.workunit.client.0.smithi063.stdout:1/26: read - d9/fe zero size 2024-05-09T06:37:59.269 INFO:tasks.workunit.client.0.smithi063.stdout:1/27: write f8 [743477,97730] 0 2024-05-09T06:37:59.269 INFO:tasks.workunit.client.0.smithi063.stdout:8/33: creat d9/fc x:0 0 0 2024-05-09T06:37:59.272 INFO:tasks.workunit.client.0.smithi063.stdout:9/16: truncate d0/f1 2193651 0 2024-05-09T06:37:59.272 INFO:tasks.workunit.client.0.smithi063.stdout:9/17: dread - d0/d4/f5 zero size 2024-05-09T06:37:59.273 INFO:tasks.workunit.client.0.smithi063.stdout:0/36: rename f3 to f9 0 2024-05-09T06:37:59.274 INFO:tasks.workunit.client.0.smithi063.stdout:0/37: dread f9 [0,4194304] 0 2024-05-09T06:37:59.276 INFO:tasks.workunit.client.0.smithi063.stdout:1/28: symlink d9/lf 0 2024-05-09T06:37:59.276 INFO:tasks.workunit.client.0.smithi063.stdout:1/29: truncate d9/fb 17847 0 2024-05-09T06:37:59.277 INFO:tasks.workunit.client.0.smithi063.stdout:1/30: dread - d9/fa zero size 2024-05-09T06:37:59.278 INFO:tasks.workunit.client.0.smithi063.stdout:1/31: dread f3 [0,4194304] 0 2024-05-09T06:37:59.279 INFO:tasks.workunit.client.0.smithi063.stdout:7/27: truncate d0/f2 3284880 0 2024-05-09T06:37:59.282 INFO:tasks.workunit.client.0.smithi063.stdout:0/38: creat fa x:0 0 0 2024-05-09T06:37:59.287 INFO:tasks.workunit.client.0.smithi063.stdout:0/39: mknod cb 0 2024-05-09T06:37:59.287 INFO:tasks.workunit.client.0.smithi063.stdout:0/40: read - f5 zero size 2024-05-09T06:37:59.288 INFO:tasks.workunit.client.0.smithi063.stdout:0/41: chown f6 205 1 2024-05-09T06:37:59.291 INFO:tasks.workunit.client.0.smithi063.stdout:0/42: rename fa to fc 0 2024-05-09T06:37:59.293 INFO:tasks.workunit.client.0.smithi063.stdout:9/18: unlink d0/f1 0 2024-05-09T06:37:59.296 INFO:tasks.workunit.client.0.smithi063.stdout:0/43: mkdir dd 0 2024-05-09T06:37:59.297 INFO:tasks.workunit.client.0.smithi063.stdout:9/19: creat d0/d4/f7 x:0 0 0 2024-05-09T06:37:59.299 INFO:tasks.workunit.client.0.smithi063.stdout:0/44: fdatasync f4 0 2024-05-09T06:37:59.300 INFO:tasks.workunit.client.0.smithi063.stdout:0/45: dread - f5 zero size 2024-05-09T06:37:59.301 INFO:tasks.workunit.client.0.smithi063.stdout:7/28: unlink d0/f2 0 2024-05-09T06:37:59.301 INFO:tasks.workunit.client.0.smithi063.stdout:0/46: read f4 [965122,61437] 0 2024-05-09T06:37:59.302 INFO:tasks.workunit.client.0.smithi063.stdout:0/47: readlink l7 0 2024-05-09T06:37:59.304 INFO:tasks.workunit.client.0.smithi063.stdout:9/20: dwrite d0/d4/f7 [0,4194304] 0 2024-05-09T06:37:59.305 INFO:tasks.workunit.client.0.smithi063.stdout:8/34: truncate f0 347688 0 2024-05-09T06:37:59.308 INFO:tasks.workunit.client.0.smithi063.stdout:0/48: rename f4 to dd/fe 0 2024-05-09T06:37:59.319 INFO:tasks.workunit.client.0.smithi063.stdout:0/49: rmdir dd 39 2024-05-09T06:37:59.326 INFO:tasks.workunit.client.0.smithi063.stdout:7/29: rmdir d0/d3 0 2024-05-09T06:37:59.333 INFO:tasks.workunit.client.0.smithi063.stdout:8/35: dread f0 [0,4194304] 0 2024-05-09T06:37:59.340 INFO:tasks.workunit.client.0.smithi063.stdout:8/36: unlink f7 0 2024-05-09T06:37:59.344 INFO:tasks.workunit.client.0.smithi063.stdout:8/37: mknod d9/da/cd 0 2024-05-09T06:37:59.346 INFO:tasks.workunit.client.0.smithi063.stdout:8/38: creat d9/fe x:0 0 0 2024-05-09T06:37:59.346 INFO:tasks.workunit.client.0.smithi063.stdout:9/21: truncate d0/d4/f7 503557 0 2024-05-09T06:37:59.348 INFO:tasks.workunit.client.0.smithi063.stdout:8/39: creat d9/da/ff x:0 0 0 2024-05-09T06:37:59.350 INFO:tasks.workunit.client.0.smithi063.stdout:9/22: mkdir d0/d8 0 2024-05-09T06:37:59.357 INFO:tasks.workunit.client.0.smithi063.stdout:8/40: dwrite d9/da/ff [0,4194304] 0 2024-05-09T06:37:59.357 INFO:tasks.workunit.client.0.smithi063.stdout:7/30: rename d0/f1 to d0/f4 0 2024-05-09T06:37:59.359 INFO:tasks.workunit.client.0.smithi063.stdout:9/23: dwrite d0/d4/f6 [0,4194304] 0 2024-05-09T06:37:59.360 INFO:tasks.workunit.client.0.smithi063.stdout:9/24: chown d0/d4/f7 2103889580 1 2024-05-09T06:37:59.365 INFO:tasks.workunit.client.0.smithi063.stdout:8/41: symlink d9/da/l10 0 2024-05-09T06:37:59.370 INFO:tasks.workunit.client.0.smithi063.stdout:9/25: dwrite d0/d4/f6 [0,4194304] 0 2024-05-09T06:37:59.373 INFO:tasks.workunit.client.0.smithi063.stdout:8/42: mknod d9/c11 0 2024-05-09T06:37:59.373 INFO:tasks.workunit.client.0.smithi063.stdout:9/26: chown d0/d4/f5 7057 1 2024-05-09T06:37:59.373 INFO:tasks.workunit.client.0.smithi063.stdout:9/27: chown d0/f3 12 1 2024-05-09T06:37:59.378 INFO:tasks.workunit.client.0.smithi063.stdout:9/28: rename d0/d4/f5 to d0/d4/f9 0 2024-05-09T06:37:59.379 INFO:tasks.workunit.client.0.smithi063.stdout:8/43: dread f2 [0,4194304] 0 2024-05-09T06:37:59.379 INFO:tasks.workunit.client.0.smithi063.stdout:8/44: dread - d9/da/fb zero size 2024-05-09T06:37:59.382 INFO:tasks.workunit.client.0.smithi063.stdout:9/29: mkdir d0/d4/da 0 2024-05-09T06:37:59.384 INFO:tasks.workunit.client.0.smithi063.stdout:8/45: symlink d9/da/l12 0 2024-05-09T06:37:59.388 INFO:tasks.workunit.client.0.smithi063.stdout:8/46: dread f2 [0,4194304] 0 2024-05-09T06:37:59.389 INFO:tasks.workunit.client.0.smithi063.stdout:9/30: getdents d0/d8 0 2024-05-09T06:37:59.391 INFO:tasks.workunit.client.0.smithi063.stdout:8/47: fdatasync f0 0 2024-05-09T06:37:59.394 INFO:tasks.workunit.client.0.smithi063.stdout:9/31: unlink d0/f3 0 2024-05-09T06:37:59.397 INFO:tasks.workunit.client.0.smithi063.stdout:8/48: rename d9/fe to d9/f13 0 2024-05-09T06:37:59.398 INFO:tasks.workunit.client.0.smithi063.stdout:8/49: truncate d9/f13 377916 0 2024-05-09T06:37:59.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:59 smithi063 ceph-mon[27171]: pgmap v268: 65 pgs: 65 active+clean; 193 MiB data, 898 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 374 op/s 2024-05-09T06:37:59.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:59 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:59.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:59 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:59.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:59 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:37:59.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:59 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:37:59.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:59 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:59.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:59 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:37:59.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:59 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr fail", "who": "smithi063.egfguy"}]: dispatch 2024-05-09T06:37:59.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:37:59 smithi063 ceph-mon[27171]: Activating manager daemon smithi193.mmmqzh 2024-05-09T06:37:59.402 INFO:tasks.workunit.client.0.smithi063.stdout:9/32: unlink d0/d4/f7 0 2024-05-09T06:37:59.402 INFO:tasks.workunit.client.0.smithi063.stdout:9/33: dread - d0/d4/f9 zero size 2024-05-09T06:37:59.402 INFO:tasks.workunit.client.0.smithi063.stdout:9/34: chown d0/d8 7 1 2024-05-09T06:37:59.404 INFO:tasks.workunit.client.0.smithi063.stdout:8/50: rename d9/da/ff to d9/f14 0 2024-05-09T06:37:59.404 INFO:tasks.workunit.client.0.smithi063.stdout:8/51: dread - d9/fc zero size 2024-05-09T06:37:59.411 INFO:tasks.workunit.client.0.smithi063.stdout:7/31: dwrite d0/f4 [0,4194304] 0 2024-05-09T06:37:59.416 INFO:tasks.workunit.client.0.smithi063.stdout:7/32: creat d0/f5 x:0 0 0 2024-05-09T06:37:59.423 INFO:tasks.workunit.client.0.smithi063.stdout:7/33: creat d0/f6 x:0 0 0 2024-05-09T06:37:59.430 INFO:tasks.workunit.client.0.smithi063.stdout:7/34: dwrite d0/f4 [0,4194304] 0 2024-05-09T06:37:59.432 INFO:tasks.workunit.client.0.smithi063.stdout:9/35: truncate d0/d4/f6 1920605 0 2024-05-09T06:37:59.433 INFO:tasks.workunit.client.0.smithi063.stdout:9/36: chown d0/c2 1115392 1 2024-05-09T06:37:59.434 INFO:tasks.workunit.client.0.smithi063.stdout:8/52: dwrite f0 [0,4194304] 0 2024-05-09T06:37:59.435 INFO:tasks.workunit.client.0.smithi063.stdout:7/35: mknod d0/c7 0 2024-05-09T06:37:59.446 INFO:tasks.workunit.client.0.smithi063.stdout:9/37: mknod d0/d4/da/cb 0 2024-05-09T06:37:59.448 INFO:tasks.workunit.client.0.smithi063.stdout:8/53: dwrite d9/f13 [0,4194304] 0 2024-05-09T06:37:59.448 INFO:tasks.workunit.client.0.smithi063.stdout:8/54: stat c8 0 2024-05-09T06:37:59.450 INFO:tasks.workunit.client.0.smithi063.stdout:7/36: unlink d0/f6 0 2024-05-09T06:37:59.452 INFO:tasks.workunit.client.0.smithi063.stdout:9/38: symlink d0/lc 0 2024-05-09T06:37:59.453 INFO:tasks.workunit.client.0.smithi063.stdout:9/39: chown d0/d4/f9 514526096 1 2024-05-09T06:37:59.454 INFO:tasks.workunit.client.0.smithi063.stdout:9/40: dread - d0/d4/f9 zero size 2024-05-09T06:37:59.455 INFO:tasks.workunit.client.0.smithi063.stdout:8/55: mkdir d9/da/d15 0 2024-05-09T06:37:59.455 INFO:tasks.workunit.client.0.smithi063.stdout:8/56: readlink l1 0 2024-05-09T06:37:59.456 INFO:tasks.workunit.client.0.smithi063.stdout:7/37: creat d0/f8 x:0 0 0 2024-05-09T06:37:59.474 INFO:tasks.workunit.client.0.smithi063.stdout:8/57: dread d9/f13 [0,4194304] 0 2024-05-09T06:37:59.590 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:59 smithi193 ceph-mon[35141]: pgmap v268: 65 pgs: 65 active+clean; 193 MiB data, 898 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 374 op/s 2024-05-09T06:37:59.590 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:59 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:59.590 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:59 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:59.590 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:59 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:37:59.591 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:59 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:37:59.591 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:59 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' 2024-05-09T06:37:59.591 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:59 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:37:59.591 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:59 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr fail", "who": "smithi063.egfguy"}]: dispatch 2024-05-09T06:37:59.591 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:37:59 smithi193 ceph-mon[35141]: Activating manager daemon smithi193.mmmqzh 2024-05-09T06:37:59.698 INFO:tasks.workunit.client.0.smithi063.stdout:9/41: link d0/lc d0/d4/da/ld 0 2024-05-09T06:37:59.699 INFO:tasks.workunit.client.0.smithi063.stdout:8/58: rename c6 to d9/c16 0 2024-05-09T06:37:59.702 INFO:tasks.workunit.client.0.smithi063.stdout:8/59: mknod d9/c17 0 2024-05-09T06:37:59.702 INFO:tasks.workunit.client.0.smithi063.stdout:8/60: chown d9 17 1 2024-05-09T06:37:59.705 INFO:tasks.workunit.client.0.smithi063.stdout:8/61: rename d9/da/d15 to d9/da/d18 0 2024-05-09T06:37:59.710 INFO:tasks.workunit.client.0.smithi063.stdout:8/62: dwrite f4 [0,4194304] 0 2024-05-09T06:37:59.715 INFO:tasks.workunit.client.0.smithi063.stdout:8/63: link d9/da/fb d9/da/f19 0 2024-05-09T06:37:59.717 INFO:tasks.workunit.client.0.smithi063.stdout:8/64: write d9/f13 [2824082,98327] 0 2024-05-09T06:37:59.722 INFO:tasks.workunit.client.0.smithi063.stdout:8/65: rename d9/fc to d9/da/f1a 0 2024-05-09T06:37:59.723 INFO:tasks.workunit.client.0.smithi063.stdout:8/66: read d9/f13 [1463382,123378] 0 2024-05-09T06:37:59.724 INFO:tasks.workunit.client.0.smithi063.stdout:8/67: write f4 [1676747,60341] 0 2024-05-09T06:37:59.725 INFO:tasks.workunit.client.0.smithi063.stdout:8/68: write f0 [3403070,3801] 0 2024-05-09T06:37:59.726 INFO:tasks.workunit.client.0.smithi063.stdout:8/69: truncate d9/da/f1a 552121 0 2024-05-09T06:37:59.730 INFO:tasks.workunit.client.0.smithi063.stdout:8/70: dread f0 [0,4194304] 0 2024-05-09T06:37:59.848 INFO:tasks.workunit.client.0.smithi063.stdout:9/42: dread d0/d4/f6 [0,4194304] 0 2024-05-09T06:37:59.850 INFO:tasks.workunit.client.0.smithi063.stdout:9/43: dread d0/d4/f6 [0,4194304] 0 2024-05-09T06:37:59.853 INFO:tasks.workunit.client.0.smithi063.stdout:9/44: fdatasync d0/d4/f6 0 2024-05-09T06:37:59.857 INFO:tasks.workunit.client.0.smithi063.stdout:8/71: fsync d9/da/fb 0 2024-05-09T06:37:59.862 INFO:tasks.workunit.client.0.smithi063.stdout:8/72: creat d9/da/d18/f1b x:0 0 0 2024-05-09T06:37:59.895 INFO:tasks.workunit.client.0.smithi063.stdout:8/73: rmdir d9/da 39 2024-05-09T06:37:59.982 INFO:tasks.workunit.client.0.smithi063.stdout:8/74: fsync d9/da/f19 0 2024-05-09T06:38:00.033 INFO:tasks.workunit.client.0.smithi063.stdout:8/75: dwrite d9/f13 [4194304,4194304] 0 2024-05-09T06:38:00.038 INFO:tasks.workunit.client.0.smithi063.stdout:8/76: dread f2 [0,4194304] 0 2024-05-09T06:38:00.232 INFO:tasks.workunit.client.0.smithi063.stdout:8/77: dwrite d9/da/d18/f1b [0,4194304] 0 2024-05-09T06:38:00.233 INFO:tasks.workunit.client.0.smithi063.stdout:8/78: rename d9/da/d18 to d9/da/d18/d1c 22 2024-05-09T06:38:00.234 INFO:tasks.workunit.client.0.smithi063.stdout:8/79: write d9/f13 [8504495,130625] 0 2024-05-09T06:38:00.237 INFO:tasks.workunit.client.0.smithi063.stdout:8/80: dread d9/f14 [0,4194304] 0 2024-05-09T06:38:00.238 INFO:tasks.workunit.client.0.smithi063.stdout:8/81: dread - d9/da/f19 zero size 2024-05-09T06:38:00.238 INFO:tasks.workunit.client.0.smithi063.stdout:8/82: stat d9/da/l12 0 2024-05-09T06:38:00.240 INFO:tasks.workunit.client.0.smithi063.stdout:8/83: dread d9/da/f1a [0,4194304] 0 2024-05-09T06:38:00.245 INFO:tasks.workunit.client.0.smithi063.stdout:8/84: creat d9/da/d18/f1d x:0 0 0 2024-05-09T06:38:00.248 INFO:tasks.workunit.client.0.smithi063.stdout:8/85: write d9/f14 [3954467,56351] 0 2024-05-09T06:38:00.254 INFO:tasks.workunit.client.0.smithi063.stdout:8/86: creat d9/da/d18/f1e x:0 0 0 2024-05-09T06:38:00.254 INFO:tasks.workunit.client.0.smithi063.stdout:8/87: readlink l1 0 2024-05-09T06:38:00.255 INFO:tasks.workunit.client.0.smithi063.stdout:8/88: chown d9/c17 113 1 2024-05-09T06:38:00.255 INFO:tasks.workunit.client.0.smithi063.stdout:8/89: dread - d9/da/fb zero size 2024-05-09T06:38:00.255 INFO:tasks.workunit.client.0.smithi063.stdout:8/90: chown d9/da/d18 2029 1 2024-05-09T06:38:00.258 INFO:tasks.workunit.client.0.smithi063.stdout:8/91: rename d9/da/cd to d9/da/d18/c1f 0 2024-05-09T06:38:00.260 INFO:tasks.workunit.client.0.smithi063.stdout:8/92: creat d9/da/f20 x:0 0 0 2024-05-09T06:38:00.262 INFO:tasks.workunit.client.0.smithi063.stdout:8/93: symlink d9/da/d18/l21 0 2024-05-09T06:38:00.265 INFO:tasks.workunit.client.0.smithi063.stdout:8/94: creat d9/da/d18/f22 x:0 0 0 2024-05-09T06:38:00.265 INFO:tasks.workunit.client.0.smithi063.stdout:8/95: chown d9/da/l12 61 1 2024-05-09T06:38:00.385 INFO:tasks.workunit.client.0.smithi063.stdout:8/96: dwrite d9/da/fb [0,4194304] 0 2024-05-09T06:38:00.388 INFO:tasks.workunit.client.0.smithi063.stdout:8/97: dread d9/da/f19 [0,4194304] 0 2024-05-09T06:38:00.469 INFO:tasks.workunit.client.0.smithi063.stdout:8/98: dwrite d9/f13 [0,4194304] 0 2024-05-09T06:38:00.471 INFO:tasks.workunit.client.0.smithi063.stdout:8/99: creat d9/da/d18/f23 x:0 0 0 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: osdmap e48: 6 total, 6 up, 6 in 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "mgr fail", "who": "smithi063.egfguy"}]': finished 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: mgrmap e21: smithi193.mmmqzh(active, starting, since 0.00705512s) 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:38:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mmmqzh", "id": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: Manager daemon smithi193.mmmqzh is now available 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mmmqzh/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mmmqzh/trash_purge_schedule"}]: dispatch 2024-05-09T06:38:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: Deploying cephadm binary to smithi063 2024-05-09T06:38:00.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: Deploying cephadm binary to smithi193 2024-05-09T06:38:00.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:00 smithi063 ceph-mon[27171]: [09/May/2024:06:38:00] ENGINE Bus STARTING 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: osdmap e48: 6 total, 6 up, 6 in 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.14215 172.21.15.63:0/1854631598' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "mgr fail", "who": "smithi063.egfguy"}]': finished 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: mgrmap e21: smithi193.mmmqzh(active, starting, since 0.00705512s) 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mmmqzh", "id": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:38:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: Manager daemon smithi193.mmmqzh is now available 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mmmqzh/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mmmqzh/trash_purge_schedule"}]: dispatch 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: Deploying cephadm binary to smithi063 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: Deploying cephadm binary to smithi193 2024-05-09T06:38:00.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:00 smithi193 ceph-mon[35141]: [09/May/2024:06:38:00] ENGINE Bus STARTING 2024-05-09T06:38:00.828 INFO:tasks.workunit.client.0.smithi063.stdout:8/100: truncate d9/da/f19 1155825 0 2024-05-09T06:38:00.829 INFO:tasks.workunit.client.0.smithi063.stdout:8/101: fdatasync d9/da/d18/f1b 0 2024-05-09T06:38:00.832 INFO:tasks.workunit.client.0.smithi063.stdout:8/102: mkdir d9/da/d18/d24 0 2024-05-09T06:38:00.833 INFO:tasks.workunit.client.0.smithi063.stdout:8/103: read - d9/da/f20 zero size 2024-05-09T06:38:00.840 INFO:tasks.workunit.client.0.smithi063.stdout:8/104: dwrite d9/da/d18/f1b [0,4194304] 0 2024-05-09T06:38:00.849 INFO:tasks.workunit.client.0.smithi063.stdout:8/105: dwrite d9/f14 [0,4194304] 0 2024-05-09T06:38:00.853 INFO:tasks.workunit.client.0.smithi063.stdout:8/106: mkdir d9/da/d18/d25 0 2024-05-09T06:38:00.856 INFO:tasks.workunit.client.0.smithi063.stdout:8/107: getdents d9/da/d18/d25 0 2024-05-09T06:38:00.861 INFO:tasks.workunit.client.0.smithi063.stdout:8/108: dread f4 [0,4194304] 0 2024-05-09T06:38:00.937 INFO:tasks.workunit.client.0.smithi063.stdout:5/4: sync 2024-05-09T06:38:00.937 INFO:tasks.workunit.client.0.smithi063.stdout:5/5: chown . 39157057 1 2024-05-09T06:38:00.937 INFO:tasks.workunit.client.0.smithi063.stdout:5/6: dread - no filename 2024-05-09T06:38:00.937 INFO:tasks.workunit.client.0.smithi063.stdout:5/7: write - no filename 2024-05-09T06:38:00.937 INFO:tasks.workunit.client.0.smithi063.stdout:5/8: rename - no filename 2024-05-09T06:38:00.939 INFO:tasks.workunit.client.0.smithi063.stdout:5/9: mknod c0 0 2024-05-09T06:38:00.942 INFO:tasks.workunit.client.0.smithi063.stdout:5/10: rename c0 to c1 0 2024-05-09T06:38:00.963 INFO:tasks.workunit.client.0.smithi063.stdout:5/11: chown c1 3 1 2024-05-09T06:38:00.966 INFO:tasks.workunit.client.0.smithi063.stdout:5/12: rename c1 to c2 0 2024-05-09T06:38:00.966 INFO:tasks.workunit.client.0.smithi063.stdout:5/13: truncate - no filename 2024-05-09T06:38:00.974 INFO:tasks.workunit.client.0.smithi063.stdout:5/14: chown c2 6410441 1 2024-05-09T06:38:01.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/7: sync 2024-05-09T06:38:01.061 INFO:tasks.workunit.client.0.smithi063.stdout:3/9: sync 2024-05-09T06:38:01.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/8: dread - no filename 2024-05-09T06:38:01.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/9: dwrite - no filename 2024-05-09T06:38:01.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/10: dwrite - no filename 2024-05-09T06:38:01.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/11: truncate - no filename 2024-05-09T06:38:01.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/12: dread - no filename 2024-05-09T06:38:01.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/13: read - no filename 2024-05-09T06:38:01.062 INFO:tasks.workunit.client.0.smithi063.stdout:3/10: write f1 [1815457,22018] 0 2024-05-09T06:38:01.062 INFO:tasks.workunit.client.0.smithi063.stdout:3/11: readlink - no filename 2024-05-09T06:38:01.063 INFO:tasks.workunit.client.0.smithi063.stdout:4/14: stat l0 0 2024-05-09T06:38:01.068 INFO:tasks.workunit.client.0.smithi063.stdout:4/15: getdents . 0 2024-05-09T06:38:01.068 INFO:tasks.workunit.client.0.smithi063.stdout:4/16: dwrite - no filename 2024-05-09T06:38:01.068 INFO:tasks.workunit.client.0.smithi063.stdout:4/17: dwrite - no filename 2024-05-09T06:38:01.068 INFO:tasks.workunit.client.0.smithi063.stdout:4/18: dwrite - no filename 2024-05-09T06:38:01.072 INFO:tasks.workunit.client.0.smithi063.stdout:4/19: link l2 l3 0 2024-05-09T06:38:01.072 INFO:tasks.workunit.client.0.smithi063.stdout:4/20: dread - no filename 2024-05-09T06:38:01.072 INFO:tasks.workunit.client.0.smithi063.stdout:4/21: dwrite - no filename 2024-05-09T06:38:01.074 INFO:tasks.workunit.client.0.smithi063.stdout:4/22: creat f4 x:0 0 0 2024-05-09T06:38:01.075 INFO:tasks.workunit.client.0.smithi063.stdout:4/23: dread - f4 zero size 2024-05-09T06:38:01.080 INFO:tasks.workunit.client.0.smithi063.stdout:4/24: dwrite f4 [0,4194304] 0 2024-05-09T06:38:01.081 INFO:tasks.workunit.client.0.smithi063.stdout:4/25: write f4 [785707,120948] 0 2024-05-09T06:38:01.085 INFO:tasks.workunit.client.0.smithi063.stdout:4/26: dread f4 [0,4194304] 0 2024-05-09T06:38:01.087 INFO:tasks.workunit.client.0.smithi063.stdout:4/27: creat f5 x:0 0 0 2024-05-09T06:38:01.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/28: dwrite f5 [0,4194304] 0 2024-05-09T06:38:01.099 INFO:tasks.workunit.client.0.smithi063.stdout:9/45: sync 2024-05-09T06:38:01.099 INFO:tasks.workunit.client.0.smithi063.stdout:6/10: sync 2024-05-09T06:38:01.102 INFO:tasks.workunit.client.0.smithi063.stdout:9/46: readlink d0/d4/da/ld 0 2024-05-09T06:38:01.105 INFO:tasks.workunit.client.0.smithi063.stdout:6/11: creat d2/d4/f5 x:0 0 0 2024-05-09T06:38:01.106 INFO:tasks.workunit.client.0.smithi063.stdout:3/12: write f2 [766859,35778] 0 2024-05-09T06:38:01.108 INFO:tasks.workunit.client.0.smithi063.stdout:9/47: truncate d0/d4/f6 833144 0 2024-05-09T06:38:01.109 INFO:tasks.workunit.client.0.smithi063.stdout:6/12: write f0 [4021595,33777] 0 2024-05-09T06:38:01.112 INFO:tasks.workunit.client.0.smithi063.stdout:4/29: fsync f4 0 2024-05-09T06:38:01.116 INFO:tasks.workunit.client.0.smithi063.stdout:3/13: dread f2 [0,4194304] 0 2024-05-09T06:38:01.122 INFO:tasks.workunit.client.0.smithi063.stdout:9/48: dwrite d0/d4/f9 [0,4194304] 0 2024-05-09T06:38:01.122 INFO:tasks.workunit.client.0.smithi063.stdout:4/30: dread f4 [0,4194304] 0 2024-05-09T06:38:01.123 INFO:tasks.workunit.client.0.smithi063.stdout:3/14: link f1 f3 0 2024-05-09T06:38:01.123 INFO:tasks.workunit.client.0.smithi063.stdout:4/31: chown f4 251099 1 2024-05-09T06:38:01.128 INFO:tasks.workunit.client.0.smithi063.stdout:3/15: creat f4 x:0 0 0 2024-05-09T06:38:01.129 INFO:tasks.workunit.client.0.smithi063.stdout:4/32: creat f6 x:0 0 0 2024-05-09T06:38:01.135 INFO:tasks.workunit.client.0.smithi063.stdout:3/16: creat f5 x:0 0 0 2024-05-09T06:38:01.136 INFO:tasks.workunit.client.0.smithi063.stdout:4/33: mkdir d7 0 2024-05-09T06:38:01.139 INFO:tasks.workunit.client.0.smithi063.stdout:3/17: symlink l6 0 2024-05-09T06:38:01.144 INFO:tasks.workunit.client.0.smithi063.stdout:4/34: mkdir d7/d8 0 2024-05-09T06:38:01.144 INFO:tasks.workunit.client.0.smithi063.stdout:4/35: fdatasync f6 0 2024-05-09T06:38:01.148 INFO:tasks.workunit.client.0.smithi063.stdout:4/36: creat d7/f9 x:0 0 0 2024-05-09T06:38:01.153 INFO:tasks.workunit.client.0.smithi063.stdout:4/37: readlink l3 0 2024-05-09T06:38:01.153 INFO:tasks.workunit.client.0.smithi063.stdout:9/49: dwrite d0/d4/f6 [0,4194304] 0 2024-05-09T06:38:01.153 INFO:tasks.workunit.client.0.smithi063.stdout:3/18: getdents . 0 2024-05-09T06:38:01.155 INFO:tasks.workunit.client.0.smithi063.stdout:9/50: truncate d0/d4/f6 4980695 0 2024-05-09T06:38:01.156 INFO:tasks.workunit.client.0.smithi063.stdout:9/51: readlink d0/d4/da/ld 0 2024-05-09T06:38:01.156 INFO:tasks.workunit.client.0.smithi063.stdout:3/19: mkdir d7 0 2024-05-09T06:38:01.166 INFO:tasks.workunit.client.0.smithi063.stdout:9/52: dwrite d0/d4/f6 [0,4194304] 0 2024-05-09T06:38:01.167 INFO:tasks.workunit.client.0.smithi063.stdout:9/53: stat d0/d4/da/ld 0 2024-05-09T06:38:01.170 INFO:tasks.workunit.client.0.smithi063.stdout:4/38: truncate f5 1732612 0 2024-05-09T06:38:01.170 INFO:tasks.workunit.client.0.smithi063.stdout:9/54: rmdir d0/d4/da 39 2024-05-09T06:38:01.173 INFO:tasks.workunit.client.0.smithi063.stdout:4/39: unlink l2 0 2024-05-09T06:38:01.175 INFO:tasks.workunit.client.0.smithi063.stdout:4/40: mknod d7/ca 0 2024-05-09T06:38:01.177 INFO:tasks.workunit.client.0.smithi063.stdout:9/55: unlink d0/d4/da/cb 0 2024-05-09T06:38:01.179 INFO:tasks.workunit.client.0.smithi063.stdout:3/20: dwrite f3 [0,4194304] 0 2024-05-09T06:38:01.179 INFO:tasks.workunit.client.0.smithi063.stdout:3/21: fsync f5 0 2024-05-09T06:38:01.180 INFO:tasks.workunit.client.0.smithi063.stdout:4/41: creat d7/fb x:0 0 0 2024-05-09T06:38:01.180 INFO:tasks.workunit.client.0.smithi063.stdout:3/22: chown f4 42 1 2024-05-09T06:38:01.185 INFO:tasks.workunit.client.0.smithi063.stdout:3/23: dread f3 [0,4194304] 0 2024-05-09T06:38:01.185 INFO:tasks.workunit.client.0.smithi063.stdout:9/56: fsync d0/d4/f6 0 2024-05-09T06:38:01.193 INFO:tasks.workunit.client.0.smithi063.stdout:4/42: rmdir d7/d8 0 2024-05-09T06:38:01.194 INFO:tasks.workunit.client.0.smithi063.stdout:3/24: creat d7/f8 x:0 0 0 2024-05-09T06:38:01.198 INFO:tasks.workunit.client.0.smithi063.stdout:4/43: stat l3 0 2024-05-09T06:38:01.198 INFO:tasks.workunit.client.0.smithi063.stdout:4/44: dread - d7/fb zero size 2024-05-09T06:38:01.200 INFO:tasks.workunit.client.0.smithi063.stdout:3/25: unlink f4 0 2024-05-09T06:38:01.201 INFO:tasks.workunit.client.0.smithi063.stdout:3/26: dread - f5 zero size 2024-05-09T06:38:01.201 INFO:tasks.workunit.client.0.smithi063.stdout:1/32: sync 2024-05-09T06:38:01.201 INFO:tasks.workunit.client.0.smithi063.stdout:2/14: sync 2024-05-09T06:38:01.201 INFO:tasks.workunit.client.0.smithi063.stdout:0/50: sync 2024-05-09T06:38:01.202 INFO:tasks.workunit.client.0.smithi063.stdout:7/38: sync 2024-05-09T06:38:01.202 INFO:tasks.workunit.client.0.smithi063.stdout:8/109: sync 2024-05-09T06:38:01.202 INFO:tasks.workunit.client.0.smithi063.stdout:5/15: sync 2024-05-09T06:38:01.203 INFO:tasks.workunit.client.0.smithi063.stdout:9/57: link d0/lc d0/d8/le 0 2024-05-09T06:38:01.206 INFO:tasks.workunit.client.0.smithi063.stdout:8/110: write d9/da/d18/f22 [74737,27759] 0 2024-05-09T06:38:01.209 INFO:tasks.workunit.client.0.smithi063.stdout:2/15: mknod c7 0 2024-05-09T06:38:01.211 INFO:tasks.workunit.client.0.smithi063.stdout:2/16: dread - f3 zero size 2024-05-09T06:38:01.211 INFO:tasks.workunit.client.0.smithi063.stdout:3/27: write f2 [705069,119783] 0 2024-05-09T06:38:01.212 INFO:tasks.workunit.client.0.smithi063.stdout:5/16: mknod c3 0 2024-05-09T06:38:01.215 INFO:tasks.workunit.client.0.smithi063.stdout:9/58: rename d0/d4/f6 to d0/d4/da/ff 0 2024-05-09T06:38:01.216 INFO:tasks.workunit.client.0.smithi063.stdout:7/39: dwrite d0/f8 [0,4194304] 0 2024-05-09T06:38:01.218 INFO:tasks.workunit.client.0.smithi063.stdout:4/45: link d7/fb d7/fc 0 2024-05-09T06:38:01.218 INFO:tasks.workunit.client.0.smithi063.stdout:1/33: dwrite d9/fb [0,4194304] 0 2024-05-09T06:38:01.231 INFO:tasks.workunit.client.0.smithi063.stdout:5/17: creat f4 x:0 0 0 2024-05-09T06:38:01.238 INFO:tasks.workunit.client.0.smithi063.stdout:9/59: unlink d0/d4/f9 0 2024-05-09T06:38:01.241 INFO:tasks.workunit.client.0.smithi063.stdout:3/28: dwrite f1 [0,4194304] 0 2024-05-09T06:38:01.242 INFO:tasks.workunit.client.0.smithi063.stdout:1/34: mkdir d9/d10 0 2024-05-09T06:38:01.243 INFO:tasks.workunit.client.0.smithi063.stdout:3/29: write f5 [612463,9855] 0 2024-05-09T06:38:01.245 INFO:tasks.workunit.client.0.smithi063.stdout:3/30: write f0 [717234,81777] 0 2024-05-09T06:38:01.246 INFO:tasks.workunit.client.0.smithi063.stdout:5/18: symlink l5 0 2024-05-09T06:38:01.247 INFO:tasks.workunit.client.0.smithi063.stdout:3/31: dread - d7/f8 zero size 2024-05-09T06:38:01.247 INFO:tasks.workunit.client.0.smithi063.stdout:5/19: stat c2 0 2024-05-09T06:38:01.249 INFO:tasks.workunit.client.0.smithi063.stdout:4/46: dwrite d7/fb [0,4194304] 0 2024-05-09T06:38:01.252 INFO:tasks.workunit.client.0.smithi063.stdout:8/111: symlink d9/da/d18/d24/l26 0 2024-05-09T06:38:01.258 INFO:tasks.workunit.client.0.smithi063.stdout:5/20: dwrite f4 [0,4194304] 0 2024-05-09T06:38:01.266 INFO:tasks.workunit.client.0.smithi063.stdout:3/32: mknod d7/c9 0 2024-05-09T06:38:01.269 INFO:tasks.workunit.client.0.smithi063.stdout:4/47: rmdir d7 39 2024-05-09T06:38:01.272 INFO:tasks.workunit.client.0.smithi063.stdout:4/48: chown f4 2 1 2024-05-09T06:38:01.275 INFO:tasks.workunit.client.0.smithi063.stdout:5/21: creat f6 x:0 0 0 2024-05-09T06:38:01.276 INFO:tasks.workunit.client.0.smithi063.stdout:8/112: dwrite d9/da/f20 [0,4194304] 0 2024-05-09T06:38:01.279 INFO:tasks.workunit.client.0.smithi063.stdout:5/22: readlink l5 0 2024-05-09T06:38:01.279 INFO:tasks.workunit.client.0.smithi063.stdout:3/33: dread f5 [0,4194304] 0 2024-05-09T06:38:01.280 INFO:tasks.workunit.client.0.smithi063.stdout:2/17: write f0 [1000559,24873] 0 2024-05-09T06:38:01.280 INFO:tasks.workunit.client.0.smithi063.stdout:0/51: dwrite dd/fe [0,4194304] 0 2024-05-09T06:38:01.281 INFO:tasks.workunit.client.0.smithi063.stdout:9/60: dwrite d0/d4/da/ff [4194304,4194304] 0 2024-05-09T06:38:01.289 INFO:tasks.workunit.client.0.smithi063.stdout:5/23: creat f7 x:0 0 0 2024-05-09T06:38:01.291 INFO:tasks.workunit.client.0.smithi063.stdout:5/24: chown c2 0 1 2024-05-09T06:38:01.292 INFO:tasks.workunit.client.0.smithi063.stdout:9/61: dread d0/d4/da/ff [4194304,4194304] 0 2024-05-09T06:38:01.292 INFO:tasks.workunit.client.0.smithi063.stdout:2/18: dwrite f3 [0,4194304] 0 2024-05-09T06:38:01.294 INFO:tasks.workunit.client.0.smithi063.stdout:3/34: mkdir d7/da 0 2024-05-09T06:38:01.298 INFO:tasks.workunit.client.0.smithi063.stdout:8/113: creat d9/da/d18/d24/f27 x:0 0 0 2024-05-09T06:38:01.301 INFO:tasks.workunit.client.0.smithi063.stdout:0/52: mknod dd/cf 0 2024-05-09T06:38:01.302 INFO:tasks.workunit.client.0.smithi063.stdout:4/49: truncate f5 2418137 0 2024-05-09T06:38:01.304 INFO:tasks.workunit.client.0.smithi063.stdout:1/35: getdents d9 0 2024-05-09T06:38:01.305 INFO:tasks.workunit.client.0.smithi063.stdout:4/50: read f4 [1650199,103019] 0 2024-05-09T06:38:01.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/19: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:01.307 INFO:tasks.workunit.client.0.smithi063.stdout:9/62: stat d0/d4/da/ld 0 2024-05-09T06:38:01.312 INFO:tasks.workunit.client.0.smithi063.stdout:0/53: mknod dd/c10 0 2024-05-09T06:38:01.313 INFO:tasks.workunit.client.0.smithi063.stdout:0/54: readlink l7 0 2024-05-09T06:38:01.314 INFO:tasks.workunit.client.0.smithi063.stdout:1/36: creat d9/f11 x:0 0 0 2024-05-09T06:38:01.315 INFO:tasks.workunit.client.0.smithi063.stdout:1/37: truncate f3 1995878 0 2024-05-09T06:38:01.316 INFO:tasks.workunit.client.0.smithi063.stdout:4/51: fdatasync d7/f9 0 2024-05-09T06:38:01.316 INFO:tasks.workunit.client.0.smithi063.stdout:2/20: symlink l8 0 2024-05-09T06:38:01.317 INFO:tasks.workunit.client.0.smithi063.stdout:4/52: write d7/f9 [71768,120424] 0 2024-05-09T06:38:01.318 INFO:tasks.workunit.client.0.smithi063.stdout:9/63: fsync d0/d4/da/ff 0 2024-05-09T06:38:01.318 INFO:tasks.workunit.client.0.smithi063.stdout:3/35: creat d7/da/fb x:0 0 0 2024-05-09T06:38:01.320 INFO:tasks.workunit.client.0.smithi063.stdout:9/64: read d0/d4/da/ff [4328527,69612] 0 2024-05-09T06:38:01.321 INFO:tasks.workunit.client.0.smithi063.stdout:8/114: unlink d9/c16 0 2024-05-09T06:38:01.324 INFO:tasks.workunit.client.0.smithi063.stdout:1/38: unlink d9/lc 0 2024-05-09T06:38:01.326 INFO:tasks.workunit.client.0.smithi063.stdout:2/21: unlink f0 0 2024-05-09T06:38:01.329 INFO:tasks.workunit.client.0.smithi063.stdout:3/36: dwrite d7/da/fb [0,4194304] 0 2024-05-09T06:38:01.336 INFO:tasks.workunit.client.0.smithi063.stdout:2/22: creat f9 x:0 0 0 2024-05-09T06:38:01.337 INFO:tasks.workunit.client.0.smithi063.stdout:4/53: link d7/fc d7/fd 0 2024-05-09T06:38:01.339 INFO:tasks.workunit.client.0.smithi063.stdout:2/23: dread f6 [0,4194304] 0 2024-05-09T06:38:01.340 INFO:tasks.workunit.client.0.smithi063.stdout:3/37: creat d7/da/fc x:0 0 0 2024-05-09T06:38:01.342 INFO:tasks.workunit.client.0.smithi063.stdout:0/55: link dd/c10 dd/c11 0 2024-05-09T06:38:01.342 INFO:tasks.workunit.client.0.smithi063.stdout:2/24: dread f3 [0,4194304] 0 2024-05-09T06:38:01.343 INFO:tasks.workunit.client.0.smithi063.stdout:0/56: chown dd/fe 64208782 1 2024-05-09T06:38:01.344 INFO:tasks.workunit.client.0.smithi063.stdout:2/25: write f9 [345081,117950] 0 2024-05-09T06:38:01.344 INFO:tasks.workunit.client.0.smithi063.stdout:0/57: dread f2 [0,4194304] 0 2024-05-09T06:38:01.345 INFO:tasks.workunit.client.0.smithi063.stdout:1/39: unlink c6 0 2024-05-09T06:38:01.350 INFO:tasks.workunit.client.0.smithi063.stdout:4/54: symlink d7/le 0 2024-05-09T06:38:01.353 INFO:tasks.workunit.client.0.smithi063.stdout:4/55: dread d7/fd [0,4194304] 0 2024-05-09T06:38:01.355 INFO:tasks.workunit.client.0.smithi063.stdout:3/38: rename d7/c9 to d7/cd 0 2024-05-09T06:38:01.355 INFO:tasks.workunit.client.0.smithi063.stdout:4/56: dread d7/f9 [0,4194304] 0 2024-05-09T06:38:01.356 INFO:tasks.workunit.client.0.smithi063.stdout:3/39: chown d7/f8 31907015 1 2024-05-09T06:38:01.357 INFO:tasks.workunit.client.0.smithi063.stdout:2/26: creat fa x:0 0 0 2024-05-09T06:38:01.358 INFO:tasks.workunit.client.0.smithi063.stdout:1/40: creat d9/f12 x:0 0 0 2024-05-09T06:38:01.361 INFO:tasks.workunit.client.0.smithi063.stdout:0/58: unlink l7 0 2024-05-09T06:38:01.364 INFO:tasks.workunit.client.0.smithi063.stdout:9/65: getdents d0 0 2024-05-09T06:38:01.368 INFO:tasks.workunit.client.0.smithi063.stdout:4/57: write d7/fd [3485569,41111] 0 2024-05-09T06:38:01.368 INFO:tasks.workunit.client.0.smithi063.stdout:4/58: dread - f6 zero size 2024-05-09T06:38:01.375 INFO:tasks.workunit.client.0.smithi063.stdout:4/59: dwrite f4 [0,4194304] 0 2024-05-09T06:38:01.375 INFO:tasks.workunit.client.0.smithi063.stdout:1/41: rename l7 to d9/l13 0 2024-05-09T06:38:01.378 INFO:tasks.workunit.client.0.smithi063.stdout:0/59: creat dd/f12 x:0 0 0 2024-05-09T06:38:01.378 INFO:tasks.workunit.client.0.smithi063.stdout:0/60: chown c8 10 1 2024-05-09T06:38:01.379 INFO:tasks.workunit.client.0.smithi063.stdout:0/61: write dd/f12 [55730,62990] 0 2024-05-09T06:38:01.388 INFO:tasks.workunit.client.0.smithi063.stdout:1/42: mknod d9/c14 0 2024-05-09T06:38:01.392 INFO:tasks.workunit.client.0.smithi063.stdout:1/43: dread f8 [0,4194304] 0 2024-05-09T06:38:01.393 INFO:tasks.workunit.client.0.smithi063.stdout:1/44: chown d9/f11 4163 1 2024-05-09T06:38:01.393 INFO:tasks.workunit.client.0.smithi063.stdout:9/66: truncate d0/d4/da/ff 6168823 0 2024-05-09T06:38:01.393 INFO:tasks.workunit.client.0.smithi063.stdout:1/45: chown d9/fe 410337 1 2024-05-09T06:38:01.395 INFO:tasks.workunit.client.0.smithi063.stdout:0/62: rename c8 to dd/c13 0 2024-05-09T06:38:01.408 INFO:tasks.workunit.client.0.smithi063.stdout:3/40: getdents d7 0 2024-05-09T06:38:01.411 INFO:tasks.workunit.client.0.smithi063.stdout:2/27: getdents . 0 2024-05-09T06:38:01.417 INFO:tasks.workunit.client.0.smithi063.stdout:1/46: creat d9/d10/f15 x:0 0 0 2024-05-09T06:38:01.417 INFO:tasks.workunit.client.0.smithi063.stdout:1/47: chown d9/fb 474117023 1 2024-05-09T06:38:01.418 INFO:tasks.workunit.client.0.smithi063.stdout:1/48: read - d9/f12 zero size 2024-05-09T06:38:01.418 INFO:tasks.workunit.client.0.smithi063.stdout:1/49: readlink l4 0 2024-05-09T06:38:01.418 INFO:tasks.workunit.client.0.smithi063.stdout:1/50: chown d9/fb 130 1 2024-05-09T06:38:01.420 INFO:tasks.workunit.client.0.smithi063.stdout:0/63: rename dd/c11 to dd/c14 0 2024-05-09T06:38:01.422 INFO:tasks.workunit.client.0.smithi063.stdout:3/41: creat d7/da/fe x:0 0 0 2024-05-09T06:38:01.423 INFO:tasks.workunit.client.0.smithi063.stdout:2/28: fsync f3 0 2024-05-09T06:38:01.425 INFO:tasks.workunit.client.0.smithi063.stdout:4/60: getdents d7 0 2024-05-09T06:38:01.428 INFO:tasks.workunit.client.0.smithi063.stdout:1/51: mknod d9/d10/c16 0 2024-05-09T06:38:01.428 INFO:tasks.workunit.client.0.smithi063.stdout:1/52: write d9/fa [699561,17551] 0 2024-05-09T06:38:01.428 INFO:tasks.workunit.client.0.smithi063.stdout:1/53: readlink d9/lf 0 2024-05-09T06:38:01.429 INFO:tasks.workunit.client.0.smithi063.stdout:0/64: creat dd/f15 x:0 0 0 2024-05-09T06:38:01.431 INFO:tasks.workunit.client.0.smithi063.stdout:3/42: symlink d7/da/lf 0 2024-05-09T06:38:01.434 INFO:tasks.workunit.client.0.smithi063.stdout:2/29: symlink lb 0 2024-05-09T06:38:01.435 INFO:tasks.workunit.client.0.smithi063.stdout:4/61: mkdir d7/df 0 2024-05-09T06:38:01.439 INFO:tasks.workunit.client.0.smithi063.stdout:0/65: write f2 [1010271,83702] 0 2024-05-09T06:38:01.440 INFO:tasks.workunit.client.0.smithi063.stdout:0/66: write f2 [1041317,64425] 0 2024-05-09T06:38:01.441 INFO:tasks.workunit.client.0.smithi063.stdout:3/43: rename d7/da/fc to d7/f10 0 2024-05-09T06:38:01.443 INFO:tasks.workunit.client.0.smithi063.stdout:2/30: mkdir dc 0 2024-05-09T06:38:01.446 INFO:tasks.workunit.client.0.smithi063.stdout:4/62: dread f5 [0,4194304] 0 2024-05-09T06:38:01.450 INFO:tasks.workunit.client.0.smithi063.stdout:3/44: write d7/f10 [376576,126132] 0 2024-05-09T06:38:01.451 INFO:tasks.workunit.client.0.smithi063.stdout:3/45: chown f0 2 1 2024-05-09T06:38:01.452 INFO:tasks.workunit.client.0.smithi063.stdout:0/67: rmdir dd 39 2024-05-09T06:38:01.454 INFO:tasks.workunit.client.0.smithi063.stdout:4/63: unlink d7/fd 0 2024-05-09T06:38:01.454 INFO:tasks.workunit.client.0.smithi063.stdout:3/46: dread f3 [0,4194304] 0 2024-05-09T06:38:01.457 INFO:tasks.workunit.client.0.smithi063.stdout:2/31: mkdir dc/dd 0 2024-05-09T06:38:01.458 INFO:tasks.workunit.client.0.smithi063.stdout:0/68: chown dd 7462 1 2024-05-09T06:38:01.466 INFO:tasks.workunit.client.0.smithi063.stdout:2/32: rmdir dc 39 2024-05-09T06:38:01.467 INFO:tasks.workunit.client.0.smithi063.stdout:7/40: sync 2024-05-09T06:38:01.467 INFO:tasks.workunit.client.0.smithi063.stdout:6/13: sync 2024-05-09T06:38:01.467 INFO:tasks.workunit.client.0.smithi063.stdout:2/33: write f2 [822794,111995] 0 2024-05-09T06:38:01.468 INFO:tasks.workunit.client.0.smithi063.stdout:1/54: truncate f3 586976 0 2024-05-09T06:38:01.469 INFO:tasks.workunit.client.0.smithi063.stdout:0/69: write fc [745692,112115] 0 2024-05-09T06:38:01.474 INFO:tasks.workunit.client.0.smithi063.stdout:1/55: truncate f8 1877031 0 2024-05-09T06:38:01.475 INFO:tasks.workunit.client.0.smithi063.stdout:1/56: readlink l4 0 2024-05-09T06:38:01.475 INFO:tasks.workunit.client.0.smithi063.stdout:7/41: dwrite d0/f5 [0,4194304] 0 2024-05-09T06:38:01.476 INFO:tasks.workunit.client.0.smithi063.stdout:1/57: truncate d9/f12 566426 0 2024-05-09T06:38:01.477 INFO:tasks.workunit.client.0.smithi063.stdout:1/58: readlink d9/lf 0 2024-05-09T06:38:01.477 INFO:tasks.workunit.client.0.smithi063.stdout:0/70: mknod dd/c16 0 2024-05-09T06:38:01.491 INFO:tasks.workunit.client.0.smithi063.stdout:7/42: rename d0/f8 to d0/f9 0 2024-05-09T06:38:01.492 INFO:tasks.workunit.client.0.smithi063.stdout:1/59: mkdir d9/d17 0 2024-05-09T06:38:01.493 INFO:tasks.workunit.client.0.smithi063.stdout:3/47: getdents d7/da 0 2024-05-09T06:38:01.495 INFO:tasks.workunit.client.0.smithi063.stdout:0/71: dread f6 [0,4194304] 0 2024-05-09T06:38:01.497 INFO:tasks.workunit.client.0.smithi063.stdout:0/72: dread - dd/f15 zero size 2024-05-09T06:38:01.501 INFO:tasks.workunit.client.0.smithi063.stdout:1/60: dwrite d9/f11 [0,4194304] 0 2024-05-09T06:38:01.502 INFO:tasks.workunit.client.0.smithi063.stdout:1/61: write d9/fa [1156469,98717] 0 2024-05-09T06:38:01.505 INFO:tasks.workunit.client.0.smithi063.stdout:2/34: dwrite f3 [0,4194304] 0 2024-05-09T06:38:01.513 INFO:tasks.workunit.client.0.smithi063.stdout:1/62: dwrite d9/f12 [0,4194304] 0 2024-05-09T06:38:01.541 INFO:tasks.workunit.client.0.smithi063.stdout:5/25: sync 2024-05-09T06:38:01.544 INFO:tasks.workunit.client.0.smithi063.stdout:5/26: dread f4 [0,4194304] 0 2024-05-09T06:38:01.592 INFO:tasks.workunit.client.0.smithi063.stdout:9/67: sync 2024-05-09T06:38:01.592 INFO:tasks.workunit.client.0.smithi063.stdout:8/115: sync 2024-05-09T06:38:01.592 INFO:tasks.workunit.client.0.smithi063.stdout:8/116: stat d9/da/d18/f22 0 2024-05-09T06:38:01.593 INFO:tasks.workunit.client.0.smithi063.stdout:8/117: readlink d9/da/l12 0 2024-05-09T06:38:01.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:01 smithi063 ceph-mon[27171]: [09/May/2024:06:38:00] ENGINE Serving on http://172.21.15.193:8765 2024-05-09T06:38:01.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:01 smithi063 ceph-mon[27171]: mgrmap e22: smithi193.mmmqzh(active, since 1.02015s) 2024-05-09T06:38:01.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:01 smithi063 ceph-mon[27171]: pgmap v3: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:38:01.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:01 smithi063 ceph-mon[27171]: [09/May/2024:06:38:00] ENGINE Serving on https://172.21.15.193:7150 2024-05-09T06:38:01.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:01 smithi063 ceph-mon[27171]: [09/May/2024:06:38:00] ENGINE Bus STARTED 2024-05-09T06:38:01.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:01 smithi193 ceph-mon[35141]: [09/May/2024:06:38:00] ENGINE Serving on http://172.21.15.193:8765 2024-05-09T06:38:01.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:01 smithi193 ceph-mon[35141]: mgrmap e22: smithi193.mmmqzh(active, since 1.02015s) 2024-05-09T06:38:01.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:01 smithi193 ceph-mon[35141]: pgmap v3: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:38:01.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:01 smithi193 ceph-mon[35141]: [09/May/2024:06:38:00] ENGINE Serving on https://172.21.15.193:7150 2024-05-09T06:38:01.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:01 smithi193 ceph-mon[35141]: [09/May/2024:06:38:00] ENGINE Bus STARTED 2024-05-09T06:38:01.624 INFO:tasks.workunit.client.0.smithi063.stdout:6/14: sync 2024-05-09T06:38:01.627 INFO:tasks.workunit.client.0.smithi063.stdout:6/15: dread f0 [0,4194304] 0 2024-05-09T06:38:01.627 INFO:tasks.workunit.client.0.smithi063.stdout:6/16: stat f0 0 2024-05-09T06:38:01.676 INFO:tasks.workunit.client.0.smithi063.stdout:7/43: mknod d0/ca 0 2024-05-09T06:38:01.677 INFO:tasks.workunit.client.0.smithi063.stdout:4/64: truncate d7/fc 4463992 0 2024-05-09T06:38:01.678 INFO:tasks.workunit.client.0.smithi063.stdout:3/48: creat d7/f11 x:0 0 0 2024-05-09T06:38:01.679 INFO:tasks.workunit.client.0.smithi063.stdout:3/49: stat d7/f8 0 2024-05-09T06:38:01.682 INFO:tasks.workunit.client.0.smithi063.stdout:0/73: mknod dd/c17 0 2024-05-09T06:38:01.683 INFO:tasks.workunit.client.0.smithi063.stdout:4/65: dwrite f4 [0,4194304] 0 2024-05-09T06:38:01.686 INFO:tasks.workunit.client.0.smithi063.stdout:2/35: chown dc 463908 1 2024-05-09T06:38:01.691 INFO:tasks.workunit.client.0.smithi063.stdout:0/74: dwrite dd/f12 [0,4194304] 0 2024-05-09T06:38:01.692 INFO:tasks.workunit.client.0.smithi063.stdout:1/63: rmdir d9/d10 39 2024-05-09T06:38:01.693 INFO:tasks.workunit.client.0.smithi063.stdout:0/75: stat dd 0 2024-05-09T06:38:01.697 INFO:tasks.workunit.client.0.smithi063.stdout:2/36: dwrite fa [0,4194304] 0 2024-05-09T06:38:01.698 INFO:tasks.workunit.client.0.smithi063.stdout:5/27: mkdir d8 0 2024-05-09T06:38:01.698 INFO:tasks.workunit.client.0.smithi063.stdout:5/28: fsync f7 0 2024-05-09T06:38:01.699 INFO:tasks.workunit.client.0.smithi063.stdout:5/29: readlink l5 0 2024-05-09T06:38:01.700 INFO:tasks.workunit.client.0.smithi063.stdout:5/30: write f7 [1001631,6834] 0 2024-05-09T06:38:01.701 INFO:tasks.workunit.client.0.smithi063.stdout:9/68: rename d0/lc to d0/l10 0 2024-05-09T06:38:01.704 INFO:tasks.workunit.client.0.smithi063.stdout:8/118: symlink d9/da/d18/l28 0 2024-05-09T06:38:01.707 INFO:tasks.workunit.client.0.smithi063.stdout:6/17: mkdir d2/d6 0 2024-05-09T06:38:01.710 INFO:tasks.workunit.client.0.smithi063.stdout:7/44: creat d0/fb x:0 0 0 2024-05-09T06:38:01.716 INFO:tasks.workunit.client.0.smithi063.stdout:4/66: rename d7/f9 to d7/f10 0 2024-05-09T06:38:01.719 INFO:tasks.workunit.client.0.smithi063.stdout:0/76: mknod dd/c18 0 2024-05-09T06:38:01.719 INFO:tasks.workunit.client.0.smithi063.stdout:0/77: chown f9 2 1 2024-05-09T06:38:01.723 INFO:tasks.workunit.client.0.smithi063.stdout:0/78: dread dd/fe [0,4194304] 0 2024-05-09T06:38:01.728 INFO:tasks.workunit.client.0.smithi063.stdout:6/18: creat d2/f7 x:0 0 0 2024-05-09T06:38:01.728 INFO:tasks.workunit.client.0.smithi063.stdout:7/45: fdatasync d0/f9 0 2024-05-09T06:38:01.729 INFO:tasks.workunit.client.0.smithi063.stdout:6/19: truncate d2/f7 457665 0 2024-05-09T06:38:01.729 INFO:tasks.workunit.client.0.smithi063.stdout:6/20: chown f1 0 1 2024-05-09T06:38:01.731 INFO:tasks.workunit.client.0.smithi063.stdout:4/67: mknod d7/c11 0 2024-05-09T06:38:01.732 INFO:tasks.workunit.client.0.smithi063.stdout:4/68: stat d7/ca 0 2024-05-09T06:38:01.734 INFO:tasks.workunit.client.0.smithi063.stdout:2/37: symlink dc/dd/le 0 2024-05-09T06:38:01.735 INFO:tasks.workunit.client.0.smithi063.stdout:5/31: getdents d8 0 2024-05-09T06:38:01.737 INFO:tasks.workunit.client.0.smithi063.stdout:2/38: dread f9 [0,4194304] 0 2024-05-09T06:38:01.738 INFO:tasks.workunit.client.0.smithi063.stdout:9/69: truncate d0/d4/da/ff 3097 0 2024-05-09T06:38:01.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/46: rename d0/fb to d0/fc 0 2024-05-09T06:38:01.741 INFO:tasks.workunit.client.0.smithi063.stdout:6/21: creat d2/d4/f8 x:0 0 0 2024-05-09T06:38:01.741 INFO:tasks.workunit.client.0.smithi063.stdout:6/22: readlink - no filename 2024-05-09T06:38:01.747 INFO:tasks.workunit.client.0.smithi063.stdout:5/32: write f4 [1292329,23475] 0 2024-05-09T06:38:01.747 INFO:tasks.workunit.client.0.smithi063.stdout:5/33: fsync f6 0 2024-05-09T06:38:01.747 INFO:tasks.workunit.client.0.smithi063.stdout:8/119: creat d9/f29 x:0 0 0 2024-05-09T06:38:01.748 INFO:tasks.workunit.client.0.smithi063.stdout:5/34: write f4 [1017534,52002] 0 2024-05-09T06:38:01.749 INFO:tasks.workunit.client.0.smithi063.stdout:8/120: readlink d9/da/d18/l21 0 2024-05-09T06:38:01.751 INFO:tasks.workunit.client.0.smithi063.stdout:2/39: creat dc/dd/ff x:0 0 0 2024-05-09T06:38:01.754 INFO:tasks.workunit.client.0.smithi063.stdout:7/47: creat d0/fd x:0 0 0 2024-05-09T06:38:01.758 INFO:tasks.workunit.client.0.smithi063.stdout:4/69: creat d7/df/f12 x:0 0 0 2024-05-09T06:38:01.759 INFO:tasks.workunit.client.0.smithi063.stdout:4/70: readlink l0 0 2024-05-09T06:38:01.762 INFO:tasks.workunit.client.0.smithi063.stdout:6/23: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:01.763 INFO:tasks.workunit.client.0.smithi063.stdout:0/79: link dd/c18 dd/c19 0 2024-05-09T06:38:01.764 INFO:tasks.workunit.client.0.smithi063.stdout:2/40: symlink dc/l10 0 2024-05-09T06:38:01.767 INFO:tasks.workunit.client.0.smithi063.stdout:1/64: sync 2024-05-09T06:38:01.768 INFO:tasks.workunit.client.0.smithi063.stdout:7/48: rename d0/ca to d0/ce 0 2024-05-09T06:38:01.770 INFO:tasks.workunit.client.0.smithi063.stdout:4/71: mknod d7/df/c13 0 2024-05-09T06:38:01.771 INFO:tasks.workunit.client.0.smithi063.stdout:1/65: dread d9/f12 [0,4194304] 0 2024-05-09T06:38:01.773 INFO:tasks.workunit.client.0.smithi063.stdout:5/35: creat d8/f9 x:0 0 0 2024-05-09T06:38:01.773 INFO:tasks.workunit.client.0.smithi063.stdout:5/36: chown c3 55760 1 2024-05-09T06:38:01.775 INFO:tasks.workunit.client.0.smithi063.stdout:0/80: mkdir dd/d1a 0 2024-05-09T06:38:01.781 INFO:tasks.workunit.client.0.smithi063.stdout:8/121: creat d9/f2a x:0 0 0 2024-05-09T06:38:01.785 INFO:tasks.workunit.client.0.smithi063.stdout:1/66: truncate d9/d10/f15 68125 0 2024-05-09T06:38:01.785 INFO:tasks.workunit.client.0.smithi063.stdout:1/67: fdatasync d9/fb 0 2024-05-09T06:38:01.785 INFO:tasks.workunit.client.0.smithi063.stdout:6/24: link d2/f7 d2/d4/f9 0 2024-05-09T06:38:01.786 INFO:tasks.workunit.client.0.smithi063.stdout:1/68: chown d9/fb 27137 1 2024-05-09T06:38:01.786 INFO:tasks.workunit.client.0.smithi063.stdout:6/25: rename d2 to d2/d4/da 22 2024-05-09T06:38:01.787 INFO:tasks.workunit.client.0.smithi063.stdout:6/26: write f0 [4520138,85136] 0 2024-05-09T06:38:01.788 INFO:tasks.workunit.client.0.smithi063.stdout:5/37: rename d8/f9 to d8/fa 0 2024-05-09T06:38:01.790 INFO:tasks.workunit.client.0.smithi063.stdout:6/27: fdatasync f1 0 2024-05-09T06:38:01.793 INFO:tasks.workunit.client.0.smithi063.stdout:7/49: link d0/f4 d0/ff 0 2024-05-09T06:38:01.800 INFO:tasks.workunit.client.0.smithi063.stdout:1/69: creat d9/d10/f18 x:0 0 0 2024-05-09T06:38:01.803 INFO:tasks.workunit.client.0.smithi063.stdout:5/38: unlink f7 0 2024-05-09T06:38:01.803 INFO:tasks.workunit.client.0.smithi063.stdout:5/39: chown c2 30681 1 2024-05-09T06:38:01.804 INFO:tasks.workunit.client.0.smithi063.stdout:0/81: creat dd/d1a/f1b x:0 0 0 2024-05-09T06:38:01.804 INFO:tasks.workunit.client.0.smithi063.stdout:0/82: readlink - no filename 2024-05-09T06:38:01.806 INFO:tasks.workunit.client.0.smithi063.stdout:6/28: mknod d2/d4/cb 0 2024-05-09T06:38:01.811 INFO:tasks.workunit.client.0.smithi063.stdout:5/40: mknod d8/cb 0 2024-05-09T06:38:01.812 INFO:tasks.workunit.client.0.smithi063.stdout:5/41: dread - d8/fa zero size 2024-05-09T06:38:01.812 INFO:tasks.workunit.client.0.smithi063.stdout:5/42: stat d8/fa 0 2024-05-09T06:38:01.815 INFO:tasks.workunit.client.0.smithi063.stdout:6/29: rename f0 to d2/d6/fc 0 2024-05-09T06:38:01.818 INFO:tasks.workunit.client.0.smithi063.stdout:1/70: symlink d9/l19 0 2024-05-09T06:38:01.823 INFO:tasks.workunit.client.0.smithi063.stdout:6/30: rename d2/d4/f9 to d2/fd 0 2024-05-09T06:38:01.826 INFO:tasks.workunit.client.0.smithi063.stdout:1/71: mkdir d9/d1a 0 2024-05-09T06:38:01.846 INFO:tasks.workunit.client.0.smithi063.stdout:6/31: creat d2/d4/fe x:0 0 0 2024-05-09T06:38:01.846 INFO:tasks.workunit.client.0.smithi063.stdout:1/72: dread d9/d10/f15 [0,4194304] 0 2024-05-09T06:38:01.849 INFO:tasks.workunit.client.0.smithi063.stdout:6/32: mkdir d2/df 0 2024-05-09T06:38:01.853 INFO:tasks.workunit.client.0.smithi063.stdout:6/33: rename d2/d4/fe to d2/d4/f10 0 2024-05-09T06:38:01.853 INFO:tasks.workunit.client.0.smithi063.stdout:1/73: dwrite d9/fa [0,4194304] 0 2024-05-09T06:38:01.854 INFO:tasks.workunit.client.0.smithi063.stdout:6/34: write d2/d4/f5 [1009012,74009] 0 2024-05-09T06:38:01.855 INFO:tasks.workunit.client.0.smithi063.stdout:1/74: write d9/d10/f18 [371348,12087] 0 2024-05-09T06:38:01.858 INFO:tasks.workunit.client.0.smithi063.stdout:6/35: mknod d2/c11 0 2024-05-09T06:38:01.858 INFO:tasks.workunit.client.0.smithi063.stdout:6/36: write d2/d4/f8 [779372,100481] 0 2024-05-09T06:38:01.863 INFO:tasks.workunit.client.0.smithi063.stdout:6/37: mkdir d2/d12 0 2024-05-09T06:38:01.866 INFO:tasks.workunit.client.0.smithi063.stdout:6/38: mknod d2/d4/c13 0 2024-05-09T06:38:01.867 INFO:tasks.workunit.client.0.smithi063.stdout:1/75: link d9/l19 d9/d1a/l1b 0 2024-05-09T06:38:01.871 INFO:tasks.workunit.client.0.smithi063.stdout:6/39: dread d2/d4/f5 [0,4194304] 0 2024-05-09T06:38:01.871 INFO:tasks.workunit.client.0.smithi063.stdout:4/72: sync 2024-05-09T06:38:01.874 INFO:tasks.workunit.client.0.smithi063.stdout:6/40: fsync d2/d4/f5 0 2024-05-09T06:38:01.877 INFO:tasks.workunit.client.0.smithi063.stdout:4/73: creat d7/f14 x:0 0 0 2024-05-09T06:38:01.877 INFO:tasks.workunit.client.0.smithi063.stdout:4/74: truncate f6 661164 0 2024-05-09T06:38:01.883 INFO:tasks.workunit.client.0.smithi063.stdout:6/41: dwrite d2/d6/fc [4194304,4194304] 0 2024-05-09T06:38:01.892 INFO:tasks.workunit.client.0.smithi063.stdout:6/42: dwrite d2/d4/f8 [0,4194304] 0 2024-05-09T06:38:01.901 INFO:tasks.workunit.client.0.smithi063.stdout:0/83: sync 2024-05-09T06:38:01.903 INFO:tasks.workunit.client.0.smithi063.stdout:0/84: truncate f9 1753714 0 2024-05-09T06:38:01.904 INFO:tasks.workunit.client.0.smithi063.stdout:6/43: dwrite d2/f7 [0,4194304] 0 2024-05-09T06:38:01.907 INFO:tasks.workunit.client.0.smithi063.stdout:0/85: mknod dd/d1a/c1c 0 2024-05-09T06:38:01.911 INFO:tasks.workunit.client.0.smithi063.stdout:0/86: creat dd/f1d x:0 0 0 2024-05-09T06:38:01.911 INFO:tasks.workunit.client.0.smithi063.stdout:0/87: readlink - no filename 2024-05-09T06:38:01.912 INFO:tasks.workunit.client.0.smithi063.stdout:0/88: write dd/f1d [548592,53836] 0 2024-05-09T06:38:01.920 INFO:tasks.workunit.client.0.smithi063.stdout:0/89: link dd/fe dd/d1a/f1e 0 2024-05-09T06:38:01.923 INFO:tasks.workunit.client.0.smithi063.stdout:0/90: unlink f9 0 2024-05-09T06:38:01.925 INFO:tasks.workunit.client.0.smithi063.stdout:6/44: fdatasync d2/f7 0 2024-05-09T06:38:01.927 INFO:tasks.workunit.client.0.smithi063.stdout:0/91: write f6 [2450579,79694] 0 2024-05-09T06:38:01.929 INFO:tasks.workunit.client.0.smithi063.stdout:6/45: creat d2/df/f14 x:0 0 0 2024-05-09T06:38:01.930 INFO:tasks.workunit.client.0.smithi063.stdout:0/92: fdatasync dd/d1a/f1e 0 2024-05-09T06:38:01.931 INFO:tasks.workunit.client.0.smithi063.stdout:0/93: write dd/f12 [1782002,76911] 0 2024-05-09T06:38:01.932 INFO:tasks.workunit.client.0.smithi063.stdout:6/46: unlink d2/c11 0 2024-05-09T06:38:01.933 INFO:tasks.workunit.client.0.smithi063.stdout:6/47: write d2/fd [3708626,20123] 0 2024-05-09T06:38:01.936 INFO:tasks.workunit.client.0.smithi063.stdout:0/94: dread dd/f1d [0,4194304] 0 2024-05-09T06:38:01.938 INFO:tasks.workunit.client.0.smithi063.stdout:6/48: dread d2/d4/f8 [0,4194304] 0 2024-05-09T06:38:01.938 INFO:tasks.workunit.client.0.smithi063.stdout:6/49: fdatasync d2/d6/fc 0 2024-05-09T06:38:01.939 INFO:tasks.workunit.client.1.smithi193.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-09T06:38:01.939 INFO:tasks.workunit.client.0.smithi063.stdout:6/50: dread - d2/d4/f10 zero size 2024-05-09T06:38:01.940 INFO:tasks.workunit.client.0.smithi063.stdout:0/95: symlink dd/l1f 0 2024-05-09T06:38:01.941 INFO:tasks.workunit.client.0.smithi063.stdout:0/96: truncate dd/f15 165478 0 2024-05-09T06:38:01.941 INFO:tasks.workunit.client.0.smithi063.stdout:0/97: truncate f5 394412 0 2024-05-09T06:38:01.944 INFO:tasks.workunit.client.0.smithi063.stdout:0/98: symlink dd/d1a/l20 0 2024-05-09T06:38:01.945 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-09T06:38:01.946 INFO:tasks.workunit.client.1.smithi193.stderr:+ make 2024-05-09T06:38:01.946 INFO:tasks.workunit.client.0.smithi063.stdout:3/50: rmdir d7 39 2024-05-09T06:38:01.950 INFO:tasks.workunit.client.0.smithi063.stdout:3/51: mknod d7/c12 0 2024-05-09T06:38:01.951 INFO:tasks.workunit.client.0.smithi063.stdout:0/99: dwrite dd/f12 [0,4194304] 0 2024-05-09T06:38:01.951 INFO:tasks.workunit.client.0.smithi063.stdout:3/52: readlink l6 0 2024-05-09T06:38:01.952 INFO:tasks.workunit.client.0.smithi063.stdout:0/100: chown dd/f15 32116 1 2024-05-09T06:38:01.959 INFO:tasks.workunit.client.0.smithi063.stdout:3/53: getdents d7/da 0 2024-05-09T06:38:01.960 INFO:tasks.workunit.client.0.smithi063.stdout:3/54: chown d7/f8 465172 1 2024-05-09T06:38:01.964 INFO:tasks.workunit.client.0.smithi063.stdout:3/55: creat d7/da/f13 x:0 0 0 2024-05-09T06:38:01.967 INFO:tasks.workunit.client.0.smithi063.stdout:3/56: unlink d7/c12 0 2024-05-09T06:38:01.968 INFO:tasks.workunit.client.0.smithi063.stdout:3/57: dread - d7/da/f13 zero size 2024-05-09T06:38:01.972 INFO:tasks.workunit.client.0.smithi063.stdout:3/58: creat d7/da/f14 x:0 0 0 2024-05-09T06:38:01.973 INFO:tasks.workunit.client.0.smithi063.stdout:4/75: truncate d7/f10 70535 0 2024-05-09T06:38:01.975 INFO:tasks.workunit.client.0.smithi063.stdout:2/41: dwrite fa [4194304,4194304] 0 2024-05-09T06:38:01.979 INFO:tasks.workunit.client.0.smithi063.stdout:4/76: dread f4 [0,4194304] 0 2024-05-09T06:38:01.979 INFO:tasks.workunit.client.0.smithi063.stdout:2/42: creat dc/dd/f11 x:0 0 0 2024-05-09T06:38:01.983 INFO:tasks.workunit.client.0.smithi063.stdout:9/70: dread d0/d4/da/ff [0,4194304] 0 2024-05-09T06:38:01.984 INFO:tasks.workunit.client.0.smithi063.stdout:9/71: chown d0/d4 0 1 2024-05-09T06:38:01.985 INFO:tasks.workunit.client.0.smithi063.stdout:7/50: fsync d0/fc 0 2024-05-09T06:38:01.986 INFO:tasks.workunit.client.0.smithi063.stdout:4/77: creat d7/f15 x:0 0 0 2024-05-09T06:38:01.987 INFO:tasks.workunit.client.0.smithi063.stdout:2/43: dwrite f3 [0,4194304] 0 2024-05-09T06:38:01.988 INFO:tasks.workunit.client.0.smithi063.stdout:7/51: creat d0/f10 x:0 0 0 2024-05-09T06:38:01.991 INFO:tasks.workunit.client.0.smithi063.stdout:9/72: symlink d0/d4/l11 0 2024-05-09T06:38:01.992 INFO:tasks.workunit.client.0.smithi063.stdout:2/44: creat dc/f12 x:0 0 0 2024-05-09T06:38:01.992 INFO:tasks.workunit.client.0.smithi063.stdout:2/45: dread - dc/f12 zero size 2024-05-09T06:38:01.995 INFO:tasks.workunit.client.0.smithi063.stdout:7/52: rename d0/f4 to d0/f11 0 2024-05-09T06:38:01.996 INFO:tasks.workunit.client.0.smithi063.stdout:9/73: readlink d0/l10 0 2024-05-09T06:38:01.997 INFO:tasks.workunit.client.0.smithi063.stdout:2/46: rename dc/dd/f11 to dc/dd/f13 0 2024-05-09T06:38:01.999 INFO:tasks.workunit.client.0.smithi063.stdout:2/47: dread - dc/dd/f13 zero size 2024-05-09T06:38:01.999 INFO:tasks.workunit.client.0.smithi063.stdout:9/74: symlink d0/l12 0 2024-05-09T06:38:02.001 INFO:tasks.workunit.client.0.smithi063.stdout:2/48: chown dc/dd/f13 203465091 1 2024-05-09T06:38:02.003 INFO:tasks.workunit.client.0.smithi063.stdout:7/53: dwrite d0/f10 [0,4194304] 0 2024-05-09T06:38:02.004 INFO:tasks.workunit.client.0.smithi063.stdout:7/54: truncate d0/fc 935661 0 2024-05-09T06:38:02.004 INFO:tasks.workunit.client.0.smithi063.stdout:7/55: readlink - no filename 2024-05-09T06:38:02.007 INFO:tasks.workunit.client.0.smithi063.stdout:7/56: symlink d0/l12 0 2024-05-09T06:38:02.007 INFO:tasks.workunit.client.0.smithi063.stdout:7/57: readlink d0/l12 0 2024-05-09T06:38:02.010 INFO:tasks.workunit.client.0.smithi063.stdout:7/58: symlink d0/l13 0 2024-05-09T06:38:02.017 INFO:tasks.workunit.client.0.smithi063.stdout:8/122: dwrite f4 [0,4194304] 0 2024-05-09T06:38:02.017 INFO:tasks.workunit.client.0.smithi063.stdout:5/43: rmdir d8 39 2024-05-09T06:38:02.021 INFO:tasks.workunit.client.0.smithi063.stdout:5/44: creat d8/fc x:0 0 0 2024-05-09T06:38:02.022 INFO:tasks.workunit.client.0.smithi063.stdout:8/123: fsync d9/da/fb 0 2024-05-09T06:38:02.023 INFO:tasks.workunit.client.0.smithi063.stdout:8/124: write d9/da/f20 [312995,28690] 0 2024-05-09T06:38:02.024 INFO:tasks.workunit.client.0.smithi063.stdout:5/45: mknod d8/cd 0 2024-05-09T06:38:02.025 INFO:tasks.workunit.client.0.smithi063.stdout:8/125: read - d9/da/d18/f1d zero size 2024-05-09T06:38:02.025 INFO:tasks.workunit.client.0.smithi063.stdout:5/46: write f4 [292776,122171] 0 2024-05-09T06:38:02.026 INFO:tasks.workunit.client.0.smithi063.stdout:5/47: readlink l5 0 2024-05-09T06:38:02.026 INFO:tasks.workunit.client.0.smithi063.stdout:8/126: write d9/f2a [385159,116269] 0 2024-05-09T06:38:02.027 INFO:tasks.workunit.client.0.smithi063.stdout:8/127: fsync d9/f29 0 2024-05-09T06:38:02.028 INFO:tasks.workunit.client.0.smithi063.stdout:5/48: chown d8/cd 7961661 1 2024-05-09T06:38:02.029 INFO:tasks.workunit.client.0.smithi063.stdout:5/49: dread - d8/fa zero size 2024-05-09T06:38:02.029 INFO:tasks.workunit.client.0.smithi063.stdout:5/50: chown d8/fc 153 1 2024-05-09T06:38:02.030 INFO:tasks.workunit.client.0.smithi063.stdout:8/128: rmdir d9/da 39 2024-05-09T06:38:02.031 INFO:tasks.workunit.client.0.smithi063.stdout:8/129: write d9/f29 [112294,74631] 0 2024-05-09T06:38:02.035 INFO:tasks.workunit.client.0.smithi063.stdout:8/130: symlink d9/da/d18/d25/l2b 0 2024-05-09T06:38:02.043 INFO:tasks.workunit.client.0.smithi063.stdout:1/76: truncate d9/fa 4142912 0 2024-05-09T06:38:02.043 INFO:tasks.workunit.client.0.smithi063.stdout:1/77: chown d9/fb 0 1 2024-05-09T06:38:02.048 INFO:tasks.workunit.client.0.smithi063.stdout:1/78: rmdir d9/d17 0 2024-05-09T06:38:02.052 INFO:tasks.workunit.client.0.smithi063.stdout:1/79: link d9/fb d9/d10/f1c 0 2024-05-09T06:38:02.053 INFO:tasks.workunit.client.0.smithi063.stdout:1/80: rename d9/d1a to d9/d1a/d1d 22 2024-05-09T06:38:02.053 INFO:tasks.workunit.client.0.smithi063.stdout:1/81: chown d9 70141 1 2024-05-09T06:38:02.060 INFO:tasks.workunit.client.0.smithi063.stdout:6/51: getdents d2/df 0 2024-05-09T06:38:02.062 INFO:tasks.workunit.client.0.smithi063.stdout:6/52: symlink d2/d12/l15 0 2024-05-09T06:38:02.063 INFO:tasks.workunit.client.0.smithi063.stdout:6/53: fsync d2/f7 0 2024-05-09T06:38:02.064 INFO:tasks.workunit.client.0.smithi063.stdout:6/54: truncate d2/f7 4284945 0 2024-05-09T06:38:02.064 INFO:tasks.workunit.client.0.smithi063.stdout:6/55: chown d2 954072 1 2024-05-09T06:38:02.066 INFO:tasks.workunit.client.0.smithi063.stdout:0/101: dwrite dd/f12 [4194304,4194304] 0 2024-05-09T06:38:02.068 INFO:tasks.workunit.client.0.smithi063.stdout:6/56: symlink d2/d6/l16 0 2024-05-09T06:38:02.071 INFO:tasks.workunit.client.0.smithi063.stdout:6/57: write d2/d4/f8 [4734109,114635] 0 2024-05-09T06:38:02.073 INFO:tasks.workunit.client.0.smithi063.stdout:3/59: write f5 [591159,114702] 0 2024-05-09T06:38:02.074 INFO:tasks.workunit.client.0.smithi063.stdout:6/58: mknod d2/d4/c17 0 2024-05-09T06:38:02.077 INFO:tasks.workunit.client.0.smithi063.stdout:6/59: rmdir d2/d6 39 2024-05-09T06:38:02.079 INFO:tasks.workunit.client.0.smithi063.stdout:9/75: truncate d0/d4/da/ff 134818 0 2024-05-09T06:38:02.082 INFO:tasks.workunit.client.0.smithi063.stdout:2/49: getdents dc 0 2024-05-09T06:38:02.082 INFO:tasks.workunit.client.1.smithi193.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-09T06:38:02.083 INFO:tasks.workunit.client.0.smithi063.stdout:3/60: dwrite f0 [0,4194304] 0 2024-05-09T06:38:02.085 INFO:tasks.workunit.client.0.smithi063.stdout:3/61: fsync d7/f10 0 2024-05-09T06:38:02.089 INFO:tasks.workunit.client.0.smithi063.stdout:4/78: dwrite d7/fc [0,4194304] 0 2024-05-09T06:38:02.090 INFO:tasks.workunit.client.0.smithi063.stdout:4/79: dread - d7/df/f12 zero size 2024-05-09T06:38:02.091 INFO:tasks.workunit.client.0.smithi063.stdout:6/60: truncate d2/d6/fc 830590 0 2024-05-09T06:38:02.093 INFO:tasks.workunit.client.0.smithi063.stdout:2/50: symlink dc/dd/l14 0 2024-05-09T06:38:02.093 INFO:tasks.workunit.client.0.smithi063.stdout:7/59: write d0/ff [1105276,99462] 0 2024-05-09T06:38:02.094 INFO:tasks.workunit.client.0.smithi063.stdout:3/62: creat d7/da/f15 x:0 0 0 2024-05-09T06:38:02.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/80: mknod d7/df/c16 0 2024-05-09T06:38:02.098 INFO:tasks.workunit.client.0.smithi063.stdout:4/81: chown d7/df/c16 310374 1 2024-05-09T06:38:02.098 INFO:tasks.workunit.client.0.smithi063.stdout:8/131: truncate d9/da/fb 15435 0 2024-05-09T06:38:02.100 INFO:tasks.workunit.client.0.smithi063.stdout:2/51: dwrite f2 [0,4194304] 0 2024-05-09T06:38:02.102 INFO:tasks.workunit.client.0.smithi063.stdout:7/60: unlink d0/l12 0 2024-05-09T06:38:02.104 INFO:tasks.workunit.client.0.smithi063.stdout:6/61: mkdir d2/d18 0 2024-05-09T06:38:02.107 INFO:tasks.workunit.client.0.smithi063.stdout:1/82: truncate d9/d10/f1c 1547015 0 2024-05-09T06:38:02.109 INFO:tasks.workunit.client.0.smithi063.stdout:5/51: sync 2024-05-09T06:38:02.109 INFO:tasks.workunit.client.0.smithi063.stdout:8/132: creat d9/da/d18/d25/f2c x:0 0 0 2024-05-09T06:38:02.111 INFO:tasks.workunit.client.0.smithi063.stdout:2/52: unlink l8 0 2024-05-09T06:38:02.114 INFO:tasks.workunit.client.0.smithi063.stdout:7/61: creat d0/f14 x:0 0 0 2024-05-09T06:38:02.117 INFO:tasks.workunit.client.0.smithi063.stdout:6/62: dwrite d2/f7 [0,4194304] 0 2024-05-09T06:38:02.118 INFO:tasks.workunit.client.0.smithi063.stdout:1/83: fsync d9/f12 0 2024-05-09T06:38:02.119 INFO:tasks.workunit.client.0.smithi063.stdout:4/82: dwrite f4 [0,4194304] 0 2024-05-09T06:38:02.120 INFO:tasks.workunit.client.0.smithi063.stdout:5/52: creat d8/fe x:0 0 0 2024-05-09T06:38:02.122 INFO:tasks.workunit.client.0.smithi063.stdout:2/53: mknod dc/c15 0 2024-05-09T06:38:02.123 INFO:tasks.workunit.client.0.smithi063.stdout:6/63: chown d2/d4/f8 1 1 2024-05-09T06:38:02.124 INFO:tasks.workunit.client.0.smithi063.stdout:0/102: truncate dd/f12 7996925 0 2024-05-09T06:38:02.126 INFO:tasks.workunit.client.0.smithi063.stdout:8/133: dwrite d9/f2a [0,4194304] 0 2024-05-09T06:38:02.128 INFO:tasks.workunit.client.0.smithi063.stdout:2/54: creat dc/f16 x:0 0 0 2024-05-09T06:38:02.128 INFO:tasks.workunit.client.0.smithi063.stdout:8/134: write f4 [2133526,55312] 0 2024-05-09T06:38:02.129 INFO:tasks.workunit.client.0.smithi063.stdout:9/76: dread d0/d4/da/ff [0,4194304] 0 2024-05-09T06:38:02.129 INFO:tasks.workunit.client.0.smithi063.stdout:5/53: creat d8/ff x:0 0 0 2024-05-09T06:38:02.130 INFO:tasks.workunit.client.0.smithi063.stdout:5/54: chown f4 631 1 2024-05-09T06:38:02.134 INFO:tasks.workunit.client.0.smithi063.stdout:8/135: write d9/da/d18/f23 [722165,36535] 0 2024-05-09T06:38:02.134 INFO:tasks.workunit.client.0.smithi063.stdout:4/83: creat d7/df/f17 x:0 0 0 2024-05-09T06:38:02.138 INFO:tasks.workunit.client.0.smithi063.stdout:3/63: getdents d7 0 2024-05-09T06:38:02.140 INFO:tasks.workunit.client.0.smithi063.stdout:3/64: chown d7/da/f14 48 1 2024-05-09T06:38:02.141 INFO:tasks.workunit.client.0.smithi063.stdout:4/84: dread d7/fb [0,4194304] 0 2024-05-09T06:38:02.142 INFO:tasks.workunit.client.0.smithi063.stdout:4/85: chown d7/ca 36423183 1 2024-05-09T06:38:02.143 INFO:tasks.workunit.client.0.smithi063.stdout:2/55: creat dc/dd/f17 x:0 0 0 2024-05-09T06:38:02.144 INFO:tasks.workunit.client.0.smithi063.stdout:9/77: creat d0/f13 x:0 0 0 2024-05-09T06:38:02.147 INFO:tasks.workunit.client.0.smithi063.stdout:8/136: creat d9/da/d18/f2d x:0 0 0 2024-05-09T06:38:02.153 INFO:tasks.workunit.client.0.smithi063.stdout:8/137: truncate d9/da/d18/f1e 963050 0 2024-05-09T06:38:02.154 INFO:tasks.workunit.client.0.smithi063.stdout:9/78: dwrite d0/f13 [0,4194304] 0 2024-05-09T06:38:02.154 INFO:tasks.workunit.client.0.smithi063.stdout:4/86: mknod d7/df/c18 0 2024-05-09T06:38:02.161 INFO:tasks.workunit.client.0.smithi063.stdout:4/87: truncate f5 24623 0 2024-05-09T06:38:02.164 INFO:tasks.workunit.client.0.smithi063.stdout:8/138: rename d9/da/d18/d25/l2b to d9/da/d18/d25/l2e 0 2024-05-09T06:38:02.166 INFO:tasks.workunit.client.0.smithi063.stdout:0/103: dwrite dd/f12 [8388608,4194304] 0 2024-05-09T06:38:02.167 INFO:tasks.workunit.client.0.smithi063.stdout:8/139: dread d9/f29 [0,4194304] 0 2024-05-09T06:38:02.171 INFO:tasks.workunit.client.0.smithi063.stdout:5/55: getdents d8 0 2024-05-09T06:38:02.174 INFO:tasks.workunit.client.0.smithi063.stdout:8/140: truncate d9/da/d18/d25/f2c 556417 0 2024-05-09T06:38:02.174 INFO:tasks.workunit.client.0.smithi063.stdout:3/65: write f1 [5048594,17818] 0 2024-05-09T06:38:02.180 INFO:tasks.workunit.client.0.smithi063.stdout:9/79: write d0/d4/da/ff [397871,8904] 0 2024-05-09T06:38:02.185 INFO:tasks.workunit.client.0.smithi063.stdout:3/66: creat d7/f16 x:0 0 0 2024-05-09T06:38:02.194 INFO:tasks.workunit.client.0.smithi063.stdout:0/104: dwrite f6 [0,4194304] 0 2024-05-09T06:38:02.198 INFO:tasks.workunit.client.0.smithi063.stdout:8/141: link d9/da/d18/d24/f27 d9/da/f2f 0 2024-05-09T06:38:02.201 INFO:tasks.workunit.client.0.smithi063.stdout:4/88: dwrite d7/fb [4194304,4194304] 0 2024-05-09T06:38:02.202 INFO:tasks.workunit.client.0.smithi063.stdout:3/67: write d7/da/fe [633565,79485] 0 2024-05-09T06:38:02.203 INFO:tasks.workunit.client.0.smithi063.stdout:1/84: dwrite d9/d10/f1c [0,4194304] 0 2024-05-09T06:38:02.206 INFO:tasks.workunit.client.0.smithi063.stdout:3/68: readlink l6 0 2024-05-09T06:38:02.206 INFO:tasks.workunit.client.0.smithi063.stdout:6/64: dwrite f1 [0,4194304] 0 2024-05-09T06:38:02.208 INFO:tasks.workunit.client.0.smithi063.stdout:5/56: dwrite d8/fe [0,4194304] 0 2024-05-09T06:38:02.211 INFO:tasks.workunit.client.0.smithi063.stdout:4/89: creat d7/df/f19 x:0 0 0 2024-05-09T06:38:02.212 INFO:tasks.workunit.client.0.smithi063.stdout:5/57: write f4 [320018,44974] 0 2024-05-09T06:38:02.214 INFO:tasks.workunit.client.0.smithi063.stdout:8/142: creat d9/da/f30 x:0 0 0 2024-05-09T06:38:02.214 INFO:tasks.workunit.client.0.smithi063.stdout:9/80: link d0/d4/da/ld d0/d4/da/l14 0 2024-05-09T06:38:02.216 INFO:tasks.workunit.client.0.smithi063.stdout:3/69: creat d7/f17 x:0 0 0 2024-05-09T06:38:02.216 INFO:tasks.workunit.client.0.smithi063.stdout:6/65: write d2/d4/f8 [2327309,1324] 0 2024-05-09T06:38:02.217 INFO:tasks.workunit.client.0.smithi063.stdout:4/90: write f4 [1156965,69961] 0 2024-05-09T06:38:02.219 INFO:tasks.workunit.client.0.smithi063.stdout:3/70: chown f2 10368 1 2024-05-09T06:38:02.219 INFO:tasks.workunit.client.0.smithi063.stdout:8/143: stat d9/da/f20 0 2024-05-09T06:38:02.221 INFO:tasks.workunit.client.0.smithi063.stdout:5/58: creat d8/f10 x:0 0 0 2024-05-09T06:38:02.221 INFO:tasks.workunit.client.0.smithi063.stdout:6/66: fdatasync d2/df/f14 0 2024-05-09T06:38:02.222 INFO:tasks.workunit.client.0.smithi063.stdout:4/91: write f4 [3249000,47511] 0 2024-05-09T06:38:02.225 INFO:tasks.workunit.client.0.smithi063.stdout:0/105: dwrite fc [0,4194304] 0 2024-05-09T06:38:02.228 INFO:tasks.workunit.client.0.smithi063.stdout:3/71: creat d7/da/f18 x:0 0 0 2024-05-09T06:38:02.230 INFO:tasks.workunit.client.0.smithi063.stdout:5/59: mknod d8/c11 0 2024-05-09T06:38:02.238 INFO:tasks.workunit.client.0.smithi063.stdout:5/60: write d8/fc [730666,42639] 0 2024-05-09T06:38:02.240 INFO:tasks.workunit.client.0.smithi063.stdout:9/81: dwrite d0/d4/da/ff [0,4194304] 0 2024-05-09T06:38:02.241 INFO:tasks.workunit.client.0.smithi063.stdout:4/92: chown d7/f10 72204001 1 2024-05-09T06:38:02.243 INFO:tasks.workunit.client.0.smithi063.stdout:5/61: dread - d8/f10 zero size 2024-05-09T06:38:02.248 INFO:tasks.workunit.client.0.smithi063.stdout:6/67: dread d2/f7 [0,4194304] 0 2024-05-09T06:38:02.248 INFO:tasks.workunit.client.0.smithi063.stdout:8/144: creat d9/f31 x:0 0 0 2024-05-09T06:38:02.249 INFO:tasks.workunit.client.0.smithi063.stdout:1/85: truncate d9/fb 816111 0 2024-05-09T06:38:02.250 INFO:tasks.workunit.client.0.smithi063.stdout:1/86: stat d9/d1a 0 2024-05-09T06:38:02.250 INFO:tasks.workunit.client.0.smithi063.stdout:8/145: fdatasync d9/f2a 0 2024-05-09T06:38:02.251 INFO:tasks.workunit.client.0.smithi063.stdout:4/93: mkdir d7/df/d1a 0 2024-05-09T06:38:02.251 INFO:tasks.workunit.client.0.smithi063.stdout:5/62: dread - d8/f10 zero size 2024-05-09T06:38:02.253 INFO:tasks.workunit.client.0.smithi063.stdout:3/72: rmdir d7/da 39 2024-05-09T06:38:02.253 INFO:tasks.workunit.client.0.smithi063.stdout:5/63: read - d8/fa zero size 2024-05-09T06:38:02.253 INFO:tasks.workunit.client.0.smithi063.stdout:3/73: chown f0 1 1 2024-05-09T06:38:02.255 INFO:tasks.workunit.client.0.smithi063.stdout:1/87: read f8 [610394,59491] 0 2024-05-09T06:38:02.259 INFO:tasks.workunit.client.0.smithi063.stdout:2/56: sync 2024-05-09T06:38:02.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/62: sync 2024-05-09T06:38:02.261 INFO:tasks.workunit.client.0.smithi063.stdout:7/63: write d0/f5 [5122316,56229] 0 2024-05-09T06:38:02.265 INFO:tasks.workunit.client.0.smithi063.stdout:8/146: symlink d9/da/d18/l32 0 2024-05-09T06:38:02.268 INFO:tasks.workunit.client.0.smithi063.stdout:5/64: mkdir d8/d12 0 2024-05-09T06:38:02.271 INFO:tasks.workunit.client.0.smithi063.stdout:7/64: dwrite d0/f14 [0,4194304] 0 2024-05-09T06:38:02.277 INFO:tasks.workunit.client.0.smithi063.stdout:1/88: dread d9/d10/f15 [0,4194304] 0 2024-05-09T06:38:02.278 INFO:tasks.workunit.client.0.smithi063.stdout:0/106: link dd/c13 dd/c21 0 2024-05-09T06:38:02.280 INFO:tasks.workunit.client.0.smithi063.stdout:5/65: dwrite d8/fc [0,4194304] 0 2024-05-09T06:38:02.281 INFO:tasks.workunit.client.0.smithi063.stdout:1/89: dread d9/d10/f15 [0,4194304] 0 2024-05-09T06:38:02.286 INFO:tasks.workunit.client.0.smithi063.stdout:2/57: mknod dc/dd/c18 0 2024-05-09T06:38:02.288 INFO:tasks.workunit.client.0.smithi063.stdout:8/147: symlink d9/da/l33 0 2024-05-09T06:38:02.289 INFO:tasks.workunit.client.0.smithi063.stdout:3/74: symlink d7/da/l19 0 2024-05-09T06:38:02.289 INFO:tasks.workunit.client.0.smithi063.stdout:2/58: stat l5 0 2024-05-09T06:38:02.295 INFO:tasks.workunit.client.0.smithi063.stdout:7/65: chown d0/ce 1477240589 1 2024-05-09T06:38:02.297 INFO:tasks.workunit.client.0.smithi063.stdout:4/94: dwrite d7/f10 [0,4194304] 0 2024-05-09T06:38:02.298 INFO:tasks.workunit.client.0.smithi063.stdout:0/107: mkdir dd/d1a/d22 0 2024-05-09T06:38:02.300 INFO:tasks.workunit.client.0.smithi063.stdout:9/82: getdents d0/d8 0 2024-05-09T06:38:02.301 INFO:tasks.workunit.client.0.smithi063.stdout:8/148: mkdir d9/da/d34 0 2024-05-09T06:38:02.301 INFO:tasks.workunit.client.0.smithi063.stdout:6/68: truncate d2/d4/f8 4694024 0 2024-05-09T06:38:02.302 INFO:tasks.workunit.client.0.smithi063.stdout:3/75: rmdir d7/da 39 2024-05-09T06:38:02.303 INFO:tasks.workunit.client.0.smithi063.stdout:5/66: dwrite d8/ff [0,4194304] 0 2024-05-09T06:38:02.303 INFO:tasks.workunit.client.0.smithi063.stdout:6/69: chown d2/d6 151981147 1 2024-05-09T06:38:02.310 INFO:tasks.workunit.client.0.smithi063.stdout:7/66: mkdir d0/d15 0 2024-05-09T06:38:02.312 INFO:tasks.workunit.client.0.smithi063.stdout:4/95: mknod d7/c1b 0 2024-05-09T06:38:02.316 INFO:tasks.workunit.client.0.smithi063.stdout:6/70: dwrite d2/d4/f5 [0,4194304] 0 2024-05-09T06:38:02.317 INFO:tasks.workunit.client.0.smithi063.stdout:6/71: readlink d2/d12/l15 0 2024-05-09T06:38:02.318 INFO:tasks.workunit.client.0.smithi063.stdout:9/83: creat d0/d4/da/f15 x:0 0 0 2024-05-09T06:38:02.320 INFO:tasks.workunit.client.0.smithi063.stdout:6/72: truncate d2/d6/fc 4627532 0 2024-05-09T06:38:02.326 INFO:tasks.workunit.client.0.smithi063.stdout:8/149: symlink d9/da/d18/d25/l35 0 2024-05-09T06:38:02.326 INFO:tasks.workunit.client.0.smithi063.stdout:1/90: write d9/f12 [3652267,82941] 0 2024-05-09T06:38:02.330 INFO:tasks.workunit.client.0.smithi063.stdout:1/91: dread d9/d10/f18 [0,4194304] 0 2024-05-09T06:38:02.334 INFO:tasks.workunit.client.0.smithi063.stdout:4/96: mknod d7/c1c 0 2024-05-09T06:38:02.335 INFO:tasks.workunit.client.0.smithi063.stdout:0/108: mkdir dd/d1a/d22/d23 0 2024-05-09T06:38:02.342 INFO:tasks.workunit.client.0.smithi063.stdout:9/84: rmdir d0/d4/da 39 2024-05-09T06:38:02.343 INFO:tasks.workunit.client.0.smithi063.stdout:4/97: dwrite d7/f10 [0,4194304] 0 2024-05-09T06:38:02.345 INFO:tasks.workunit.client.0.smithi063.stdout:2/59: truncate f2 2233166 0 2024-05-09T06:38:02.362 INFO:tasks.workunit.client.0.smithi063.stdout:5/67: rmdir d8/d12 0 2024-05-09T06:38:02.367 INFO:tasks.workunit.client.0.smithi063.stdout:2/60: symlink dc/l19 0 2024-05-09T06:38:02.367 INFO:tasks.workunit.client.0.smithi063.stdout:2/61: write dc/dd/ff [692391,7718] 0 2024-05-09T06:38:02.369 INFO:tasks.workunit.client.0.smithi063.stdout:7/67: truncate d0/ff 1526758 0 2024-05-09T06:38:02.371 INFO:tasks.workunit.client.0.smithi063.stdout:7/68: write d0/f9 [4259975,54000] 0 2024-05-09T06:38:02.373 INFO:tasks.workunit.client.0.smithi063.stdout:1/92: mkdir d9/d1a/d1e 0 2024-05-09T06:38:02.374 INFO:tasks.workunit.client.0.smithi063.stdout:7/69: dread d0/f10 [0,4194304] 0 2024-05-09T06:38:02.374 INFO:tasks.workunit.client.0.smithi063.stdout:1/93: dread d9/d10/f1c [0,4194304] 0 2024-05-09T06:38:02.375 INFO:tasks.workunit.client.0.smithi063.stdout:7/70: fdatasync d0/fc 0 2024-05-09T06:38:02.375 INFO:tasks.workunit.client.0.smithi063.stdout:6/73: truncate d2/fd 2269776 0 2024-05-09T06:38:02.376 INFO:tasks.workunit.client.0.smithi063.stdout:7/71: read d0/fc [781554,26264] 0 2024-05-09T06:38:02.377 INFO:tasks.workunit.client.0.smithi063.stdout:7/72: chown d0/d15 52955563 1 2024-05-09T06:38:02.377 INFO:tasks.workunit.client.0.smithi063.stdout:6/74: truncate d2/d4/f10 542368 0 2024-05-09T06:38:02.377 INFO:tasks.workunit.client.0.smithi063.stdout:6/75: chown d2/d6/fc 2733 1 2024-05-09T06:38:02.379 INFO:tasks.workunit.client.0.smithi063.stdout:5/68: symlink d8/l13 0 2024-05-09T06:38:02.379 INFO:tasks.workunit.client.0.smithi063.stdout:5/69: readlink l5 0 2024-05-09T06:38:02.381 INFO:tasks.workunit.client.0.smithi063.stdout:4/98: mknod d7/df/d1a/c1d 0 2024-05-09T06:38:02.381 INFO:tasks.workunit.client.0.smithi063.stdout:4/99: write f6 [1198083,64259] 0 2024-05-09T06:38:02.383 INFO:tasks.workunit.client.0.smithi063.stdout:2/62: read f9 [334279,79678] 0 2024-05-09T06:38:02.385 INFO:tasks.workunit.client.0.smithi063.stdout:8/150: rename d9/f2a to d9/f36 0 2024-05-09T06:38:02.385 INFO:tasks.workunit.client.0.smithi063.stdout:2/63: chown dc/dd/ff 187099 1 2024-05-09T06:38:02.387 INFO:tasks.workunit.client.0.smithi063.stdout:2/64: fsync f3 0 2024-05-09T06:38:02.387 INFO:tasks.workunit.client.0.smithi063.stdout:8/151: stat d9/da/d18/d25/l35 0 2024-05-09T06:38:02.389 INFO:tasks.workunit.client.0.smithi063.stdout:0/109: link dd/c13 dd/d1a/d22/c24 0 2024-05-09T06:38:02.389 INFO:tasks.workunit.client.0.smithi063.stdout:2/65: truncate dc/dd/f13 2456 0 2024-05-09T06:38:02.390 INFO:tasks.workunit.client.0.smithi063.stdout:4/100: dwrite d7/df/f19 [0,4194304] 0 2024-05-09T06:38:02.390 INFO:tasks.workunit.client.0.smithi063.stdout:0/110: write dd/f12 [12465886,7304] 0 2024-05-09T06:38:02.391 INFO:tasks.workunit.client.0.smithi063.stdout:0/111: chown dd 4302719 1 2024-05-09T06:38:02.392 INFO:tasks.workunit.client.0.smithi063.stdout:4/101: write d7/df/f19 [3327751,15703] 0 2024-05-09T06:38:02.393 INFO:tasks.workunit.client.0.smithi063.stdout:4/102: stat l3 0 2024-05-09T06:38:02.393 INFO:tasks.workunit.client.0.smithi063.stdout:1/94: mknod d9/d1a/c1f 0 2024-05-09T06:38:02.396 INFO:tasks.workunit.client.0.smithi063.stdout:7/73: creat d0/f16 x:0 0 0 2024-05-09T06:38:02.398 INFO:tasks.workunit.client.0.smithi063.stdout:9/85: write d0/d4/da/ff [4430598,95328] 0 2024-05-09T06:38:02.398 INFO:tasks.workunit.client.0.smithi063.stdout:6/76: symlink d2/d12/l19 0 2024-05-09T06:38:02.399 INFO:tasks.workunit.client.0.smithi063.stdout:7/74: read d0/f5 [4213641,31035] 0 2024-05-09T06:38:02.401 INFO:tasks.workunit.client.0.smithi063.stdout:2/66: dwrite fa [0,4194304] 0 2024-05-09T06:38:02.402 INFO:tasks.workunit.client.0.smithi063.stdout:5/70: creat d8/f14 x:0 0 0 2024-05-09T06:38:02.406 INFO:tasks.workunit.client.0.smithi063.stdout:2/67: chown l5 9072606 1 2024-05-09T06:38:02.407 INFO:tasks.workunit.client.0.smithi063.stdout:5/71: write d8/fc [3744644,24098] 0 2024-05-09T06:38:02.407 INFO:tasks.workunit.client.0.smithi063.stdout:1/95: dread d9/f11 [0,4194304] 0 2024-05-09T06:38:02.410 INFO:tasks.workunit.client.0.smithi063.stdout:9/86: dwrite d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:02.411 INFO:tasks.workunit.client.0.smithi063.stdout:9/87: fdatasync d0/f13 0 2024-05-09T06:38:02.419 INFO:tasks.workunit.client.0.smithi063.stdout:8/152: unlink c8 0 2024-05-09T06:38:02.419 INFO:tasks.workunit.client.0.smithi063.stdout:8/153: stat d9/da/l10 0 2024-05-09T06:38:02.424 INFO:tasks.workunit.client.0.smithi063.stdout:0/112: creat dd/d1a/f25 x:0 0 0 2024-05-09T06:38:02.426 INFO:tasks.workunit.client.0.smithi063.stdout:4/103: unlink l3 0 2024-05-09T06:38:02.426 INFO:tasks.workunit.client.0.smithi063.stdout:4/104: chown d7/df 9529 1 2024-05-09T06:38:02.428 INFO:tasks.workunit.client.0.smithi063.stdout:0/113: dread fc [0,4194304] 0 2024-05-09T06:38:02.429 INFO:tasks.workunit.client.0.smithi063.stdout:0/114: dread - dd/d1a/f1b zero size 2024-05-09T06:38:02.434 INFO:tasks.workunit.client.0.smithi063.stdout:7/75: dread d0/f10 [0,4194304] 0 2024-05-09T06:38:02.434 INFO:tasks.workunit.client.0.smithi063.stdout:7/76: fsync d0/fd 0 2024-05-09T06:38:02.435 INFO:tasks.workunit.client.0.smithi063.stdout:3/76: sync 2024-05-09T06:38:02.438 INFO:tasks.workunit.client.0.smithi063.stdout:5/72: mknod d8/c15 0 2024-05-09T06:38:02.438 INFO:tasks.workunit.client.0.smithi063.stdout:5/73: readlink l5 0 2024-05-09T06:38:02.439 INFO:tasks.workunit.client.0.smithi063.stdout:5/74: fsync d8/fc 0 2024-05-09T06:38:02.441 INFO:tasks.workunit.client.0.smithi063.stdout:2/68: dwrite dc/dd/f13 [0,4194304] 0 2024-05-09T06:38:02.441 INFO:tasks.workunit.client.0.smithi063.stdout:1/96: readlink d9/l19 0 2024-05-09T06:38:02.442 INFO:tasks.workunit.client.0.smithi063.stdout:7/77: dread d0/f9 [0,4194304] 0 2024-05-09T06:38:02.446 INFO:tasks.workunit.client.0.smithi063.stdout:9/88: symlink d0/l16 0 2024-05-09T06:38:02.447 INFO:tasks.workunit.client.0.smithi063.stdout:9/89: fsync d0/d4/da/ff 0 2024-05-09T06:38:02.450 INFO:tasks.workunit.client.0.smithi063.stdout:9/90: dread d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:02.460 INFO:tasks.workunit.client.0.smithi063.stdout:4/105: rename d7/fc to d7/df/d1a/f1e 0 2024-05-09T06:38:02.462 INFO:tasks.workunit.client.0.smithi063.stdout:0/115: creat dd/d1a/d22/f26 x:0 0 0 2024-05-09T06:38:02.463 INFO:tasks.workunit.client.0.smithi063.stdout:0/116: chown dd/d1a 177647 1 2024-05-09T06:38:02.475 INFO:tasks.workunit.client.0.smithi063.stdout:2/69: mknod dc/c1a 0 2024-05-09T06:38:02.475 INFO:tasks.workunit.client.0.smithi063.stdout:2/70: fsync f6 0 2024-05-09T06:38:02.478 INFO:tasks.workunit.client.0.smithi063.stdout:7/78: mkdir d0/d17 0 2024-05-09T06:38:02.482 INFO:tasks.workunit.client.0.smithi063.stdout:8/154: mkdir d9/d37 0 2024-05-09T06:38:02.483 INFO:tasks.workunit.client.0.smithi063.stdout:2/71: dwrite dc/dd/ff [0,4194304] 0 2024-05-09T06:38:02.484 INFO:tasks.workunit.client.0.smithi063.stdout:8/155: chown d9/f13 6785 1 2024-05-09T06:38:02.487 INFO:tasks.workunit.client.0.smithi063.stdout:9/91: rename d0/d4 to d0/d4/d17 22 2024-05-09T06:38:02.490 INFO:tasks.workunit.client.0.smithi063.stdout:4/106: mkdir d7/df/d1f 0 2024-05-09T06:38:02.490 INFO:tasks.workunit.client.0.smithi063.stdout:4/107: dread - d7/f15 zero size 2024-05-09T06:38:02.498 INFO:tasks.workunit.client.0.smithi063.stdout:0/117: symlink dd/l27 0 2024-05-09T06:38:02.499 INFO:tasks.workunit.client.0.smithi063.stdout:7/79: rename d0/fd to d0/f18 0 2024-05-09T06:38:02.501 INFO:tasks.workunit.client.0.smithi063.stdout:6/77: truncate d2/fd 464561 0 2024-05-09T06:38:02.503 INFO:tasks.workunit.client.0.smithi063.stdout:8/156: rename d9/da/f30 to d9/da/d34/f38 0 2024-05-09T06:38:02.504 INFO:tasks.workunit.client.0.smithi063.stdout:8/157: chown d9/da/d18/f1b 105 1 2024-05-09T06:38:02.585 INFO:tasks.workunit.client.0.smithi063.stdout:5/75: sync 2024-05-09T06:38:02.585 INFO:tasks.workunit.client.0.smithi063.stdout:5/76: chown d8 10932321 1 2024-05-09T06:38:02.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:02 smithi063 ceph-mon[27171]: pgmap v4: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:38:02.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:02 smithi063 ceph-mon[27171]: mgrmap e23: smithi193.mmmqzh(active, since 2s) 2024-05-09T06:38:02.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:02 smithi193 ceph-mon[35141]: pgmap v4: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:38:02.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:02 smithi193 ceph-mon[35141]: mgrmap e23: smithi193.mmmqzh(active, since 2s) 2024-05-09T06:38:02.642 INFO:tasks.workunit.client.0.smithi063.stdout:0/118: rmdir dd/d1a/d22 39 2024-05-09T06:38:02.644 INFO:tasks.workunit.client.0.smithi063.stdout:0/119: write f6 [2170213,6680] 0 2024-05-09T06:38:02.644 INFO:tasks.workunit.client.0.smithi063.stdout:7/80: write d0/f18 [897068,24087] 0 2024-05-09T06:38:02.645 INFO:tasks.workunit.client.0.smithi063.stdout:6/78: symlink d2/d4/l1a 0 2024-05-09T06:38:02.647 INFO:tasks.workunit.client.0.smithi063.stdout:4/108: rename d7/df/f17 to d7/df/d1a/f20 0 2024-05-09T06:38:02.648 INFO:tasks.workunit.client.0.smithi063.stdout:4/109: truncate d7/f14 671647 0 2024-05-09T06:38:02.649 INFO:tasks.workunit.client.0.smithi063.stdout:9/92: truncate d0/d4/da/ff 1916608 0 2024-05-09T06:38:02.650 INFO:tasks.workunit.client.0.smithi063.stdout:8/158: truncate d9/f29 514876 0 2024-05-09T06:38:02.651 INFO:tasks.workunit.client.0.smithi063.stdout:5/77: rmdir d8 39 2024-05-09T06:38:02.651 INFO:tasks.workunit.client.0.smithi063.stdout:8/159: truncate d9/da/d18/f22 330096 0 2024-05-09T06:38:02.653 INFO:tasks.workunit.client.0.smithi063.stdout:2/72: sync 2024-05-09T06:38:02.653 INFO:tasks.workunit.client.0.smithi063.stdout:1/97: getdents d9 0 2024-05-09T06:38:02.654 INFO:tasks.workunit.client.0.smithi063.stdout:2/73: chown lb 0 1 2024-05-09T06:38:02.655 INFO:tasks.workunit.client.0.smithi063.stdout:3/77: link d7/da/f14 d7/f1a 0 2024-05-09T06:38:02.658 INFO:tasks.workunit.client.0.smithi063.stdout:0/120: symlink dd/l28 0 2024-05-09T06:38:02.659 INFO:tasks.workunit.client.0.smithi063.stdout:8/160: dwrite d9/da/d18/f2d [0,4194304] 0 2024-05-09T06:38:02.659 INFO:tasks.workunit.client.0.smithi063.stdout:6/79: creat d2/d6/f1b x:0 0 0 2024-05-09T06:38:02.660 INFO:tasks.workunit.client.0.smithi063.stdout:8/161: readlink d9/da/d18/l21 0 2024-05-09T06:38:02.661 INFO:tasks.workunit.client.0.smithi063.stdout:8/162: rename d9/da/d18 to d9/da/d18/d39 22 2024-05-09T06:38:02.661 INFO:tasks.workunit.client.0.smithi063.stdout:7/81: rename d0/c7 to d0/c19 0 2024-05-09T06:38:02.666 INFO:tasks.workunit.client.0.smithi063.stdout:1/98: symlink d9/d1a/l20 0 2024-05-09T06:38:02.670 INFO:tasks.workunit.client.0.smithi063.stdout:5/78: dwrite d8/fc [0,4194304] 0 2024-05-09T06:38:02.672 INFO:tasks.workunit.client.0.smithi063.stdout:6/80: creat d2/d4/f1c x:0 0 0 2024-05-09T06:38:02.673 INFO:tasks.workunit.client.0.smithi063.stdout:6/81: fsync f1 0 2024-05-09T06:38:02.675 INFO:tasks.workunit.client.0.smithi063.stdout:8/163: unlink f0 0 2024-05-09T06:38:02.677 INFO:tasks.workunit.client.0.smithi063.stdout:9/93: mknod d0/d4/c18 0 2024-05-09T06:38:02.678 INFO:tasks.workunit.client.0.smithi063.stdout:1/99: rmdir d9 39 2024-05-09T06:38:02.686 INFO:tasks.workunit.client.0.smithi063.stdout:7/82: fdatasync d0/ff 0 2024-05-09T06:38:02.687 INFO:tasks.workunit.client.0.smithi063.stdout:3/78: creat d7/f1b x:0 0 0 2024-05-09T06:38:02.689 INFO:tasks.workunit.client.0.smithi063.stdout:5/79: link d8/fa d8/f16 0 2024-05-09T06:38:02.689 INFO:tasks.workunit.client.0.smithi063.stdout:0/121: getdents dd/d1a/d22/d23 0 2024-05-09T06:38:02.690 INFO:tasks.workunit.client.0.smithi063.stdout:8/164: creat d9/d37/f3a x:0 0 0 2024-05-09T06:38:02.691 INFO:tasks.workunit.client.0.smithi063.stdout:9/94: dread d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:02.695 INFO:tasks.workunit.client.0.smithi063.stdout:7/83: creat d0/d17/f1a x:0 0 0 2024-05-09T06:38:02.696 INFO:tasks.workunit.client.0.smithi063.stdout:3/79: dwrite d7/da/fe [0,4194304] 0 2024-05-09T06:38:02.696 INFO:tasks.workunit.client.0.smithi063.stdout:9/95: creat d0/f19 x:0 0 0 2024-05-09T06:38:02.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/80: read - d7/f17 zero size 2024-05-09T06:38:02.698 INFO:tasks.workunit.client.0.smithi063.stdout:3/81: write d7/f16 [227114,123972] 0 2024-05-09T06:38:02.699 INFO:tasks.workunit.client.0.smithi063.stdout:8/165: unlink d9/da/f20 0 2024-05-09T06:38:02.700 INFO:tasks.workunit.client.0.smithi063.stdout:3/82: truncate d7/f1b 960667 0 2024-05-09T06:38:02.702 INFO:tasks.workunit.client.0.smithi063.stdout:9/96: dread d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:02.707 INFO:tasks.workunit.client.0.smithi063.stdout:9/97: mkdir d0/d4/da/d1a 0 2024-05-09T06:38:02.707 INFO:tasks.workunit.client.0.smithi063.stdout:9/98: read - d0/f19 zero size 2024-05-09T06:38:02.710 INFO:tasks.workunit.client.0.smithi063.stdout:3/83: mknod d7/c1c 0 2024-05-09T06:38:02.710 INFO:tasks.workunit.client.0.smithi063.stdout:3/84: fsync f2 0 2024-05-09T06:38:02.711 INFO:tasks.workunit.client.0.smithi063.stdout:3/85: truncate f0 5003311 0 2024-05-09T06:38:02.712 INFO:tasks.workunit.client.0.smithi063.stdout:9/99: unlink d0/l16 0 2024-05-09T06:38:02.715 INFO:tasks.workunit.client.0.smithi063.stdout:8/166: rename d9/da/d18/d25 to d9/d3b 0 2024-05-09T06:38:02.718 INFO:tasks.workunit.client.0.smithi063.stdout:9/100: truncate d0/d4/da/f15 1113820 0 2024-05-09T06:38:02.744 INFO:tasks.workunit.client.1.smithi193.stderr:++ readlink -f fsstress 2024-05-09T06:38:02.746 INFO:tasks.workunit.client.1.smithi193.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-09T06:38:02.746 INFO:tasks.workunit.client.1.smithi193.stderr:+ popd 2024-05-09T06:38:02.748 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-09T06:38:02.748 INFO:tasks.workunit.client.1.smithi193.stderr:+ popd 2024-05-09T06:38:02.749 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp 2024-05-09T06:38:02.749 INFO:tasks.workunit.client.1.smithi193.stderr:++ mktemp -d -p . 2024-05-09T06:38:02.751 INFO:tasks.workunit.client.1.smithi193.stderr:+ T=./tmp.CAkSB7U89v 2024-05-09T06:38:02.751 INFO:tasks.workunit.client.1.smithi193.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.CAkSB7U89v -l 1 -n 1000 -p 10 -v 2024-05-09T06:38:02.756 INFO:tasks.workunit.client.1.smithi193.stdout:seed = 1714926523 2024-05-09T06:38:02.760 INFO:tasks.workunit.client.1.smithi193.stdout:0/0: write - no filename 2024-05-09T06:38:02.763 INFO:tasks.workunit.client.1.smithi193.stdout:2/0: dread - no filename 2024-05-09T06:38:02.764 INFO:tasks.workunit.client.1.smithi193.stdout:0/1: symlink l0 0 2024-05-09T06:38:02.764 INFO:tasks.workunit.client.1.smithi193.stdout:0/2: fdatasync - no filename 2024-05-09T06:38:02.764 INFO:tasks.workunit.client.1.smithi193.stdout:0/3: write - no filename 2024-05-09T06:38:02.765 INFO:tasks.workunit.client.1.smithi193.stdout:0/4: truncate - no filename 2024-05-09T06:38:02.766 INFO:tasks.workunit.client.1.smithi193.stdout:3/0: chown . 320228 1 2024-05-09T06:38:02.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/5: mknod c1 0 2024-05-09T06:38:02.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/6: dwrite - no filename 2024-05-09T06:38:02.769 INFO:tasks.workunit.client.1.smithi193.stdout:4/0: dread - no filename 2024-05-09T06:38:02.770 INFO:tasks.workunit.client.1.smithi193.stdout:2/1: mknod c0 0 2024-05-09T06:38:02.773 INFO:tasks.workunit.client.1.smithi193.stdout:5/0: rename - no filename 2024-05-09T06:38:02.773 INFO:tasks.workunit.client.1.smithi193.stdout:5/1: rename - no filename 2024-05-09T06:38:02.775 INFO:tasks.workunit.client.1.smithi193.stdout:2/2: creat f1 x:0 0 0 2024-05-09T06:38:02.775 INFO:tasks.workunit.client.1.smithi193.stdout:2/3: rmdir - no directory 2024-05-09T06:38:02.775 INFO:tasks.workunit.client.1.smithi193.stdout:2/4: dread - f1 zero size 2024-05-09T06:38:02.776 INFO:tasks.workunit.client.1.smithi193.stdout:3/1: creat f0 x:0 0 0 2024-05-09T06:38:02.776 INFO:tasks.workunit.client.1.smithi193.stdout:3/2: write f0 [110330,114754] 0 2024-05-09T06:38:02.780 INFO:tasks.workunit.client.1.smithi193.stdout:4/1: creat f0 x:0 0 0 2024-05-09T06:38:02.780 INFO:tasks.workunit.client.1.smithi193.stdout:0/7: link l0 l2 0 2024-05-09T06:38:02.782 INFO:tasks.workunit.client.1.smithi193.stdout:7/0: unlink - no file 2024-05-09T06:38:02.782 INFO:tasks.workunit.client.1.smithi193.stdout:7/1: fsync - no filename 2024-05-09T06:38:02.783 INFO:tasks.workunit.client.1.smithi193.stdout:4/2: unlink f0 0 2024-05-09T06:38:02.784 INFO:tasks.workunit.client.1.smithi193.stdout:5/2: mkdir d0 0 2024-05-09T06:38:02.785 INFO:tasks.workunit.client.1.smithi193.stdout:0/8: creat f3 x:0 0 0 2024-05-09T06:38:02.787 INFO:tasks.workunit.client.1.smithi193.stdout:0/9: write f3 [867511,5135] 0 2024-05-09T06:38:02.787 INFO:tasks.workunit.client.1.smithi193.stdout:8/0: dwrite - no filename 2024-05-09T06:38:02.788 INFO:tasks.workunit.client.1.smithi193.stdout:6/0: getdents . 0 2024-05-09T06:38:02.788 INFO:tasks.workunit.client.1.smithi193.stdout:6/1: fsync - no filename 2024-05-09T06:38:02.788 INFO:tasks.workunit.client.1.smithi193.stdout:6/2: write - no filename 2024-05-09T06:38:02.788 INFO:tasks.workunit.client.1.smithi193.stdout:6/3: dwrite - no filename 2024-05-09T06:38:02.788 INFO:tasks.workunit.client.1.smithi193.stdout:6/4: rmdir - no directory 2024-05-09T06:38:02.788 INFO:tasks.workunit.client.1.smithi193.stdout:6/5: write - no filename 2024-05-09T06:38:02.790 INFO:tasks.workunit.client.1.smithi193.stdout:4/3: symlink l1 0 2024-05-09T06:38:02.793 INFO:tasks.workunit.client.1.smithi193.stdout:0/10: creat f4 x:0 0 0 2024-05-09T06:38:02.794 INFO:tasks.workunit.client.1.smithi193.stdout:7/2: creat f0 x:0 0 0 2024-05-09T06:38:02.795 INFO:tasks.workunit.client.1.smithi193.stdout:7/3: dread - f0 zero size 2024-05-09T06:38:02.795 INFO:tasks.workunit.client.1.smithi193.stdout:5/3: mkdir d0/d1 0 2024-05-09T06:38:02.796 INFO:tasks.workunit.client.1.smithi193.stdout:4/4: rename l1 to l2 0 2024-05-09T06:38:02.796 INFO:tasks.workunit.client.1.smithi193.stdout:4/5: truncate - no filename 2024-05-09T06:38:02.797 INFO:tasks.workunit.client.1.smithi193.stdout:8/1: creat f0 x:0 0 0 2024-05-09T06:38:02.799 INFO:tasks.workunit.client.1.smithi193.stdout:6/6: creat f0 x:0 0 0 2024-05-09T06:38:02.800 INFO:tasks.workunit.client.1.smithi193.stdout:0/11: unlink f3 0 2024-05-09T06:38:02.801 INFO:tasks.workunit.client.1.smithi193.stdout:7/4: rename f0 to f1 0 2024-05-09T06:38:02.803 INFO:tasks.workunit.client.1.smithi193.stdout:9/0: creat f0 x:0 0 0 2024-05-09T06:38:02.804 INFO:tasks.workunit.client.1.smithi193.stdout:9/1: chown f0 25591 1 2024-05-09T06:38:02.804 INFO:tasks.workunit.client.1.smithi193.stdout:8/2: mknod c1 0 2024-05-09T06:38:02.806 INFO:tasks.workunit.client.1.smithi193.stdout:0/12: readlink l0 0 2024-05-09T06:38:02.807 INFO:tasks.workunit.client.1.smithi193.stdout:0/13: dread - f4 zero size 2024-05-09T06:38:02.807 INFO:tasks.workunit.client.1.smithi193.stdout:0/14: truncate f4 583301 0 2024-05-09T06:38:02.808 INFO:tasks.workunit.client.1.smithi193.stdout:7/5: symlink l2 0 2024-05-09T06:38:02.810 INFO:tasks.workunit.client.1.smithi193.stdout:9/2: mknod c1 0 2024-05-09T06:38:02.811 INFO:tasks.workunit.client.1.smithi193.stdout:8/3: mknod c2 0 2024-05-09T06:38:02.813 INFO:tasks.workunit.client.1.smithi193.stdout:7/6: symlink l3 0 2024-05-09T06:38:02.816 INFO:tasks.workunit.client.1.smithi193.stdout:9/3: mknod c2 0 2024-05-09T06:38:02.817 INFO:tasks.workunit.client.1.smithi193.stdout:0/15: dwrite f4 [0,4194304] 0 2024-05-09T06:38:02.818 INFO:tasks.workunit.client.1.smithi193.stdout:9/4: fdatasync f0 0 2024-05-09T06:38:02.819 INFO:tasks.workunit.client.1.smithi193.stdout:8/4: creat f3 x:0 0 0 2024-05-09T06:38:02.822 INFO:tasks.workunit.client.1.smithi193.stdout:7/7: unlink l2 0 2024-05-09T06:38:02.824 INFO:tasks.workunit.client.1.smithi193.stdout:0/16: creat f5 x:0 0 0 2024-05-09T06:38:02.824 INFO:tasks.workunit.client.1.smithi193.stdout:9/5: mknod c3 0 2024-05-09T06:38:02.825 INFO:tasks.workunit.client.1.smithi193.stdout:9/6: dread - f0 zero size 2024-05-09T06:38:02.825 INFO:tasks.workunit.client.1.smithi193.stdout:9/7: readlink - no filename 2024-05-09T06:38:02.825 INFO:tasks.workunit.client.1.smithi193.stdout:9/8: chown f0 149197103 1 2024-05-09T06:38:02.829 INFO:tasks.workunit.client.0.smithi063.stdout:4/110: sync 2024-05-09T06:38:02.829 INFO:tasks.workunit.client.0.smithi063.stdout:2/74: sync 2024-05-09T06:38:02.830 INFO:tasks.workunit.client.1.smithi193.stdout:0/17: chown l0 25 1 2024-05-09T06:38:02.830 INFO:tasks.workunit.client.1.smithi193.stdout:0/18: rmdir - no directory 2024-05-09T06:38:02.831 INFO:tasks.workunit.client.1.smithi193.stdout:8/5: link c1 c4 0 2024-05-09T06:38:02.831 INFO:tasks.workunit.client.1.smithi193.stdout:8/6: chown c2 179049250 1 2024-05-09T06:38:02.832 INFO:tasks.workunit.client.0.smithi063.stdout:4/111: creat d7/df/f21 x:0 0 0 2024-05-09T06:38:02.834 INFO:tasks.workunit.client.0.smithi063.stdout:2/75: unlink dc/dd/ff 0 2024-05-09T06:38:02.837 INFO:tasks.workunit.client.1.smithi193.stdout:0/19: fdatasync f4 0 2024-05-09T06:38:02.837 INFO:tasks.workunit.client.1.smithi193.stdout:0/20: stat c1 0 2024-05-09T06:38:02.837 INFO:tasks.workunit.client.1.smithi193.stdout:8/7: symlink l5 0 2024-05-09T06:38:02.838 INFO:tasks.workunit.client.1.smithi193.stdout:0/21: fdatasync f4 0 2024-05-09T06:38:02.838 INFO:tasks.workunit.client.1.smithi193.stdout:0/22: dread - f5 zero size 2024-05-09T06:38:02.841 INFO:tasks.workunit.client.0.smithi063.stdout:2/76: creat dc/f1b x:0 0 0 2024-05-09T06:38:02.842 INFO:tasks.workunit.client.0.smithi063.stdout:2/77: chown dc/f16 21741 1 2024-05-09T06:38:02.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/78: write f6 [311213,26984] 0 2024-05-09T06:38:02.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/79: fsync fa 0 2024-05-09T06:38:02.844 INFO:tasks.workunit.client.1.smithi193.stdout:0/23: mknod c6 0 2024-05-09T06:38:02.846 INFO:tasks.workunit.client.1.smithi193.stdout:0/24: write f5 [39491,89881] 0 2024-05-09T06:38:02.846 INFO:tasks.workunit.client.1.smithi193.stdout:8/8: dwrite f0 [0,4194304] 0 2024-05-09T06:38:02.847 INFO:tasks.workunit.client.1.smithi193.stdout:8/9: dread - f3 zero size 2024-05-09T06:38:02.852 INFO:tasks.workunit.client.0.smithi063.stdout:2/80: rename dc/f16 to dc/f1c 0 2024-05-09T06:38:02.853 INFO:tasks.workunit.client.0.smithi063.stdout:2/81: chown f3 11835 1 2024-05-09T06:38:02.853 INFO:tasks.workunit.client.0.smithi063.stdout:2/82: chown dc/l10 1407 1 2024-05-09T06:38:02.855 INFO:tasks.workunit.client.1.smithi193.stdout:8/10: unlink f3 0 2024-05-09T06:38:02.855 INFO:tasks.workunit.client.1.smithi193.stdout:0/25: dwrite f4 [0,4194304] 0 2024-05-09T06:38:02.856 INFO:tasks.workunit.client.0.smithi063.stdout:6/82: fsync d2/d6/f1b 0 2024-05-09T06:38:02.856 INFO:tasks.workunit.client.1.smithi193.stdout:0/26: chown f5 3142 1 2024-05-09T06:38:02.857 INFO:tasks.workunit.client.0.smithi063.stdout:6/83: chown d2/d4/c13 214 1 2024-05-09T06:38:02.864 INFO:tasks.workunit.client.0.smithi063.stdout:2/83: mknod dc/dd/c1d 0 2024-05-09T06:38:02.865 INFO:tasks.workunit.client.1.smithi193.stdout:0/27: dwrite f4 [0,4194304] 0 2024-05-09T06:38:02.869 INFO:tasks.workunit.client.0.smithi063.stdout:2/84: mknod dc/c1e 0 2024-05-09T06:38:02.872 INFO:tasks.workunit.client.0.smithi063.stdout:6/84: unlink d2/d6/fc 0 2024-05-09T06:38:02.873 INFO:tasks.workunit.client.0.smithi063.stdout:6/85: write d2/d4/f1c [571423,125927] 0 2024-05-09T06:38:02.873 INFO:tasks.workunit.client.0.smithi063.stdout:2/85: creat dc/f1f x:0 0 0 2024-05-09T06:38:02.874 INFO:tasks.workunit.client.1.smithi193.stdout:0/28: chown l2 86471187 1 2024-05-09T06:38:02.880 INFO:tasks.workunit.client.0.smithi063.stdout:7/84: dwrite d0/f11 [0,4194304] 0 2024-05-09T06:38:02.881 INFO:tasks.workunit.client.0.smithi063.stdout:0/122: dwrite fc [0,4194304] 0 2024-05-09T06:38:02.881 INFO:tasks.workunit.client.0.smithi063.stdout:6/86: creat d2/df/f1d x:0 0 0 2024-05-09T06:38:02.883 INFO:tasks.workunit.client.0.smithi063.stdout:6/87: chown d2/d18 101688 1 2024-05-09T06:38:02.884 INFO:tasks.workunit.client.0.smithi063.stdout:6/88: chown d2/d4/f10 474775363 1 2024-05-09T06:38:02.884 INFO:tasks.workunit.client.0.smithi063.stdout:0/123: read dd/f15 [90977,86802] 0 2024-05-09T06:38:02.885 INFO:tasks.workunit.client.0.smithi063.stdout:8/167: rmdir d9/da/d18 39 2024-05-09T06:38:02.889 INFO:tasks.workunit.client.0.smithi063.stdout:9/101: dread d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:02.892 INFO:tasks.workunit.client.0.smithi063.stdout:7/85: creat d0/d15/f1b x:0 0 0 2024-05-09T06:38:02.895 INFO:tasks.workunit.client.0.smithi063.stdout:5/80: sync 2024-05-09T06:38:02.895 INFO:tasks.workunit.client.0.smithi063.stdout:1/100: sync 2024-05-09T06:38:02.895 INFO:tasks.workunit.client.0.smithi063.stdout:3/86: sync 2024-05-09T06:38:02.896 INFO:tasks.workunit.client.0.smithi063.stdout:5/81: write d8/f14 [586618,20541] 0 2024-05-09T06:38:02.898 INFO:tasks.workunit.client.0.smithi063.stdout:8/168: unlink d9/da/d18/f1e 0 2024-05-09T06:38:02.900 INFO:tasks.workunit.client.0.smithi063.stdout:5/82: dread d8/fc [0,4194304] 0 2024-05-09T06:38:02.900 INFO:tasks.workunit.client.0.smithi063.stdout:0/124: truncate dd/fe 1301652 0 2024-05-09T06:38:02.902 INFO:tasks.workunit.client.0.smithi063.stdout:7/86: rename d0/f9 to d0/f1c 0 2024-05-09T06:38:02.903 INFO:tasks.workunit.client.0.smithi063.stdout:7/87: write d0/d17/f1a [1005453,27956] 0 2024-05-09T06:38:02.903 INFO:tasks.workunit.client.0.smithi063.stdout:6/89: rmdir d2/d18 0 2024-05-09T06:38:02.906 INFO:tasks.workunit.client.0.smithi063.stdout:3/87: rmdir d7/da 39 2024-05-09T06:38:02.910 INFO:tasks.workunit.client.0.smithi063.stdout:6/90: dwrite d2/d6/f1b [0,4194304] 0 2024-05-09T06:38:02.912 INFO:tasks.workunit.client.0.smithi063.stdout:5/83: rmdir d8 39 2024-05-09T06:38:02.914 INFO:tasks.workunit.client.0.smithi063.stdout:0/125: symlink dd/d1a/d22/l29 0 2024-05-09T06:38:02.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/88: rmdir d0/d17 39 2024-05-09T06:38:02.918 INFO:tasks.workunit.client.0.smithi063.stdout:1/101: write d9/d10/f1c [1800146,127932] 0 2024-05-09T06:38:02.918 INFO:tasks.workunit.client.0.smithi063.stdout:1/102: stat d9/l19 0 2024-05-09T06:38:02.927 INFO:tasks.workunit.client.0.smithi063.stdout:6/91: dread d2/d4/f8 [0,4194304] 0 2024-05-09T06:38:02.928 INFO:tasks.workunit.client.0.smithi063.stdout:6/92: stat d2/d4/c17 0 2024-05-09T06:38:02.928 INFO:tasks.workunit.client.0.smithi063.stdout:7/89: mknod d0/d15/c1d 0 2024-05-09T06:38:02.929 INFO:tasks.workunit.client.0.smithi063.stdout:1/103: creat d9/d10/f21 x:0 0 0 2024-05-09T06:38:02.929 INFO:tasks.workunit.client.0.smithi063.stdout:6/93: truncate d2/df/f14 903754 0 2024-05-09T06:38:02.930 INFO:tasks.workunit.client.0.smithi063.stdout:3/88: mknod d7/da/c1d 0 2024-05-09T06:38:02.931 INFO:tasks.workunit.client.0.smithi063.stdout:9/102: getdents d0/d4/da 0 2024-05-09T06:38:02.931 INFO:tasks.workunit.client.0.smithi063.stdout:9/103: fsync d0/f13 0 2024-05-09T06:38:02.932 INFO:tasks.workunit.client.0.smithi063.stdout:5/84: mkdir d8/d17 0 2024-05-09T06:38:02.932 INFO:tasks.workunit.client.0.smithi063.stdout:9/104: rename d0/d4/da to d0/d4/da/d1a/d1b 22 2024-05-09T06:38:02.934 INFO:tasks.workunit.client.1.smithi193.stdout:5/4: getdents d0 0 2024-05-09T06:38:02.934 INFO:tasks.workunit.client.1.smithi193.stdout:5/5: write - no filename 2024-05-09T06:38:02.934 INFO:tasks.workunit.client.1.smithi193.stdout:5/6: truncate - no filename 2024-05-09T06:38:02.934 INFO:tasks.workunit.client.0.smithi063.stdout:1/104: creat d9/d10/f22 x:0 0 0 2024-05-09T06:38:02.935 INFO:tasks.workunit.client.0.smithi063.stdout:9/105: dread d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:02.935 INFO:tasks.workunit.client.0.smithi063.stdout:6/94: creat d2/d6/f1e x:0 0 0 2024-05-09T06:38:02.936 INFO:tasks.workunit.client.0.smithi063.stdout:6/95: fdatasync d2/df/f1d 0 2024-05-09T06:38:02.937 INFO:tasks.workunit.client.1.smithi193.stdout:5/7: rmdir d0 39 2024-05-09T06:38:02.938 INFO:tasks.workunit.client.1.smithi193.stdout:5/8: read - no filename 2024-05-09T06:38:02.938 INFO:tasks.workunit.client.1.smithi193.stdout:5/9: link - no file 2024-05-09T06:38:02.939 INFO:tasks.workunit.client.1.smithi193.stdout:6/7: getdents . 0 2024-05-09T06:38:02.939 INFO:tasks.workunit.client.1.smithi193.stdout:6/8: readlink - no filename 2024-05-09T06:38:02.939 INFO:tasks.workunit.client.1.smithi193.stdout:6/9: chown f0 20 1 2024-05-09T06:38:02.940 INFO:tasks.workunit.client.1.smithi193.stdout:7/8: fsync f1 0 2024-05-09T06:38:02.941 INFO:tasks.workunit.client.1.smithi193.stdout:7/9: fsync f1 0 2024-05-09T06:38:02.941 INFO:tasks.workunit.client.1.smithi193.stdout:7/10: rmdir - no directory 2024-05-09T06:38:02.941 INFO:tasks.workunit.client.1.smithi193.stdout:7/11: fsync f1 0 2024-05-09T06:38:02.942 INFO:tasks.workunit.client.0.smithi063.stdout:7/90: symlink d0/l1e 0 2024-05-09T06:38:02.943 INFO:tasks.workunit.client.0.smithi063.stdout:9/106: dwrite d0/f19 [0,4194304] 0 2024-05-09T06:38:02.943 INFO:tasks.workunit.client.0.smithi063.stdout:7/91: dread - d0/f16 zero size 2024-05-09T06:38:02.943 INFO:tasks.workunit.client.0.smithi063.stdout:9/107: read d0/f13 [1980492,14021] 0 2024-05-09T06:38:02.944 INFO:tasks.workunit.client.0.smithi063.stdout:9/108: chown d0/d4/da 4 1 2024-05-09T06:38:02.945 INFO:tasks.workunit.client.1.smithi193.stdout:6/10: mkdir d1 0 2024-05-09T06:38:02.948 INFO:tasks.workunit.client.0.smithi063.stdout:1/105: dread f3 [0,4194304] 0 2024-05-09T06:38:02.948 INFO:tasks.workunit.client.0.smithi063.stdout:1/106: readlink d9/l13 0 2024-05-09T06:38:02.948 INFO:tasks.workunit.client.0.smithi063.stdout:1/107: chown d9/f11 454470217 1 2024-05-09T06:38:02.949 INFO:tasks.workunit.client.1.smithi193.stdout:7/12: creat f4 x:0 0 0 2024-05-09T06:38:02.954 INFO:tasks.workunit.client.1.smithi193.stdout:0/29: fsync f5 0 2024-05-09T06:38:02.954 INFO:tasks.workunit.client.0.smithi063.stdout:5/85: mkdir d8/d17/d18 0 2024-05-09T06:38:02.954 INFO:tasks.workunit.client.1.smithi193.stdout:4/6: link l2 l3 0 2024-05-09T06:38:02.955 INFO:tasks.workunit.client.1.smithi193.stdout:9/9: getdents . 0 2024-05-09T06:38:02.956 INFO:tasks.workunit.client.0.smithi063.stdout:1/108: dwrite d9/d10/f1c [0,4194304] 0 2024-05-09T06:38:02.958 INFO:tasks.workunit.client.1.smithi193.stdout:7/13: creat f5 x:0 0 0 2024-05-09T06:38:02.961 INFO:tasks.workunit.client.1.smithi193.stdout:5/10: mknod d0/d1/c2 0 2024-05-09T06:38:02.966 INFO:tasks.workunit.client.1.smithi193.stdout:6/11: creat d1/f2 x:0 0 0 2024-05-09T06:38:02.966 INFO:tasks.workunit.client.1.smithi193.stdout:6/12: chown d1/f2 228120784 1 2024-05-09T06:38:02.968 INFO:tasks.workunit.client.1.smithi193.stdout:0/30: symlink l7 0 2024-05-09T06:38:02.972 INFO:tasks.workunit.client.1.smithi193.stdout:9/10: symlink l4 0 2024-05-09T06:38:02.973 INFO:tasks.workunit.client.1.smithi193.stdout:9/11: truncate f0 192947 0 2024-05-09T06:38:02.974 INFO:tasks.workunit.client.1.smithi193.stdout:8/11: write f0 [4314685,15830] 0 2024-05-09T06:38:02.975 INFO:tasks.workunit.client.1.smithi193.stdout:8/12: read f0 [3682817,63651] 0 2024-05-09T06:38:02.977 INFO:tasks.workunit.client.0.smithi063.stdout:4/112: dwrite f5 [0,4194304] 0 2024-05-09T06:38:02.978 INFO:tasks.workunit.client.0.smithi063.stdout:4/113: fsync d7/f14 0 2024-05-09T06:38:02.978 INFO:tasks.workunit.client.1.smithi193.stdout:6/13: unlink f0 0 2024-05-09T06:38:02.979 INFO:tasks.workunit.client.1.smithi193.stdout:6/14: chown d1 190545 1 2024-05-09T06:38:02.979 INFO:tasks.workunit.client.0.smithi063.stdout:4/114: write d7/df/f12 [547811,114736] 0 2024-05-09T06:38:02.980 INFO:tasks.workunit.client.1.smithi193.stdout:6/15: dread - d1/f2 zero size 2024-05-09T06:38:02.981 INFO:tasks.workunit.client.0.smithi063.stdout:4/115: write d7/df/f12 [1090629,30452] 0 2024-05-09T06:38:02.981 INFO:tasks.workunit.client.0.smithi063.stdout:1/109: link d9/fe d9/f23 0 2024-05-09T06:38:02.982 INFO:tasks.workunit.client.1.smithi193.stdout:8/13: dwrite f0 [0,4194304] 0 2024-05-09T06:38:02.984 INFO:tasks.workunit.client.1.smithi193.stdout:0/31: creat f8 x:0 0 0 2024-05-09T06:38:02.987 INFO:tasks.workunit.client.0.smithi063.stdout:4/116: dwrite d7/f14 [0,4194304] 0 2024-05-09T06:38:02.991 INFO:tasks.workunit.client.1.smithi193.stdout:6/16: dwrite d1/f2 [0,4194304] 0 2024-05-09T06:38:02.995 INFO:tasks.workunit.client.0.smithi063.stdout:4/117: dwrite d7/df/f12 [0,4194304] 0 2024-05-09T06:38:03.021 INFO:tasks.workunit.client.0.smithi063.stdout:7/92: sync 2024-05-09T06:38:03.024 INFO:tasks.workunit.client.0.smithi063.stdout:7/93: dread d0/f10 [0,4194304] 0 2024-05-09T06:38:03.027 INFO:tasks.workunit.client.0.smithi063.stdout:7/94: dread d0/f10 [0,4194304] 0 2024-05-09T06:38:03.028 INFO:tasks.workunit.client.0.smithi063.stdout:7/95: rename d0 to d0/d15/d1f 22 2024-05-09T06:38:03.037 INFO:tasks.workunit.client.1.smithi193.stdout:4/7: stat l3 0 2024-05-09T06:38:03.037 INFO:tasks.workunit.client.1.smithi193.stdout:4/8: write - no filename 2024-05-09T06:38:03.037 INFO:tasks.workunit.client.1.smithi193.stdout:4/9: write - no filename 2024-05-09T06:38:03.039 INFO:tasks.workunit.client.0.smithi063.stdout:2/86: getdents dc/dd 0 2024-05-09T06:38:03.039 INFO:tasks.workunit.client.0.smithi063.stdout:2/87: read - dc/dd/f17 zero size 2024-05-09T06:38:03.041 INFO:tasks.workunit.client.0.smithi063.stdout:6/96: dread d2/fd [0,4194304] 0 2024-05-09T06:38:03.043 INFO:tasks.workunit.client.1.smithi193.stdout:9/12: symlink l5 0 2024-05-09T06:38:03.050 INFO:tasks.workunit.client.1.smithi193.stdout:8/14: mkdir d6 0 2024-05-09T06:38:03.052 INFO:tasks.workunit.client.1.smithi193.stdout:0/32: creat f9 x:0 0 0 2024-05-09T06:38:03.052 INFO:tasks.workunit.client.1.smithi193.stdout:0/33: chown f5 191935 1 2024-05-09T06:38:03.054 INFO:tasks.workunit.client.0.smithi063.stdout:7/96: truncate d0/f5 3601440 0 2024-05-09T06:38:03.056 INFO:tasks.workunit.client.0.smithi063.stdout:7/97: read d0/f14 [44823,15037] 0 2024-05-09T06:38:03.056 INFO:tasks.workunit.client.0.smithi063.stdout:8/169: rmdir d9/da 39 2024-05-09T06:38:03.057 INFO:tasks.workunit.client.0.smithi063.stdout:7/98: chown d0/f10 1 1 2024-05-09T06:38:03.058 INFO:tasks.workunit.client.0.smithi063.stdout:8/170: fsync d9/d3b/f2c 0 2024-05-09T06:38:03.059 INFO:tasks.workunit.client.0.smithi063.stdout:7/99: write d0/f14 [1762495,35910] 0 2024-05-09T06:38:03.059 INFO:tasks.workunit.client.0.smithi063.stdout:2/88: creat dc/dd/f20 x:0 0 0 2024-05-09T06:38:03.059 INFO:tasks.workunit.client.0.smithi063.stdout:4/118: dwrite d7/df/d1a/f1e [4194304,4194304] 0 2024-05-09T06:38:03.059 INFO:tasks.workunit.client.1.smithi193.stdout:0/34: dwrite f5 [0,4194304] 0 2024-05-09T06:38:03.061 INFO:tasks.workunit.client.0.smithi063.stdout:2/89: chown dc/dd/l14 0 1 2024-05-09T06:38:03.065 INFO:tasks.workunit.client.0.smithi063.stdout:6/97: truncate d2/fd 848674 0 2024-05-09T06:38:03.069 INFO:tasks.workunit.client.1.smithi193.stdout:9/13: unlink c2 0 2024-05-09T06:38:03.069 INFO:tasks.workunit.client.1.smithi193.stdout:0/35: dwrite f4 [0,4194304] 0 2024-05-09T06:38:03.071 INFO:tasks.workunit.client.1.smithi193.stdout:9/14: write f0 [120184,81071] 0 2024-05-09T06:38:03.071 INFO:tasks.workunit.client.1.smithi193.stdout:9/15: rmdir - no directory 2024-05-09T06:38:03.072 INFO:tasks.workunit.client.1.smithi193.stdout:9/16: read f0 [170100,105552] 0 2024-05-09T06:38:03.073 INFO:tasks.workunit.client.1.smithi193.stdout:9/17: chown f0 48395835 1 2024-05-09T06:38:03.081 INFO:tasks.workunit.client.1.smithi193.stdout:9/18: dwrite f0 [0,4194304] 0 2024-05-09T06:38:03.083 INFO:tasks.workunit.client.1.smithi193.stdout:9/19: write f0 [742820,8680] 0 2024-05-09T06:38:03.083 INFO:tasks.workunit.client.1.smithi193.stdout:6/17: link d1/f2 d1/f3 0 2024-05-09T06:38:03.087 INFO:tasks.workunit.client.1.smithi193.stdout:0/36: fdatasync f5 0 2024-05-09T06:38:03.087 INFO:tasks.workunit.client.1.smithi193.stdout:9/20: dread f0 [0,4194304] 0 2024-05-09T06:38:03.093 INFO:tasks.workunit.client.0.smithi063.stdout:4/119: creat d7/df/d1a/f22 x:0 0 0 2024-05-09T06:38:03.095 INFO:tasks.workunit.client.0.smithi063.stdout:3/89: dwrite d7/da/f14 [0,4194304] 0 2024-05-09T06:38:03.096 INFO:tasks.workunit.client.1.smithi193.stdout:0/37: dwrite f9 [0,4194304] 0 2024-05-09T06:38:03.098 INFO:tasks.workunit.client.1.smithi193.stdout:7/14: fsync f4 0 2024-05-09T06:38:03.099 INFO:tasks.workunit.client.1.smithi193.stdout:7/15: truncate f4 812086 0 2024-05-09T06:38:03.100 INFO:tasks.workunit.client.0.smithi063.stdout:2/90: rename f9 to dc/f21 0 2024-05-09T06:38:03.101 INFO:tasks.workunit.client.0.smithi063.stdout:2/91: dread - dc/f12 zero size 2024-05-09T06:38:03.102 INFO:tasks.workunit.client.0.smithi063.stdout:2/92: chown dc/f1f 49455 1 2024-05-09T06:38:03.102 INFO:tasks.workunit.client.1.smithi193.stdout:8/15: creat d6/f7 x:0 0 0 2024-05-09T06:38:03.103 INFO:tasks.workunit.client.0.smithi063.stdout:2/93: write dc/dd/f20 [541298,63551] 0 2024-05-09T06:38:03.105 INFO:tasks.workunit.client.0.smithi063.stdout:9/109: dwrite d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:03.107 INFO:tasks.workunit.client.0.smithi063.stdout:9/110: chown d0/f19 12202944 1 2024-05-09T06:38:03.111 INFO:tasks.workunit.client.1.smithi193.stdout:6/18: write d1/f2 [4016188,12905] 0 2024-05-09T06:38:03.111 INFO:tasks.workunit.client.1.smithi193.stdout:8/16: dwrite d6/f7 [0,4194304] 0 2024-05-09T06:38:03.117 INFO:tasks.workunit.client.1.smithi193.stdout:9/21: creat f6 x:0 0 0 2024-05-09T06:38:03.118 INFO:tasks.workunit.client.0.smithi063.stdout:5/86: truncate d8/fe 1623270 0 2024-05-09T06:38:03.118 INFO:tasks.workunit.client.1.smithi193.stdout:9/22: dread - f6 zero size 2024-05-09T06:38:03.120 INFO:tasks.workunit.client.1.smithi193.stdout:9/23: chown f0 1730104 1 2024-05-09T06:38:03.121 INFO:tasks.workunit.client.1.smithi193.stdout:4/10: link l3 l4 0 2024-05-09T06:38:03.124 INFO:tasks.workunit.client.0.smithi063.stdout:4/120: rename d7/f10 to d7/df/d1f/f23 0 2024-05-09T06:38:03.125 INFO:tasks.workunit.client.1.smithi193.stdout:6/19: dwrite d1/f3 [0,4194304] 0 2024-05-09T06:38:03.132 INFO:tasks.workunit.client.1.smithi193.stdout:0/38: rename f5 to fa 0 2024-05-09T06:38:03.132 INFO:tasks.workunit.client.1.smithi193.stdout:9/24: dwrite f6 [0,4194304] 0 2024-05-09T06:38:03.133 INFO:tasks.workunit.client.1.smithi193.stdout:0/39: fdatasync f9 0 2024-05-09T06:38:03.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/94: mknod dc/dd/c22 0 2024-05-09T06:38:03.135 INFO:tasks.workunit.client.0.smithi063.stdout:2/95: write fa [6734822,117240] 0 2024-05-09T06:38:03.136 INFO:tasks.workunit.client.0.smithi063.stdout:2/96: write f3 [191571,112412] 0 2024-05-09T06:38:03.137 INFO:tasks.workunit.client.0.smithi063.stdout:8/171: sync 2024-05-09T06:38:03.137 INFO:tasks.workunit.client.0.smithi063.stdout:2/97: write dc/f1f [722325,61371] 0 2024-05-09T06:38:03.139 INFO:tasks.workunit.client.1.smithi193.stdout:0/40: dread f9 [0,4194304] 0 2024-05-09T06:38:03.141 INFO:tasks.workunit.client.1.smithi193.stdout:7/16: mkdir d6 0 2024-05-09T06:38:03.145 INFO:tasks.workunit.client.0.smithi063.stdout:2/98: dwrite dc/f1b [0,4194304] 0 2024-05-09T06:38:03.145 INFO:tasks.workunit.client.0.smithi063.stdout:9/111: creat d0/d4/da/f1c x:0 0 0 2024-05-09T06:38:03.151 INFO:tasks.workunit.client.0.smithi063.stdout:3/90: mkdir d7/d1e 0 2024-05-09T06:38:03.152 INFO:tasks.workunit.client.0.smithi063.stdout:2/99: dwrite dc/f1f [0,4194304] 0 2024-05-09T06:38:03.159 INFO:tasks.workunit.client.1.smithi193.stdout:4/11: creat f5 x:0 0 0 2024-05-09T06:38:03.160 INFO:tasks.workunit.client.1.smithi193.stdout:4/12: dread - f5 zero size 2024-05-09T06:38:03.160 INFO:tasks.workunit.client.0.smithi063.stdout:2/100: dwrite f3 [0,4194304] 0 2024-05-09T06:38:03.170 INFO:tasks.workunit.client.0.smithi063.stdout:7/100: getdents d0/d15 0 2024-05-09T06:38:03.171 INFO:tasks.workunit.client.0.smithi063.stdout:4/121: dwrite d7/df/d1a/f20 [0,4194304] 0 2024-05-09T06:38:03.172 INFO:tasks.workunit.client.1.smithi193.stdout:9/25: rename c1 to c7 0 2024-05-09T06:38:03.174 INFO:tasks.workunit.client.0.smithi063.stdout:8/172: readlink d9/da/d18/l21 0 2024-05-09T06:38:03.183 INFO:tasks.workunit.client.1.smithi193.stdout:0/41: dwrite fa [0,4194304] 0 2024-05-09T06:38:03.190 INFO:tasks.workunit.client.1.smithi193.stdout:6/20: link d1/f2 d1/f4 0 2024-05-09T06:38:03.193 INFO:tasks.workunit.client.1.smithi193.stdout:9/26: symlink l8 0 2024-05-09T06:38:03.194 INFO:tasks.workunit.client.0.smithi063.stdout:2/101: creat dc/dd/f23 x:0 0 0 2024-05-09T06:38:03.195 INFO:tasks.workunit.client.1.smithi193.stdout:7/17: symlink d6/l7 0 2024-05-09T06:38:03.195 INFO:tasks.workunit.client.1.smithi193.stdout:7/18: dread - f1 zero size 2024-05-09T06:38:03.200 INFO:tasks.workunit.client.1.smithi193.stdout:4/13: stat l4 0 2024-05-09T06:38:03.203 INFO:tasks.workunit.client.0.smithi063.stdout:4/122: rename d7/f15 to d7/f24 0 2024-05-09T06:38:03.204 INFO:tasks.workunit.client.0.smithi063.stdout:4/123: fsync d7/df/d1a/f20 0 2024-05-09T06:38:03.204 INFO:tasks.workunit.client.0.smithi063.stdout:7/101: dwrite d0/f14 [0,4194304] 0 2024-05-09T06:38:03.204 INFO:tasks.workunit.client.1.smithi193.stdout:7/19: dwrite f5 [0,4194304] 0 2024-05-09T06:38:03.207 INFO:tasks.workunit.client.0.smithi063.stdout:8/173: fsync d9/da/f1a 0 2024-05-09T06:38:03.208 INFO:tasks.workunit.client.1.smithi193.stdout:9/27: mknod c9 0 2024-05-09T06:38:03.209 INFO:tasks.workunit.client.0.smithi063.stdout:1/110: write f8 [2330013,32286] 0 2024-05-09T06:38:03.210 INFO:tasks.workunit.client.0.smithi063.stdout:9/112: creat d0/d4/da/d1a/f1d x:0 0 0 2024-05-09T06:38:03.212 INFO:tasks.workunit.client.0.smithi063.stdout:3/91: symlink d7/l1f 0 2024-05-09T06:38:03.212 INFO:tasks.workunit.client.1.smithi193.stdout:4/14: dwrite f5 [0,4194304] 0 2024-05-09T06:38:03.213 INFO:tasks.workunit.client.0.smithi063.stdout:3/92: dread - d7/da/f13 zero size 2024-05-09T06:38:03.214 INFO:tasks.workunit.client.0.smithi063.stdout:4/124: dwrite d7/df/d1a/f22 [0,4194304] 0 2024-05-09T06:38:03.215 INFO:tasks.workunit.client.0.smithi063.stdout:3/93: chown d7/da/c1d 8391 1 2024-05-09T06:38:03.216 INFO:tasks.workunit.client.0.smithi063.stdout:4/125: write d7/f14 [260692,73822] 0 2024-05-09T06:38:03.216 INFO:tasks.workunit.client.0.smithi063.stdout:2/102: symlink dc/l24 0 2024-05-09T06:38:03.216 INFO:tasks.workunit.client.0.smithi063.stdout:3/94: fsync d7/da/f13 0 2024-05-09T06:38:03.217 INFO:tasks.workunit.client.0.smithi063.stdout:2/103: stat f3 0 2024-05-09T06:38:03.217 INFO:tasks.workunit.client.1.smithi193.stdout:7/20: mkdir d6/d8 0 2024-05-09T06:38:03.218 INFO:tasks.workunit.client.0.smithi063.stdout:3/95: stat d7/f16 0 2024-05-09T06:38:03.219 INFO:tasks.workunit.client.1.smithi193.stdout:7/21: chown d6/l7 1048122 1 2024-05-09T06:38:03.219 INFO:tasks.workunit.client.0.smithi063.stdout:3/96: fdatasync d7/f10 0 2024-05-09T06:38:03.226 INFO:tasks.workunit.client.0.smithi063.stdout:2/104: dwrite f6 [0,4194304] 0 2024-05-09T06:38:03.227 INFO:tasks.workunit.client.0.smithi063.stdout:0/126: truncate dd/d1a/f1e 308215 0 2024-05-09T06:38:03.231 INFO:tasks.workunit.client.1.smithi193.stdout:9/28: creat fa x:0 0 0 2024-05-09T06:38:03.231 INFO:tasks.workunit.client.1.smithi193.stdout:7/22: dwrite f1 [0,4194304] 0 2024-05-09T06:38:03.233 INFO:tasks.workunit.client.0.smithi063.stdout:6/98: dread d2/f7 [0,4194304] 0 2024-05-09T06:38:03.236 INFO:tasks.workunit.client.1.smithi193.stdout:7/23: dread f5 [0,4194304] 0 2024-05-09T06:38:03.238 INFO:tasks.workunit.client.0.smithi063.stdout:8/174: creat d9/da/d18/d24/f3c x:0 0 0 2024-05-09T06:38:03.248 INFO:tasks.workunit.client.0.smithi063.stdout:9/113: creat d0/d4/da/f1e x:0 0 0 2024-05-09T06:38:03.251 INFO:tasks.workunit.client.1.smithi193.stdout:9/29: symlink lb 0 2024-05-09T06:38:03.252 INFO:tasks.workunit.client.1.smithi193.stdout:9/30: write fa [660824,79843] 0 2024-05-09T06:38:03.255 INFO:tasks.workunit.client.0.smithi063.stdout:4/126: unlink d7/f14 0 2024-05-09T06:38:03.256 INFO:tasks.workunit.client.0.smithi063.stdout:4/127: write d7/df/f21 [24279,103949] 0 2024-05-09T06:38:03.257 INFO:tasks.workunit.client.0.smithi063.stdout:3/97: mknod d7/da/c20 0 2024-05-09T06:38:03.257 INFO:tasks.workunit.client.0.smithi063.stdout:4/128: write d7/df/f21 [558010,4134] 0 2024-05-09T06:38:03.258 INFO:tasks.workunit.client.0.smithi063.stdout:4/129: dread - d7/f24 zero size 2024-05-09T06:38:03.260 INFO:tasks.workunit.client.0.smithi063.stdout:2/105: chown f2 4 1 2024-05-09T06:38:03.262 INFO:tasks.workunit.client.0.smithi063.stdout:2/106: dread f6 [0,4194304] 0 2024-05-09T06:38:03.263 INFO:tasks.workunit.client.0.smithi063.stdout:2/107: write dc/dd/f17 [841353,29175] 0 2024-05-09T06:38:03.271 INFO:tasks.workunit.client.1.smithi193.stdout:0/42: getdents . 0 2024-05-09T06:38:03.272 INFO:tasks.workunit.client.0.smithi063.stdout:6/99: dwrite d2/d4/f10 [0,4194304] 0 2024-05-09T06:38:03.280 INFO:tasks.workunit.client.0.smithi063.stdout:3/98: mkdir d7/da/d21 0 2024-05-09T06:38:03.280 INFO:tasks.workunit.client.0.smithi063.stdout:3/99: fdatasync d7/da/f18 0 2024-05-09T06:38:03.280 INFO:tasks.workunit.client.0.smithi063.stdout:3/100: stat d7/f10 0 2024-05-09T06:38:03.281 INFO:tasks.workunit.client.0.smithi063.stdout:5/87: write d8/fa [953552,86035] 0 2024-05-09T06:38:03.294 INFO:tasks.workunit.client.0.smithi063.stdout:1/111: mkdir d9/d1a/d24 0 2024-05-09T06:38:03.299 INFO:tasks.workunit.client.1.smithi193.stdout:8/17: truncate d6/f7 1393376 0 2024-05-09T06:38:03.301 INFO:tasks.workunit.client.0.smithi063.stdout:9/114: unlink d0/l10 0 2024-05-09T06:38:03.304 INFO:tasks.workunit.client.1.smithi193.stdout:6/21: rmdir d1 39 2024-05-09T06:38:03.306 INFO:tasks.workunit.client.0.smithi063.stdout:4/130: unlink l0 0 2024-05-09T06:38:03.314 INFO:tasks.workunit.client.1.smithi193.stdout:7/24: write f5 [4849870,60493] 0 2024-05-09T06:38:03.314 INFO:tasks.workunit.client.0.smithi063.stdout:3/101: mkdir d7/d22 0 2024-05-09T06:38:03.315 INFO:tasks.workunit.client.0.smithi063.stdout:1/112: dwrite d9/d10/f22 [0,4194304] 0 2024-05-09T06:38:03.316 INFO:tasks.workunit.client.0.smithi063.stdout:4/131: mknod d7/c25 0 2024-05-09T06:38:03.324 INFO:tasks.workunit.client.1.smithi193.stdout:9/31: fsync fa 0 2024-05-09T06:38:03.325 INFO:tasks.workunit.client.1.smithi193.stdout:9/32: write f6 [2403989,61390] 0 2024-05-09T06:38:03.326 INFO:tasks.workunit.client.0.smithi063.stdout:3/102: mkdir d7/da/d23 0 2024-05-09T06:38:03.330 INFO:tasks.workunit.client.0.smithi063.stdout:4/132: creat d7/df/f26 x:0 0 0 2024-05-09T06:38:03.331 INFO:tasks.workunit.client.1.smithi193.stdout:9/33: dread f0 [0,4194304] 0 2024-05-09T06:38:03.333 INFO:tasks.workunit.client.1.smithi193.stdout:7/25: dwrite f1 [4194304,4194304] 0 2024-05-09T06:38:03.334 INFO:tasks.workunit.client.0.smithi063.stdout:7/102: unlink d0/f5 0 2024-05-09T06:38:03.335 INFO:tasks.workunit.client.1.smithi193.stdout:7/26: dread f4 [0,4194304] 0 2024-05-09T06:38:03.338 INFO:tasks.workunit.client.0.smithi063.stdout:2/108: getdents dc 0 2024-05-09T06:38:03.340 INFO:tasks.workunit.client.0.smithi063.stdout:0/127: write dd/f15 [1132799,57091] 0 2024-05-09T06:38:03.346 INFO:tasks.workunit.client.0.smithi063.stdout:1/113: creat d9/d1a/d1e/f25 x:0 0 0 2024-05-09T06:38:03.347 INFO:tasks.workunit.client.0.smithi063.stdout:0/128: fdatasync dd/f15 0 2024-05-09T06:38:03.350 INFO:tasks.workunit.client.0.smithi063.stdout:7/103: creat d0/d15/f20 x:0 0 0 2024-05-09T06:38:03.351 INFO:tasks.workunit.client.0.smithi063.stdout:7/104: write d0/ff [4919671,100863] 0 2024-05-09T06:38:03.351 INFO:tasks.workunit.client.0.smithi063.stdout:2/109: truncate f2 692773 0 2024-05-09T06:38:03.352 INFO:tasks.workunit.client.0.smithi063.stdout:2/110: truncate dc/f12 344063 0 2024-05-09T06:38:03.354 INFO:tasks.workunit.client.0.smithi063.stdout:9/115: getdents d0/d4 0 2024-05-09T06:38:03.357 INFO:tasks.workunit.client.0.smithi063.stdout:2/111: dread dc/dd/f13 [0,4194304] 0 2024-05-09T06:38:03.358 INFO:tasks.workunit.client.0.smithi063.stdout:2/112: fdatasync dc/dd/f23 0 2024-05-09T06:38:03.358 INFO:tasks.workunit.client.0.smithi063.stdout:2/113: chown dc/dd/f17 0 1 2024-05-09T06:38:03.360 INFO:tasks.workunit.client.0.smithi063.stdout:3/103: mknod d7/d22/c24 0 2024-05-09T06:38:03.364 INFO:tasks.workunit.client.0.smithi063.stdout:8/175: truncate d9/da/d18/f2d 2784842 0 2024-05-09T06:38:03.366 INFO:tasks.workunit.client.0.smithi063.stdout:0/129: symlink dd/l2a 0 2024-05-09T06:38:03.367 INFO:tasks.workunit.client.0.smithi063.stdout:3/104: dwrite f5 [0,4194304] 0 2024-05-09T06:38:03.369 INFO:tasks.workunit.client.0.smithi063.stdout:4/133: link f4 d7/df/d1f/f27 0 2024-05-09T06:38:03.372 INFO:tasks.workunit.client.0.smithi063.stdout:5/88: sync 2024-05-09T06:38:03.376 INFO:tasks.workunit.client.0.smithi063.stdout:3/105: dread d7/f1a [0,4194304] 0 2024-05-09T06:38:03.382 INFO:tasks.workunit.client.0.smithi063.stdout:2/114: creat dc/f25 x:0 0 0 2024-05-09T06:38:03.383 INFO:tasks.workunit.client.0.smithi063.stdout:6/100: dwrite d2/d4/f8 [0,4194304] 0 2024-05-09T06:38:03.384 INFO:tasks.workunit.client.0.smithi063.stdout:2/115: dread - dc/f25 zero size 2024-05-09T06:38:03.384 INFO:tasks.workunit.client.0.smithi063.stdout:4/134: dwrite d7/df/f21 [0,4194304] 0 2024-05-09T06:38:03.385 INFO:tasks.workunit.client.0.smithi063.stdout:2/116: read fa [7315348,29661] 0 2024-05-09T06:38:03.388 INFO:tasks.workunit.client.0.smithi063.stdout:8/176: dread d9/da/d18/f1b [0,4194304] 0 2024-05-09T06:38:03.388 INFO:tasks.workunit.client.0.smithi063.stdout:0/130: creat dd/d1a/d22/f2b x:0 0 0 2024-05-09T06:38:03.389 INFO:tasks.workunit.client.0.smithi063.stdout:8/177: chown d9/da/d18 933838 1 2024-05-09T06:38:03.396 INFO:tasks.workunit.client.0.smithi063.stdout:5/89: mkdir d8/d19 0 2024-05-09T06:38:03.425 INFO:tasks.workunit.client.0.smithi063.stdout:5/90: rename d8/ff to d8/d19/f1a 0 2024-05-09T06:38:03.426 INFO:tasks.workunit.client.0.smithi063.stdout:7/105: creat d0/f21 x:0 0 0 2024-05-09T06:38:03.427 INFO:tasks.workunit.client.0.smithi063.stdout:7/106: chown d0/f18 4 1 2024-05-09T06:38:03.427 INFO:tasks.workunit.client.0.smithi063.stdout:7/107: fdatasync d0/f14 0 2024-05-09T06:38:03.427 INFO:tasks.workunit.client.0.smithi063.stdout:3/106: mkdir d7/d1e/d25 0 2024-05-09T06:38:03.429 INFO:tasks.workunit.client.0.smithi063.stdout:1/114: write d9/f23 [856917,75285] 0 2024-05-09T06:38:03.432 INFO:tasks.workunit.client.0.smithi063.stdout:1/115: dread f3 [0,4194304] 0 2024-05-09T06:38:03.433 INFO:tasks.workunit.client.0.smithi063.stdout:6/101: mkdir d2/d1f 0 2024-05-09T06:38:03.437 INFO:tasks.workunit.client.0.smithi063.stdout:6/102: write d2/df/f14 [1095851,994] 0 2024-05-09T06:38:03.438 INFO:tasks.workunit.client.0.smithi063.stdout:0/131: mkdir dd/d1a/d2c 0 2024-05-09T06:38:03.439 INFO:tasks.workunit.client.0.smithi063.stdout:8/178: symlink d9/da/l3d 0 2024-05-09T06:38:03.440 INFO:tasks.workunit.client.0.smithi063.stdout:7/108: dwrite d0/fc [0,4194304] 0 2024-05-09T06:38:03.441 INFO:tasks.workunit.client.0.smithi063.stdout:8/179: readlink l1 0 2024-05-09T06:38:03.442 INFO:tasks.workunit.client.0.smithi063.stdout:7/109: stat d0/l13 0 2024-05-09T06:38:03.444 INFO:tasks.workunit.client.0.smithi063.stdout:3/107: dwrite d7/da/fe [0,4194304] 0 2024-05-09T06:38:03.448 INFO:tasks.workunit.client.0.smithi063.stdout:3/108: write d7/f10 [12556,109175] 0 2024-05-09T06:38:03.449 INFO:tasks.workunit.client.0.smithi063.stdout:1/116: dwrite d9/d1a/d1e/f25 [0,4194304] 0 2024-05-09T06:38:03.453 INFO:tasks.workunit.client.0.smithi063.stdout:7/110: dread d0/f11 [0,4194304] 0 2024-05-09T06:38:03.454 INFO:tasks.workunit.client.0.smithi063.stdout:7/111: truncate d0/f21 277384 0 2024-05-09T06:38:03.462 INFO:tasks.workunit.client.0.smithi063.stdout:6/103: dread f1 [0,4194304] 0 2024-05-09T06:38:03.463 INFO:tasks.workunit.client.0.smithi063.stdout:8/180: unlink d9/c11 0 2024-05-09T06:38:03.464 INFO:tasks.workunit.client.0.smithi063.stdout:6/104: write d2/d6/f1b [2569904,93137] 0 2024-05-09T06:38:03.465 INFO:tasks.workunit.client.0.smithi063.stdout:8/181: write d9/da/f1a [282688,114130] 0 2024-05-09T06:38:03.465 INFO:tasks.workunit.client.0.smithi063.stdout:6/105: chown d2/d4/f8 5012 1 2024-05-09T06:38:03.466 INFO:tasks.workunit.client.0.smithi063.stdout:6/106: chown d2/d6/l16 8364 1 2024-05-09T06:38:03.467 INFO:tasks.workunit.client.0.smithi063.stdout:6/107: readlink d2/d12/l15 0 2024-05-09T06:38:03.471 INFO:tasks.workunit.client.0.smithi063.stdout:9/116: getdents d0/d4/da 0 2024-05-09T06:38:03.474 INFO:tasks.workunit.client.0.smithi063.stdout:9/117: dread d0/f19 [0,4194304] 0 2024-05-09T06:38:03.475 INFO:tasks.workunit.client.0.smithi063.stdout:3/109: fsync d7/da/f14 0 2024-05-09T06:38:03.476 INFO:tasks.workunit.client.0.smithi063.stdout:9/118: write d0/d4/da/d1a/f1d [768206,78200] 0 2024-05-09T06:38:03.479 INFO:tasks.workunit.client.0.smithi063.stdout:2/117: write f3 [3894485,31225] 0 2024-05-09T06:38:03.482 INFO:tasks.workunit.client.0.smithi063.stdout:3/110: dwrite d7/f11 [0,4194304] 0 2024-05-09T06:38:03.484 INFO:tasks.workunit.client.0.smithi063.stdout:8/182: creat d9/da/d18/d24/f3e x:0 0 0 2024-05-09T06:38:03.484 INFO:tasks.workunit.client.0.smithi063.stdout:2/118: dread f6 [0,4194304] 0 2024-05-09T06:38:03.485 INFO:tasks.workunit.client.0.smithi063.stdout:5/91: link d8/l13 d8/d17/l1b 0 2024-05-09T06:38:03.486 INFO:tasks.workunit.client.0.smithi063.stdout:2/119: readlink dc/dd/l14 0 2024-05-09T06:38:03.486 INFO:tasks.workunit.client.0.smithi063.stdout:2/120: chown l5 21 1 2024-05-09T06:38:03.491 INFO:tasks.workunit.client.0.smithi063.stdout:8/183: dwrite d9/da/f1a [0,4194304] 0 2024-05-09T06:38:03.493 INFO:tasks.workunit.client.0.smithi063.stdout:2/121: dread dc/f1f [0,4194304] 0 2024-05-09T06:38:03.493 INFO:tasks.workunit.client.0.smithi063.stdout:2/122: stat lb 0 2024-05-09T06:38:03.494 INFO:tasks.workunit.client.0.smithi063.stdout:2/123: chown dc/dd/l14 38557 1 2024-05-09T06:38:03.495 INFO:tasks.workunit.client.0.smithi063.stdout:2/124: truncate f6 4314260 0 2024-05-09T06:38:03.496 INFO:tasks.workunit.client.0.smithi063.stdout:9/119: readlink d0/d4/da/l14 0 2024-05-09T06:38:03.497 INFO:tasks.workunit.client.0.smithi063.stdout:9/120: stat d0 0 2024-05-09T06:38:03.497 INFO:tasks.workunit.client.0.smithi063.stdout:4/135: sync 2024-05-09T06:38:03.497 INFO:tasks.workunit.client.0.smithi063.stdout:4/136: stat d7 0 2024-05-09T06:38:03.502 INFO:tasks.workunit.client.0.smithi063.stdout:4/137: fdatasync d7/df/d1a/f22 0 2024-05-09T06:38:03.503 INFO:tasks.workunit.client.0.smithi063.stdout:4/138: dread - d7/df/f26 zero size 2024-05-09T06:38:03.504 INFO:tasks.workunit.client.0.smithi063.stdout:9/121: dread d0/f13 [0,4194304] 0 2024-05-09T06:38:03.510 INFO:tasks.workunit.client.0.smithi063.stdout:5/92: fsync d8/fc 0 2024-05-09T06:38:03.514 INFO:tasks.workunit.client.0.smithi063.stdout:6/108: mkdir d2/d20 0 2024-05-09T06:38:03.520 INFO:tasks.workunit.client.0.smithi063.stdout:2/125: mkdir dc/dd/d26 0 2024-05-09T06:38:03.521 INFO:tasks.workunit.client.0.smithi063.stdout:2/126: write dc/f12 [1167404,121527] 0 2024-05-09T06:38:03.523 INFO:tasks.workunit.client.0.smithi063.stdout:7/112: link d0/f21 d0/d17/f22 0 2024-05-09T06:38:03.528 INFO:tasks.workunit.client.0.smithi063.stdout:1/117: dwrite d9/d10/f18 [0,4194304] 0 2024-05-09T06:38:03.528 INFO:tasks.workunit.client.0.smithi063.stdout:0/132: write dd/fe [1029891,40718] 0 2024-05-09T06:38:03.530 INFO:tasks.workunit.client.0.smithi063.stdout:0/133: truncate dd/d1a/f1b 830240 0 2024-05-09T06:38:03.530 INFO:tasks.workunit.client.0.smithi063.stdout:4/139: mkdir d7/df/d1f/d28 0 2024-05-09T06:38:03.532 INFO:tasks.workunit.client.0.smithi063.stdout:9/122: creat d0/f1f x:0 0 0 2024-05-09T06:38:03.534 INFO:tasks.workunit.client.0.smithi063.stdout:3/111: creat d7/d1e/d25/f26 x:0 0 0 2024-05-09T06:38:03.534 INFO:tasks.workunit.client.0.smithi063.stdout:3/112: chown d7/d1e/d25/f26 1042761917 1 2024-05-09T06:38:03.537 INFO:tasks.workunit.client.0.smithi063.stdout:5/93: creat d8/d17/f1c x:0 0 0 2024-05-09T06:38:03.549 INFO:tasks.workunit.client.0.smithi063.stdout:7/113: creat d0/d17/f23 x:0 0 0 2024-05-09T06:38:03.551 INFO:tasks.workunit.client.0.smithi063.stdout:7/114: dread d0/d17/f22 [0,4194304] 0 2024-05-09T06:38:03.554 INFO:tasks.workunit.client.0.smithi063.stdout:0/134: write dd/f1d [842615,49617] 0 2024-05-09T06:38:03.556 INFO:tasks.workunit.client.0.smithi063.stdout:4/140: symlink d7/df/l29 0 2024-05-09T06:38:03.559 INFO:tasks.workunit.client.0.smithi063.stdout:4/141: dread d7/df/d1f/f27 [0,4194304] 0 2024-05-09T06:38:03.559 INFO:tasks.workunit.client.0.smithi063.stdout:3/113: creat d7/d22/f27 x:0 0 0 2024-05-09T06:38:03.560 INFO:tasks.workunit.client.0.smithi063.stdout:4/142: readlink d7/le 0 2024-05-09T06:38:03.560 INFO:tasks.workunit.client.0.smithi063.stdout:3/114: truncate d7/f8 111067 0 2024-05-09T06:38:03.560 INFO:tasks.workunit.client.0.smithi063.stdout:4/143: chown d7/df/c13 0 1 2024-05-09T06:38:03.561 INFO:tasks.workunit.client.0.smithi063.stdout:4/144: write d7/df/f21 [335692,42353] 0 2024-05-09T06:38:03.561 INFO:tasks.workunit.client.0.smithi063.stdout:5/94: creat d8/f1d x:0 0 0 2024-05-09T06:38:03.562 INFO:tasks.workunit.client.0.smithi063.stdout:5/95: readlink l5 0 2024-05-09T06:38:03.564 INFO:tasks.workunit.client.0.smithi063.stdout:5/96: write d8/d17/f1c [757492,31063] 0 2024-05-09T06:38:03.568 INFO:tasks.workunit.client.0.smithi063.stdout:3/115: dwrite d7/da/fe [0,4194304] 0 2024-05-09T06:38:03.570 INFO:tasks.workunit.client.0.smithi063.stdout:3/116: read - d7/da/f13 zero size 2024-05-09T06:38:03.573 INFO:tasks.workunit.client.0.smithi063.stdout:7/115: mknod d0/d15/c24 0 2024-05-09T06:38:03.573 INFO:tasks.workunit.client.0.smithi063.stdout:5/97: dwrite f6 [0,4194304] 0 2024-05-09T06:38:03.574 INFO:tasks.workunit.client.0.smithi063.stdout:5/98: chown c3 224760664 1 2024-05-09T06:38:03.584 INFO:tasks.workunit.client.0.smithi063.stdout:6/109: write f1 [3945382,98157] 0 2024-05-09T06:38:03.586 INFO:tasks.workunit.client.0.smithi063.stdout:6/110: write d2/d4/f5 [917688,119199] 0 2024-05-09T06:38:03.589 INFO:tasks.workunit.client.0.smithi063.stdout:6/111: dread d2/d4/f10 [0,4194304] 0 2024-05-09T06:38:03.593 INFO:tasks.workunit.client.0.smithi063.stdout:4/145: symlink d7/df/d1f/l2a 0 2024-05-09T06:38:03.599 INFO:tasks.workunit.client.0.smithi063.stdout:8/184: dread d9/da/d18/f2d [0,4194304] 0 2024-05-09T06:38:03.600 INFO:tasks.workunit.client.0.smithi063.stdout:2/127: rmdir dc/dd/d26 0 2024-05-09T06:38:03.604 INFO:tasks.workunit.client.0.smithi063.stdout:8/185: dwrite d9/da/d18/f23 [0,4194304] 0 2024-05-09T06:38:03.605 INFO:tasks.workunit.client.0.smithi063.stdout:8/186: dread - d9/da/d18/d24/f27 zero size 2024-05-09T06:38:03.605 INFO:tasks.workunit.client.0.smithi063.stdout:3/117: unlink f0 0 2024-05-09T06:38:03.608 INFO:tasks.workunit.client.0.smithi063.stdout:7/116: creat d0/d15/f25 x:0 0 0 2024-05-09T06:38:03.608 INFO:tasks.workunit.client.0.smithi063.stdout:7/117: readlink d0/l13 0 2024-05-09T06:38:03.610 INFO:tasks.workunit.client.0.smithi063.stdout:5/99: mkdir d8/d17/d1e 0 2024-05-09T06:38:03.613 INFO:tasks.workunit.client.0.smithi063.stdout:1/118: link d9/f12 d9/d1a/d24/f26 0 2024-05-09T06:38:03.617 INFO:tasks.workunit.client.0.smithi063.stdout:9/123: link d0/d4/da/l14 d0/d4/da/l20 0 2024-05-09T06:38:03.627 INFO:tasks.workunit.client.0.smithi063.stdout:4/146: creat d7/df/f2b x:0 0 0 2024-05-09T06:38:03.633 INFO:tasks.workunit.client.0.smithi063.stdout:2/128: dread f3 [0,4194304] 0 2024-05-09T06:38:03.634 INFO:tasks.workunit.client.1.smithi193.stdout:4/15: chown l4 1141035509 1 2024-05-09T06:38:03.640 INFO:tasks.workunit.client.0.smithi063.stdout:3/118: rename d7/da/d23 to d7/d22/d28 0 2024-05-09T06:38:03.643 INFO:tasks.workunit.client.1.smithi193.stdout:0/43: mknod cb 0 2024-05-09T06:38:03.644 INFO:tasks.workunit.client.0.smithi063.stdout:0/135: truncate dd/fe 1031677 0 2024-05-09T06:38:03.646 INFO:tasks.workunit.client.0.smithi063.stdout:1/119: rmdir d9/d1a/d1e 39 2024-05-09T06:38:03.647 INFO:tasks.workunit.client.1.smithi193.stdout:8/18: creat d6/f8 x:0 0 0 2024-05-09T06:38:03.654 INFO:tasks.workunit.client.0.smithi063.stdout:9/124: dwrite d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:03.654 INFO:tasks.workunit.client.0.smithi063.stdout:2/129: creat dc/f27 x:0 0 0 2024-05-09T06:38:03.655 INFO:tasks.workunit.client.0.smithi063.stdout:8/187: link f4 d9/da/d18/d24/f3f 0 2024-05-09T06:38:03.658 INFO:tasks.workunit.client.0.smithi063.stdout:6/112: dwrite d2/fd [0,4194304] 0 2024-05-09T06:38:03.661 INFO:tasks.workunit.client.0.smithi063.stdout:6/113: write d2/d6/f1b [70248,46238] 0 2024-05-09T06:38:03.664 INFO:tasks.workunit.client.0.smithi063.stdout:9/125: dwrite d0/f1f [0,4194304] 0 2024-05-09T06:38:03.667 INFO:tasks.workunit.client.1.smithi193.stdout:4/16: creat f6 x:0 0 0 2024-05-09T06:38:03.670 INFO:tasks.workunit.client.1.smithi193.stdout:0/44: chown l2 863758 1 2024-05-09T06:38:03.670 INFO:tasks.workunit.client.1.smithi193.stdout:0/45: chown f8 284211888 1 2024-05-09T06:38:03.671 INFO:tasks.workunit.client.1.smithi193.stdout:0/46: chown f9 7 1 2024-05-09T06:38:03.671 INFO:tasks.workunit.client.1.smithi193.stdout:0/47: stat l7 0 2024-05-09T06:38:03.673 INFO:tasks.workunit.client.0.smithi063.stdout:5/100: creat d8/d17/d18/f1f x:0 0 0 2024-05-09T06:38:03.673 INFO:tasks.workunit.client.1.smithi193.stdout:4/17: dread f5 [0,4194304] 0 2024-05-09T06:38:03.677 INFO:tasks.workunit.client.1.smithi193.stdout:6/22: creat d1/f5 x:0 0 0 2024-05-09T06:38:03.685 INFO:tasks.workunit.client.1.smithi193.stdout:7/27: link f5 d6/d8/f9 0 2024-05-09T06:38:03.686 INFO:tasks.workunit.client.1.smithi193.stdout:8/19: stat d6/f7 0 2024-05-09T06:38:03.690 INFO:tasks.workunit.client.1.smithi193.stdout:0/48: rename c1 to cc 0 2024-05-09T06:38:03.693 INFO:tasks.workunit.client.1.smithi193.stdout:6/23: rename d1/f4 to d1/f6 0 2024-05-09T06:38:03.694 INFO:tasks.workunit.client.0.smithi063.stdout:8/188: creat d9/d3b/f40 x:0 0 0 2024-05-09T06:38:03.695 INFO:tasks.workunit.client.1.smithi193.stdout:9/34: link l8 lc 0 2024-05-09T06:38:03.696 INFO:tasks.workunit.client.0.smithi063.stdout:4/147: truncate d7/df/d1a/f1e 2165101 0 2024-05-09T06:38:03.697 INFO:tasks.workunit.client.0.smithi063.stdout:4/148: stat d7/df/d1a/c1d 0 2024-05-09T06:38:03.697 INFO:tasks.workunit.client.0.smithi063.stdout:4/149: chown f6 323231559 1 2024-05-09T06:38:03.697 INFO:tasks.workunit.client.0.smithi063.stdout:4/150: fdatasync d7/df/f12 0 2024-05-09T06:38:03.698 INFO:tasks.workunit.client.0.smithi063.stdout:4/151: read - d7/f24 zero size 2024-05-09T06:38:03.700 INFO:tasks.workunit.client.1.smithi193.stdout:6/24: dwrite d1/f5 [0,4194304] 0 2024-05-09T06:38:03.702 INFO:tasks.workunit.client.1.smithi193.stdout:8/20: unlink l5 0 2024-05-09T06:38:03.706 INFO:tasks.workunit.client.1.smithi193.stdout:0/49: mknod cd 0 2024-05-09T06:38:03.707 INFO:tasks.workunit.client.1.smithi193.stdout:0/50: fdatasync f4 0 2024-05-09T06:38:03.707 INFO:tasks.workunit.client.1.smithi193.stdout:0/51: rmdir - no directory 2024-05-09T06:38:03.707 INFO:tasks.workunit.client.1.smithi193.stdout:0/52: fdatasync f4 0 2024-05-09T06:38:03.707 INFO:tasks.workunit.client.1.smithi193.stdout:0/53: stat cd 0 2024-05-09T06:38:03.708 INFO:tasks.workunit.client.0.smithi063.stdout:3/119: creat d7/da/f29 x:0 0 0 2024-05-09T06:38:03.710 INFO:tasks.workunit.client.1.smithi193.stdout:9/35: mkdir dd 0 2024-05-09T06:38:03.711 INFO:tasks.workunit.client.1.smithi193.stdout:9/36: stat f6 0 2024-05-09T06:38:03.712 INFO:tasks.workunit.client.0.smithi063.stdout:9/126: dwrite d0/d4/da/f1c [0,4194304] 0 2024-05-09T06:38:03.714 INFO:tasks.workunit.client.0.smithi063.stdout:5/101: symlink d8/l20 0 2024-05-09T06:38:03.715 INFO:tasks.workunit.client.1.smithi193.stdout:7/28: creat d6/d8/fa x:0 0 0 2024-05-09T06:38:03.718 INFO:tasks.workunit.client.1.smithi193.stdout:6/25: fdatasync d1/f2 0 2024-05-09T06:38:03.721 INFO:tasks.workunit.client.0.smithi063.stdout:1/120: creat d9/d1a/d1e/f27 x:0 0 0 2024-05-09T06:38:03.721 INFO:tasks.workunit.client.0.smithi063.stdout:1/121: dread - d9/d10/f21 zero size 2024-05-09T06:38:03.722 INFO:tasks.workunit.client.1.smithi193.stdout:8/21: rmdir d6 39 2024-05-09T06:38:03.726 INFO:tasks.workunit.client.1.smithi193.stdout:0/54: rename fa to fe 0 2024-05-09T06:38:03.727 INFO:tasks.workunit.client.1.smithi193.stdout:0/55: write f4 [2089094,123107] 0 2024-05-09T06:38:03.731 INFO:tasks.workunit.client.0.smithi063.stdout:6/114: creat d2/d20/f21 x:0 0 0 2024-05-09T06:38:03.734 INFO:tasks.workunit.client.1.smithi193.stdout:0/56: dwrite f4 [0,4194304] 0 2024-05-09T06:38:03.735 INFO:tasks.workunit.client.1.smithi193.stdout:0/57: read f9 [1912811,62800] 0 2024-05-09T06:38:03.736 INFO:tasks.workunit.client.1.smithi193.stdout:6/26: creat d1/f7 x:0 0 0 2024-05-09T06:38:03.736 INFO:tasks.workunit.client.0.smithi063.stdout:6/115: dwrite f1 [0,4194304] 0 2024-05-09T06:38:03.737 INFO:tasks.workunit.client.0.smithi063.stdout:3/120: rmdir d7/da 39 2024-05-09T06:38:03.737 INFO:tasks.workunit.client.0.smithi063.stdout:6/116: chown d2/df/f1d 0 1 2024-05-09T06:38:03.738 INFO:tasks.workunit.client.0.smithi063.stdout:6/117: chown d2/d4/l1a 272927 1 2024-05-09T06:38:03.742 INFO:tasks.workunit.client.0.smithi063.stdout:7/118: getdents d0 0 2024-05-09T06:38:03.747 INFO:tasks.workunit.client.1.smithi193.stdout:9/37: creat dd/fe x:0 0 0 2024-05-09T06:38:03.747 INFO:tasks.workunit.client.1.smithi193.stdout:7/29: write f5 [5739340,6720] 0 2024-05-09T06:38:03.748 INFO:tasks.workunit.client.1.smithi193.stdout:7/30: readlink l3 0 2024-05-09T06:38:03.751 INFO:tasks.workunit.client.0.smithi063.stdout:5/102: creat d8/f21 x:0 0 0 2024-05-09T06:38:03.751 INFO:tasks.workunit.client.0.smithi063.stdout:5/103: chown f6 2234323 1 2024-05-09T06:38:03.754 INFO:tasks.workunit.client.1.smithi193.stdout:9/38: dwrite f6 [0,4194304] 0 2024-05-09T06:38:03.756 INFO:tasks.workunit.client.1.smithi193.stdout:0/58: creat ff x:0 0 0 2024-05-09T06:38:03.756 INFO:tasks.workunit.client.1.smithi193.stdout:0/59: dread - f8 zero size 2024-05-09T06:38:03.758 INFO:tasks.workunit.client.1.smithi193.stdout:6/27: mknod d1/c8 0 2024-05-09T06:38:03.760 INFO:tasks.workunit.client.0.smithi063.stdout:1/122: fsync d9/d1a/d24/f26 0 2024-05-09T06:38:03.760 INFO:tasks.workunit.client.0.smithi063.stdout:2/130: link lb dc/dd/l28 0 2024-05-09T06:38:03.761 INFO:tasks.workunit.client.0.smithi063.stdout:2/131: readlink dc/dd/le 0 2024-05-09T06:38:03.762 INFO:tasks.workunit.client.0.smithi063.stdout:8/189: mkdir d9/d41 0 2024-05-09T06:38:03.764 INFO:tasks.workunit.client.1.smithi193.stdout:7/31: mkdir d6/d8/db 0 2024-05-09T06:38:03.764 INFO:tasks.workunit.client.1.smithi193.stdout:8/22: dwrite f0 [0,4194304] 0 2024-05-09T06:38:03.764 INFO:tasks.workunit.client.0.smithi063.stdout:4/152: mknod d7/df/d1f/d28/c2c 0 2024-05-09T06:38:03.765 INFO:tasks.workunit.client.1.smithi193.stdout:7/32: dread - d6/d8/fa zero size 2024-05-09T06:38:03.765 INFO:tasks.workunit.client.1.smithi193.stdout:9/39: mkdir dd/df 0 2024-05-09T06:38:03.766 INFO:tasks.workunit.client.0.smithi063.stdout:4/153: dread - d7/f24 zero size 2024-05-09T06:38:03.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/60: write fe [4284130,97516] 0 2024-05-09T06:38:03.768 INFO:tasks.workunit.client.0.smithi063.stdout:1/123: dwrite d9/d10/f21 [0,4194304] 0 2024-05-09T06:38:03.769 INFO:tasks.workunit.client.1.smithi193.stdout:0/61: truncate fe 4656449 0 2024-05-09T06:38:03.772 INFO:tasks.workunit.client.1.smithi193.stdout:9/40: dread f0 [0,4194304] 0 2024-05-09T06:38:03.773 INFO:tasks.workunit.client.0.smithi063.stdout:6/118: mknod d2/d12/c22 0 2024-05-09T06:38:03.773 INFO:tasks.workunit.client.1.smithi193.stdout:9/41: stat dd/fe 0 2024-05-09T06:38:03.774 INFO:tasks.workunit.client.1.smithi193.stdout:0/62: dread f9 [0,4194304] 0 2024-05-09T06:38:03.776 INFO:tasks.workunit.client.1.smithi193.stdout:0/63: write f4 [2371644,68516] 0 2024-05-09T06:38:03.777 INFO:tasks.workunit.client.1.smithi193.stdout:7/33: dread f4 [0,4194304] 0 2024-05-09T06:38:03.780 INFO:tasks.workunit.client.0.smithi063.stdout:3/121: dread d7/da/f14 [0,4194304] 0 2024-05-09T06:38:03.780 INFO:tasks.workunit.client.0.smithi063.stdout:5/104: dread f4 [0,4194304] 0 2024-05-09T06:38:03.781 INFO:tasks.workunit.client.1.smithi193.stdout:8/23: unlink d6/f8 0 2024-05-09T06:38:03.781 INFO:tasks.workunit.client.0.smithi063.stdout:1/124: dwrite d9/d1a/d1e/f27 [0,4194304] 0 2024-05-09T06:38:03.785 INFO:tasks.workunit.client.1.smithi193.stdout:0/64: mknod c10 0 2024-05-09T06:38:03.788 INFO:tasks.workunit.client.1.smithi193.stdout:9/42: creat dd/df/f10 x:0 0 0 2024-05-09T06:38:03.788 INFO:tasks.workunit.client.0.smithi063.stdout:2/132: dread fa [4194304,4194304] 0 2024-05-09T06:38:03.790 INFO:tasks.workunit.client.1.smithi193.stdout:7/34: unlink l3 0 2024-05-09T06:38:03.794 INFO:tasks.workunit.client.0.smithi063.stdout:2/133: dread dc/f12 [0,4194304] 0 2024-05-09T06:38:03.794 INFO:tasks.workunit.client.1.smithi193.stdout:8/24: mknod d6/c9 0 2024-05-09T06:38:03.797 INFO:tasks.workunit.client.0.smithi063.stdout:7/119: link d0/f14 d0/d15/f26 0 2024-05-09T06:38:03.797 INFO:tasks.workunit.client.0.smithi063.stdout:7/120: readlink d0/l1e 0 2024-05-09T06:38:03.799 INFO:tasks.workunit.client.1.smithi193.stdout:0/65: rename l0 to l11 0 2024-05-09T06:38:03.799 INFO:tasks.workunit.client.1.smithi193.stdout:7/35: dwrite d6/d8/fa [0,4194304] 0 2024-05-09T06:38:03.801 INFO:tasks.workunit.client.0.smithi063.stdout:9/127: rename d0/f19 to d0/d4/da/f21 0 2024-05-09T06:38:03.801 INFO:tasks.workunit.client.0.smithi063.stdout:9/128: chown d0/d4/da/f1c 7930647 1 2024-05-09T06:38:03.805 INFO:tasks.workunit.client.1.smithi193.stdout:6/28: dwrite d1/f6 [4194304,4194304] 0 2024-05-09T06:38:03.805 INFO:tasks.workunit.client.1.smithi193.stdout:9/43: creat dd/f11 x:0 0 0 2024-05-09T06:38:03.808 INFO:tasks.workunit.client.1.smithi193.stdout:7/36: dread f1 [0,4194304] 0 2024-05-09T06:38:03.812 INFO:tasks.workunit.client.1.smithi193.stdout:0/66: creat f12 x:0 0 0 2024-05-09T06:38:03.813 INFO:tasks.workunit.client.1.smithi193.stdout:7/37: dread d6/d8/f9 [0,4194304] 0 2024-05-09T06:38:03.818 INFO:tasks.workunit.client.1.smithi193.stdout:6/29: creat d1/f9 x:0 0 0 2024-05-09T06:38:03.818 INFO:tasks.workunit.client.1.smithi193.stdout:7/38: dread d6/d8/fa [0,4194304] 0 2024-05-09T06:38:03.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/125: dread d9/fb [0,4194304] 0 2024-05-09T06:38:03.821 INFO:tasks.workunit.client.0.smithi063.stdout:4/154: link d7/f24 d7/f2d 0 2024-05-09T06:38:03.822 INFO:tasks.workunit.client.0.smithi063.stdout:6/119: mknod d2/c23 0 2024-05-09T06:38:03.823 INFO:tasks.workunit.client.0.smithi063.stdout:6/120: fdatasync d2/d4/f8 0 2024-05-09T06:38:03.824 INFO:tasks.workunit.client.0.smithi063.stdout:2/134: symlink dc/dd/l29 0 2024-05-09T06:38:03.830 INFO:tasks.workunit.client.1.smithi193.stdout:8/25: rmdir d6 39 2024-05-09T06:38:03.832 INFO:tasks.workunit.client.1.smithi193.stdout:6/30: creat d1/fa x:0 0 0 2024-05-09T06:38:03.833 INFO:tasks.workunit.client.1.smithi193.stdout:6/31: truncate d1/f9 655093 0 2024-05-09T06:38:03.834 INFO:tasks.workunit.client.1.smithi193.stdout:6/32: write d1/f3 [4683519,49041] 0 2024-05-09T06:38:03.835 INFO:tasks.workunit.client.1.smithi193.stdout:6/33: write d1/f9 [266856,92724] 0 2024-05-09T06:38:03.837 INFO:tasks.workunit.client.0.smithi063.stdout:0/136: dread dd/fe [0,4194304] 0 2024-05-09T06:38:03.838 INFO:tasks.workunit.client.0.smithi063.stdout:0/137: truncate dd/d1a/f25 433190 0 2024-05-09T06:38:03.838 INFO:tasks.workunit.client.0.smithi063.stdout:0/138: fdatasync f6 0 2024-05-09T06:38:03.839 INFO:tasks.workunit.client.1.smithi193.stdout:0/67: fsync f12 0 2024-05-09T06:38:03.840 INFO:tasks.workunit.client.1.smithi193.stdout:8/26: stat d6/f7 0 2024-05-09T06:38:03.841 INFO:tasks.workunit.client.1.smithi193.stdout:8/27: write f0 [1747164,52077] 0 2024-05-09T06:38:03.844 INFO:tasks.workunit.client.0.smithi063.stdout:1/126: rmdir d9/d1a 39 2024-05-09T06:38:03.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/139: dwrite dd/f15 [0,4194304] 0 2024-05-09T06:38:03.846 INFO:tasks.workunit.client.0.smithi063.stdout:0/140: chown f6 58 1 2024-05-09T06:38:03.846 INFO:tasks.workunit.client.1.smithi193.stdout:6/34: mknod d1/cb 0 2024-05-09T06:38:03.847 INFO:tasks.workunit.client.0.smithi063.stdout:0/141: readlink dd/l1f 0 2024-05-09T06:38:03.848 INFO:tasks.workunit.client.0.smithi063.stdout:8/190: dwrite d9/da/fb [0,4194304] 0 2024-05-09T06:38:03.848 INFO:tasks.workunit.client.1.smithi193.stdout:0/68: creat f13 x:0 0 0 2024-05-09T06:38:03.848 INFO:tasks.workunit.client.1.smithi193.stdout:0/69: fsync fe 0 2024-05-09T06:38:03.849 INFO:tasks.workunit.client.1.smithi193.stdout:0/70: write f8 [465289,25085] 0 2024-05-09T06:38:03.851 INFO:tasks.workunit.client.0.smithi063.stdout:6/121: creat d2/d12/f24 x:0 0 0 2024-05-09T06:38:03.853 INFO:tasks.workunit.client.1.smithi193.stdout:9/44: truncate f6 849754 0 2024-05-09T06:38:03.853 INFO:tasks.workunit.client.1.smithi193.stdout:9/45: fdatasync fa 0 2024-05-09T06:38:03.854 INFO:tasks.workunit.client.0.smithi063.stdout:3/122: write f5 [4482875,25499] 0 2024-05-09T06:38:03.855 INFO:tasks.workunit.client.0.smithi063.stdout:0/142: dread dd/f1d [0,4194304] 0 2024-05-09T06:38:03.856 INFO:tasks.workunit.client.0.smithi063.stdout:0/143: chown dd/d1a/l20 73446521 1 2024-05-09T06:38:03.856 INFO:tasks.workunit.client.0.smithi063.stdout:0/144: readlink dd/l28 0 2024-05-09T06:38:03.856 INFO:tasks.workunit.client.0.smithi063.stdout:5/105: truncate d8/fa 608338 0 2024-05-09T06:38:03.857 INFO:tasks.workunit.client.0.smithi063.stdout:0/145: dread dd/fe [0,4194304] 0 2024-05-09T06:38:03.859 INFO:tasks.workunit.client.0.smithi063.stdout:9/129: mknod d0/d4/da/d1a/c22 0 2024-05-09T06:38:03.859 INFO:tasks.workunit.client.0.smithi063.stdout:0/146: write dd/d1a/f1b [1877093,3516] 0 2024-05-09T06:38:03.860 INFO:tasks.workunit.client.0.smithi063.stdout:0/147: chown dd/cf 7 1 2024-05-09T06:38:03.863 INFO:tasks.workunit.client.1.smithi193.stdout:0/71: rename c6 to c14 0 2024-05-09T06:38:03.865 INFO:tasks.workunit.client.1.smithi193.stdout:9/46: write f0 [3639713,24781] 0 2024-05-09T06:38:03.866 INFO:tasks.workunit.client.1.smithi193.stdout:7/39: getdents d6/d8 0 2024-05-09T06:38:03.874 INFO:tasks.workunit.client.1.smithi193.stdout:0/72: unlink ff 0 2024-05-09T06:38:03.883 INFO:tasks.workunit.client.0.smithi063.stdout:8/191: rmdir d9/da/d18 39 2024-05-09T06:38:03.885 INFO:tasks.workunit.client.0.smithi063.stdout:6/122: creat d2/d6/f25 x:0 0 0 2024-05-09T06:38:03.885 INFO:tasks.workunit.client.0.smithi063.stdout:6/123: fdatasync f1 0 2024-05-09T06:38:03.886 INFO:tasks.workunit.client.1.smithi193.stdout:0/73: rename f4 to f15 0 2024-05-09T06:38:03.888 INFO:tasks.workunit.client.0.smithi063.stdout:2/135: link dc/dd/f23 dc/f2a 0 2024-05-09T06:38:03.890 INFO:tasks.workunit.client.1.smithi193.stdout:7/40: write f1 [8233421,85993] 0 2024-05-09T06:38:03.891 INFO:tasks.workunit.client.1.smithi193.stdout:6/35: getdents d1 0 2024-05-09T06:38:03.892 INFO:tasks.workunit.client.1.smithi193.stdout:6/36: fdatasync d1/fa 0 2024-05-09T06:38:03.893 INFO:tasks.workunit.client.1.smithi193.stdout:6/37: write d1/f6 [7485759,41808] 0 2024-05-09T06:38:03.894 INFO:tasks.workunit.client.1.smithi193.stdout:0/74: readlink l11 0 2024-05-09T06:38:03.897 INFO:tasks.workunit.client.1.smithi193.stdout:7/41: mknod d6/d8/cc 0 2024-05-09T06:38:03.898 INFO:tasks.workunit.client.1.smithi193.stdout:0/75: dread f9 [0,4194304] 0 2024-05-09T06:38:03.898 INFO:tasks.workunit.client.0.smithi063.stdout:0/148: creat dd/f2d x:0 0 0 2024-05-09T06:38:03.901 INFO:tasks.workunit.client.1.smithi193.stdout:6/38: symlink d1/lc 0 2024-05-09T06:38:03.902 INFO:tasks.workunit.client.1.smithi193.stdout:0/76: dread f9 [0,4194304] 0 2024-05-09T06:38:03.902 INFO:tasks.workunit.client.0.smithi063.stdout:9/130: dwrite d0/d4/da/f1e [0,4194304] 0 2024-05-09T06:38:03.902 INFO:tasks.workunit.client.1.smithi193.stdout:6/39: chown d1/f9 17 1 2024-05-09T06:38:03.903 INFO:tasks.workunit.client.1.smithi193.stdout:0/77: dread - f13 zero size 2024-05-09T06:38:03.904 INFO:tasks.workunit.client.1.smithi193.stdout:7/42: mkdir d6/dd 0 2024-05-09T06:38:03.908 INFO:tasks.workunit.client.1.smithi193.stdout:9/47: chown f6 42077721 1 2024-05-09T06:38:03.908 INFO:tasks.workunit.client.1.smithi193.stdout:6/40: dwrite d1/f2 [4194304,4194304] 0 2024-05-09T06:38:03.908 INFO:tasks.workunit.client.1.smithi193.stdout:9/48: readlink l8 0 2024-05-09T06:38:03.909 INFO:tasks.workunit.client.1.smithi193.stdout:0/78: creat f16 x:0 0 0 2024-05-09T06:38:03.911 INFO:tasks.workunit.client.1.smithi193.stdout:7/43: fsync d6/d8/f9 0 2024-05-09T06:38:03.911 INFO:tasks.workunit.client.1.smithi193.stdout:9/49: chown dd/df/f10 8459 1 2024-05-09T06:38:03.915 INFO:tasks.workunit.client.1.smithi193.stdout:9/50: truncate dd/df/f10 944354 0 2024-05-09T06:38:03.915 INFO:tasks.workunit.client.1.smithi193.stdout:6/41: dread d1/f2 [0,4194304] 0 2024-05-09T06:38:03.915 INFO:tasks.workunit.client.0.smithi063.stdout:2/136: mknod dc/c2b 0 2024-05-09T06:38:03.916 INFO:tasks.workunit.client.0.smithi063.stdout:2/137: readlink dc/dd/l14 0 2024-05-09T06:38:03.916 INFO:tasks.workunit.client.1.smithi193.stdout:0/79: creat f17 x:0 0 0 2024-05-09T06:38:03.920 INFO:tasks.workunit.client.0.smithi063.stdout:3/123: truncate d7/f1a 147858 0 2024-05-09T06:38:03.921 INFO:tasks.workunit.client.0.smithi063.stdout:3/124: chown d7/d22 12785 1 2024-05-09T06:38:03.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/121: getdents d0 0 2024-05-09T06:38:03.924 INFO:tasks.workunit.client.0.smithi063.stdout:5/106: rename d8/fa to d8/d17/d18/f22 0 2024-05-09T06:38:03.925 INFO:tasks.workunit.client.1.smithi193.stdout:6/42: creat d1/fd x:0 0 0 2024-05-09T06:38:03.926 INFO:tasks.workunit.client.1.smithi193.stdout:0/80: mkdir d18 0 2024-05-09T06:38:03.927 INFO:tasks.workunit.client.1.smithi193.stdout:0/81: stat l7 0 2024-05-09T06:38:03.927 INFO:tasks.workunit.client.1.smithi193.stdout:0/82: dread - f13 zero size 2024-05-09T06:38:03.930 INFO:tasks.workunit.client.0.smithi063.stdout:1/127: symlink d9/d1a/l28 0 2024-05-09T06:38:03.931 INFO:tasks.workunit.client.0.smithi063.stdout:4/155: getdents d7/df/d1f 0 2024-05-09T06:38:03.932 INFO:tasks.workunit.client.1.smithi193.stdout:6/43: fsync d1/f3 0 2024-05-09T06:38:03.932 INFO:tasks.workunit.client.0.smithi063.stdout:4/156: truncate d7/df/f26 407540 0 2024-05-09T06:38:03.933 INFO:tasks.workunit.client.0.smithi063.stdout:4/157: chown d7/df/d1f/f23 105879 1 2024-05-09T06:38:03.933 INFO:tasks.workunit.client.0.smithi063.stdout:6/124: write d2/d4/f10 [91394,93046] 0 2024-05-09T06:38:03.934 INFO:tasks.workunit.client.1.smithi193.stdout:7/44: link f4 d6/dd/fe 0 2024-05-09T06:38:03.936 INFO:tasks.workunit.client.1.smithi193.stdout:6/44: mkdir d1/de 0 2024-05-09T06:38:03.939 INFO:tasks.workunit.client.1.smithi193.stdout:0/83: mknod d18/c19 0 2024-05-09T06:38:03.940 INFO:tasks.workunit.client.1.smithi193.stdout:0/84: write f9 [572379,24397] 0 2024-05-09T06:38:03.941 INFO:tasks.workunit.client.1.smithi193.stdout:9/51: write f6 [1458649,60435] 0 2024-05-09T06:38:03.941 INFO:tasks.workunit.client.1.smithi193.stdout:7/45: fsync d6/d8/fa 0 2024-05-09T06:38:03.943 INFO:tasks.workunit.client.0.smithi063.stdout:7/122: symlink d0/d15/l27 0 2024-05-09T06:38:03.944 INFO:tasks.workunit.client.1.smithi193.stdout:9/52: stat dd/fe 0 2024-05-09T06:38:03.945 INFO:tasks.workunit.client.1.smithi193.stdout:6/45: dwrite d1/fa [0,4194304] 0 2024-05-09T06:38:03.946 INFO:tasks.workunit.client.1.smithi193.stdout:6/46: readlink d1/lc 0 2024-05-09T06:38:03.946 INFO:tasks.workunit.client.1.smithi193.stdout:6/47: chown d1/f2 17 1 2024-05-09T06:38:03.947 INFO:tasks.workunit.client.1.smithi193.stdout:6/48: fsync d1/f2 0 2024-05-09T06:38:03.947 INFO:tasks.workunit.client.0.smithi063.stdout:5/107: rename f4 to d8/d17/d1e/f23 0 2024-05-09T06:38:03.949 INFO:tasks.workunit.client.1.smithi193.stdout:0/85: symlink d18/l1a 0 2024-05-09T06:38:03.952 INFO:tasks.workunit.client.0.smithi063.stdout:5/108: dwrite d8/f21 [0,4194304] 0 2024-05-09T06:38:03.954 INFO:tasks.workunit.client.0.smithi063.stdout:5/109: chown d8/d17/f1c 0 1 2024-05-09T06:38:03.955 INFO:tasks.workunit.client.1.smithi193.stdout:9/53: mknod dd/df/c12 0 2024-05-09T06:38:03.956 INFO:tasks.workunit.client.0.smithi063.stdout:2/138: write fa [8907793,85511] 0 2024-05-09T06:38:03.957 INFO:tasks.workunit.client.0.smithi063.stdout:9/131: unlink d0/d4/da/ff 0 2024-05-09T06:38:03.962 INFO:tasks.workunit.client.1.smithi193.stdout:0/86: creat d18/f1b x:0 0 0 2024-05-09T06:38:03.963 INFO:tasks.workunit.client.1.smithi193.stdout:0/87: readlink l2 0 2024-05-09T06:38:03.963 INFO:tasks.workunit.client.1.smithi193.stdout:7/46: creat d6/dd/ff x:0 0 0 2024-05-09T06:38:03.964 INFO:tasks.workunit.client.0.smithi063.stdout:6/125: mkdir d2/d20/d26 0 2024-05-09T06:38:03.965 INFO:tasks.workunit.client.0.smithi063.stdout:6/126: fsync d2/d12/f24 0 2024-05-09T06:38:03.965 INFO:tasks.workunit.client.1.smithi193.stdout:9/54: creat dd/f13 x:0 0 0 2024-05-09T06:38:03.965 INFO:tasks.workunit.client.1.smithi193.stdout:9/55: readlink l8 0 2024-05-09T06:38:03.967 INFO:tasks.workunit.client.1.smithi193.stdout:6/49: creat d1/de/ff x:0 0 0 2024-05-09T06:38:03.968 INFO:tasks.workunit.client.0.smithi063.stdout:1/128: write d9/fb [3028742,75843] 0 2024-05-09T06:38:03.968 INFO:tasks.workunit.client.1.smithi193.stdout:0/88: mknod d18/c1c 0 2024-05-09T06:38:03.969 INFO:tasks.workunit.client.0.smithi063.stdout:6/127: dread f1 [0,4194304] 0 2024-05-09T06:38:03.969 INFO:tasks.workunit.client.0.smithi063.stdout:6/128: chown d2/d6 1025 1 2024-05-09T06:38:03.970 INFO:tasks.workunit.client.0.smithi063.stdout:1/129: truncate d9/d1a/d1e/f25 5152302 0 2024-05-09T06:38:03.970 INFO:tasks.workunit.client.0.smithi063.stdout:6/129: write d2/d4/f1c [720135,11284] 0 2024-05-09T06:38:03.971 INFO:tasks.workunit.client.0.smithi063.stdout:6/130: dread - d2/d6/f25 zero size 2024-05-09T06:38:03.972 INFO:tasks.workunit.client.1.smithi193.stdout:7/47: rename f1 to d6/d8/f10 0 2024-05-09T06:38:03.972 INFO:tasks.workunit.client.1.smithi193.stdout:0/89: dread f9 [0,4194304] 0 2024-05-09T06:38:03.972 INFO:tasks.workunit.client.0.smithi063.stdout:6/131: rename d2 to d2/df/d27 22 2024-05-09T06:38:03.973 INFO:tasks.workunit.client.0.smithi063.stdout:3/125: mkdir d7/d2a 0 2024-05-09T06:38:03.974 INFO:tasks.workunit.client.1.smithi193.stdout:9/56: mknod dd/df/c14 0 2024-05-09T06:38:03.975 INFO:tasks.workunit.client.0.smithi063.stdout:7/123: creat d0/d15/f28 x:0 0 0 2024-05-09T06:38:03.975 INFO:tasks.workunit.client.1.smithi193.stdout:0/90: dread fe [0,4194304] 0 2024-05-09T06:38:03.976 INFO:tasks.workunit.client.1.smithi193.stdout:0/91: write f12 [849501,66879] 0 2024-05-09T06:38:03.978 INFO:tasks.workunit.client.0.smithi063.stdout:0/149: creat dd/d1a/f2e x:0 0 0 2024-05-09T06:38:03.978 INFO:tasks.workunit.client.0.smithi063.stdout:4/158: dwrite d7/f2d [0,4194304] 0 2024-05-09T06:38:03.979 INFO:tasks.workunit.client.0.smithi063.stdout:4/159: stat d7/df/d1f/d28 0 2024-05-09T06:38:03.981 INFO:tasks.workunit.client.1.smithi193.stdout:7/48: symlink d6/l11 0 2024-05-09T06:38:03.983 INFO:tasks.workunit.client.1.smithi193.stdout:9/57: mkdir dd/df/d15 0 2024-05-09T06:38:03.983 INFO:tasks.workunit.client.1.smithi193.stdout:6/50: rmdir d1 39 2024-05-09T06:38:03.985 INFO:tasks.workunit.client.0.smithi063.stdout:4/160: dwrite d7/df/f2b [0,4194304] 0 2024-05-09T06:38:03.986 INFO:tasks.workunit.client.0.smithi063.stdout:4/161: chown d7/df/d1f/f27 228 1 2024-05-09T06:38:03.990 INFO:tasks.workunit.client.0.smithi063.stdout:2/139: rmdir dc 39 2024-05-09T06:38:03.993 INFO:tasks.workunit.client.1.smithi193.stdout:0/92: write f15 [255549,94826] 0 2024-05-09T06:38:03.993 INFO:tasks.workunit.client.1.smithi193.stdout:9/58: dwrite dd/f13 [0,4194304] 0 2024-05-09T06:38:03.996 INFO:tasks.workunit.client.1.smithi193.stdout:9/59: chown f0 230657068 1 2024-05-09T06:38:03.996 INFO:tasks.workunit.client.1.smithi193.stdout:9/60: write dd/f13 [786789,107278] 0 2024-05-09T06:38:03.996 INFO:tasks.workunit.client.1.smithi193.stdout:7/49: rename d6/d8/f10 to d6/d8/db/f12 0 2024-05-09T06:38:03.999 INFO:tasks.workunit.client.1.smithi193.stdout:9/61: truncate fa 1780840 0 2024-05-09T06:38:03.999 INFO:tasks.workunit.client.0.smithi063.stdout:8/192: getdents d9 0 2024-05-09T06:38:04.003 INFO:tasks.workunit.client.1.smithi193.stdout:0/93: rmdir d18 39 2024-05-09T06:38:04.007 INFO:tasks.workunit.client.1.smithi193.stdout:6/51: dwrite d1/f7 [0,4194304] 0 2024-05-09T06:38:04.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/132: creat d2/df/f28 x:0 0 0 2024-05-09T06:38:04.008 INFO:tasks.workunit.client.0.smithi063.stdout:6/133: dread - d2/df/f28 zero size 2024-05-09T06:38:04.009 INFO:tasks.workunit.client.1.smithi193.stdout:7/50: creat d6/f13 x:0 0 0 2024-05-09T06:38:04.010 INFO:tasks.workunit.client.1.smithi193.stdout:0/94: dread fe [0,4194304] 0 2024-05-09T06:38:04.010 INFO:tasks.workunit.client.1.smithi193.stdout:7/51: fdatasync d6/d8/fa 0 2024-05-09T06:38:04.011 INFO:tasks.workunit.client.0.smithi063.stdout:3/126: mkdir d7/da/d2b 0 2024-05-09T06:38:04.016 INFO:tasks.workunit.client.0.smithi063.stdout:7/124: creat d0/d17/f29 x:0 0 0 2024-05-09T06:38:04.017 INFO:tasks.workunit.client.0.smithi063.stdout:7/125: write d0/d17/f1a [1381208,52738] 0 2024-05-09T06:38:04.017 INFO:tasks.workunit.client.0.smithi063.stdout:7/126: chown d0/f10 2709745 1 2024-05-09T06:38:04.018 INFO:tasks.workunit.client.1.smithi193.stdout:9/62: mkdir dd/df/d15/d16 0 2024-05-09T06:38:04.018 INFO:tasks.workunit.client.1.smithi193.stdout:9/63: dread - dd/fe zero size 2024-05-09T06:38:04.020 INFO:tasks.workunit.client.1.smithi193.stdout:7/52: fdatasync d6/d8/f9 0 2024-05-09T06:38:04.022 INFO:tasks.workunit.client.0.smithi063.stdout:4/162: creat d7/df/d1f/d28/f2e x:0 0 0 2024-05-09T06:38:04.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/140: dread - dc/f1c zero size 2024-05-09T06:38:04.027 INFO:tasks.workunit.client.1.smithi193.stdout:9/64: creat dd/df/d15/f17 x:0 0 0 2024-05-09T06:38:04.028 INFO:tasks.workunit.client.1.smithi193.stdout:9/65: write dd/f11 [982037,123055] 0 2024-05-09T06:38:04.030 INFO:tasks.workunit.client.0.smithi063.stdout:8/193: dread d9/da/d18/f22 [0,4194304] 0 2024-05-09T06:38:04.031 INFO:tasks.workunit.client.1.smithi193.stdout:7/53: creat d6/dd/f14 x:0 0 0 2024-05-09T06:38:04.033 INFO:tasks.workunit.client.0.smithi063.stdout:5/110: write d8/fe [2487195,58195] 0 2024-05-09T06:38:04.035 INFO:tasks.workunit.client.0.smithi063.stdout:0/150: creat dd/d1a/d22/d23/f2f x:0 0 0 2024-05-09T06:38:04.035 INFO:tasks.workunit.client.1.smithi193.stdout:9/66: creat dd/f18 x:0 0 0 2024-05-09T06:38:04.035 INFO:tasks.workunit.client.0.smithi063.stdout:0/151: fdatasync f5 0 2024-05-09T06:38:04.037 INFO:tasks.workunit.client.0.smithi063.stdout:7/127: unlink d0/d15/f28 0 2024-05-09T06:38:04.038 INFO:tasks.workunit.client.1.smithi193.stdout:6/52: getdents d1/de 0 2024-05-09T06:38:04.038 INFO:tasks.workunit.client.1.smithi193.stdout:6/53: dread - d1/de/ff zero size 2024-05-09T06:38:04.038 INFO:tasks.workunit.client.0.smithi063.stdout:7/128: write d0/f16 [71939,106211] 0 2024-05-09T06:38:04.043 INFO:tasks.workunit.client.1.smithi193.stdout:9/67: symlink dd/df/d15/l19 0 2024-05-09T06:38:04.043 INFO:tasks.workunit.client.1.smithi193.stdout:9/68: write dd/f13 [3737365,120993] 0 2024-05-09T06:38:04.043 INFO:tasks.workunit.client.0.smithi063.stdout:5/111: dwrite d8/d17/d18/f1f [0,4194304] 0 2024-05-09T06:38:04.043 INFO:tasks.workunit.client.0.smithi063.stdout:4/163: fsync f4 0 2024-05-09T06:38:04.043 INFO:tasks.workunit.client.0.smithi063.stdout:4/164: stat d7/f24 0 2024-05-09T06:38:04.043 INFO:tasks.workunit.client.0.smithi063.stdout:5/112: fdatasync f6 0 2024-05-09T06:38:04.044 INFO:tasks.workunit.client.1.smithi193.stdout:0/95: getdents d18 0 2024-05-09T06:38:04.047 INFO:tasks.workunit.client.1.smithi193.stdout:6/54: rmdir d1/de 39 2024-05-09T06:38:04.047 INFO:tasks.workunit.client.1.smithi193.stdout:0/96: dread f8 [0,4194304] 0 2024-05-09T06:38:04.050 INFO:tasks.workunit.client.0.smithi063.stdout:8/194: creat d9/da/d18/d24/f42 x:0 0 0 2024-05-09T06:38:04.051 INFO:tasks.workunit.client.0.smithi063.stdout:8/195: chown d9/da/d18/d24/l26 18 1 2024-05-09T06:38:04.052 INFO:tasks.workunit.client.0.smithi063.stdout:6/134: mkdir d2/d1f/d29 0 2024-05-09T06:38:04.052 INFO:tasks.workunit.client.0.smithi063.stdout:6/135: dread - d2/df/f28 zero size 2024-05-09T06:38:04.053 INFO:tasks.workunit.client.1.smithi193.stdout:9/69: mknod dd/c1a 0 2024-05-09T06:38:04.053 INFO:tasks.workunit.client.0.smithi063.stdout:6/136: write d2/fd [3391687,35960] 0 2024-05-09T06:38:04.054 INFO:tasks.workunit.client.1.smithi193.stdout:9/70: write dd/df/d15/f17 [964056,25070] 0 2024-05-09T06:38:04.058 INFO:tasks.workunit.client.0.smithi063.stdout:1/130: truncate d9/fa 2046011 0 2024-05-09T06:38:04.060 INFO:tasks.workunit.client.1.smithi193.stdout:0/97: mkdir d18/d1d 0 2024-05-09T06:38:04.060 INFO:tasks.workunit.client.1.smithi193.stdout:9/71: dread dd/df/d15/f17 [0,4194304] 0 2024-05-09T06:38:04.061 INFO:tasks.workunit.client.1.smithi193.stdout:9/72: write dd/f11 [758996,70931] 0 2024-05-09T06:38:04.061 INFO:tasks.workunit.client.0.smithi063.stdout:0/152: chown dd/c19 12 1 2024-05-09T06:38:04.063 INFO:tasks.workunit.client.1.smithi193.stdout:7/54: getdents d6/d8 0 2024-05-09T06:38:04.064 INFO:tasks.workunit.client.0.smithi063.stdout:1/131: dwrite d9/fb [4194304,4194304] 0 2024-05-09T06:38:04.064 INFO:tasks.workunit.client.0.smithi063.stdout:0/153: dread dd/fe [0,4194304] 0 2024-05-09T06:38:04.065 INFO:tasks.workunit.client.0.smithi063.stdout:3/127: truncate d7/da/fb 232719 0 2024-05-09T06:38:04.069 INFO:tasks.workunit.client.1.smithi193.stdout:6/55: creat d1/de/f10 x:0 0 0 2024-05-09T06:38:04.070 INFO:tasks.workunit.client.1.smithi193.stdout:9/73: dwrite dd/f11 [0,4194304] 0 2024-05-09T06:38:04.072 INFO:tasks.workunit.client.1.smithi193.stdout:0/98: unlink cd 0 2024-05-09T06:38:04.073 INFO:tasks.workunit.client.1.smithi193.stdout:0/99: chown d18 4517 1 2024-05-09T06:38:04.074 INFO:tasks.workunit.client.1.smithi193.stdout:0/100: write f15 [504565,58179] 0 2024-05-09T06:38:04.079 INFO:tasks.workunit.client.1.smithi193.stdout:6/56: dread d1/f9 [0,4194304] 0 2024-05-09T06:38:04.080 INFO:tasks.workunit.client.0.smithi063.stdout:9/132: link d0/d8/le d0/d4/da/d1a/l23 0 2024-05-09T06:38:04.081 INFO:tasks.workunit.client.0.smithi063.stdout:5/113: creat d8/d17/f24 x:0 0 0 2024-05-09T06:38:04.085 INFO:tasks.workunit.client.1.smithi193.stdout:9/74: dwrite dd/fe [0,4194304] 0 2024-05-09T06:38:04.087 INFO:tasks.workunit.client.1.smithi193.stdout:6/57: dread d1/f7 [0,4194304] 0 2024-05-09T06:38:04.090 INFO:tasks.workunit.client.1.smithi193.stdout:7/55: truncate d6/dd/fe 312560 0 2024-05-09T06:38:04.090 INFO:tasks.workunit.client.1.smithi193.stdout:7/56: chown d6/l7 257008 1 2024-05-09T06:38:04.096 INFO:tasks.workunit.client.0.smithi063.stdout:5/114: write d8/d17/d18/f1f [4325968,58595] 0 2024-05-09T06:38:04.099 INFO:tasks.workunit.client.1.smithi193.stdout:0/101: creat d18/d1d/f1e x:0 0 0 2024-05-09T06:38:04.100 INFO:tasks.workunit.client.1.smithi193.stdout:0/102: write fe [5388082,87245] 0 2024-05-09T06:38:04.100 INFO:tasks.workunit.client.1.smithi193.stdout:0/103: fdatasync f16 0 2024-05-09T06:38:04.100 INFO:tasks.workunit.client.1.smithi193.stdout:6/58: mkdir d1/de/d11 0 2024-05-09T06:38:04.103 INFO:tasks.workunit.client.1.smithi193.stdout:7/57: mknod d6/d8/db/c15 0 2024-05-09T06:38:04.103 INFO:tasks.workunit.client.0.smithi063.stdout:7/129: dwrite d0/ff [4194304,4194304] 0 2024-05-09T06:38:04.104 INFO:tasks.workunit.client.0.smithi063.stdout:9/133: rmdir d0/d8 39 2024-05-09T06:38:04.105 INFO:tasks.workunit.client.0.smithi063.stdout:9/134: read d0/f1f [1509172,15090] 0 2024-05-09T06:38:04.106 INFO:tasks.workunit.client.1.smithi193.stdout:0/104: creat d18/d1d/f1f x:0 0 0 2024-05-09T06:38:04.107 INFO:tasks.workunit.client.0.smithi063.stdout:4/165: truncate d7/df/d1a/f1e 2237326 0 2024-05-09T06:38:04.108 INFO:tasks.workunit.client.0.smithi063.stdout:9/135: write d0/d4/da/d1a/f1d [1013084,99982] 0 2024-05-09T06:38:04.109 INFO:tasks.workunit.client.1.smithi193.stdout:6/59: write d1/f9 [420458,92032] 0 2024-05-09T06:38:04.110 INFO:tasks.workunit.client.0.smithi063.stdout:2/141: dwrite f2 [0,4194304] 0 2024-05-09T06:38:04.110 INFO:tasks.workunit.client.1.smithi193.stdout:6/60: chown d1/f2 398 1 2024-05-09T06:38:04.112 INFO:tasks.workunit.client.1.smithi193.stdout:6/61: read d1/fa [2849924,129496] 0 2024-05-09T06:38:04.112 INFO:tasks.workunit.client.1.smithi193.stdout:7/58: creat d6/f16 x:0 0 0 2024-05-09T06:38:04.114 INFO:tasks.workunit.client.1.smithi193.stdout:6/62: dread d1/f9 [0,4194304] 0 2024-05-09T06:38:04.116 INFO:tasks.workunit.client.1.smithi193.stdout:0/105: mknod d18/d1d/c20 0 2024-05-09T06:38:04.119 INFO:tasks.workunit.client.0.smithi063.stdout:9/136: dwrite d0/d4/da/f15 [4194304,4194304] 0 2024-05-09T06:38:04.119 INFO:tasks.workunit.client.0.smithi063.stdout:6/137: link d2/d4/f10 d2/d1f/d29/f2a 0 2024-05-09T06:38:04.120 INFO:tasks.workunit.client.0.smithi063.stdout:8/196: dwrite d9/da/d18/f2d [0,4194304] 0 2024-05-09T06:38:04.120 INFO:tasks.workunit.client.1.smithi193.stdout:0/106: chown d18/d1d/f1e 22146 1 2024-05-09T06:38:04.126 INFO:tasks.workunit.client.1.smithi193.stdout:0/107: write d18/d1d/f1f [744434,62761] 0 2024-05-09T06:38:04.129 INFO:tasks.workunit.client.0.smithi063.stdout:0/154: symlink dd/d1a/d2c/l30 0 2024-05-09T06:38:04.129 INFO:tasks.workunit.client.1.smithi193.stdout:7/59: dwrite d6/f16 [0,4194304] 0 2024-05-09T06:38:04.132 INFO:tasks.workunit.client.1.smithi193.stdout:6/63: dwrite d1/f5 [0,4194304] 0 2024-05-09T06:38:04.133 INFO:tasks.workunit.client.0.smithi063.stdout:3/128: link d7/d1e/d25/f26 d7/da/d21/f2c 0 2024-05-09T06:38:04.133 INFO:tasks.workunit.client.0.smithi063.stdout:3/129: dread - d7/da/d21/f2c zero size 2024-05-09T06:38:04.139 INFO:tasks.workunit.client.0.smithi063.stdout:3/130: dwrite d7/d22/f27 [0,4194304] 0 2024-05-09T06:38:04.142 INFO:tasks.workunit.client.0.smithi063.stdout:7/130: rename d0/l1e to d0/d17/l2a 0 2024-05-09T06:38:04.149 INFO:tasks.workunit.client.1.smithi193.stdout:0/108: dread f9 [0,4194304] 0 2024-05-09T06:38:04.151 INFO:tasks.workunit.client.1.smithi193.stdout:0/109: write fe [3673893,61536] 0 2024-05-09T06:38:04.154 INFO:tasks.workunit.client.1.smithi193.stdout:6/64: dread d1/fa [0,4194304] 0 2024-05-09T06:38:04.159 INFO:tasks.workunit.client.1.smithi193.stdout:0/110: creat d18/d1d/f21 x:0 0 0 2024-05-09T06:38:04.162 INFO:tasks.workunit.client.1.smithi193.stdout:0/111: mknod d18/d1d/c22 0 2024-05-09T06:38:04.164 INFO:tasks.workunit.client.0.smithi063.stdout:1/132: creat d9/d1a/f29 x:0 0 0 2024-05-09T06:38:04.166 INFO:tasks.workunit.client.0.smithi063.stdout:0/155: mkdir dd/d1a/d22/d23/d31 0 2024-05-09T06:38:04.167 INFO:tasks.workunit.client.0.smithi063.stdout:0/156: readlink dd/l27 0 2024-05-09T06:38:04.169 INFO:tasks.workunit.client.1.smithi193.stdout:6/65: getdents d1/de 0 2024-05-09T06:38:04.171 INFO:tasks.workunit.client.1.smithi193.stdout:6/66: mknod d1/de/c12 0 2024-05-09T06:38:04.173 INFO:tasks.workunit.client.0.smithi063.stdout:4/166: rename f4 to d7/df/f2f 0 2024-05-09T06:38:04.174 INFO:tasks.workunit.client.1.smithi193.stdout:0/112: getdents d18 0 2024-05-09T06:38:04.174 INFO:tasks.workunit.client.1.smithi193.stdout:0/113: truncate f13 327665 0 2024-05-09T06:38:04.175 INFO:tasks.workunit.client.0.smithi063.stdout:7/131: creat d0/d15/f2b x:0 0 0 2024-05-09T06:38:04.176 INFO:tasks.workunit.client.1.smithi193.stdout:6/67: unlink d1/cb 0 2024-05-09T06:38:04.177 INFO:tasks.workunit.client.1.smithi193.stdout:6/68: write d1/de/f10 [763034,33256] 0 2024-05-09T06:38:04.178 INFO:tasks.workunit.client.1.smithi193.stdout:0/114: creat d18/d1d/f23 x:0 0 0 2024-05-09T06:38:04.181 INFO:tasks.workunit.client.1.smithi193.stdout:0/115: rmdir d18 39 2024-05-09T06:38:04.182 INFO:tasks.workunit.client.1.smithi193.stdout:7/60: write d6/d8/f9 [258815,30942] 0 2024-05-09T06:38:04.182 INFO:tasks.workunit.client.1.smithi193.stdout:6/69: dread d1/f7 [0,4194304] 0 2024-05-09T06:38:04.183 INFO:tasks.workunit.client.0.smithi063.stdout:8/197: creat d9/d41/f43 x:0 0 0 2024-05-09T06:38:04.186 INFO:tasks.workunit.client.0.smithi063.stdout:2/142: write f3 [1393067,52506] 0 2024-05-09T06:38:04.189 INFO:tasks.workunit.client.0.smithi063.stdout:3/131: mkdir d7/d2a/d2d 0 2024-05-09T06:38:04.192 INFO:tasks.workunit.client.0.smithi063.stdout:3/132: write d7/da/fe [1642138,65643] 0 2024-05-09T06:38:04.192 INFO:tasks.workunit.client.0.smithi063.stdout:4/167: unlink d7/f24 0 2024-05-09T06:38:04.192 INFO:tasks.workunit.client.0.smithi063.stdout:8/198: dwrite d9/da/d18/d24/f3c [0,4194304] 0 2024-05-09T06:38:04.193 INFO:tasks.workunit.client.1.smithi193.stdout:7/61: truncate d6/dd/fe 466427 0 2024-05-09T06:38:04.194 INFO:tasks.workunit.client.0.smithi063.stdout:4/168: fsync d7/df/d1a/f20 0 2024-05-09T06:38:04.194 INFO:tasks.workunit.client.0.smithi063.stdout:2/143: dread f2 [0,4194304] 0 2024-05-09T06:38:04.194 INFO:tasks.workunit.client.1.smithi193.stdout:7/62: dread - d6/f13 zero size 2024-05-09T06:38:04.195 INFO:tasks.workunit.client.1.smithi193.stdout:7/63: fdatasync d6/f13 0 2024-05-09T06:38:04.195 INFO:tasks.workunit.client.0.smithi063.stdout:2/144: read - dc/f1c zero size 2024-05-09T06:38:04.195 INFO:tasks.workunit.client.0.smithi063.stdout:9/137: creat d0/f24 x:0 0 0 2024-05-09T06:38:04.196 INFO:tasks.workunit.client.1.smithi193.stdout:7/64: write d6/f13 [661137,79204] 0 2024-05-09T06:38:04.201 INFO:tasks.workunit.client.0.smithi063.stdout:1/133: mkdir d9/d2a 0 2024-05-09T06:38:04.203 INFO:tasks.workunit.client.0.smithi063.stdout:0/157: rename dd/c14 to dd/d1a/d22/c32 0 2024-05-09T06:38:04.209 INFO:tasks.workunit.client.0.smithi063.stdout:7/132: link d0/d15/f25 d0/d17/f2c 0 2024-05-09T06:38:04.210 INFO:tasks.workunit.client.1.smithi193.stdout:0/116: rmdir d18/d1d 39 2024-05-09T06:38:04.212 INFO:tasks.workunit.client.0.smithi063.stdout:4/169: symlink d7/df/d1f/l30 0 2024-05-09T06:38:04.212 INFO:tasks.workunit.client.0.smithi063.stdout:6/138: truncate d2/df/f14 323713 0 2024-05-09T06:38:04.213 INFO:tasks.workunit.client.0.smithi063.stdout:2/145: symlink dc/dd/l2c 0 2024-05-09T06:38:04.215 INFO:tasks.workunit.client.0.smithi063.stdout:6/139: dread - d2/d20/f21 zero size 2024-05-09T06:38:04.216 INFO:tasks.workunit.client.0.smithi063.stdout:7/133: dwrite d0/f18 [0,4194304] 0 2024-05-09T06:38:04.217 INFO:tasks.workunit.client.0.smithi063.stdout:9/138: read d0/d4/da/f21 [1962605,73860] 0 2024-05-09T06:38:04.218 INFO:tasks.workunit.client.0.smithi063.stdout:9/139: chown d0/f13 68120287 1 2024-05-09T06:38:04.218 INFO:tasks.workunit.client.1.smithi193.stdout:0/117: rename f12 to d18/d1d/f24 0 2024-05-09T06:38:04.219 INFO:tasks.workunit.client.1.smithi193.stdout:0/118: write d18/d1d/f23 [533989,115881] 0 2024-05-09T06:38:04.221 INFO:tasks.workunit.client.0.smithi063.stdout:1/134: mkdir d9/d10/d2b 0 2024-05-09T06:38:04.226 INFO:tasks.workunit.client.0.smithi063.stdout:0/158: rmdir dd/d1a/d22 39 2024-05-09T06:38:04.226 INFO:tasks.workunit.client.0.smithi063.stdout:0/159: readlink dd/l1f 0 2024-05-09T06:38:04.229 INFO:tasks.workunit.client.1.smithi193.stdout:7/65: truncate f5 3051753 0 2024-05-09T06:38:04.230 INFO:tasks.workunit.client.1.smithi193.stdout:7/66: chown d6/dd/f14 1664815 1 2024-05-09T06:38:04.231 INFO:tasks.workunit.client.0.smithi063.stdout:4/170: mkdir d7/df/d1f/d28/d31 0 2024-05-09T06:38:04.231 INFO:tasks.workunit.client.0.smithi063.stdout:3/133: dread f2 [0,4194304] 0 2024-05-09T06:38:04.233 INFO:tasks.workunit.client.0.smithi063.stdout:2/146: fdatasync dc/f2a 0 2024-05-09T06:38:04.237 INFO:tasks.workunit.client.1.smithi193.stdout:7/67: dwrite d6/f13 [0,4194304] 0 2024-05-09T06:38:04.237 INFO:tasks.workunit.client.0.smithi063.stdout:4/171: dwrite f6 [0,4194304] 0 2024-05-09T06:38:04.238 INFO:tasks.workunit.client.1.smithi193.stdout:0/119: creat d18/f25 x:0 0 0 2024-05-09T06:38:04.238 INFO:tasks.workunit.client.1.smithi193.stdout:7/68: chown d6/f13 38271 1 2024-05-09T06:38:04.239 INFO:tasks.workunit.client.1.smithi193.stdout:7/69: readlink d6/l7 0 2024-05-09T06:38:04.244 INFO:tasks.workunit.client.0.smithi063.stdout:7/134: rename d0/d15/l27 to d0/l2d 0 2024-05-09T06:38:04.250 INFO:tasks.workunit.client.1.smithi193.stdout:7/70: dwrite d6/f16 [4194304,4194304] 0 2024-05-09T06:38:04.262 INFO:tasks.workunit.client.0.smithi063.stdout:8/199: link d9/da/d18/l28 d9/da/d34/l44 0 2024-05-09T06:38:04.262 INFO:tasks.workunit.client.0.smithi063.stdout:8/200: readlink d9/da/d34/l44 0 2024-05-09T06:38:04.271 INFO:tasks.workunit.client.0.smithi063.stdout:2/147: rmdir dc/dd 39 2024-05-09T06:38:04.272 INFO:tasks.workunit.client.0.smithi063.stdout:7/135: dread d0/f14 [0,4194304] 0 2024-05-09T06:38:04.272 INFO:tasks.workunit.client.1.smithi193.stdout:7/71: dwrite d6/dd/ff [0,4194304] 0 2024-05-09T06:38:04.274 INFO:tasks.workunit.client.0.smithi063.stdout:6/140: symlink d2/d20/d26/l2b 0 2024-05-09T06:38:04.278 INFO:tasks.workunit.client.0.smithi063.stdout:6/141: dread d2/d1f/d29/f2a [0,4194304] 0 2024-05-09T06:38:04.282 INFO:tasks.workunit.client.0.smithi063.stdout:1/135: mknod d9/d1a/c2c 0 2024-05-09T06:38:04.283 INFO:tasks.workunit.client.0.smithi063.stdout:0/160: mkdir dd/d1a/d22/d23/d33 0 2024-05-09T06:38:04.284 INFO:tasks.workunit.client.0.smithi063.stdout:8/201: readlink d9/d3b/l2e 0 2024-05-09T06:38:04.285 INFO:tasks.workunit.client.0.smithi063.stdout:3/134: symlink d7/d2a/d2d/l2e 0 2024-05-09T06:38:04.286 INFO:tasks.workunit.client.0.smithi063.stdout:3/135: write d7/da/f18 [228345,85317] 0 2024-05-09T06:38:04.288 INFO:tasks.workunit.client.0.smithi063.stdout:5/115: sync 2024-05-09T06:38:04.293 INFO:tasks.workunit.client.0.smithi063.stdout:7/136: creat d0/d17/f2e x:0 0 0 2024-05-09T06:38:04.294 INFO:tasks.workunit.client.0.smithi063.stdout:7/137: chown d0/f10 675681 1 2024-05-09T06:38:04.298 INFO:tasks.workunit.client.0.smithi063.stdout:9/140: truncate d0/d4/da/f1e 2294109 0 2024-05-09T06:38:04.299 INFO:tasks.workunit.client.0.smithi063.stdout:8/202: dread d9/f14 [0,4194304] 0 2024-05-09T06:38:04.303 INFO:tasks.workunit.client.0.smithi063.stdout:6/142: mknod d2/d4/c2c 0 2024-05-09T06:38:04.306 INFO:tasks.workunit.client.0.smithi063.stdout:6/143: dread f1 [0,4194304] 0 2024-05-09T06:38:04.307 INFO:tasks.workunit.client.0.smithi063.stdout:6/144: truncate d2/df/f1d 176723 0 2024-05-09T06:38:04.321 INFO:tasks.workunit.client.0.smithi063.stdout:1/136: mknod d9/d1a/c2d 0 2024-05-09T06:38:04.321 INFO:tasks.workunit.client.0.smithi063.stdout:1/137: chown d9/d2a 248050 1 2024-05-09T06:38:04.324 INFO:tasks.workunit.client.0.smithi063.stdout:6/145: mkdir d2/d4/d2d 0 2024-05-09T06:38:04.325 INFO:tasks.workunit.client.0.smithi063.stdout:1/138: dread d9/f23 [0,4194304] 0 2024-05-09T06:38:04.326 INFO:tasks.workunit.client.0.smithi063.stdout:2/148: truncate f6 2926113 0 2024-05-09T06:38:04.329 INFO:tasks.workunit.client.0.smithi063.stdout:7/138: link d0/d17/f23 d0/f2f 0 2024-05-09T06:38:04.329 INFO:tasks.workunit.client.0.smithi063.stdout:7/139: chown d0/d15/f20 0 1 2024-05-09T06:38:04.330 INFO:tasks.workunit.client.0.smithi063.stdout:4/172: mkdir d7/df/d1f/d28/d31/d32 0 2024-05-09T06:38:04.334 INFO:tasks.workunit.client.0.smithi063.stdout:0/161: link dd/c18 dd/d1a/d22/d23/d31/c34 0 2024-05-09T06:38:04.339 INFO:tasks.workunit.client.0.smithi063.stdout:3/136: creat d7/f2f x:0 0 0 2024-05-09T06:38:04.339 INFO:tasks.workunit.client.0.smithi063.stdout:3/137: dread - d7/da/f29 zero size 2024-05-09T06:38:04.340 INFO:tasks.workunit.client.0.smithi063.stdout:2/149: truncate dc/f12 394522 0 2024-05-09T06:38:04.340 INFO:tasks.workunit.client.0.smithi063.stdout:2/150: dread - dc/f1c zero size 2024-05-09T06:38:04.349 INFO:tasks.workunit.client.0.smithi063.stdout:4/173: dread d7/df/f2f [0,4194304] 0 2024-05-09T06:38:04.350 INFO:tasks.workunit.client.0.smithi063.stdout:9/141: truncate d0/d4/da/f15 3945527 0 2024-05-09T06:38:04.350 INFO:tasks.workunit.client.0.smithi063.stdout:4/174: stat d7/df/d1a/f20 0 2024-05-09T06:38:04.351 INFO:tasks.workunit.client.0.smithi063.stdout:0/162: chown dd/d1a/d22/c24 124654 1 2024-05-09T06:38:04.352 INFO:tasks.workunit.client.0.smithi063.stdout:4/175: write d7/df/d1f/d28/f2e [639746,28943] 0 2024-05-09T06:38:04.352 INFO:tasks.workunit.client.0.smithi063.stdout:6/146: symlink d2/d4/d2d/l2e 0 2024-05-09T06:38:04.354 INFO:tasks.workunit.client.0.smithi063.stdout:6/147: chown d2/d20/f21 6961 1 2024-05-09T06:38:04.354 INFO:tasks.workunit.client.0.smithi063.stdout:1/139: mkdir d9/d2a/d2e 0 2024-05-09T06:38:04.355 INFO:tasks.workunit.client.0.smithi063.stdout:6/148: fdatasync d2/d6/f1b 0 2024-05-09T06:38:04.356 INFO:tasks.workunit.client.0.smithi063.stdout:1/140: write d9/d10/f21 [922678,36914] 0 2024-05-09T06:38:04.359 INFO:tasks.workunit.client.0.smithi063.stdout:0/163: dwrite dd/d1a/f25 [0,4194304] 0 2024-05-09T06:38:04.362 INFO:tasks.workunit.client.0.smithi063.stdout:1/141: dread d9/d10/f1c [0,4194304] 0 2024-05-09T06:38:04.366 INFO:tasks.workunit.client.0.smithi063.stdout:1/142: dread d9/d1a/d1e/f25 [0,4194304] 0 2024-05-09T06:38:04.380 INFO:tasks.workunit.client.0.smithi063.stdout:6/149: symlink d2/df/l2f 0 2024-05-09T06:38:04.384 INFO:tasks.workunit.client.0.smithi063.stdout:9/142: write d0/f1f [790719,23612] 0 2024-05-09T06:38:04.384 INFO:tasks.workunit.client.0.smithi063.stdout:2/151: mkdir dc/dd/d2d 0 2024-05-09T06:38:04.389 INFO:tasks.workunit.client.0.smithi063.stdout:2/152: dwrite dc/f1b [0,4194304] 0 2024-05-09T06:38:04.392 INFO:tasks.workunit.client.0.smithi063.stdout:3/138: truncate d7/da/f18 197605 0 2024-05-09T06:38:04.394 INFO:tasks.workunit.client.0.smithi063.stdout:3/139: dread d7/f1b [0,4194304] 0 2024-05-09T06:38:04.395 INFO:tasks.workunit.client.0.smithi063.stdout:1/143: dread d9/f23 [0,4194304] 0 2024-05-09T06:38:04.395 INFO:tasks.workunit.client.0.smithi063.stdout:1/144: fsync f8 0 2024-05-09T06:38:04.400 INFO:tasks.workunit.client.0.smithi063.stdout:3/140: dwrite d7/f17 [0,4194304] 0 2024-05-09T06:38:04.405 INFO:tasks.workunit.client.0.smithi063.stdout:7/140: getdents d0 0 2024-05-09T06:38:04.406 INFO:tasks.workunit.client.0.smithi063.stdout:7/141: dread d0/d17/f22 [0,4194304] 0 2024-05-09T06:38:04.407 INFO:tasks.workunit.client.0.smithi063.stdout:7/142: dread - d0/f2f zero size 2024-05-09T06:38:04.408 INFO:tasks.workunit.client.0.smithi063.stdout:7/143: chown d0/f21 11 1 2024-05-09T06:38:04.413 INFO:tasks.workunit.client.0.smithi063.stdout:2/153: chown f3 42976 1 2024-05-09T06:38:04.424 INFO:tasks.workunit.client.0.smithi063.stdout:1/145: symlink d9/d1a/d1e/l2f 0 2024-05-09T06:38:04.437 INFO:tasks.workunit.client.0.smithi063.stdout:7/144: unlink d0/d17/f2c 0 2024-05-09T06:38:04.440 INFO:tasks.workunit.client.0.smithi063.stdout:0/164: creat dd/d1a/f35 x:0 0 0 2024-05-09T06:38:04.443 INFO:tasks.workunit.client.0.smithi063.stdout:9/143: truncate d0/d4/da/d1a/f1d 513512 0 2024-05-09T06:38:04.446 INFO:tasks.workunit.client.0.smithi063.stdout:4/176: getdents d7 0 2024-05-09T06:38:04.453 INFO:tasks.workunit.client.0.smithi063.stdout:3/141: mkdir d7/da/d21/d30 0 2024-05-09T06:38:04.453 INFO:tasks.workunit.client.0.smithi063.stdout:3/142: fsync f5 0 2024-05-09T06:38:04.454 INFO:tasks.workunit.client.0.smithi063.stdout:5/116: sync 2024-05-09T06:38:04.455 INFO:tasks.workunit.client.0.smithi063.stdout:5/117: write f6 [2018346,120708] 0 2024-05-09T06:38:04.456 INFO:tasks.workunit.client.0.smithi063.stdout:7/145: creat d0/d17/f30 x:0 0 0 2024-05-09T06:38:04.456 INFO:tasks.workunit.client.0.smithi063.stdout:7/146: dread - d0/d17/f30 zero size 2024-05-09T06:38:04.459 INFO:tasks.workunit.client.0.smithi063.stdout:2/154: mknod dc/dd/d2d/c2e 0 2024-05-09T06:38:04.461 INFO:tasks.workunit.client.0.smithi063.stdout:4/177: dread f5 [0,4194304] 0 2024-05-09T06:38:04.465 INFO:tasks.workunit.client.0.smithi063.stdout:0/165: fsync dd/d1a/f35 0 2024-05-09T06:38:04.467 INFO:tasks.workunit.client.0.smithi063.stdout:3/143: read d7/f1a [8176,58120] 0 2024-05-09T06:38:04.468 INFO:tasks.workunit.client.0.smithi063.stdout:3/144: write d7/da/f15 [277616,91572] 0 2024-05-09T06:38:04.469 INFO:tasks.workunit.client.0.smithi063.stdout:3/145: write d7/f2f [482069,95179] 0 2024-05-09T06:38:04.470 INFO:tasks.workunit.client.0.smithi063.stdout:5/118: readlink d8/d17/l1b 0 2024-05-09T06:38:04.471 INFO:tasks.workunit.client.0.smithi063.stdout:5/119: truncate d8/d19/f1a 4281786 0 2024-05-09T06:38:04.477 INFO:tasks.workunit.client.0.smithi063.stdout:2/155: creat dc/dd/f2f x:0 0 0 2024-05-09T06:38:04.482 INFO:tasks.workunit.client.0.smithi063.stdout:2/156: dread dc/dd/f17 [0,4194304] 0 2024-05-09T06:38:04.482 INFO:tasks.workunit.client.0.smithi063.stdout:0/166: mknod dd/d1a/d22/d23/c36 0 2024-05-09T06:38:04.484 INFO:tasks.workunit.client.0.smithi063.stdout:5/120: write d8/d17/d1e/f23 [4087169,59542] 0 2024-05-09T06:38:04.487 INFO:tasks.workunit.client.0.smithi063.stdout:9/144: rename d0/d4/l11 to d0/l25 0 2024-05-09T06:38:04.489 INFO:tasks.workunit.client.0.smithi063.stdout:1/146: link d9/l19 d9/d2a/l30 0 2024-05-09T06:38:04.490 INFO:tasks.workunit.client.0.smithi063.stdout:1/147: chown c0 12199 1 2024-05-09T06:38:04.491 INFO:tasks.workunit.client.0.smithi063.stdout:2/157: mknod dc/dd/c30 0 2024-05-09T06:38:04.492 INFO:tasks.workunit.client.0.smithi063.stdout:2/158: dread - dc/dd/f2f zero size 2024-05-09T06:38:04.496 INFO:tasks.workunit.client.0.smithi063.stdout:5/121: stat d8/f16 0 2024-05-09T06:38:04.496 INFO:tasks.workunit.client.0.smithi063.stdout:7/147: dwrite d0/f14 [0,4194304] 0 2024-05-09T06:38:04.497 INFO:tasks.workunit.client.0.smithi063.stdout:7/148: fsync d0/f18 0 2024-05-09T06:38:04.506 INFO:tasks.workunit.client.0.smithi063.stdout:2/159: fsync dc/dd/f17 0 2024-05-09T06:38:04.507 INFO:tasks.workunit.client.0.smithi063.stdout:0/167: symlink dd/d1a/d22/d23/d31/l37 0 2024-05-09T06:38:04.508 INFO:tasks.workunit.client.0.smithi063.stdout:0/168: truncate dd/f2d 967520 0 2024-05-09T06:38:04.513 INFO:tasks.workunit.client.0.smithi063.stdout:7/149: mknod d0/c31 0 2024-05-09T06:38:04.514 INFO:tasks.workunit.client.0.smithi063.stdout:6/150: sync 2024-05-09T06:38:04.514 INFO:tasks.workunit.client.0.smithi063.stdout:8/203: sync 2024-05-09T06:38:04.515 INFO:tasks.workunit.client.0.smithi063.stdout:9/145: truncate d0/d4/da/f1e 888548 0 2024-05-09T06:38:04.515 INFO:tasks.workunit.client.0.smithi063.stdout:3/146: truncate d7/f11 3261562 0 2024-05-09T06:38:04.516 INFO:tasks.workunit.client.0.smithi063.stdout:5/122: dwrite d8/d19/f1a [0,4194304] 0 2024-05-09T06:38:04.521 INFO:tasks.workunit.client.0.smithi063.stdout:1/148: rename d9/lf to d9/d2a/l31 0 2024-05-09T06:38:04.523 INFO:tasks.workunit.client.0.smithi063.stdout:5/123: dread d8/d17/d1e/f23 [0,4194304] 0 2024-05-09T06:38:04.528 INFO:tasks.workunit.client.0.smithi063.stdout:6/151: mknod d2/d20/c30 0 2024-05-09T06:38:04.530 INFO:tasks.workunit.client.0.smithi063.stdout:9/146: readlink d0/d4/da/l14 0 2024-05-09T06:38:04.531 INFO:tasks.workunit.client.0.smithi063.stdout:3/147: mkdir d7/d1e/d31 0 2024-05-09T06:38:04.531 INFO:tasks.workunit.client.0.smithi063.stdout:3/148: stat f2 0 2024-05-09T06:38:04.533 INFO:tasks.workunit.client.0.smithi063.stdout:3/149: write d7/da/f15 [752202,79736] 0 2024-05-09T06:38:04.533 INFO:tasks.workunit.client.0.smithi063.stdout:2/160: mknod dc/c31 0 2024-05-09T06:38:04.533 INFO:tasks.workunit.client.0.smithi063.stdout:2/161: readlink l5 0 2024-05-09T06:38:04.534 INFO:tasks.workunit.client.0.smithi063.stdout:0/169: symlink dd/d1a/d22/d23/d33/l38 0 2024-05-09T06:38:04.534 INFO:tasks.workunit.client.0.smithi063.stdout:2/162: dread - dc/f27 zero size 2024-05-09T06:38:04.535 INFO:tasks.workunit.client.0.smithi063.stdout:1/149: unlink d9/d1a/l28 0 2024-05-09T06:38:04.536 INFO:tasks.workunit.client.0.smithi063.stdout:2/163: fdatasync dc/dd/f17 0 2024-05-09T06:38:04.537 INFO:tasks.workunit.client.0.smithi063.stdout:1/150: chown d9/d10/c16 30142352 1 2024-05-09T06:38:04.539 INFO:tasks.workunit.client.0.smithi063.stdout:6/152: symlink d2/d1f/l31 0 2024-05-09T06:38:04.540 INFO:tasks.workunit.client.0.smithi063.stdout:9/147: mkdir d0/d4/d26 0 2024-05-09T06:38:04.541 INFO:tasks.workunit.client.0.smithi063.stdout:1/151: read d9/d10/f22 [1872598,68142] 0 2024-05-09T06:38:04.542 INFO:tasks.workunit.client.0.smithi063.stdout:9/148: fsync d0/f13 0 2024-05-09T06:38:04.542 INFO:tasks.workunit.client.0.smithi063.stdout:8/204: dread d9/f13 [4194304,4194304] 0 2024-05-09T06:38:04.544 INFO:tasks.workunit.client.0.smithi063.stdout:2/164: rmdir dc 39 2024-05-09T06:38:04.545 INFO:tasks.workunit.client.0.smithi063.stdout:0/170: dwrite dd/d1a/d22/f26 [0,4194304] 0 2024-05-09T06:38:04.545 INFO:tasks.workunit.client.0.smithi063.stdout:3/150: chown d7/da/f18 184471 1 2024-05-09T06:38:04.545 INFO:tasks.workunit.client.0.smithi063.stdout:7/150: link d0/f2f d0/f32 0 2024-05-09T06:38:04.547 INFO:tasks.workunit.client.0.smithi063.stdout:8/205: chown d9/da/d18/d24/f3e 900 1 2024-05-09T06:38:04.547 INFO:tasks.workunit.client.0.smithi063.stdout:0/171: chown f6 120486379 1 2024-05-09T06:38:04.547 INFO:tasks.workunit.client.0.smithi063.stdout:4/178: sync 2024-05-09T06:38:04.549 INFO:tasks.workunit.client.0.smithi063.stdout:1/152: chown d9/l19 84 1 2024-05-09T06:38:04.549 INFO:tasks.workunit.client.0.smithi063.stdout:4/179: chown d7/df/d1f/d28/c2c 47 1 2024-05-09T06:38:04.557 INFO:tasks.workunit.client.0.smithi063.stdout:2/165: truncate dc/dd/f2f 779441 0 2024-05-09T06:38:04.560 INFO:tasks.workunit.client.0.smithi063.stdout:3/151: mknod d7/d22/c32 0 2024-05-09T06:38:04.561 INFO:tasks.workunit.client.0.smithi063.stdout:6/153: dread d2/d6/f1b [0,4194304] 0 2024-05-09T06:38:04.561 INFO:tasks.workunit.client.0.smithi063.stdout:3/152: chown d7/d22 860 1 2024-05-09T06:38:04.562 INFO:tasks.workunit.client.0.smithi063.stdout:6/154: chown d2/d6/f1b 0 1 2024-05-09T06:38:04.567 INFO:tasks.workunit.client.0.smithi063.stdout:7/151: rename d0/d15/c24 to d0/d15/c33 0 2024-05-09T06:38:04.572 INFO:tasks.workunit.client.0.smithi063.stdout:0/172: fdatasync dd/d1a/d22/f26 0 2024-05-09T06:38:04.573 INFO:tasks.workunit.client.0.smithi063.stdout:5/124: getdents d8/d17/d1e 0 2024-05-09T06:38:04.578 INFO:tasks.workunit.client.0.smithi063.stdout:9/149: symlink d0/d4/da/l27 0 2024-05-09T06:38:04.578 INFO:tasks.workunit.client.0.smithi063.stdout:5/125: dwrite d8/d17/f1c [0,4194304] 0 2024-05-09T06:38:04.579 INFO:tasks.workunit.client.0.smithi063.stdout:5/126: readlink d8/l13 0 2024-05-09T06:38:04.579 INFO:tasks.workunit.client.0.smithi063.stdout:2/166: rmdir dc 39 2024-05-09T06:38:04.581 INFO:tasks.workunit.client.0.smithi063.stdout:3/153: fdatasync d7/f1a 0 2024-05-09T06:38:04.583 INFO:tasks.workunit.client.0.smithi063.stdout:3/154: write d7/d22/f27 [5209596,104323] 0 2024-05-09T06:38:04.584 INFO:tasks.workunit.client.0.smithi063.stdout:6/155: symlink d2/d20/d26/l32 0 2024-05-09T06:38:04.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/206: mkdir d9/da/d45 0 2024-05-09T06:38:04.590 INFO:tasks.workunit.client.0.smithi063.stdout:8/207: dread d9/f13 [4194304,4194304] 0 2024-05-09T06:38:04.591 INFO:tasks.workunit.client.0.smithi063.stdout:7/152: symlink d0/d17/l34 0 2024-05-09T06:38:04.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/208: stat d9/da/d34 0 2024-05-09T06:38:04.591 INFO:tasks.workunit.client.0.smithi063.stdout:7/153: chown d0/d17/l34 61 1 2024-05-09T06:38:04.595 INFO:tasks.workunit.client.0.smithi063.stdout:0/173: mknod dd/d1a/d22/c39 0 2024-05-09T06:38:04.603 INFO:tasks.workunit.client.0.smithi063.stdout:2/167: write dc/f1b [3398067,84590] 0 2024-05-09T06:38:04.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:04 smithi063 ceph-mon[27171]: pgmap v5: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:38:04.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:04 smithi063 ceph-mon[27171]: mgrmap e24: smithi193.mmmqzh(active, since 4s) 2024-05-09T06:38:04.611 INFO:tasks.workunit.client.0.smithi063.stdout:3/155: rename d7/d1e/d25/f26 to d7/d2a/f33 0 2024-05-09T06:38:04.613 INFO:tasks.workunit.client.0.smithi063.stdout:6/156: mknod d2/d20/d26/c33 0 2024-05-09T06:38:04.614 INFO:tasks.workunit.client.0.smithi063.stdout:8/209: mkdir d9/da/d18/d46 0 2024-05-09T06:38:04.618 INFO:tasks.workunit.client.0.smithi063.stdout:0/174: mknod dd/d1a/d22/d23/c3a 0 2024-05-09T06:38:04.619 INFO:tasks.workunit.client.0.smithi063.stdout:9/150: rmdir d0/d8 39 2024-05-09T06:38:04.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:04 smithi193 ceph-mon[35141]: pgmap v5: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:38:04.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:04 smithi193 ceph-mon[35141]: mgrmap e24: smithi193.mmmqzh(active, since 4s) 2024-05-09T06:38:04.624 INFO:tasks.workunit.client.0.smithi063.stdout:1/153: write d9/d10/f15 [606558,87418] 0 2024-05-09T06:38:04.625 INFO:tasks.workunit.client.0.smithi063.stdout:4/180: dwrite f5 [0,4194304] 0 2024-05-09T06:38:04.626 INFO:tasks.workunit.client.0.smithi063.stdout:1/154: write d9/d10/f21 [105278,22927] 0 2024-05-09T06:38:04.631 INFO:tasks.workunit.client.0.smithi063.stdout:3/156: write d7/da/f14 [788137,29682] 0 2024-05-09T06:38:04.636 INFO:tasks.workunit.client.0.smithi063.stdout:6/157: creat d2/d12/f34 x:0 0 0 2024-05-09T06:38:04.638 INFO:tasks.workunit.client.0.smithi063.stdout:8/210: write f2 [1339641,58682] 0 2024-05-09T06:38:04.641 INFO:tasks.workunit.client.0.smithi063.stdout:6/158: dread d2/d4/f10 [0,4194304] 0 2024-05-09T06:38:04.643 INFO:tasks.workunit.client.0.smithi063.stdout:0/175: symlink dd/l3b 0 2024-05-09T06:38:04.648 INFO:tasks.workunit.client.0.smithi063.stdout:6/159: dwrite d2/d4/f1c [0,4194304] 0 2024-05-09T06:38:04.650 INFO:tasks.workunit.client.0.smithi063.stdout:1/155: rmdir d9/d2a 39 2024-05-09T06:38:04.650 INFO:tasks.workunit.client.0.smithi063.stdout:0/176: dwrite dd/d1a/f25 [0,4194304] 0 2024-05-09T06:38:04.654 INFO:tasks.workunit.client.0.smithi063.stdout:3/157: mknod d7/d2a/d2d/c34 0 2024-05-09T06:38:04.657 INFO:tasks.workunit.client.0.smithi063.stdout:3/158: read - d7/da/f29 zero size 2024-05-09T06:38:04.658 INFO:tasks.workunit.client.0.smithi063.stdout:8/211: creat d9/f47 x:0 0 0 2024-05-09T06:38:04.668 INFO:tasks.workunit.client.0.smithi063.stdout:5/127: getdents d8/d19 0 2024-05-09T06:38:04.669 INFO:tasks.workunit.client.0.smithi063.stdout:2/168: dwrite f6 [0,4194304] 0 2024-05-09T06:38:04.669 INFO:tasks.workunit.client.0.smithi063.stdout:5/128: write d8/d17/f24 [643967,103810] 0 2024-05-09T06:38:04.672 INFO:tasks.workunit.client.0.smithi063.stdout:2/169: write dc/dd/f17 [244209,53890] 0 2024-05-09T06:38:04.673 INFO:tasks.workunit.client.0.smithi063.stdout:0/177: dread fc [0,4194304] 0 2024-05-09T06:38:04.677 INFO:tasks.workunit.client.0.smithi063.stdout:7/154: write d0/d15/f25 [306923,90292] 0 2024-05-09T06:38:04.678 INFO:tasks.workunit.client.0.smithi063.stdout:5/129: dread d8/f21 [0,4194304] 0 2024-05-09T06:38:04.681 INFO:tasks.workunit.client.0.smithi063.stdout:7/155: dread d0/f14 [0,4194304] 0 2024-05-09T06:38:04.682 INFO:tasks.workunit.client.0.smithi063.stdout:9/151: creat d0/f28 x:0 0 0 2024-05-09T06:38:04.682 INFO:tasks.workunit.client.0.smithi063.stdout:7/156: fdatasync d0/d15/f26 0 2024-05-09T06:38:04.683 INFO:tasks.workunit.client.0.smithi063.stdout:7/157: chown d0/d15/f25 491 1 2024-05-09T06:38:04.690 INFO:tasks.workunit.client.0.smithi063.stdout:1/156: mknod d9/d10/d2b/c32 0 2024-05-09T06:38:04.693 INFO:tasks.workunit.client.0.smithi063.stdout:0/178: dread f6 [0,4194304] 0 2024-05-09T06:38:04.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/159: mkdir d7/d1e/d31/d35 0 2024-05-09T06:38:04.695 INFO:tasks.workunit.client.0.smithi063.stdout:3/160: readlink d7/da/l19 0 2024-05-09T06:38:04.696 INFO:tasks.workunit.client.0.smithi063.stdout:0/179: dread f6 [0,4194304] 0 2024-05-09T06:38:04.697 INFO:tasks.workunit.client.0.smithi063.stdout:5/130: mknod d8/d17/c25 0 2024-05-09T06:38:04.697 INFO:tasks.workunit.client.0.smithi063.stdout:4/181: write d7/fb [1431664,111327] 0 2024-05-09T06:38:04.699 INFO:tasks.workunit.client.0.smithi063.stdout:4/182: write d7/df/f26 [456491,130157] 0 2024-05-09T06:38:04.700 INFO:tasks.workunit.client.0.smithi063.stdout:9/152: chown d0/d4/da/f1e 1912 1 2024-05-09T06:38:04.700 INFO:tasks.workunit.client.0.smithi063.stdout:4/183: write d7/df/f12 [2456569,78670] 0 2024-05-09T06:38:04.702 INFO:tasks.workunit.client.0.smithi063.stdout:5/131: dread d8/d17/d18/f22 [0,4194304] 0 2024-05-09T06:38:04.702 INFO:tasks.workunit.client.0.smithi063.stdout:7/158: creat d0/d15/f35 x:0 0 0 2024-05-09T06:38:04.703 INFO:tasks.workunit.client.0.smithi063.stdout:5/132: readlink l5 0 2024-05-09T06:38:04.703 INFO:tasks.workunit.client.0.smithi063.stdout:7/159: fdatasync d0/d17/f2e 0 2024-05-09T06:38:04.704 INFO:tasks.workunit.client.0.smithi063.stdout:7/160: chown d0/d15/f26 29101672 1 2024-05-09T06:38:04.710 INFO:tasks.workunit.client.0.smithi063.stdout:3/161: mkdir d7/d2a/d2d/d36 0 2024-05-09T06:38:04.711 INFO:tasks.workunit.client.0.smithi063.stdout:6/160: write d2/df/f14 [631899,102276] 0 2024-05-09T06:38:04.712 INFO:tasks.workunit.client.0.smithi063.stdout:0/180: unlink fc 0 2024-05-09T06:38:04.716 INFO:tasks.workunit.client.0.smithi063.stdout:4/184: unlink d7/df/f26 0 2024-05-09T06:38:04.717 INFO:tasks.workunit.client.0.smithi063.stdout:4/185: chown d7/df/d1a/f1e 5 1 2024-05-09T06:38:04.717 INFO:tasks.workunit.client.0.smithi063.stdout:4/186: write d7/df/d1a/f22 [2077633,15053] 0 2024-05-09T06:38:04.719 INFO:tasks.workunit.client.0.smithi063.stdout:4/187: rename d7/df/d1f/d28/d31/d32 to d7/df/d1f/d28/d31/d32/d33 22 2024-05-09T06:38:04.719 INFO:tasks.workunit.client.0.smithi063.stdout:7/161: mkdir d0/d17/d36 0 2024-05-09T06:38:04.720 INFO:tasks.workunit.client.0.smithi063.stdout:7/162: readlink d0/d17/l34 0 2024-05-09T06:38:04.722 INFO:tasks.workunit.client.0.smithi063.stdout:6/161: symlink d2/d1f/l35 0 2024-05-09T06:38:04.723 INFO:tasks.workunit.client.0.smithi063.stdout:6/162: fdatasync d2/fd 0 2024-05-09T06:38:04.724 INFO:tasks.workunit.client.0.smithi063.stdout:0/181: mknod dd/d1a/d22/c3c 0 2024-05-09T06:38:04.725 INFO:tasks.workunit.client.0.smithi063.stdout:8/212: dwrite d9/f36 [0,4194304] 0 2024-05-09T06:38:04.725 INFO:tasks.workunit.client.0.smithi063.stdout:9/153: symlink d0/d8/l29 0 2024-05-09T06:38:04.726 INFO:tasks.workunit.client.1.smithi193.stdout:0/120: getdents d18/d1d 0 2024-05-09T06:38:04.728 INFO:tasks.workunit.client.0.smithi063.stdout:9/154: read d0/d4/da/f21 [688552,78720] 0 2024-05-09T06:38:04.729 INFO:tasks.workunit.client.0.smithi063.stdout:5/133: link d8/f21 d8/d17/f26 0 2024-05-09T06:38:04.730 INFO:tasks.workunit.client.0.smithi063.stdout:2/170: getdents dc/dd/d2d 0 2024-05-09T06:38:04.733 INFO:tasks.workunit.client.0.smithi063.stdout:6/163: dwrite d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:04.733 INFO:tasks.workunit.client.0.smithi063.stdout:9/155: dread d0/d4/da/f1e [0,4194304] 0 2024-05-09T06:38:04.737 INFO:tasks.workunit.client.0.smithi063.stdout:3/162: symlink d7/d1e/d31/d35/l37 0 2024-05-09T06:38:04.737 INFO:tasks.workunit.client.0.smithi063.stdout:3/163: truncate d7/f1a 1360369 0 2024-05-09T06:38:04.738 INFO:tasks.workunit.client.1.smithi193.stdout:0/121: rename d18/d1d/f21 to d18/f26 0 2024-05-09T06:38:04.744 INFO:tasks.workunit.client.0.smithi063.stdout:3/164: dwrite d7/f10 [0,4194304] 0 2024-05-09T06:38:04.746 INFO:tasks.workunit.client.1.smithi193.stdout:0/122: mkdir d18/d27 0 2024-05-09T06:38:04.746 INFO:tasks.workunit.client.0.smithi063.stdout:3/165: read d7/f8 [54643,124666] 0 2024-05-09T06:38:04.748 INFO:tasks.workunit.client.0.smithi063.stdout:4/188: mkdir d7/df/d1f/d28/d31/d32/d34 0 2024-05-09T06:38:04.752 INFO:tasks.workunit.client.0.smithi063.stdout:3/166: dread d7/f2f [0,4194304] 0 2024-05-09T06:38:04.753 INFO:tasks.workunit.client.1.smithi193.stdout:0/123: dread d18/d1d/f24 [0,4194304] 0 2024-05-09T06:38:04.753 INFO:tasks.workunit.client.0.smithi063.stdout:3/167: read - d7/da/f29 zero size 2024-05-09T06:38:04.753 INFO:tasks.workunit.client.0.smithi063.stdout:6/164: mknod d2/d1f/c36 0 2024-05-09T06:38:04.754 INFO:tasks.workunit.client.0.smithi063.stdout:6/165: read d2/df/f14 [392907,96356] 0 2024-05-09T06:38:04.755 INFO:tasks.workunit.client.0.smithi063.stdout:1/157: getdents d9/d1a 0 2024-05-09T06:38:04.755 INFO:tasks.workunit.client.0.smithi063.stdout:6/166: stat d2/c23 0 2024-05-09T06:38:04.758 INFO:tasks.workunit.client.0.smithi063.stdout:8/213: mknod d9/d41/c48 0 2024-05-09T06:38:04.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/124: rename d18/f1b to d18/f28 0 2024-05-09T06:38:04.768 INFO:tasks.workunit.client.0.smithi063.stdout:7/163: rmdir d0/d17/d36 0 2024-05-09T06:38:04.768 INFO:tasks.workunit.client.0.smithi063.stdout:1/158: dread f8 [0,4194304] 0 2024-05-09T06:38:04.769 INFO:tasks.workunit.client.0.smithi063.stdout:4/189: mkdir d7/df/d35 0 2024-05-09T06:38:04.771 INFO:tasks.workunit.client.0.smithi063.stdout:5/134: link d8/fe d8/f27 0 2024-05-09T06:38:04.773 INFO:tasks.workunit.client.1.smithi193.stdout:0/125: mkdir d18/d29 0 2024-05-09T06:38:04.774 INFO:tasks.workunit.client.0.smithi063.stdout:3/168: write f1 [2939550,6578] 0 2024-05-09T06:38:04.776 INFO:tasks.workunit.client.0.smithi063.stdout:6/167: symlink d2/df/l37 0 2024-05-09T06:38:04.777 INFO:tasks.workunit.client.0.smithi063.stdout:6/168: write d2/fd [5112988,15807] 0 2024-05-09T06:38:04.780 INFO:tasks.workunit.client.0.smithi063.stdout:7/164: creat d0/d17/f37 x:0 0 0 2024-05-09T06:38:04.781 INFO:tasks.workunit.client.0.smithi063.stdout:1/159: creat d9/d10/d2b/f33 x:0 0 0 2024-05-09T06:38:04.782 INFO:tasks.workunit.client.0.smithi063.stdout:7/165: write d0/d15/f20 [816850,105150] 0 2024-05-09T06:38:04.783 INFO:tasks.workunit.client.0.smithi063.stdout:4/190: rename d7/df/d1a/f1e to d7/df/d35/f36 0 2024-05-09T06:38:04.784 INFO:tasks.workunit.client.0.smithi063.stdout:6/169: dwrite d2/fd [0,4194304] 0 2024-05-09T06:38:04.784 INFO:tasks.workunit.client.0.smithi063.stdout:7/166: fdatasync d0/ff 0 2024-05-09T06:38:04.785 INFO:tasks.workunit.client.0.smithi063.stdout:5/135: symlink d8/l28 0 2024-05-09T06:38:04.788 INFO:tasks.workunit.client.0.smithi063.stdout:8/214: mknod d9/d3b/c49 0 2024-05-09T06:38:04.788 INFO:tasks.workunit.client.0.smithi063.stdout:3/169: dread - d7/d2a/f33 zero size 2024-05-09T06:38:04.789 INFO:tasks.workunit.client.0.smithi063.stdout:0/182: link dd/d1a/d22/d23/d33/l38 dd/d1a/l3d 0 2024-05-09T06:38:04.790 INFO:tasks.workunit.client.0.smithi063.stdout:3/170: chown d7/d2a 3 1 2024-05-09T06:38:04.792 INFO:tasks.workunit.client.0.smithi063.stdout:2/171: getdents dc/dd 0 2024-05-09T06:38:04.793 INFO:tasks.workunit.client.0.smithi063.stdout:2/172: stat f6 0 2024-05-09T06:38:04.794 INFO:tasks.workunit.client.0.smithi063.stdout:1/160: dwrite d9/d1a/d1e/f27 [0,4194304] 0 2024-05-09T06:38:04.795 INFO:tasks.workunit.client.0.smithi063.stdout:2/173: chown dc/dd/f2f 918 1 2024-05-09T06:38:04.795 INFO:tasks.workunit.client.0.smithi063.stdout:4/191: dwrite d7/df/f21 [0,4194304] 0 2024-05-09T06:38:04.796 INFO:tasks.workunit.client.0.smithi063.stdout:7/167: rmdir d0 39 2024-05-09T06:38:04.799 INFO:tasks.workunit.client.0.smithi063.stdout:5/136: creat d8/d17/d18/f29 x:0 0 0 2024-05-09T06:38:04.800 INFO:tasks.workunit.client.0.smithi063.stdout:5/137: readlink d8/d17/l1b 0 2024-05-09T06:38:04.800 INFO:tasks.workunit.client.0.smithi063.stdout:3/171: truncate d7/f1b 316095 0 2024-05-09T06:38:04.804 INFO:tasks.workunit.client.0.smithi063.stdout:1/161: symlink d9/d10/d2b/l34 0 2024-05-09T06:38:04.807 INFO:tasks.workunit.client.0.smithi063.stdout:8/215: rename d9/da/l10 to d9/da/d45/l4a 0 2024-05-09T06:38:04.812 INFO:tasks.workunit.client.0.smithi063.stdout:6/170: link d2/df/f14 d2/f38 0 2024-05-09T06:38:04.812 INFO:tasks.workunit.client.0.smithi063.stdout:8/216: dread d9/da/f19 [0,4194304] 0 2024-05-09T06:38:04.813 INFO:tasks.workunit.client.0.smithi063.stdout:2/174: mkdir dc/d32 0 2024-05-09T06:38:04.814 INFO:tasks.workunit.client.0.smithi063.stdout:2/175: write dc/dd/f17 [1036793,66470] 0 2024-05-09T06:38:04.815 INFO:tasks.workunit.client.0.smithi063.stdout:5/138: fdatasync d8/d17/d1e/f23 0 2024-05-09T06:38:04.817 INFO:tasks.workunit.client.0.smithi063.stdout:5/139: chown d8/d17/f1c 85 1 2024-05-09T06:38:04.817 INFO:tasks.workunit.client.0.smithi063.stdout:7/168: unlink d0/d17/f22 0 2024-05-09T06:38:04.817 INFO:tasks.workunit.client.0.smithi063.stdout:0/183: dread dd/f12 [4194304,4194304] 0 2024-05-09T06:38:04.818 INFO:tasks.workunit.client.0.smithi063.stdout:5/140: chown d8/fc 18556 1 2024-05-09T06:38:04.822 INFO:tasks.workunit.client.0.smithi063.stdout:3/172: mknod d7/c38 0 2024-05-09T06:38:04.823 INFO:tasks.workunit.client.0.smithi063.stdout:5/141: dwrite d8/f10 [0,4194304] 0 2024-05-09T06:38:04.824 INFO:tasks.workunit.client.0.smithi063.stdout:6/171: symlink d2/d6/l39 0 2024-05-09T06:38:04.828 INFO:tasks.workunit.client.0.smithi063.stdout:2/176: truncate f2 4906061 0 2024-05-09T06:38:04.830 INFO:tasks.workunit.client.0.smithi063.stdout:7/169: symlink d0/l38 0 2024-05-09T06:38:04.831 INFO:tasks.workunit.client.0.smithi063.stdout:5/142: dwrite d8/d17/f24 [0,4194304] 0 2024-05-09T06:38:04.833 INFO:tasks.workunit.client.0.smithi063.stdout:1/162: rename d9/d1a to d9/d2a/d35 0 2024-05-09T06:38:04.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/192: link d7/df/l29 d7/df/l37 0 2024-05-09T06:38:04.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/143: dread d8/d17/f1c [0,4194304] 0 2024-05-09T06:38:04.842 INFO:tasks.workunit.client.0.smithi063.stdout:3/173: creat d7/d1e/d25/f39 x:0 0 0 2024-05-09T06:38:04.843 INFO:tasks.workunit.client.0.smithi063.stdout:3/174: dread - d7/da/d21/f2c zero size 2024-05-09T06:38:04.844 INFO:tasks.workunit.client.0.smithi063.stdout:6/172: mknod d2/d20/c3a 0 2024-05-09T06:38:04.845 INFO:tasks.workunit.client.0.smithi063.stdout:6/173: dread - d2/df/f28 zero size 2024-05-09T06:38:04.846 INFO:tasks.workunit.client.0.smithi063.stdout:7/170: rmdir d0/d15 39 2024-05-09T06:38:04.850 INFO:tasks.workunit.client.0.smithi063.stdout:1/163: symlink d9/d2a/l36 0 2024-05-09T06:38:04.851 INFO:tasks.workunit.client.0.smithi063.stdout:0/184: link f5 dd/d1a/d22/d23/d33/f3e 0 2024-05-09T06:38:04.852 INFO:tasks.workunit.client.0.smithi063.stdout:4/193: mknod d7/df/d1a/c38 0 2024-05-09T06:38:04.854 INFO:tasks.workunit.client.0.smithi063.stdout:5/144: rename d8/d17/c25 to d8/d17/d1e/c2a 0 2024-05-09T06:38:04.855 INFO:tasks.workunit.client.0.smithi063.stdout:3/175: unlink d7/d22/c24 0 2024-05-09T06:38:04.857 INFO:tasks.workunit.client.0.smithi063.stdout:7/171: dread - d0/d17/f23 zero size 2024-05-09T06:38:04.858 INFO:tasks.workunit.client.0.smithi063.stdout:2/177: dread dc/dd/f13 [0,4194304] 0 2024-05-09T06:38:04.858 INFO:tasks.workunit.client.0.smithi063.stdout:3/176: chown d7/da/l19 787 1 2024-05-09T06:38:04.859 INFO:tasks.workunit.client.0.smithi063.stdout:1/164: readlink d9/l13 0 2024-05-09T06:38:04.859 INFO:tasks.workunit.client.0.smithi063.stdout:7/172: readlink d0/l13 0 2024-05-09T06:38:04.859 INFO:tasks.workunit.client.0.smithi063.stdout:0/185: mkdir dd/d1a/d22/d3f 0 2024-05-09T06:38:04.861 INFO:tasks.workunit.client.0.smithi063.stdout:5/145: mknod d8/d17/c2b 0 2024-05-09T06:38:04.862 INFO:tasks.workunit.client.0.smithi063.stdout:6/174: mkdir d2/d1f/d29/d3b 0 2024-05-09T06:38:04.864 INFO:tasks.workunit.client.0.smithi063.stdout:0/186: read - dd/d1a/f35 zero size 2024-05-09T06:38:04.867 INFO:tasks.workunit.client.0.smithi063.stdout:8/217: dwrite d9/da/f2f [0,4194304] 0 2024-05-09T06:38:04.867 INFO:tasks.workunit.client.0.smithi063.stdout:6/175: chown d2/d4/f1c 879327308 1 2024-05-09T06:38:04.868 INFO:tasks.workunit.client.0.smithi063.stdout:3/177: rename d7/da/f15 to d7/d22/d28/f3a 0 2024-05-09T06:38:04.871 INFO:tasks.workunit.client.0.smithi063.stdout:7/173: mknod d0/c39 0 2024-05-09T06:38:04.872 INFO:tasks.workunit.client.0.smithi063.stdout:0/187: chown dd/d1a/l3d 24 1 2024-05-09T06:38:04.873 INFO:tasks.workunit.client.0.smithi063.stdout:2/178: dwrite f6 [0,4194304] 0 2024-05-09T06:38:04.876 INFO:tasks.workunit.client.0.smithi063.stdout:2/179: fsync fa 0 2024-05-09T06:38:04.878 INFO:tasks.workunit.client.0.smithi063.stdout:7/174: write d0/d17/f30 [368988,57385] 0 2024-05-09T06:38:04.878 INFO:tasks.workunit.client.0.smithi063.stdout:0/188: write dd/d1a/f25 [2516921,75798] 0 2024-05-09T06:38:04.882 INFO:tasks.workunit.client.0.smithi063.stdout:6/176: rename d2/d4/f10 to d2/d4/f3c 0 2024-05-09T06:38:04.882 INFO:tasks.workunit.client.0.smithi063.stdout:4/194: dread d7/df/d1a/f22 [0,4194304] 0 2024-05-09T06:38:04.882 INFO:tasks.workunit.client.0.smithi063.stdout:1/165: dwrite d9/d10/d2b/f33 [0,4194304] 0 2024-05-09T06:38:04.885 INFO:tasks.workunit.client.0.smithi063.stdout:5/146: dwrite d8/d17/d18/f1f [0,4194304] 0 2024-05-09T06:38:04.888 INFO:tasks.workunit.client.0.smithi063.stdout:7/175: chown d0/d15/f2b 338 1 2024-05-09T06:38:04.889 INFO:tasks.workunit.client.0.smithi063.stdout:4/195: creat d7/f39 x:0 0 0 2024-05-09T06:38:04.892 INFO:tasks.workunit.client.0.smithi063.stdout:6/177: mknod d2/d4/d2d/c3d 0 2024-05-09T06:38:04.892 INFO:tasks.workunit.client.0.smithi063.stdout:0/189: dread dd/d1a/d22/f26 [0,4194304] 0 2024-05-09T06:38:04.893 INFO:tasks.workunit.client.0.smithi063.stdout:2/180: mkdir dc/d32/d33 0 2024-05-09T06:38:04.894 INFO:tasks.workunit.client.0.smithi063.stdout:0/190: fsync dd/d1a/f1b 0 2024-05-09T06:38:04.895 INFO:tasks.workunit.client.0.smithi063.stdout:1/166: mknod d9/d2a/d35/d1e/c37 0 2024-05-09T06:38:04.897 INFO:tasks.workunit.client.0.smithi063.stdout:2/181: read dc/dd/f2f [227031,103876] 0 2024-05-09T06:38:04.899 INFO:tasks.workunit.client.0.smithi063.stdout:0/191: mknod dd/d1a/d22/c40 0 2024-05-09T06:38:04.900 INFO:tasks.workunit.client.0.smithi063.stdout:2/182: creat dc/dd/d2d/f34 x:0 0 0 2024-05-09T06:38:04.900 INFO:tasks.workunit.client.0.smithi063.stdout:6/178: dwrite f1 [0,4194304] 0 2024-05-09T06:38:04.902 INFO:tasks.workunit.client.0.smithi063.stdout:0/192: chown dd/d1a/d22/d23/d31/c34 65843232 1 2024-05-09T06:38:04.902 INFO:tasks.workunit.client.0.smithi063.stdout:6/179: mknod d2/d4/c3e 0 2024-05-09T06:38:04.903 INFO:tasks.workunit.client.0.smithi063.stdout:2/183: symlink dc/dd/d2d/l35 0 2024-05-09T06:38:04.906 INFO:tasks.workunit.client.0.smithi063.stdout:2/184: mkdir dc/d36 0 2024-05-09T06:38:04.907 INFO:tasks.workunit.client.0.smithi063.stdout:6/180: dread d2/d4/f1c [0,4194304] 0 2024-05-09T06:38:04.908 INFO:tasks.workunit.client.0.smithi063.stdout:2/185: symlink dc/d32/d33/l37 0 2024-05-09T06:38:04.910 INFO:tasks.workunit.client.0.smithi063.stdout:2/186: write dc/f25 [67264,107974] 0 2024-05-09T06:38:04.911 INFO:tasks.workunit.client.0.smithi063.stdout:6/181: dread d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:04.925 INFO:tasks.workunit.client.0.smithi063.stdout:9/156: sync 2024-05-09T06:38:04.927 INFO:tasks.workunit.client.0.smithi063.stdout:9/157: unlink d0/d4/da/d1a/l23 0 2024-05-09T06:38:04.938 INFO:tasks.workunit.client.0.smithi063.stdout:9/158: dread d0/f13 [0,4194304] 0 2024-05-09T06:38:04.953 INFO:tasks.workunit.client.0.smithi063.stdout:7/176: write d0/f32 [77669,105066] 0 2024-05-09T06:38:04.954 INFO:tasks.workunit.client.0.smithi063.stdout:7/177: fsync d0/d17/f29 0 2024-05-09T06:38:04.956 INFO:tasks.workunit.client.0.smithi063.stdout:0/193: write dd/d1a/f1e [862576,32028] 0 2024-05-09T06:38:04.958 INFO:tasks.workunit.client.0.smithi063.stdout:7/178: creat d0/d15/f3a x:0 0 0 2024-05-09T06:38:04.959 INFO:tasks.workunit.client.0.smithi063.stdout:6/182: write d2/d4/f3c [2992210,129180] 0 2024-05-09T06:38:04.960 INFO:tasks.workunit.client.0.smithi063.stdout:0/194: unlink dd/d1a/f1e 0 2024-05-09T06:38:04.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/147: dwrite d8/d17/d1e/f23 [0,4194304] 0 2024-05-09T06:38:04.962 INFO:tasks.workunit.client.0.smithi063.stdout:1/167: dwrite d9/d10/f22 [0,4194304] 0 2024-05-09T06:38:04.966 INFO:tasks.workunit.client.0.smithi063.stdout:5/148: symlink d8/l2c 0 2024-05-09T06:38:04.967 INFO:tasks.workunit.client.0.smithi063.stdout:6/183: chown d2/d6/l16 231 1 2024-05-09T06:38:04.967 INFO:tasks.workunit.client.0.smithi063.stdout:1/168: rmdir d9/d10 39 2024-05-09T06:38:04.967 INFO:tasks.workunit.client.0.smithi063.stdout:7/179: write d0/f1c [992517,70402] 0 2024-05-09T06:38:04.970 INFO:tasks.workunit.client.0.smithi063.stdout:0/195: creat dd/d1a/d22/d23/d31/f41 x:0 0 0 2024-05-09T06:38:04.972 INFO:tasks.workunit.client.0.smithi063.stdout:5/149: creat d8/d17/d18/f2d x:0 0 0 2024-05-09T06:38:04.972 INFO:tasks.workunit.client.0.smithi063.stdout:9/159: dwrite d0/d4/da/f1e [0,4194304] 0 2024-05-09T06:38:04.974 INFO:tasks.workunit.client.0.smithi063.stdout:1/169: fdatasync d9/f11 0 2024-05-09T06:38:04.975 INFO:tasks.workunit.client.0.smithi063.stdout:0/196: mkdir dd/d1a/d2c/d42 0 2024-05-09T06:38:04.978 INFO:tasks.workunit.client.0.smithi063.stdout:9/160: chown d0/c2 4268446 1 2024-05-09T06:38:04.978 INFO:tasks.workunit.client.0.smithi063.stdout:1/170: readlink d9/l13 0 2024-05-09T06:38:04.981 INFO:tasks.workunit.client.0.smithi063.stdout:7/180: dwrite d0/ff [4194304,4194304] 0 2024-05-09T06:38:04.987 INFO:tasks.workunit.client.0.smithi063.stdout:9/161: dwrite d0/f28 [0,4194304] 0 2024-05-09T06:38:05.004 INFO:tasks.workunit.client.0.smithi063.stdout:7/181: dread d0/fc [0,4194304] 0 2024-05-09T06:38:05.030 INFO:tasks.workunit.client.0.smithi063.stdout:8/218: sync 2024-05-09T06:38:05.030 INFO:tasks.workunit.client.0.smithi063.stdout:8/219: stat d9/da/l3d 0 2024-05-09T06:38:05.090 INFO:tasks.workunit.client.0.smithi063.stdout:3/178: sync 2024-05-09T06:38:05.090 INFO:tasks.workunit.client.0.smithi063.stdout:3/179: chown d7/d22 243786227 1 2024-05-09T06:38:05.116 INFO:tasks.workunit.client.0.smithi063.stdout:6/184: rename d2/d6 to d2/d1f/d29/d3f 0 2024-05-09T06:38:05.117 INFO:tasks.workunit.client.0.smithi063.stdout:6/185: write d2/d12/f24 [654013,14339] 0 2024-05-09T06:38:05.117 INFO:tasks.workunit.client.0.smithi063.stdout:5/150: mknod d8/c2e 0 2024-05-09T06:38:05.119 INFO:tasks.workunit.client.0.smithi063.stdout:5/151: write d8/f10 [5016317,45792] 0 2024-05-09T06:38:05.120 INFO:tasks.workunit.client.0.smithi063.stdout:0/197: creat dd/d1a/d2c/f43 x:0 0 0 2024-05-09T06:38:05.122 INFO:tasks.workunit.client.0.smithi063.stdout:4/196: sync 2024-05-09T06:38:05.123 INFO:tasks.workunit.client.0.smithi063.stdout:2/187: sync 2024-05-09T06:38:05.123 INFO:tasks.workunit.client.0.smithi063.stdout:9/162: sync 2024-05-09T06:38:05.124 INFO:tasks.workunit.client.0.smithi063.stdout:4/197: dread - d7/f39 zero size 2024-05-09T06:38:05.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/186: dwrite d2/d1f/d29/f2a [4194304,4194304] 0 2024-05-09T06:38:05.130 INFO:tasks.workunit.client.0.smithi063.stdout:9/163: dread d0/d4/da/f1e [0,4194304] 0 2024-05-09T06:38:05.130 INFO:tasks.workunit.client.0.smithi063.stdout:9/164: readlink d0/d4/da/l27 0 2024-05-09T06:38:05.131 INFO:tasks.workunit.client.0.smithi063.stdout:9/165: stat d0/d4/da/f21 0 2024-05-09T06:38:05.138 INFO:tasks.workunit.client.0.smithi063.stdout:9/166: dwrite d0/f24 [0,4194304] 0 2024-05-09T06:38:05.142 INFO:tasks.workunit.client.0.smithi063.stdout:1/171: fsync d9/f12 0 2024-05-09T06:38:05.143 INFO:tasks.workunit.client.0.smithi063.stdout:1/172: read d9/f23 [771926,100415] 0 2024-05-09T06:38:05.143 INFO:tasks.workunit.client.0.smithi063.stdout:9/167: dread d0/f13 [0,4194304] 0 2024-05-09T06:38:05.146 INFO:tasks.workunit.client.0.smithi063.stdout:8/220: write d9/d37/f3a [823597,21347] 0 2024-05-09T06:38:05.146 INFO:tasks.workunit.client.0.smithi063.stdout:3/180: rmdir d7/d22/d28 39 2024-05-09T06:38:05.148 INFO:tasks.workunit.client.0.smithi063.stdout:8/221: write d9/d3b/f2c [563684,40651] 0 2024-05-09T06:38:05.151 INFO:tasks.workunit.client.0.smithi063.stdout:5/152: creat d8/d17/d18/f2f x:0 0 0 2024-05-09T06:38:05.153 INFO:tasks.workunit.client.0.smithi063.stdout:1/173: dread f3 [0,4194304] 0 2024-05-09T06:38:05.153 INFO:tasks.workunit.client.0.smithi063.stdout:6/187: symlink d2/d4/l40 0 2024-05-09T06:38:05.154 INFO:tasks.workunit.client.0.smithi063.stdout:2/188: dread dc/f12 [0,4194304] 0 2024-05-09T06:38:05.154 INFO:tasks.workunit.client.0.smithi063.stdout:4/198: truncate d7/df/f2f 3649308 0 2024-05-09T06:38:05.155 INFO:tasks.workunit.client.0.smithi063.stdout:2/189: readlink dc/dd/le 0 2024-05-09T06:38:05.162 INFO:tasks.workunit.client.0.smithi063.stdout:8/222: creat d9/da/d34/f4b x:0 0 0 2024-05-09T06:38:05.165 INFO:tasks.workunit.client.0.smithi063.stdout:5/153: mkdir d8/d19/d30 0 2024-05-09T06:38:05.166 INFO:tasks.workunit.client.0.smithi063.stdout:5/154: readlink d8/d17/l1b 0 2024-05-09T06:38:05.167 INFO:tasks.workunit.client.0.smithi063.stdout:6/188: symlink d2/d1f/l41 0 2024-05-09T06:38:05.168 INFO:tasks.workunit.client.0.smithi063.stdout:6/189: fdatasync d2/d12/f34 0 2024-05-09T06:38:05.170 INFO:tasks.workunit.client.0.smithi063.stdout:9/168: link d0/f13 d0/d4/d26/f2a 0 2024-05-09T06:38:05.171 INFO:tasks.workunit.client.0.smithi063.stdout:3/181: creat d7/da/d21/d30/f3b x:0 0 0 2024-05-09T06:38:05.172 INFO:tasks.workunit.client.0.smithi063.stdout:3/182: stat f5 0 2024-05-09T06:38:05.172 INFO:tasks.workunit.client.0.smithi063.stdout:6/190: dread d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:05.175 INFO:tasks.workunit.client.0.smithi063.stdout:0/198: sync 2024-05-09T06:38:05.175 INFO:tasks.workunit.client.0.smithi063.stdout:0/199: write dd/d1a/f1b [1862675,121998] 0 2024-05-09T06:38:05.180 INFO:tasks.workunit.client.0.smithi063.stdout:7/182: getdents d0/d15 0 2024-05-09T06:38:05.180 INFO:tasks.workunit.client.0.smithi063.stdout:2/190: truncate f6 2558175 0 2024-05-09T06:38:05.180 INFO:tasks.workunit.client.0.smithi063.stdout:9/169: creat d0/d4/da/d1a/f2b x:0 0 0 2024-05-09T06:38:05.182 INFO:tasks.workunit.client.0.smithi063.stdout:6/191: mkdir d2/d4/d2d/d42 0 2024-05-09T06:38:05.185 INFO:tasks.workunit.client.0.smithi063.stdout:4/199: getdents d7/df/d35 0 2024-05-09T06:38:05.186 INFO:tasks.workunit.client.0.smithi063.stdout:2/191: dread dc/f25 [0,4194304] 0 2024-05-09T06:38:05.187 INFO:tasks.workunit.client.0.smithi063.stdout:3/183: creat d7/d22/d28/f3c x:0 0 0 2024-05-09T06:38:05.187 INFO:tasks.workunit.client.0.smithi063.stdout:1/174: getdents d9/d10/d2b 0 2024-05-09T06:38:05.189 INFO:tasks.workunit.client.0.smithi063.stdout:4/200: readlink d7/df/d1f/l30 0 2024-05-09T06:38:05.193 INFO:tasks.workunit.client.0.smithi063.stdout:3/184: fdatasync d7/da/f14 0 2024-05-09T06:38:05.193 INFO:tasks.workunit.client.0.smithi063.stdout:8/223: dwrite d9/da/d18/d24/f3f [0,4194304] 0 2024-05-09T06:38:05.193 INFO:tasks.workunit.client.0.smithi063.stdout:1/175: readlink d9/d2a/d35/l20 0 2024-05-09T06:38:05.195 INFO:tasks.workunit.client.0.smithi063.stdout:0/200: dwrite dd/d1a/d22/d23/f2f [0,4194304] 0 2024-05-09T06:38:05.196 INFO:tasks.workunit.client.0.smithi063.stdout:5/155: dwrite d8/d17/f26 [0,4194304] 0 2024-05-09T06:38:05.204 INFO:tasks.workunit.client.0.smithi063.stdout:4/201: creat d7/df/d1f/f3a x:0 0 0 2024-05-09T06:38:05.207 INFO:tasks.workunit.client.0.smithi063.stdout:7/183: dwrite d0/d15/f26 [4194304,4194304] 0 2024-05-09T06:38:05.209 INFO:tasks.workunit.client.0.smithi063.stdout:1/176: chown d9/fa 1852 1 2024-05-09T06:38:05.212 INFO:tasks.workunit.client.0.smithi063.stdout:9/170: getdents d0 0 2024-05-09T06:38:05.215 INFO:tasks.workunit.client.0.smithi063.stdout:6/192: getdents d2/d1f/d29/d3f 0 2024-05-09T06:38:05.217 INFO:tasks.workunit.client.0.smithi063.stdout:4/202: creat d7/df/d1a/f3b x:0 0 0 2024-05-09T06:38:05.218 INFO:tasks.workunit.client.0.smithi063.stdout:0/201: mkdir dd/d1a/d22/d3f/d44 0 2024-05-09T06:38:05.219 INFO:tasks.workunit.client.0.smithi063.stdout:2/192: dwrite dc/f1f [0,4194304] 0 2024-05-09T06:38:05.221 INFO:tasks.workunit.client.0.smithi063.stdout:7/184: creat d0/d17/f3b x:0 0 0 2024-05-09T06:38:05.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/177: symlink d9/d2a/d35/d1e/l38 0 2024-05-09T06:38:05.224 INFO:tasks.workunit.client.0.smithi063.stdout:5/156: getdents d8/d17/d18 0 2024-05-09T06:38:05.224 INFO:tasks.workunit.client.0.smithi063.stdout:4/203: mknod d7/df/c3c 0 2024-05-09T06:38:05.226 INFO:tasks.workunit.client.0.smithi063.stdout:2/193: write fa [6715757,121198] 0 2024-05-09T06:38:05.226 INFO:tasks.workunit.client.0.smithi063.stdout:8/224: dwrite d9/da/d34/f38 [0,4194304] 0 2024-05-09T06:38:05.228 INFO:tasks.workunit.client.0.smithi063.stdout:5/157: chown d8/f10 14506729 1 2024-05-09T06:38:05.228 INFO:tasks.workunit.client.0.smithi063.stdout:6/193: getdents d2/d4/d2d/d42 0 2024-05-09T06:38:05.229 INFO:tasks.workunit.client.0.smithi063.stdout:9/171: dread d0/f24 [0,4194304] 0 2024-05-09T06:38:05.232 INFO:tasks.workunit.client.0.smithi063.stdout:5/158: rmdir d8/d19 39 2024-05-09T06:38:05.233 INFO:tasks.workunit.client.0.smithi063.stdout:2/194: creat dc/dd/d2d/f38 x:0 0 0 2024-05-09T06:38:05.235 INFO:tasks.workunit.client.0.smithi063.stdout:6/194: dread d2/df/f1d [0,4194304] 0 2024-05-09T06:38:05.235 INFO:tasks.workunit.client.0.smithi063.stdout:7/185: rename d0/f21 to d0/d17/f3c 0 2024-05-09T06:38:05.238 INFO:tasks.workunit.client.0.smithi063.stdout:5/159: write d8/d17/d1e/f23 [3282634,78701] 0 2024-05-09T06:38:05.239 INFO:tasks.workunit.client.0.smithi063.stdout:2/195: creat dc/dd/d2d/f39 x:0 0 0 2024-05-09T06:38:05.240 INFO:tasks.workunit.client.0.smithi063.stdout:5/160: chown l5 197947056 1 2024-05-09T06:38:05.240 INFO:tasks.workunit.client.0.smithi063.stdout:7/186: truncate d0/f32 238746 0 2024-05-09T06:38:05.241 INFO:tasks.workunit.client.0.smithi063.stdout:0/202: rename dd/d1a/d22/l29 to dd/d1a/d22/d23/l45 0 2024-05-09T06:38:05.242 INFO:tasks.workunit.client.0.smithi063.stdout:1/178: dwrite d9/d2a/d35/f29 [0,4194304] 0 2024-05-09T06:38:05.242 INFO:tasks.workunit.client.0.smithi063.stdout:6/195: truncate d2/d12/f34 519362 0 2024-05-09T06:38:05.244 INFO:tasks.workunit.client.0.smithi063.stdout:9/172: rename d0/d4/da/d1a to d0/d4/da/d1a/d2c 22 2024-05-09T06:38:05.244 INFO:tasks.workunit.client.0.smithi063.stdout:7/187: chown d0/d17/f2e 16 1 2024-05-09T06:38:05.246 INFO:tasks.workunit.client.0.smithi063.stdout:1/179: write d9/d10/f21 [4845355,20535] 0 2024-05-09T06:38:05.247 INFO:tasks.workunit.client.0.smithi063.stdout:6/196: chown d2/d20/f21 10 1 2024-05-09T06:38:05.248 INFO:tasks.workunit.client.0.smithi063.stdout:7/188: fdatasync d0/d15/f20 0 2024-05-09T06:38:05.249 INFO:tasks.workunit.client.0.smithi063.stdout:0/203: mkdir dd/d1a/d22/d23/d46 0 2024-05-09T06:38:05.250 INFO:tasks.workunit.client.0.smithi063.stdout:5/161: symlink d8/l31 0 2024-05-09T06:38:05.250 INFO:tasks.workunit.client.0.smithi063.stdout:3/185: dread d7/f11 [0,4194304] 0 2024-05-09T06:38:05.252 INFO:tasks.workunit.client.0.smithi063.stdout:6/197: chown d2/d1f/c36 1 1 2024-05-09T06:38:05.252 INFO:tasks.workunit.client.0.smithi063.stdout:8/225: dread d9/da/d18/f1b [0,4194304] 0 2024-05-09T06:38:05.252 INFO:tasks.workunit.client.0.smithi063.stdout:0/204: chown f6 258607 1 2024-05-09T06:38:05.252 INFO:tasks.workunit.client.0.smithi063.stdout:9/173: mkdir d0/d8/d2d 0 2024-05-09T06:38:05.253 INFO:tasks.workunit.client.0.smithi063.stdout:1/180: mkdir d9/d2a/d39 0 2024-05-09T06:38:05.255 INFO:tasks.workunit.client.0.smithi063.stdout:1/181: chown d9/f23 8641147 1 2024-05-09T06:38:05.255 INFO:tasks.workunit.client.0.smithi063.stdout:0/205: dread - dd/d1a/d22/d23/d31/f41 zero size 2024-05-09T06:38:05.256 INFO:tasks.workunit.client.0.smithi063.stdout:5/162: mknod d8/d17/c32 0 2024-05-09T06:38:05.257 INFO:tasks.workunit.client.0.smithi063.stdout:0/206: readlink dd/l2a 0 2024-05-09T06:38:05.258 INFO:tasks.workunit.client.0.smithi063.stdout:3/186: mknod d7/d1e/c3d 0 2024-05-09T06:38:05.258 INFO:tasks.workunit.client.0.smithi063.stdout:6/198: symlink d2/d20/d26/l43 0 2024-05-09T06:38:05.259 INFO:tasks.workunit.client.0.smithi063.stdout:5/163: stat d8/d17/d18 0 2024-05-09T06:38:05.260 INFO:tasks.workunit.client.0.smithi063.stdout:0/207: chown dd/f12 737300211 1 2024-05-09T06:38:05.260 INFO:tasks.workunit.client.0.smithi063.stdout:9/174: rename d0/d4/c18 to d0/d4/d26/c2e 0 2024-05-09T06:38:05.262 INFO:tasks.workunit.client.0.smithi063.stdout:8/226: truncate d9/f14 2463894 0 2024-05-09T06:38:05.264 INFO:tasks.workunit.client.0.smithi063.stdout:6/199: chown d2/d4/d2d/c3d 161785 1 2024-05-09T06:38:05.264 INFO:tasks.workunit.client.0.smithi063.stdout:7/189: dwrite d0/d15/f26 [4194304,4194304] 0 2024-05-09T06:38:05.265 INFO:tasks.workunit.client.0.smithi063.stdout:0/208: dread - dd/d1a/d22/d23/d31/f41 zero size 2024-05-09T06:38:05.266 INFO:tasks.workunit.client.0.smithi063.stdout:6/200: chown d2/df/f14 0 1 2024-05-09T06:38:05.266 INFO:tasks.workunit.client.0.smithi063.stdout:9/175: unlink d0/f13 0 2024-05-09T06:38:05.270 INFO:tasks.workunit.client.0.smithi063.stdout:0/209: chown dd/d1a/f35 11330 1 2024-05-09T06:38:05.270 INFO:tasks.workunit.client.0.smithi063.stdout:2/196: dwrite dc/f2a [0,4194304] 0 2024-05-09T06:38:05.271 INFO:tasks.workunit.client.0.smithi063.stdout:5/164: rename d8/l13 to d8/d17/d1e/l33 0 2024-05-09T06:38:05.277 INFO:tasks.workunit.client.0.smithi063.stdout:8/227: mknod d9/d37/c4c 0 2024-05-09T06:38:05.279 INFO:tasks.workunit.client.0.smithi063.stdout:1/182: dwrite f3 [0,4194304] 0 2024-05-09T06:38:05.284 INFO:tasks.workunit.client.0.smithi063.stdout:2/197: chown f6 337241 1 2024-05-09T06:38:05.284 INFO:tasks.workunit.client.0.smithi063.stdout:5/165: dwrite d8/f1d [0,4194304] 0 2024-05-09T06:38:05.286 INFO:tasks.workunit.client.0.smithi063.stdout:2/198: chown dc/dd/d2d 83 1 2024-05-09T06:38:05.289 INFO:tasks.workunit.client.0.smithi063.stdout:2/199: chown dc/dd/c30 0 1 2024-05-09T06:38:05.289 INFO:tasks.workunit.client.0.smithi063.stdout:7/190: mknod d0/c3d 0 2024-05-09T06:38:05.289 INFO:tasks.workunit.client.0.smithi063.stdout:3/187: rename d7/d22/d28/f3a to d7/d2a/d2d/f3e 0 2024-05-09T06:38:05.296 INFO:tasks.workunit.client.0.smithi063.stdout:8/228: readlink d9/da/d45/l4a 0 2024-05-09T06:38:05.296 INFO:tasks.workunit.client.0.smithi063.stdout:8/229: fsync d9/d37/f3a 0 2024-05-09T06:38:05.297 INFO:tasks.workunit.client.0.smithi063.stdout:8/230: chown d9/da/d18/l28 18389 1 2024-05-09T06:38:05.301 INFO:tasks.workunit.client.0.smithi063.stdout:0/210: mkdir dd/d1a/d2c/d42/d47 0 2024-05-09T06:38:05.307 INFO:tasks.workunit.client.0.smithi063.stdout:4/204: sync 2024-05-09T06:38:05.309 INFO:tasks.workunit.client.0.smithi063.stdout:2/200: symlink dc/d32/l3a 0 2024-05-09T06:38:05.310 INFO:tasks.workunit.client.0.smithi063.stdout:2/201: read f6 [669148,113245] 0 2024-05-09T06:38:05.311 INFO:tasks.workunit.client.0.smithi063.stdout:2/202: chown dc/d32 23 1 2024-05-09T06:38:05.311 INFO:tasks.workunit.client.0.smithi063.stdout:6/201: truncate f1 2597194 0 2024-05-09T06:38:05.313 INFO:tasks.workunit.client.0.smithi063.stdout:3/188: creat d7/d22/d28/f3f x:0 0 0 2024-05-09T06:38:05.317 INFO:tasks.workunit.client.0.smithi063.stdout:5/166: write d8/d17/f24 [5211354,53834] 0 2024-05-09T06:38:05.318 INFO:tasks.workunit.client.0.smithi063.stdout:7/191: dread d0/f10 [0,4194304] 0 2024-05-09T06:38:05.319 INFO:tasks.workunit.client.0.smithi063.stdout:8/231: creat d9/d37/f4d x:0 0 0 2024-05-09T06:38:05.320 INFO:tasks.workunit.client.0.smithi063.stdout:5/167: read d8/d17/d18/f1f [1144670,123503] 0 2024-05-09T06:38:05.321 INFO:tasks.workunit.client.0.smithi063.stdout:1/183: dwrite d9/d10/f1c [0,4194304] 0 2024-05-09T06:38:05.328 INFO:tasks.workunit.client.0.smithi063.stdout:4/205: mkdir d7/d3d 0 2024-05-09T06:38:05.328 INFO:tasks.workunit.client.0.smithi063.stdout:7/192: dwrite d0/d17/f1a [0,4194304] 0 2024-05-09T06:38:05.329 INFO:tasks.workunit.client.0.smithi063.stdout:4/206: chown d7/df/d35/f36 523353 1 2024-05-09T06:38:05.329 INFO:tasks.workunit.client.0.smithi063.stdout:7/193: chown d0/d17/f29 870179 1 2024-05-09T06:38:05.331 INFO:tasks.workunit.client.0.smithi063.stdout:7/194: write d0/d17/f23 [1194989,114037] 0 2024-05-09T06:38:05.334 INFO:tasks.workunit.client.0.smithi063.stdout:0/211: dread dd/f15 [0,4194304] 0 2024-05-09T06:38:05.337 INFO:tasks.workunit.client.0.smithi063.stdout:6/202: rename d2/d1f/d29/d3f/f1b to d2/d20/f44 0 2024-05-09T06:38:05.340 INFO:tasks.workunit.client.0.smithi063.stdout:3/189: creat d7/da/d21/d30/f40 x:0 0 0 2024-05-09T06:38:05.346 INFO:tasks.workunit.client.0.smithi063.stdout:5/168: mknod d8/d17/d1e/c34 0 2024-05-09T06:38:05.348 INFO:tasks.workunit.client.0.smithi063.stdout:9/176: symlink d0/d4/da/l2f 0 2024-05-09T06:38:05.349 INFO:tasks.workunit.client.0.smithi063.stdout:7/195: symlink d0/d15/l3e 0 2024-05-09T06:38:05.350 INFO:tasks.workunit.client.0.smithi063.stdout:2/203: symlink dc/d36/l3b 0 2024-05-09T06:38:05.353 INFO:tasks.workunit.client.0.smithi063.stdout:1/184: rename d9/fb to d9/d2a/d35/d1e/f3a 0 2024-05-09T06:38:05.354 INFO:tasks.workunit.client.0.smithi063.stdout:6/203: mkdir d2/d1f/d45 0 2024-05-09T06:38:05.355 INFO:tasks.workunit.client.0.smithi063.stdout:6/204: chown d2/d20/c3a 499945671 1 2024-05-09T06:38:05.355 INFO:tasks.workunit.client.0.smithi063.stdout:6/205: chown d2/d12/c22 5091 1 2024-05-09T06:38:05.358 INFO:tasks.workunit.client.0.smithi063.stdout:4/207: mknod d7/df/d1f/d28/d31/d32/d34/c3e 0 2024-05-09T06:38:05.360 INFO:tasks.workunit.client.0.smithi063.stdout:7/196: creat d0/d17/f3f x:0 0 0 2024-05-09T06:38:05.361 INFO:tasks.workunit.client.0.smithi063.stdout:2/204: symlink dc/d32/d33/l3c 0 2024-05-09T06:38:05.364 INFO:tasks.workunit.client.0.smithi063.stdout:3/190: mknod d7/da/d2b/c41 0 2024-05-09T06:38:05.365 INFO:tasks.workunit.client.0.smithi063.stdout:6/206: symlink d2/d1f/d29/d3f/l46 0 2024-05-09T06:38:05.366 INFO:tasks.workunit.client.0.smithi063.stdout:5/169: mkdir d8/d35 0 2024-05-09T06:38:05.366 INFO:tasks.workunit.client.0.smithi063.stdout:4/208: dwrite d7/f39 [0,4194304] 0 2024-05-09T06:38:05.369 INFO:tasks.workunit.client.0.smithi063.stdout:3/191: chown d7/d1e/d31/d35/l37 22 1 2024-05-09T06:38:05.375 INFO:tasks.workunit.client.0.smithi063.stdout:7/197: dwrite d0/d17/f30 [0,4194304] 0 2024-05-09T06:38:05.376 INFO:tasks.workunit.client.0.smithi063.stdout:6/207: dread d2/d12/f24 [0,4194304] 0 2024-05-09T06:38:05.376 INFO:tasks.workunit.client.0.smithi063.stdout:2/205: dwrite dc/dd/f17 [0,4194304] 0 2024-05-09T06:38:05.383 INFO:tasks.workunit.client.0.smithi063.stdout:6/208: readlink d2/df/l2f 0 2024-05-09T06:38:05.384 INFO:tasks.workunit.client.0.smithi063.stdout:9/177: creat d0/d4/d26/f30 x:0 0 0 2024-05-09T06:38:05.385 INFO:tasks.workunit.client.0.smithi063.stdout:2/206: write dc/f1b [2658606,7208] 0 2024-05-09T06:38:05.386 INFO:tasks.workunit.client.0.smithi063.stdout:7/198: creat d0/d15/f40 x:0 0 0 2024-05-09T06:38:05.387 INFO:tasks.workunit.client.0.smithi063.stdout:5/170: symlink d8/l36 0 2024-05-09T06:38:05.387 INFO:tasks.workunit.client.0.smithi063.stdout:4/209: dwrite d7/df/d1a/f3b [0,4194304] 0 2024-05-09T06:38:05.389 INFO:tasks.workunit.client.0.smithi063.stdout:2/207: truncate dc/dd/d2d/f39 306020 0 2024-05-09T06:38:05.390 INFO:tasks.workunit.client.0.smithi063.stdout:2/208: stat l5 0 2024-05-09T06:38:05.393 INFO:tasks.workunit.client.0.smithi063.stdout:3/192: symlink d7/l42 0 2024-05-09T06:38:05.394 INFO:tasks.workunit.client.0.smithi063.stdout:3/193: readlink d7/l1f 0 2024-05-09T06:38:05.398 INFO:tasks.workunit.client.0.smithi063.stdout:7/199: rmdir d0 39 2024-05-09T06:38:05.401 INFO:tasks.workunit.client.0.smithi063.stdout:5/171: creat d8/d17/d1e/f37 x:0 0 0 2024-05-09T06:38:05.403 INFO:tasks.workunit.client.0.smithi063.stdout:6/209: dwrite d2/d20/f44 [0,4194304] 0 2024-05-09T06:38:05.404 INFO:tasks.workunit.client.0.smithi063.stdout:2/209: unlink l5 0 2024-05-09T06:38:05.406 INFO:tasks.workunit.client.0.smithi063.stdout:2/210: truncate dc/dd/d2d/f38 171667 0 2024-05-09T06:38:05.406 INFO:tasks.workunit.client.0.smithi063.stdout:3/194: creat d7/da/d2b/f43 x:0 0 0 2024-05-09T06:38:05.407 INFO:tasks.workunit.client.0.smithi063.stdout:3/195: chown d7/c1c 2 1 2024-05-09T06:38:05.407 INFO:tasks.workunit.client.0.smithi063.stdout:9/178: mknod d0/c31 0 2024-05-09T06:38:05.411 INFO:tasks.workunit.client.0.smithi063.stdout:7/200: write d0/d15/f20 [1156908,36351] 0 2024-05-09T06:38:05.413 INFO:tasks.workunit.client.0.smithi063.stdout:7/201: truncate d0/d17/f3b 268815 0 2024-05-09T06:38:05.413 INFO:tasks.workunit.client.0.smithi063.stdout:8/232: sync 2024-05-09T06:38:05.417 INFO:tasks.workunit.client.0.smithi063.stdout:6/210: creat d2/d1f/d29/f47 x:0 0 0 2024-05-09T06:38:05.421 INFO:tasks.workunit.client.0.smithi063.stdout:3/196: mkdir d7/da/d21/d30/d44 0 2024-05-09T06:38:05.424 INFO:tasks.workunit.client.0.smithi063.stdout:7/202: dread d0/d15/f20 [0,4194304] 0 2024-05-09T06:38:05.424 INFO:tasks.workunit.client.0.smithi063.stdout:6/211: dwrite d2/d20/f44 [0,4194304] 0 2024-05-09T06:38:05.425 INFO:tasks.workunit.client.0.smithi063.stdout:6/212: chown d2/df/f14 5103 1 2024-05-09T06:38:05.427 INFO:tasks.workunit.client.0.smithi063.stdout:4/210: link d7/c1b d7/df/d35/c3f 0 2024-05-09T06:38:05.428 INFO:tasks.workunit.client.0.smithi063.stdout:8/233: rmdir d9/da 39 2024-05-09T06:38:05.430 INFO:tasks.workunit.client.0.smithi063.stdout:7/203: read d0/f1c [3883494,44360] 0 2024-05-09T06:38:05.431 INFO:tasks.workunit.client.0.smithi063.stdout:7/204: dread - d0/d15/f40 zero size 2024-05-09T06:38:05.433 INFO:tasks.workunit.client.0.smithi063.stdout:5/172: link d8/f14 d8/d17/d18/f38 0 2024-05-09T06:38:05.435 INFO:tasks.workunit.client.0.smithi063.stdout:1/185: truncate f3 3091695 0 2024-05-09T06:38:05.437 INFO:tasks.workunit.client.0.smithi063.stdout:7/205: dwrite d0/f16 [0,4194304] 0 2024-05-09T06:38:05.438 INFO:tasks.workunit.client.0.smithi063.stdout:2/211: symlink dc/l3d 0 2024-05-09T06:38:05.439 INFO:tasks.workunit.client.0.smithi063.stdout:7/206: write d0/d15/f25 [762526,115578] 0 2024-05-09T06:38:05.441 INFO:tasks.workunit.client.0.smithi063.stdout:3/197: unlink d7/f8 0 2024-05-09T06:38:05.443 INFO:tasks.workunit.client.0.smithi063.stdout:2/212: dread dc/dd/f20 [0,4194304] 0 2024-05-09T06:38:05.446 INFO:tasks.workunit.client.0.smithi063.stdout:6/213: mknod d2/df/c48 0 2024-05-09T06:38:05.452 INFO:tasks.workunit.client.0.smithi063.stdout:5/173: mknod d8/d17/d1e/c39 0 2024-05-09T06:38:05.455 INFO:tasks.workunit.client.0.smithi063.stdout:7/207: symlink d0/d15/l41 0 2024-05-09T06:38:05.455 INFO:tasks.workunit.client.0.smithi063.stdout:5/174: dread d8/d17/f24 [0,4194304] 0 2024-05-09T06:38:05.456 INFO:tasks.workunit.client.0.smithi063.stdout:7/208: fsync d0/d17/f29 0 2024-05-09T06:38:05.456 INFO:tasks.workunit.client.0.smithi063.stdout:3/198: creat d7/d1e/d31/d35/f45 x:0 0 0 2024-05-09T06:38:05.457 INFO:tasks.workunit.client.0.smithi063.stdout:7/209: stat d0/d15/l3e 0 2024-05-09T06:38:05.463 INFO:tasks.workunit.client.0.smithi063.stdout:6/214: dread d2/d4/f5 [0,4194304] 0 2024-05-09T06:38:05.464 INFO:tasks.workunit.client.0.smithi063.stdout:8/234: write d9/da/d34/f38 [812848,26806] 0 2024-05-09T06:38:05.465 INFO:tasks.workunit.client.0.smithi063.stdout:1/186: symlink d9/d2a/d2e/l3b 0 2024-05-09T06:38:05.481 INFO:tasks.workunit.client.0.smithi063.stdout:6/215: rename d2/d1f/d29/d3f/l39 to d2/df/l49 0 2024-05-09T06:38:05.482 INFO:tasks.workunit.client.0.smithi063.stdout:6/216: fsync d2/d1f/d29/d3f/f1e 0 2024-05-09T06:38:05.482 INFO:tasks.workunit.client.0.smithi063.stdout:1/187: rmdir d9/d10 39 2024-05-09T06:38:05.490 INFO:tasks.workunit.client.0.smithi063.stdout:7/210: mknod d0/c42 0 2024-05-09T06:38:05.490 INFO:tasks.workunit.client.0.smithi063.stdout:7/211: write d0/d17/f30 [974329,30967] 0 2024-05-09T06:38:05.493 INFO:tasks.workunit.client.0.smithi063.stdout:6/217: rmdir d2/d4 39 2024-05-09T06:38:05.496 INFO:tasks.workunit.client.0.smithi063.stdout:1/188: mkdir d9/d2a/d3c 0 2024-05-09T06:38:05.498 INFO:tasks.workunit.client.0.smithi063.stdout:5/175: creat d8/f3a x:0 0 0 2024-05-09T06:38:05.503 INFO:tasks.workunit.client.0.smithi063.stdout:5/176: dwrite d8/d17/d18/f29 [0,4194304] 0 2024-05-09T06:38:05.504 INFO:tasks.workunit.client.0.smithi063.stdout:8/235: creat d9/da/f4e x:0 0 0 2024-05-09T06:38:05.504 INFO:tasks.workunit.client.0.smithi063.stdout:8/236: dread - d9/da/d34/f4b zero size 2024-05-09T06:38:05.505 INFO:tasks.workunit.client.0.smithi063.stdout:7/212: symlink d0/d15/l43 0 2024-05-09T06:38:05.509 INFO:tasks.workunit.client.0.smithi063.stdout:1/189: truncate d9/f23 640023 0 2024-05-09T06:38:05.510 INFO:tasks.workunit.client.0.smithi063.stdout:8/237: chown d9/da/d18/f22 812 1 2024-05-09T06:38:05.512 INFO:tasks.workunit.client.0.smithi063.stdout:5/177: symlink d8/d17/l3b 0 2024-05-09T06:38:05.513 INFO:tasks.workunit.client.0.smithi063.stdout:9/179: write d0/d4/da/d1a/f1d [229064,23706] 0 2024-05-09T06:38:05.515 INFO:tasks.workunit.client.0.smithi063.stdout:2/213: write f3 [2089924,104140] 0 2024-05-09T06:38:05.516 INFO:tasks.workunit.client.0.smithi063.stdout:8/238: mknod d9/da/d34/c4f 0 2024-05-09T06:38:05.516 INFO:tasks.workunit.client.0.smithi063.stdout:6/218: dwrite d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:05.518 INFO:tasks.workunit.client.0.smithi063.stdout:7/213: truncate d0/d15/f20 1555252 0 2024-05-09T06:38:05.524 INFO:tasks.workunit.client.0.smithi063.stdout:0/212: sync 2024-05-09T06:38:05.532 INFO:tasks.workunit.client.0.smithi063.stdout:6/219: dread d2/d4/f1c [0,4194304] 0 2024-05-09T06:38:05.534 INFO:tasks.workunit.client.0.smithi063.stdout:8/239: rename d9/da/f19 to d9/da/d18/d46/f50 0 2024-05-09T06:38:05.536 INFO:tasks.workunit.client.0.smithi063.stdout:1/190: mkdir d9/d2a/d3c/d3d 0 2024-05-09T06:38:05.538 INFO:tasks.workunit.client.0.smithi063.stdout:7/214: mknod d0/d17/c44 0 2024-05-09T06:38:05.539 INFO:tasks.workunit.client.0.smithi063.stdout:7/215: dread - d0/d17/f37 zero size 2024-05-09T06:38:05.541 INFO:tasks.workunit.client.0.smithi063.stdout:0/213: rmdir dd/d1a/d22/d23/d31 39 2024-05-09T06:38:05.547 INFO:tasks.workunit.client.0.smithi063.stdout:2/214: link fa dc/dd/d2d/f3e 0 2024-05-09T06:38:05.549 INFO:tasks.workunit.client.0.smithi063.stdout:3/199: write d7/f1b [917038,35672] 0 2024-05-09T06:38:05.550 INFO:tasks.workunit.client.0.smithi063.stdout:6/220: write d2/d4/f5 [3939411,109416] 0 2024-05-09T06:38:05.552 INFO:tasks.workunit.client.0.smithi063.stdout:6/221: fdatasync d2/d1f/d29/f47 0 2024-05-09T06:38:05.552 INFO:tasks.workunit.client.0.smithi063.stdout:8/240: write d9/d3b/f2c [1334913,79019] 0 2024-05-09T06:38:05.553 INFO:tasks.workunit.client.0.smithi063.stdout:1/191: write d9/d10/f18 [4060184,64594] 0 2024-05-09T06:38:05.553 INFO:tasks.workunit.client.0.smithi063.stdout:8/241: dread - d9/da/d18/d24/f3e zero size 2024-05-09T06:38:05.554 INFO:tasks.workunit.client.0.smithi063.stdout:1/192: stat d9/d2a/d35/d24 0 2024-05-09T06:38:05.555 INFO:tasks.workunit.client.0.smithi063.stdout:7/216: mknod d0/d17/c45 0 2024-05-09T06:38:05.555 INFO:tasks.workunit.client.0.smithi063.stdout:1/193: readlink d9/d2a/d35/l1b 0 2024-05-09T06:38:05.556 INFO:tasks.workunit.client.0.smithi063.stdout:7/217: write d0/d17/f29 [961980,69979] 0 2024-05-09T06:38:05.562 INFO:tasks.workunit.client.0.smithi063.stdout:2/215: truncate dc/dd/f13 1253252 0 2024-05-09T06:38:05.563 INFO:tasks.workunit.client.0.smithi063.stdout:2/216: write f6 [2064169,46318] 0 2024-05-09T06:38:05.564 INFO:tasks.workunit.client.0.smithi063.stdout:2/217: truncate dc/dd/d2d/f39 395950 0 2024-05-09T06:38:05.567 INFO:tasks.workunit.client.0.smithi063.stdout:3/200: creat d7/da/d21/d30/f46 x:0 0 0 2024-05-09T06:38:05.570 INFO:tasks.workunit.client.0.smithi063.stdout:6/222: readlink d2/df/l49 0 2024-05-09T06:38:05.570 INFO:tasks.workunit.client.0.smithi063.stdout:6/223: dread - d2/d1f/d29/d3f/f25 zero size 2024-05-09T06:38:05.571 INFO:tasks.workunit.client.0.smithi063.stdout:6/224: fdatasync d2/d1f/d29/d3f/f25 0 2024-05-09T06:38:05.575 INFO:tasks.workunit.client.0.smithi063.stdout:5/178: rmdir d8 39 2024-05-09T06:38:05.581 INFO:tasks.workunit.client.0.smithi063.stdout:8/242: dwrite d9/d37/f4d [0,4194304] 0 2024-05-09T06:38:05.582 INFO:tasks.workunit.client.0.smithi063.stdout:8/243: fdatasync d9/da/d18/d24/f3f 0 2024-05-09T06:38:05.583 INFO:tasks.workunit.client.0.smithi063.stdout:2/218: creat dc/d32/d33/f3f x:0 0 0 2024-05-09T06:38:05.585 INFO:tasks.workunit.client.0.smithi063.stdout:8/244: chown d9/da/d18/f1d 330756 1 2024-05-09T06:38:05.586 INFO:tasks.workunit.client.0.smithi063.stdout:8/245: write d9/f47 [1047633,104929] 0 2024-05-09T06:38:05.587 INFO:tasks.workunit.client.0.smithi063.stdout:6/225: rename d2/d1f/d45 to d2/d20/d4a 0 2024-05-09T06:38:05.599 INFO:tasks.workunit.client.0.smithi063.stdout:2/219: fsync dc/f21 0 2024-05-09T06:38:05.599 INFO:tasks.workunit.client.0.smithi063.stdout:2/220: write dc/f1f [3527507,45382] 0 2024-05-09T06:38:05.607 INFO:tasks.workunit.client.0.smithi063.stdout:6/226: symlink d2/d12/l4b 0 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:2/221: dwrite dc/dd/d2d/f34 [0,4194304] 0 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:9/180: dwrite d0/d4/d26/f2a [0,4194304] 0 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:5/179: rename d8/cb to d8/d35/c3c 0 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:0/214: getdents dd/d1a/d22 0 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:2/222: dread dc/dd/d2d/f39 [0,4194304] 0 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:6/227: dread d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:8/246: mkdir d9/da/d51 0 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:4/211: sync 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:1/194: getdents d9/d2a/d35 0 2024-05-09T06:38:05.692 INFO:tasks.workunit.client.0.smithi063.stdout:0/215: mkdir dd/d1a/d22/d23/d33/d48 0 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:6/228: rename d2/d4/f8 to d2/d1f/f4c 0 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:8/247: rmdir d9/d37 39 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:8/248: chown d9/f47 58 1 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:4/212: mknod d7/df/d1f/d28/c40 0 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:1/195: creat d9/d10/f3e x:0 0 0 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:1/196: write d9/d2a/d35/d1e/f27 [2608885,123983] 0 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:3/201: link d7/da/fb d7/d22/f47 0 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:9/181: creat d0/f32 x:0 0 0 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:1/197: mkdir d9/d10/d2b/d3f 0 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:1/198: chown d9/d10/f21 12 1 2024-05-09T06:38:05.693 INFO:tasks.workunit.client.0.smithi063.stdout:1/199: rename d9/d2a/l30 to d9/d2a/d39/l40 0 2024-05-09T06:38:05.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/202: mkdir d7/da/d21/d30/d44/d48 0 2024-05-09T06:38:05.694 INFO:tasks.workunit.client.0.smithi063.stdout:8/249: dwrite d9/da/f1a [4194304,4194304] 0 2024-05-09T06:38:05.694 INFO:tasks.workunit.client.0.smithi063.stdout:8/250: dread - d9/da/d18/d24/f3e zero size 2024-05-09T06:38:05.694 INFO:tasks.workunit.client.0.smithi063.stdout:9/182: link d0/f1f d0/d4/d26/f33 0 2024-05-09T06:38:05.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/203: dwrite d7/d22/d28/f3f [0,4194304] 0 2024-05-09T06:38:05.696 INFO:tasks.workunit.client.0.smithi063.stdout:1/200: link d9/d2a/d35/l20 d9/d2a/d39/l41 0 2024-05-09T06:38:05.697 INFO:tasks.workunit.client.0.smithi063.stdout:4/213: write d7/f2d [4640338,28900] 0 2024-05-09T06:38:05.699 INFO:tasks.workunit.client.0.smithi063.stdout:3/204: unlink d7/da/d21/d30/f46 0 2024-05-09T06:38:05.703 INFO:tasks.workunit.client.0.smithi063.stdout:0/216: dwrite f2 [0,4194304] 0 2024-05-09T06:38:05.703 INFO:tasks.workunit.client.0.smithi063.stdout:8/251: rename d9/d41/f43 to d9/da/d18/d46/f52 0 2024-05-09T06:38:05.706 INFO:tasks.workunit.client.0.smithi063.stdout:1/201: creat d9/d2a/d35/d1e/f42 x:0 0 0 2024-05-09T06:38:05.707 INFO:tasks.workunit.client.0.smithi063.stdout:2/223: write dc/dd/f13 [944603,5493] 0 2024-05-09T06:38:05.708 INFO:tasks.workunit.client.0.smithi063.stdout:4/214: mknod d7/df/d1f/c41 0 2024-05-09T06:38:05.709 INFO:tasks.workunit.client.0.smithi063.stdout:4/215: chown f5 910 1 2024-05-09T06:38:05.709 INFO:tasks.workunit.client.0.smithi063.stdout:1/202: write d9/d10/f3e [921351,27107] 0 2024-05-09T06:38:05.710 INFO:tasks.workunit.client.0.smithi063.stdout:4/216: chown d7/df/d1a/c1d 22968 1 2024-05-09T06:38:05.712 INFO:tasks.workunit.client.0.smithi063.stdout:8/252: write d9/da/d18/f1b [3129551,125031] 0 2024-05-09T06:38:05.713 INFO:tasks.workunit.client.0.smithi063.stdout:2/224: mkdir dc/d36/d40 0 2024-05-09T06:38:05.717 INFO:tasks.workunit.client.0.smithi063.stdout:1/203: creat d9/d10/f43 x:0 0 0 2024-05-09T06:38:05.719 INFO:tasks.workunit.client.0.smithi063.stdout:4/217: readlink d7/df/l37 0 2024-05-09T06:38:05.720 INFO:tasks.workunit.client.0.smithi063.stdout:3/205: link d7/d2a/d2d/l2e d7/da/d21/d30/d44/d48/l49 0 2024-05-09T06:38:05.721 INFO:tasks.workunit.client.0.smithi063.stdout:3/206: chown d7/c38 438062929 1 2024-05-09T06:38:05.721 INFO:tasks.workunit.client.0.smithi063.stdout:2/225: dwrite dc/dd/d2d/f38 [0,4194304] 0 2024-05-09T06:38:05.722 INFO:tasks.workunit.client.0.smithi063.stdout:1/204: creat d9/d2a/d35/f44 x:0 0 0 2024-05-09T06:38:05.722 INFO:tasks.workunit.client.0.smithi063.stdout:8/253: mkdir d9/da/d53 0 2024-05-09T06:38:05.723 INFO:tasks.workunit.client.0.smithi063.stdout:0/217: getdents dd/d1a/d22/d23/d33 0 2024-05-09T06:38:05.726 INFO:tasks.workunit.client.0.smithi063.stdout:4/218: creat d7/f42 x:0 0 0 2024-05-09T06:38:05.726 INFO:tasks.workunit.client.0.smithi063.stdout:3/207: symlink d7/da/d21/l4a 0 2024-05-09T06:38:05.728 INFO:tasks.workunit.client.0.smithi063.stdout:7/218: sync 2024-05-09T06:38:05.728 INFO:tasks.workunit.client.0.smithi063.stdout:4/219: chown d7/f42 5746439 1 2024-05-09T06:38:05.729 INFO:tasks.workunit.client.0.smithi063.stdout:3/208: write d7/f17 [3136538,71483] 0 2024-05-09T06:38:05.729 INFO:tasks.workunit.client.0.smithi063.stdout:2/226: unlink dc/dd/d2d/f39 0 2024-05-09T06:38:05.730 INFO:tasks.workunit.client.0.smithi063.stdout:9/183: dread d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:05.734 INFO:tasks.workunit.client.0.smithi063.stdout:2/227: creat dc/d32/d33/f41 x:0 0 0 2024-05-09T06:38:05.735 INFO:tasks.workunit.client.0.smithi063.stdout:2/228: chown dc/dd/d2d/l35 1024449 1 2024-05-09T06:38:05.737 INFO:tasks.workunit.client.0.smithi063.stdout:3/209: rename d7/f1a to d7/d2a/f4b 0 2024-05-09T06:38:05.738 INFO:tasks.workunit.client.0.smithi063.stdout:7/219: read d0/f1c [2573894,79557] 0 2024-05-09T06:38:05.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/220: stat d0/d17/f3b 0 2024-05-09T06:38:05.741 INFO:tasks.workunit.client.0.smithi063.stdout:9/184: creat d0/d4/d26/f34 x:0 0 0 2024-05-09T06:38:05.742 INFO:tasks.workunit.client.0.smithi063.stdout:4/220: rename d7/df/c16 to d7/df/d1f/d28/d31/d32/d34/c43 0 2024-05-09T06:38:05.742 INFO:tasks.workunit.client.0.smithi063.stdout:9/185: readlink d0/d4/da/l27 0 2024-05-09T06:38:05.743 INFO:tasks.workunit.client.0.smithi063.stdout:2/229: dwrite dc/dd/d2d/f38 [0,4194304] 0 2024-05-09T06:38:05.745 INFO:tasks.workunit.client.0.smithi063.stdout:3/210: write d7/d2a/d2d/f3e [170504,24479] 0 2024-05-09T06:38:05.745 INFO:tasks.workunit.client.0.smithi063.stdout:7/221: mknod d0/d17/c46 0 2024-05-09T06:38:05.747 INFO:tasks.workunit.client.0.smithi063.stdout:3/211: chown d7/d22 9715235 1 2024-05-09T06:38:05.750 INFO:tasks.workunit.client.0.smithi063.stdout:2/230: creat dc/d32/f42 x:0 0 0 2024-05-09T06:38:05.750 INFO:tasks.workunit.client.0.smithi063.stdout:1/205: dread d9/f12 [0,4194304] 0 2024-05-09T06:38:05.753 INFO:tasks.workunit.client.0.smithi063.stdout:2/231: readlink dc/dd/d2d/l35 0 2024-05-09T06:38:05.753 INFO:tasks.workunit.client.0.smithi063.stdout:1/206: chown d9/d10/f1c 0 1 2024-05-09T06:38:05.755 INFO:tasks.workunit.client.0.smithi063.stdout:9/186: mkdir d0/d35 0 2024-05-09T06:38:05.757 INFO:tasks.workunit.client.0.smithi063.stdout:8/254: dwrite d9/f14 [0,4194304] 0 2024-05-09T06:38:05.757 INFO:tasks.workunit.client.0.smithi063.stdout:4/221: getdents d7/df/d1f/d28/d31 0 2024-05-09T06:38:05.761 INFO:tasks.workunit.client.0.smithi063.stdout:3/212: rename d7/d1e to d7/d4c 0 2024-05-09T06:38:05.763 INFO:tasks.workunit.client.0.smithi063.stdout:3/213: chown d7/da/f13 114431 1 2024-05-09T06:38:05.763 INFO:tasks.workunit.client.0.smithi063.stdout:7/222: dwrite d0/ff [4194304,4194304] 0 2024-05-09T06:38:05.765 INFO:tasks.workunit.client.0.smithi063.stdout:8/255: write d9/d37/f4d [3907851,47224] 0 2024-05-09T06:38:05.767 INFO:tasks.workunit.client.0.smithi063.stdout:7/223: dread d0/d17/f3b [0,4194304] 0 2024-05-09T06:38:05.768 INFO:tasks.workunit.client.0.smithi063.stdout:4/222: dread d7/df/d1a/f22 [0,4194304] 0 2024-05-09T06:38:05.769 INFO:tasks.workunit.client.0.smithi063.stdout:9/187: creat d0/d4/d26/f36 x:0 0 0 2024-05-09T06:38:05.771 INFO:tasks.workunit.client.0.smithi063.stdout:7/224: write d0/d15/f3a [955235,121675] 0 2024-05-09T06:38:05.774 INFO:tasks.workunit.client.0.smithi063.stdout:8/256: dwrite f2 [0,4194304] 0 2024-05-09T06:38:05.776 INFO:tasks.workunit.client.0.smithi063.stdout:9/188: creat d0/d4/f37 x:0 0 0 2024-05-09T06:38:05.777 INFO:tasks.workunit.client.0.smithi063.stdout:7/225: mknod d0/d17/c47 0 2024-05-09T06:38:05.779 INFO:tasks.workunit.client.0.smithi063.stdout:3/214: link d7/da/d21/d30/f3b d7/da/d21/f4d 0 2024-05-09T06:38:05.780 INFO:tasks.workunit.client.0.smithi063.stdout:7/226: truncate d0/d15/f1b 612153 0 2024-05-09T06:38:05.783 INFO:tasks.workunit.client.0.smithi063.stdout:8/257: dwrite d9/da/d18/f2d [0,4194304] 0 2024-05-09T06:38:05.786 INFO:tasks.workunit.client.0.smithi063.stdout:3/215: rmdir d7/da 39 2024-05-09T06:38:05.789 INFO:tasks.workunit.client.0.smithi063.stdout:2/232: truncate dc/dd/f23 3258594 0 2024-05-09T06:38:05.792 INFO:tasks.workunit.client.0.smithi063.stdout:8/258: symlink d9/da/d53/l54 0 2024-05-09T06:38:05.793 INFO:tasks.workunit.client.0.smithi063.stdout:8/259: chown d9/d41 136514 1 2024-05-09T06:38:05.794 INFO:tasks.workunit.client.0.smithi063.stdout:8/260: chown d9/da/d18/d46/f52 1 1 2024-05-09T06:38:05.794 INFO:tasks.workunit.client.0.smithi063.stdout:8/261: stat d9/da/d45 0 2024-05-09T06:38:05.795 INFO:tasks.workunit.client.0.smithi063.stdout:3/216: write d7/da/d21/f4d [925181,33005] 0 2024-05-09T06:38:05.799 INFO:tasks.workunit.client.0.smithi063.stdout:8/262: dread d9/d37/f3a [0,4194304] 0 2024-05-09T06:38:05.800 INFO:tasks.workunit.client.0.smithi063.stdout:3/217: creat d7/d2a/f4e x:0 0 0 2024-05-09T06:38:05.803 INFO:tasks.workunit.client.0.smithi063.stdout:9/189: getdents d0/d8 0 2024-05-09T06:38:05.804 INFO:tasks.workunit.client.0.smithi063.stdout:3/218: chown d7/da/lf 0 1 2024-05-09T06:38:05.804 INFO:tasks.workunit.client.0.smithi063.stdout:2/233: write f2 [4345768,27551] 0 2024-05-09T06:38:05.806 INFO:tasks.workunit.client.0.smithi063.stdout:4/223: dwrite d7/df/d1f/f27 [0,4194304] 0 2024-05-09T06:38:05.808 INFO:tasks.workunit.client.0.smithi063.stdout:4/224: dread - d7/df/d1f/f3a zero size 2024-05-09T06:38:05.809 INFO:tasks.workunit.client.0.smithi063.stdout:3/219: unlink d7/d4c/d25/f39 0 2024-05-09T06:38:05.809 INFO:tasks.workunit.client.0.smithi063.stdout:7/227: dwrite d0/f1c [0,4194304] 0 2024-05-09T06:38:05.810 INFO:tasks.workunit.client.0.smithi063.stdout:8/263: truncate d9/f14 551633 0 2024-05-09T06:38:05.813 INFO:tasks.workunit.client.0.smithi063.stdout:8/264: read - d9/da/d18/d24/f3e zero size 2024-05-09T06:38:05.815 INFO:tasks.workunit.client.0.smithi063.stdout:4/225: dread d7/df/d1f/f27 [0,4194304] 0 2024-05-09T06:38:05.815 INFO:tasks.workunit.client.0.smithi063.stdout:7/228: dread d0/d15/f26 [4194304,4194304] 0 2024-05-09T06:38:05.819 INFO:tasks.workunit.client.0.smithi063.stdout:3/220: symlink d7/d4c/d31/l4f 0 2024-05-09T06:38:05.820 INFO:tasks.workunit.client.0.smithi063.stdout:4/226: dread d7/df/d1a/f3b [0,4194304] 0 2024-05-09T06:38:05.821 INFO:tasks.workunit.client.0.smithi063.stdout:7/229: dread d0/d17/f30 [0,4194304] 0 2024-05-09T06:38:05.824 INFO:tasks.workunit.client.0.smithi063.stdout:8/265: mkdir d9/da/d18/d24/d55 0 2024-05-09T06:38:05.827 INFO:tasks.workunit.client.0.smithi063.stdout:3/221: read d7/f2f [371451,56203] 0 2024-05-09T06:38:05.827 INFO:tasks.workunit.client.0.smithi063.stdout:7/230: creat d0/d15/f48 x:0 0 0 2024-05-09T06:38:05.828 INFO:tasks.workunit.client.0.smithi063.stdout:4/227: dread d7/df/d1f/d28/f2e [0,4194304] 0 2024-05-09T06:38:05.829 INFO:tasks.workunit.client.0.smithi063.stdout:8/266: unlink d9/da/d18/d24/l26 0 2024-05-09T06:38:05.831 INFO:tasks.workunit.client.0.smithi063.stdout:7/231: mkdir d0/d15/d49 0 2024-05-09T06:38:05.834 INFO:tasks.workunit.client.0.smithi063.stdout:7/232: mkdir d0/d15/d4a 0 2024-05-09T06:38:05.835 INFO:tasks.workunit.client.0.smithi063.stdout:7/233: dread - d0/d17/f2e zero size 2024-05-09T06:38:05.837 INFO:tasks.workunit.client.0.smithi063.stdout:3/222: rename d7/da/d21/d30/f3b to d7/d22/f50 0 2024-05-09T06:38:05.840 INFO:tasks.workunit.client.0.smithi063.stdout:2/234: dwrite dc/dd/f23 [0,4194304] 0 2024-05-09T06:38:05.840 INFO:tasks.workunit.client.0.smithi063.stdout:4/228: rename d7/df/d1f/d28/f2e to d7/df/d1f/d28/d31/d32/d34/f44 0 2024-05-09T06:38:05.841 INFO:tasks.workunit.client.0.smithi063.stdout:3/223: mknod d7/d4c/c51 0 2024-05-09T06:38:05.841 INFO:tasks.workunit.client.0.smithi063.stdout:7/234: link d0/d15/l43 d0/d17/l4b 0 2024-05-09T06:38:05.843 INFO:tasks.workunit.client.0.smithi063.stdout:3/224: write d7/da/f13 [701200,99117] 0 2024-05-09T06:38:05.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/235: mkdir dc/d36/d43 0 2024-05-09T06:38:05.844 INFO:tasks.workunit.client.0.smithi063.stdout:3/225: stat d7/da/f13 0 2024-05-09T06:38:05.845 INFO:tasks.workunit.client.0.smithi063.stdout:7/235: creat d0/d15/d49/f4c x:0 0 0 2024-05-09T06:38:05.847 INFO:tasks.workunit.client.0.smithi063.stdout:7/236: fsync d0/f11 0 2024-05-09T06:38:05.847 INFO:tasks.workunit.client.0.smithi063.stdout:2/236: fdatasync dc/dd/f2f 0 2024-05-09T06:38:05.849 INFO:tasks.workunit.client.0.smithi063.stdout:2/237: chown dc/c31 0 1 2024-05-09T06:38:05.849 INFO:tasks.workunit.client.0.smithi063.stdout:6/229: sync 2024-05-09T06:38:05.850 INFO:tasks.workunit.client.0.smithi063.stdout:5/180: sync 2024-05-09T06:38:05.850 INFO:tasks.workunit.client.0.smithi063.stdout:7/237: write d0/d17/f3f [722425,41144] 0 2024-05-09T06:38:05.851 INFO:tasks.workunit.client.0.smithi063.stdout:4/229: dwrite d7/df/f21 [0,4194304] 0 2024-05-09T06:38:05.853 INFO:tasks.workunit.client.0.smithi063.stdout:8/267: dread d9/da/d18/d46/f50 [0,4194304] 0 2024-05-09T06:38:05.857 INFO:tasks.workunit.client.0.smithi063.stdout:2/238: dread f3 [0,4194304] 0 2024-05-09T06:38:05.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/238: rename d0/d17/l34 to d0/d17/l4d 0 2024-05-09T06:38:05.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/239: dread - d0/d15/f40 zero size 2024-05-09T06:38:05.862 INFO:tasks.workunit.client.0.smithi063.stdout:8/268: mkdir d9/da/d18/d56 0 2024-05-09T06:38:05.863 INFO:tasks.workunit.client.0.smithi063.stdout:6/230: creat d2/f4d x:0 0 0 2024-05-09T06:38:05.864 INFO:tasks.workunit.client.0.smithi063.stdout:6/231: chown d2/d4/cb 34856 1 2024-05-09T06:38:05.865 INFO:tasks.workunit.client.0.smithi063.stdout:2/239: mkdir dc/d36/d44 0 2024-05-09T06:38:05.866 INFO:tasks.workunit.client.0.smithi063.stdout:4/230: rename d7/df/d35/c3f to d7/df/d35/c45 0 2024-05-09T06:38:05.867 INFO:tasks.workunit.client.0.smithi063.stdout:8/269: dread f2 [0,4194304] 0 2024-05-09T06:38:05.869 INFO:tasks.workunit.client.0.smithi063.stdout:4/231: truncate d7/df/d1f/f27 5191771 0 2024-05-09T06:38:05.872 INFO:tasks.workunit.client.0.smithi063.stdout:6/232: mkdir d2/d1f/d29/d3b/d4e 0 2024-05-09T06:38:05.872 INFO:tasks.workunit.client.0.smithi063.stdout:5/181: write d8/fc [938782,72452] 0 2024-05-09T06:38:05.873 INFO:tasks.workunit.client.0.smithi063.stdout:6/233: chown d2/d1f/l35 1867950 1 2024-05-09T06:38:05.873 INFO:tasks.workunit.client.0.smithi063.stdout:4/232: symlink d7/df/d1f/d28/d31/d32/l46 0 2024-05-09T06:38:05.875 INFO:tasks.workunit.client.0.smithi063.stdout:4/233: write d7/df/f12 [2157567,10063] 0 2024-05-09T06:38:05.875 INFO:tasks.workunit.client.0.smithi063.stdout:8/270: mknod d9/da/d18/d56/c57 0 2024-05-09T06:38:05.877 INFO:tasks.workunit.client.0.smithi063.stdout:4/234: read f5 [2853871,16656] 0 2024-05-09T06:38:05.878 INFO:tasks.workunit.client.0.smithi063.stdout:3/226: dread f3 [0,4194304] 0 2024-05-09T06:38:05.878 INFO:tasks.workunit.client.0.smithi063.stdout:4/235: chown d7/df/d1f/l30 110056935 1 2024-05-09T06:38:05.879 INFO:tasks.workunit.client.0.smithi063.stdout:4/236: readlink d7/df/l37 0 2024-05-09T06:38:05.881 INFO:tasks.workunit.client.0.smithi063.stdout:8/271: fdatasync d9/da/d18/d46/f52 0 2024-05-09T06:38:05.882 INFO:tasks.workunit.client.0.smithi063.stdout:6/234: symlink d2/l4f 0 2024-05-09T06:38:05.886 INFO:tasks.workunit.client.0.smithi063.stdout:3/227: fdatasync f2 0 2024-05-09T06:38:05.886 INFO:tasks.workunit.client.0.smithi063.stdout:2/240: dwrite fa [0,4194304] 0 2024-05-09T06:38:05.886 INFO:tasks.workunit.client.0.smithi063.stdout:4/237: symlink d7/df/d1f/d28/d31/l47 0 2024-05-09T06:38:05.887 INFO:tasks.workunit.client.0.smithi063.stdout:5/182: truncate d8/fc 2720831 0 2024-05-09T06:38:05.887 INFO:tasks.workunit.client.0.smithi063.stdout:5/183: chown d8/d35 499 1 2024-05-09T06:38:05.889 INFO:tasks.workunit.client.0.smithi063.stdout:6/235: creat d2/df/f50 x:0 0 0 2024-05-09T06:38:05.892 INFO:tasks.workunit.client.0.smithi063.stdout:3/228: creat d7/d4c/f52 x:0 0 0 2024-05-09T06:38:05.894 INFO:tasks.workunit.client.0.smithi063.stdout:6/236: dwrite d2/f7 [0,4194304] 0 2024-05-09T06:38:05.896 INFO:tasks.workunit.client.0.smithi063.stdout:2/241: rmdir dc 39 2024-05-09T06:38:05.896 INFO:tasks.workunit.client.0.smithi063.stdout:4/238: fdatasync d7/df/d1f/d28/d31/d32/d34/f44 0 2024-05-09T06:38:05.897 INFO:tasks.workunit.client.0.smithi063.stdout:8/272: write f2 [1965285,72095] 0 2024-05-09T06:38:05.898 INFO:tasks.workunit.client.0.smithi063.stdout:6/237: symlink d2/d1f/d29/d3f/l51 0 2024-05-09T06:38:05.899 INFO:tasks.workunit.client.0.smithi063.stdout:3/229: dread d7/d2a/f4b [0,4194304] 0 2024-05-09T06:38:05.900 INFO:tasks.workunit.client.0.smithi063.stdout:5/184: link d8/f1d d8/d17/f3d 0 2024-05-09T06:38:05.903 INFO:tasks.workunit.client.0.smithi063.stdout:6/238: rename d2/d1f/l35 to d2/d20/d4a/l52 0 2024-05-09T06:38:05.904 INFO:tasks.workunit.client.0.smithi063.stdout:2/242: dread dc/dd/d2d/f34 [0,4194304] 0 2024-05-09T06:38:05.905 INFO:tasks.workunit.client.0.smithi063.stdout:3/230: write d7/da/f14 [212511,32875] 0 2024-05-09T06:38:05.907 INFO:tasks.workunit.client.0.smithi063.stdout:5/185: mkdir d8/d3e 0 2024-05-09T06:38:05.912 INFO:tasks.workunit.client.0.smithi063.stdout:2/243: getdents dc/d36/d40 0 2024-05-09T06:38:05.913 INFO:tasks.workunit.client.0.smithi063.stdout:8/273: getdents d9/da/d34 0 2024-05-09T06:38:05.915 INFO:tasks.workunit.client.0.smithi063.stdout:2/244: mkdir dc/dd/d45 0 2024-05-09T06:38:05.917 INFO:tasks.workunit.client.0.smithi063.stdout:6/239: getdents d2/d4 0 2024-05-09T06:38:05.917 INFO:tasks.workunit.client.0.smithi063.stdout:6/240: read d2/fd [1330598,173] 0 2024-05-09T06:38:05.917 INFO:tasks.workunit.client.0.smithi063.stdout:8/274: mkdir d9/da/d18/d56/d58 0 2024-05-09T06:38:05.918 INFO:tasks.workunit.client.0.smithi063.stdout:8/275: chown d9/da/d18/f23 986 1 2024-05-09T06:38:05.919 INFO:tasks.workunit.client.0.smithi063.stdout:3/231: getdents d7/d2a 0 2024-05-09T06:38:05.921 INFO:tasks.workunit.client.0.smithi063.stdout:6/241: creat d2/d12/f53 x:0 0 0 2024-05-09T06:38:05.923 INFO:tasks.workunit.client.0.smithi063.stdout:2/245: dwrite f6 [0,4194304] 0 2024-05-09T06:38:05.925 INFO:tasks.workunit.client.0.smithi063.stdout:8/276: rename d9/da/d18/d24/f3f to d9/da/d34/f59 0 2024-05-09T06:38:05.925 INFO:tasks.workunit.client.0.smithi063.stdout:8/277: readlink d9/da/l12 0 2024-05-09T06:38:05.926 INFO:tasks.workunit.client.0.smithi063.stdout:6/242: write d2/d1f/f4c [1351717,96389] 0 2024-05-09T06:38:05.927 INFO:tasks.workunit.client.0.smithi063.stdout:5/186: write d8/d17/d18/f1f [1766447,114572] 0 2024-05-09T06:38:05.930 INFO:tasks.workunit.client.0.smithi063.stdout:8/278: creat d9/d37/f5a x:0 0 0 2024-05-09T06:38:05.932 INFO:tasks.workunit.client.0.smithi063.stdout:5/187: dread d8/d17/f3d [0,4194304] 0 2024-05-09T06:38:05.933 INFO:tasks.workunit.client.0.smithi063.stdout:5/188: truncate d8/d17/d18/f2d 762068 0 2024-05-09T06:38:05.933 INFO:tasks.workunit.client.0.smithi063.stdout:6/243: rename d2/d1f/d29/d3f/l51 to d2/d20/l54 0 2024-05-09T06:38:05.934 INFO:tasks.workunit.client.0.smithi063.stdout:5/189: chown d8/d17/f26 26866674 1 2024-05-09T06:38:05.934 INFO:tasks.workunit.client.0.smithi063.stdout:6/244: dread - d2/df/f50 zero size 2024-05-09T06:38:05.935 INFO:tasks.workunit.client.0.smithi063.stdout:3/232: write d7/f11 [155146,95820] 0 2024-05-09T06:38:05.935 INFO:tasks.workunit.client.0.smithi063.stdout:6/245: chown d2/d20/d26/l2b 10155 1 2024-05-09T06:38:05.936 INFO:tasks.workunit.client.0.smithi063.stdout:1/207: sync 2024-05-09T06:38:05.936 INFO:tasks.workunit.client.0.smithi063.stdout:0/218: sync 2024-05-09T06:38:05.943 INFO:tasks.workunit.client.0.smithi063.stdout:1/208: fsync d9/d2a/d35/d24/f26 0 2024-05-09T06:38:05.946 INFO:tasks.workunit.client.0.smithi063.stdout:2/246: dwrite dc/dd/f17 [4194304,4194304] 0 2024-05-09T06:38:05.946 INFO:tasks.workunit.client.0.smithi063.stdout:0/219: dwrite f6 [4194304,4194304] 0 2024-05-09T06:38:05.948 INFO:tasks.workunit.client.0.smithi063.stdout:5/190: link d8/d17/f24 d8/d19/f3f 0 2024-05-09T06:38:05.952 INFO:tasks.workunit.client.0.smithi063.stdout:1/209: symlink d9/d10/l45 0 2024-05-09T06:38:05.953 INFO:tasks.workunit.client.0.smithi063.stdout:6/246: dwrite d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:05.955 INFO:tasks.workunit.client.0.smithi063.stdout:1/210: write d9/d10/f21 [1281430,130842] 0 2024-05-09T06:38:05.956 INFO:tasks.workunit.client.0.smithi063.stdout:6/247: chown d2/d1f/d29/d3f/l46 634770705 1 2024-05-09T06:38:05.958 INFO:tasks.workunit.client.0.smithi063.stdout:0/220: mknod dd/d1a/c49 0 2024-05-09T06:38:05.959 INFO:tasks.workunit.client.0.smithi063.stdout:5/191: mknod d8/c40 0 2024-05-09T06:38:05.959 INFO:tasks.workunit.client.0.smithi063.stdout:5/192: readlink d8/l28 0 2024-05-09T06:38:05.960 INFO:tasks.workunit.client.0.smithi063.stdout:6/248: mknod d2/d1f/d29/d3b/c55 0 2024-05-09T06:38:05.960 INFO:tasks.workunit.client.0.smithi063.stdout:0/221: rmdir dd/d1a/d22 39 2024-05-09T06:38:05.961 INFO:tasks.workunit.client.0.smithi063.stdout:0/222: write dd/d1a/f2e [362991,23875] 0 2024-05-09T06:38:05.962 INFO:tasks.workunit.client.0.smithi063.stdout:2/247: rmdir dc/d36/d44 0 2024-05-09T06:38:05.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/248: dread - dc/f1c zero size 2024-05-09T06:38:05.963 INFO:tasks.workunit.client.0.smithi063.stdout:0/223: write dd/d1a/f25 [1820420,89525] 0 2024-05-09T06:38:05.964 INFO:tasks.workunit.client.0.smithi063.stdout:5/193: creat d8/d35/f41 x:0 0 0 2024-05-09T06:38:05.967 INFO:tasks.workunit.client.0.smithi063.stdout:6/249: mkdir d2/d4/d2d/d42/d56 0 2024-05-09T06:38:05.970 INFO:tasks.workunit.client.0.smithi063.stdout:6/250: mknod d2/d20/c57 0 2024-05-09T06:38:05.973 INFO:tasks.workunit.client.0.smithi063.stdout:0/224: dwrite dd/d1a/d2c/f43 [0,4194304] 0 2024-05-09T06:38:05.976 INFO:tasks.workunit.client.0.smithi063.stdout:6/251: creat d2/d1f/d29/d3f/f58 x:0 0 0 2024-05-09T06:38:05.977 INFO:tasks.workunit.client.0.smithi063.stdout:2/249: dwrite dc/f25 [0,4194304] 0 2024-05-09T06:38:05.977 INFO:tasks.workunit.client.0.smithi063.stdout:6/252: readlink d2/d1f/l31 0 2024-05-09T06:38:05.980 INFO:tasks.workunit.client.0.smithi063.stdout:6/253: mkdir d2/df/d59 0 2024-05-09T06:38:05.983 INFO:tasks.workunit.client.0.smithi063.stdout:6/254: write d2/d1f/d29/d3f/f58 [798420,121108] 0 2024-05-09T06:38:05.984 INFO:tasks.workunit.client.0.smithi063.stdout:0/225: creat dd/d1a/d22/d23/d31/f4a x:0 0 0 2024-05-09T06:38:05.985 INFO:tasks.workunit.client.0.smithi063.stdout:0/226: write f2 [2395703,126539] 0 2024-05-09T06:38:05.987 INFO:tasks.workunit.client.0.smithi063.stdout:6/255: read d2/fd [4594238,13450] 0 2024-05-09T06:38:05.987 INFO:tasks.workunit.client.0.smithi063.stdout:2/250: mkdir dc/d36/d43/d46 0 2024-05-09T06:38:05.990 INFO:tasks.workunit.client.0.smithi063.stdout:2/251: dread dc/f12 [0,4194304] 0 2024-05-09T06:38:05.991 INFO:tasks.workunit.client.0.smithi063.stdout:6/256: link d2/d20/c57 d2/d12/c5a 0 2024-05-09T06:38:05.991 INFO:tasks.workunit.client.0.smithi063.stdout:2/252: write dc/f27 [219323,83866] 0 2024-05-09T06:38:05.998 INFO:tasks.workunit.client.0.smithi063.stdout:2/253: rmdir dc/d36/d40 0 2024-05-09T06:38:05.998 INFO:tasks.workunit.client.0.smithi063.stdout:6/257: dwrite d2/d1f/d29/d3f/f58 [0,4194304] 0 2024-05-09T06:38:06.000 INFO:tasks.workunit.client.0.smithi063.stdout:6/258: chown d2/d1f/l31 92 1 2024-05-09T06:38:06.003 INFO:tasks.workunit.client.0.smithi063.stdout:6/259: truncate d2/f4d 979371 0 2024-05-09T06:38:06.005 INFO:tasks.workunit.client.0.smithi063.stdout:2/254: rename lb to dc/dd/d2d/l47 0 2024-05-09T06:38:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:2/255: chown dc/l10 3190775 1 2024-05-09T06:38:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:0/227: dwrite dd/fe [0,4194304] 0 2024-05-09T06:38:06.007 INFO:tasks.workunit.client.0.smithi063.stdout:2/256: write dc/f27 [1259318,54852] 0 2024-05-09T06:38:06.008 INFO:tasks.workunit.client.0.smithi063.stdout:0/228: chown dd/d1a/f1b 609527418 1 2024-05-09T06:38:06.010 INFO:tasks.workunit.client.0.smithi063.stdout:9/190: sync 2024-05-09T06:38:06.010 INFO:tasks.workunit.client.0.smithi063.stdout:7/240: sync 2024-05-09T06:38:06.013 INFO:tasks.workunit.client.0.smithi063.stdout:7/241: read d0/f16 [2597873,19464] 0 2024-05-09T06:38:06.014 INFO:tasks.workunit.client.0.smithi063.stdout:0/229: dread dd/f2d [0,4194304] 0 2024-05-09T06:38:06.015 INFO:tasks.workunit.client.0.smithi063.stdout:6/260: dwrite d2/df/f28 [0,4194304] 0 2024-05-09T06:38:06.015 INFO:tasks.workunit.client.0.smithi063.stdout:9/191: truncate d0/d4/da/f1c 4831275 0 2024-05-09T06:38:06.017 INFO:tasks.workunit.client.0.smithi063.stdout:7/242: fdatasync d0/f10 0 2024-05-09T06:38:06.019 INFO:tasks.workunit.client.0.smithi063.stdout:0/230: dread - dd/d1a/d22/d23/d31/f41 zero size 2024-05-09T06:38:06.020 INFO:tasks.workunit.client.0.smithi063.stdout:6/261: dread - d2/d1f/d29/d3f/f25 zero size 2024-05-09T06:38:06.020 INFO:tasks.workunit.client.0.smithi063.stdout:2/257: dread dc/dd/f23 [0,4194304] 0 2024-05-09T06:38:06.020 INFO:tasks.workunit.client.0.smithi063.stdout:7/243: chown d0/d17/c44 50201250 1 2024-05-09T06:38:06.023 INFO:tasks.workunit.client.0.smithi063.stdout:9/192: mknod d0/d4/d26/c38 0 2024-05-09T06:38:06.023 INFO:tasks.workunit.client.0.smithi063.stdout:2/258: dread - dc/d32/f42 zero size 2024-05-09T06:38:06.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/259: read dc/f12 [54864,18436] 0 2024-05-09T06:38:06.025 INFO:tasks.workunit.client.0.smithi063.stdout:6/262: creat d2/d1f/f5b x:0 0 0 2024-05-09T06:38:06.025 INFO:tasks.workunit.client.0.smithi063.stdout:2/260: chown f3 0 1 2024-05-09T06:38:06.028 INFO:tasks.workunit.client.0.smithi063.stdout:0/231: creat dd/d1a/f4b x:0 0 0 2024-05-09T06:38:06.029 INFO:tasks.workunit.client.0.smithi063.stdout:0/232: chown dd/d1a/d2c/d42 1629 1 2024-05-09T06:38:06.029 INFO:tasks.workunit.client.0.smithi063.stdout:6/263: mkdir d2/d1f/d29/d3f/d5c 0 2024-05-09T06:38:06.030 INFO:tasks.workunit.client.0.smithi063.stdout:7/244: dwrite d0/d17/f1a [0,4194304] 0 2024-05-09T06:38:06.030 INFO:tasks.workunit.client.0.smithi063.stdout:2/261: unlink dc/c31 0 2024-05-09T06:38:06.031 INFO:tasks.workunit.client.0.smithi063.stdout:2/262: fsync dc/f1f 0 2024-05-09T06:38:06.033 INFO:tasks.workunit.client.0.smithi063.stdout:2/263: dread - dc/f1c zero size 2024-05-09T06:38:06.033 INFO:tasks.workunit.client.0.smithi063.stdout:9/193: rename d0/d35 to d0/d4/d26/d39 0 2024-05-09T06:38:06.034 INFO:tasks.workunit.client.0.smithi063.stdout:0/233: getdents dd/d1a/d2c/d42/d47 0 2024-05-09T06:38:06.036 INFO:tasks.workunit.client.0.smithi063.stdout:7/245: rename d0/d15/f40 to d0/d15/f4e 0 2024-05-09T06:38:06.036 INFO:tasks.workunit.client.0.smithi063.stdout:0/234: write dd/d1a/f35 [118803,12614] 0 2024-05-09T06:38:06.039 INFO:tasks.workunit.client.0.smithi063.stdout:9/194: creat d0/d4/f3a x:0 0 0 2024-05-09T06:38:06.040 INFO:tasks.workunit.client.0.smithi063.stdout:2/264: dwrite dc/f27 [0,4194304] 0 2024-05-09T06:38:06.041 INFO:tasks.workunit.client.0.smithi063.stdout:7/246: read d0/d15/f25 [303745,108394] 0 2024-05-09T06:38:06.042 INFO:tasks.workunit.client.0.smithi063.stdout:9/195: mkdir d0/d3b 0 2024-05-09T06:38:06.043 INFO:tasks.workunit.client.0.smithi063.stdout:2/265: truncate dc/d32/f42 891515 0 2024-05-09T06:38:06.044 INFO:tasks.workunit.client.0.smithi063.stdout:9/196: chown d0/d4/da/d1a/c22 2187495 1 2024-05-09T06:38:06.045 INFO:tasks.workunit.client.0.smithi063.stdout:7/247: unlink d0/d17/c47 0 2024-05-09T06:38:06.046 INFO:tasks.workunit.client.0.smithi063.stdout:9/197: write d0/d4/d26/f34 [650470,12474] 0 2024-05-09T06:38:06.049 INFO:tasks.workunit.client.0.smithi063.stdout:9/198: dread - d0/d4/d26/f36 zero size 2024-05-09T06:38:06.052 INFO:tasks.workunit.client.0.smithi063.stdout:0/235: truncate dd/d1a/d2c/f43 2708412 0 2024-05-09T06:38:06.053 INFO:tasks.workunit.client.0.smithi063.stdout:2/266: dwrite f6 [0,4194304] 0 2024-05-09T06:38:06.054 INFO:tasks.workunit.client.0.smithi063.stdout:0/236: chown f6 6 1 2024-05-09T06:38:06.056 INFO:tasks.workunit.client.0.smithi063.stdout:0/237: chown dd/l1f 97 1 2024-05-09T06:38:06.056 INFO:tasks.workunit.client.0.smithi063.stdout:7/248: dwrite d0/f1c [0,4194304] 0 2024-05-09T06:38:06.059 INFO:tasks.workunit.client.0.smithi063.stdout:7/249: fdatasync d0/ff 0 2024-05-09T06:38:06.064 INFO:tasks.workunit.client.0.smithi063.stdout:9/199: dwrite d0/d4/da/f1e [0,4194304] 0 2024-05-09T06:38:06.064 INFO:tasks.workunit.client.0.smithi063.stdout:0/238: dread f2 [0,4194304] 0 2024-05-09T06:38:06.065 INFO:tasks.workunit.client.0.smithi063.stdout:7/250: mkdir d0/d15/d49/d4f 0 2024-05-09T06:38:06.067 INFO:tasks.workunit.client.0.smithi063.stdout:0/239: truncate dd/d1a/f4b 864624 0 2024-05-09T06:38:06.069 INFO:tasks.workunit.client.0.smithi063.stdout:2/267: truncate dc/dd/d2d/f3e 3317423 0 2024-05-09T06:38:06.073 INFO:tasks.workunit.client.0.smithi063.stdout:7/251: rename d0/d17 to d0/d15/d4a/d50 0 2024-05-09T06:38:06.076 INFO:tasks.workunit.client.0.smithi063.stdout:2/268: creat dc/d32/d33/f48 x:0 0 0 2024-05-09T06:38:06.076 INFO:tasks.workunit.client.0.smithi063.stdout:7/252: fsync d0/d15/d4a/d50/f23 0 2024-05-09T06:38:06.077 INFO:tasks.workunit.client.0.smithi063.stdout:0/240: dread dd/fe [0,4194304] 0 2024-05-09T06:38:06.077 INFO:tasks.workunit.client.0.smithi063.stdout:9/200: dwrite d0/f28 [0,4194304] 0 2024-05-09T06:38:06.081 INFO:tasks.workunit.client.0.smithi063.stdout:0/241: rmdir dd/d1a/d22/d23 39 2024-05-09T06:38:06.082 INFO:tasks.workunit.client.0.smithi063.stdout:2/269: mkdir dc/d36/d43/d46/d49 0 2024-05-09T06:38:06.089 INFO:tasks.workunit.client.0.smithi063.stdout:9/201: creat d0/d4/da/f3c x:0 0 0 2024-05-09T06:38:06.093 INFO:tasks.workunit.client.0.smithi063.stdout:0/242: truncate dd/d1a/d22/d23/d33/f3e 340552 0 2024-05-09T06:38:06.094 INFO:tasks.workunit.client.0.smithi063.stdout:9/202: mknod d0/d4/d26/c3d 0 2024-05-09T06:38:06.097 INFO:tasks.workunit.client.0.smithi063.stdout:9/203: mkdir d0/d4/d26/d39/d3e 0 2024-05-09T06:38:06.103 INFO:tasks.workunit.client.0.smithi063.stdout:0/243: dread dd/d1a/d22/d23/d33/f3e [0,4194304] 0 2024-05-09T06:38:06.103 INFO:tasks.workunit.client.0.smithi063.stdout:7/253: dwrite d0/d15/f26 [0,4194304] 0 2024-05-09T06:38:06.104 INFO:tasks.workunit.client.0.smithi063.stdout:9/204: rename d0/d4/da/f3c to d0/d4/da/f3f 0 2024-05-09T06:38:06.106 INFO:tasks.workunit.client.0.smithi063.stdout:2/270: dwrite dc/f12 [0,4194304] 0 2024-05-09T06:38:06.108 INFO:tasks.workunit.client.0.smithi063.stdout:2/271: chown dc/dd/l29 1611111 1 2024-05-09T06:38:06.109 INFO:tasks.workunit.client.0.smithi063.stdout:0/244: mknod dd/d1a/d22/d23/d31/c4c 0 2024-05-09T06:38:06.112 INFO:tasks.workunit.client.0.smithi063.stdout:7/254: symlink d0/d15/d49/l51 0 2024-05-09T06:38:06.112 INFO:tasks.workunit.client.0.smithi063.stdout:9/205: symlink d0/d8/d2d/l40 0 2024-05-09T06:38:06.112 INFO:tasks.workunit.client.0.smithi063.stdout:7/255: stat d0/d15/d4a/d50/f3b 0 2024-05-09T06:38:06.113 INFO:tasks.workunit.client.0.smithi063.stdout:7/256: chown d0/d15/f1b 37 1 2024-05-09T06:38:06.113 INFO:tasks.workunit.client.0.smithi063.stdout:2/272: symlink dc/dd/l4a 0 2024-05-09T06:38:06.114 INFO:tasks.workunit.client.0.smithi063.stdout:2/273: chown dc/dd/d2d/f38 44 1 2024-05-09T06:38:06.114 INFO:tasks.workunit.client.0.smithi063.stdout:9/206: creat d0/d8/d2d/f41 x:0 0 0 2024-05-09T06:38:06.117 INFO:tasks.workunit.client.0.smithi063.stdout:7/257: symlink d0/l52 0 2024-05-09T06:38:06.119 INFO:tasks.workunit.client.0.smithi063.stdout:9/207: unlink d0/d4/da/f1e 0 2024-05-09T06:38:06.124 INFO:tasks.workunit.client.0.smithi063.stdout:4/239: sync 2024-05-09T06:38:06.140 INFO:tasks.workunit.client.0.smithi063.stdout:7/258: creat d0/d15/d4a/f53 x:0 0 0 2024-05-09T06:38:06.140 INFO:tasks.workunit.client.0.smithi063.stdout:9/208: mkdir d0/d4/d26/d39/d42 0 2024-05-09T06:38:06.141 INFO:tasks.workunit.client.0.smithi063.stdout:7/259: stat d0/f1c 0 2024-05-09T06:38:06.143 INFO:tasks.workunit.client.0.smithi063.stdout:9/209: write d0/d4/d26/f34 [1570624,107453] 0 2024-05-09T06:38:06.143 INFO:tasks.workunit.client.0.smithi063.stdout:7/260: fdatasync d0/d15/f25 0 2024-05-09T06:38:06.146 INFO:tasks.workunit.client.0.smithi063.stdout:9/210: mkdir d0/d43 0 2024-05-09T06:38:06.146 INFO:tasks.workunit.client.0.smithi063.stdout:2/274: dwrite dc/dd/f2f [0,4194304] 0 2024-05-09T06:38:06.148 INFO:tasks.workunit.client.0.smithi063.stdout:2/275: rmdir dc/d36/d43 39 2024-05-09T06:38:06.150 INFO:tasks.workunit.client.0.smithi063.stdout:9/211: mknod d0/d4/da/d1a/c44 0 2024-05-09T06:38:06.151 INFO:tasks.workunit.client.0.smithi063.stdout:2/276: chown dc/d36/d43/d46 15168188 1 2024-05-09T06:38:06.154 INFO:tasks.workunit.client.0.smithi063.stdout:2/277: symlink dc/d36/d43/d46/l4b 0 2024-05-09T06:38:06.156 INFO:tasks.workunit.client.0.smithi063.stdout:2/278: read dc/dd/f20 [237178,35071] 0 2024-05-09T06:38:06.157 INFO:tasks.workunit.client.0.smithi063.stdout:4/240: truncate d7/df/d1f/f23 1691335 0 2024-05-09T06:38:06.158 INFO:tasks.workunit.client.0.smithi063.stdout:4/241: write d7/f2d [3974847,44388] 0 2024-05-09T06:38:06.159 INFO:tasks.workunit.client.0.smithi063.stdout:2/279: creat dc/d32/d33/f4c x:0 0 0 2024-05-09T06:38:06.160 INFO:tasks.workunit.client.0.smithi063.stdout:4/242: readlink d7/le 0 2024-05-09T06:38:06.161 INFO:tasks.workunit.client.0.smithi063.stdout:4/243: write f6 [3784839,93626] 0 2024-05-09T06:38:06.162 INFO:tasks.workunit.client.0.smithi063.stdout:9/212: dwrite d0/f24 [4194304,4194304] 0 2024-05-09T06:38:06.163 INFO:tasks.workunit.client.0.smithi063.stdout:2/280: symlink dc/dd/l4d 0 2024-05-09T06:38:06.166 INFO:tasks.workunit.client.0.smithi063.stdout:4/244: symlink d7/df/d1f/d28/l48 0 2024-05-09T06:38:06.168 INFO:tasks.workunit.client.0.smithi063.stdout:2/281: mkdir dc/d32/d33/d4e 0 2024-05-09T06:38:06.172 INFO:tasks.workunit.client.0.smithi063.stdout:9/213: mknod d0/d3b/c45 0 2024-05-09T06:38:06.175 INFO:tasks.workunit.client.0.smithi063.stdout:2/282: rename dc/d32/d33/f3f to dc/dd/f4f 0 2024-05-09T06:38:06.176 INFO:tasks.workunit.client.0.smithi063.stdout:4/245: creat d7/d3d/f49 x:0 0 0 2024-05-09T06:38:06.182 INFO:tasks.workunit.client.0.smithi063.stdout:9/214: rename d0/d4/d26/c2e to d0/d4/c46 0 2024-05-09T06:38:06.182 INFO:tasks.workunit.client.0.smithi063.stdout:9/215: chown d0/d4/da/d1a 15757244 1 2024-05-09T06:38:06.183 INFO:tasks.workunit.client.0.smithi063.stdout:9/216: dread - d0/d4/da/d1a/f2b zero size 2024-05-09T06:38:06.188 INFO:tasks.workunit.client.0.smithi063.stdout:2/283: truncate dc/f12 3611851 0 2024-05-09T06:38:06.189 INFO:tasks.workunit.client.0.smithi063.stdout:9/217: dwrite d0/d4/f3a [0,4194304] 0 2024-05-09T06:38:06.191 INFO:tasks.workunit.client.0.smithi063.stdout:2/284: rmdir dc/dd 39 2024-05-09T06:38:06.192 INFO:tasks.workunit.client.0.smithi063.stdout:4/246: dwrite d7/df/d1a/f3b [0,4194304] 0 2024-05-09T06:38:06.197 INFO:tasks.workunit.client.0.smithi063.stdout:2/285: dread fa [0,4194304] 0 2024-05-09T06:38:06.197 INFO:tasks.workunit.client.0.smithi063.stdout:9/218: rename d0/d4/da/d1a to d0/d4/d26/d47 0 2024-05-09T06:38:06.198 INFO:tasks.workunit.client.0.smithi063.stdout:4/247: mknod d7/df/d1f/c4a 0 2024-05-09T06:38:06.199 INFO:tasks.workunit.client.0.smithi063.stdout:4/248: dread - d7/f42 zero size 2024-05-09T06:38:06.202 INFO:tasks.workunit.client.0.smithi063.stdout:2/286: stat dc/dd/d2d/f38 0 2024-05-09T06:38:06.204 INFO:tasks.workunit.client.0.smithi063.stdout:4/249: write d7/df/d1f/d28/d31/d32/d34/f44 [1270004,91486] 0 2024-05-09T06:38:06.205 INFO:tasks.workunit.client.0.smithi063.stdout:9/219: dwrite d0/d4/d26/f36 [0,4194304] 0 2024-05-09T06:38:06.206 INFO:tasks.workunit.client.0.smithi063.stdout:2/287: mknod dc/d32/d33/d4e/c50 0 2024-05-09T06:38:06.207 INFO:tasks.workunit.client.0.smithi063.stdout:9/220: chown d0/f28 203 1 2024-05-09T06:38:06.207 INFO:tasks.workunit.client.0.smithi063.stdout:2/288: chown dc/f1f 0 1 2024-05-09T06:38:06.209 INFO:tasks.workunit.client.0.smithi063.stdout:2/289: fdatasync dc/dd/f23 0 2024-05-09T06:38:06.210 INFO:tasks.workunit.client.0.smithi063.stdout:9/221: mkdir d0/d4/d26/d39/d48 0 2024-05-09T06:38:06.213 INFO:tasks.workunit.client.0.smithi063.stdout:4/250: dwrite d7/df/d1a/f22 [0,4194304] 0 2024-05-09T06:38:06.214 INFO:tasks.workunit.client.0.smithi063.stdout:4/251: chown d7/df/d1a 1084909919 1 2024-05-09T06:38:06.215 INFO:tasks.workunit.client.0.smithi063.stdout:4/252: readlink d7/df/d1f/d28/d31/l47 0 2024-05-09T06:38:06.216 INFO:tasks.workunit.client.0.smithi063.stdout:9/222: creat d0/d4/d26/f49 x:0 0 0 2024-05-09T06:38:06.217 INFO:tasks.workunit.client.0.smithi063.stdout:3/233: sync 2024-05-09T06:38:06.217 INFO:tasks.workunit.client.0.smithi063.stdout:1/211: sync 2024-05-09T06:38:06.217 INFO:tasks.workunit.client.0.smithi063.stdout:8/279: sync 2024-05-09T06:38:06.217 INFO:tasks.workunit.client.0.smithi063.stdout:8/280: chown d9/d37 14711 1 2024-05-09T06:38:06.218 INFO:tasks.workunit.client.0.smithi063.stdout:9/223: fdatasync d0/f1f 0 2024-05-09T06:38:06.220 INFO:tasks.workunit.client.0.smithi063.stdout:4/253: rename d7/df/d1a/c1d to d7/df/d35/c4b 0 2024-05-09T06:38:06.220 INFO:tasks.workunit.client.0.smithi063.stdout:1/212: write d9/d2a/d35/f44 [747453,94456] 0 2024-05-09T06:38:06.221 INFO:tasks.workunit.client.0.smithi063.stdout:9/224: read d0/d4/da/f15 [758422,125262] 0 2024-05-09T06:38:06.222 INFO:tasks.workunit.client.0.smithi063.stdout:3/234: creat d7/da/d21/d30/d44/d48/f53 x:0 0 0 2024-05-09T06:38:06.223 INFO:tasks.workunit.client.0.smithi063.stdout:3/235: dread - d7/d2a/f33 zero size 2024-05-09T06:38:06.224 INFO:tasks.workunit.client.0.smithi063.stdout:4/254: mknod d7/df/d1a/c4c 0 2024-05-09T06:38:06.224 INFO:tasks.workunit.client.0.smithi063.stdout:8/281: creat d9/da/d18/d24/d55/f5b x:0 0 0 2024-05-09T06:38:06.225 INFO:tasks.workunit.client.0.smithi063.stdout:4/255: write d7/d3d/f49 [148088,97583] 0 2024-05-09T06:38:06.226 INFO:tasks.workunit.client.0.smithi063.stdout:1/213: mkdir d9/d46 0 2024-05-09T06:38:06.226 INFO:tasks.workunit.client.0.smithi063.stdout:4/256: fdatasync d7/f39 0 2024-05-09T06:38:06.227 INFO:tasks.workunit.client.0.smithi063.stdout:1/214: stat l4 0 2024-05-09T06:38:06.228 INFO:tasks.workunit.client.0.smithi063.stdout:1/215: chown d9 0 1 2024-05-09T06:38:06.228 INFO:tasks.workunit.client.0.smithi063.stdout:3/236: creat d7/d4c/f54 x:0 0 0 2024-05-09T06:38:06.228 INFO:tasks.workunit.client.0.smithi063.stdout:9/225: symlink d0/d43/l4a 0 2024-05-09T06:38:06.230 INFO:tasks.workunit.client.0.smithi063.stdout:2/290: truncate f3 55244 0 2024-05-09T06:38:06.233 INFO:tasks.workunit.client.0.smithi063.stdout:4/257: dread d7/df/f21 [0,4194304] 0 2024-05-09T06:38:06.233 INFO:tasks.workunit.client.0.smithi063.stdout:9/226: creat d0/d43/f4b x:0 0 0 2024-05-09T06:38:06.234 INFO:tasks.workunit.client.0.smithi063.stdout:1/216: symlink d9/d46/l47 0 2024-05-09T06:38:06.235 INFO:tasks.workunit.client.0.smithi063.stdout:2/291: getdents dc/d36/d43/d46/d49 0 2024-05-09T06:38:06.236 INFO:tasks.workunit.client.0.smithi063.stdout:3/237: link d7/c38 d7/d2a/d2d/d36/c55 0 2024-05-09T06:38:06.236 INFO:tasks.workunit.client.0.smithi063.stdout:9/227: truncate d0/d4/d26/d47/f1d 1282229 0 2024-05-09T06:38:06.237 INFO:tasks.workunit.client.0.smithi063.stdout:1/217: dread - d9/d2a/d35/d1e/f42 zero size 2024-05-09T06:38:06.237 INFO:tasks.workunit.client.0.smithi063.stdout:4/258: rename d7/df/d1f/d28/d31/d32/l46 to d7/df/d1f/d28/d31/d32/d34/l4d 0 2024-05-09T06:38:06.238 INFO:tasks.workunit.client.0.smithi063.stdout:1/218: fsync d9/d2a/d35/d1e/f27 0 2024-05-09T06:38:06.239 INFO:tasks.workunit.client.0.smithi063.stdout:2/292: unlink dc/dd/l4d 0 2024-05-09T06:38:06.239 INFO:tasks.workunit.client.0.smithi063.stdout:3/238: symlink d7/da/d21/d30/d44/l56 0 2024-05-09T06:38:06.240 INFO:tasks.workunit.client.0.smithi063.stdout:4/259: chown d7/df/d1f/d28/d31/d32/d34/f44 24728 1 2024-05-09T06:38:06.242 INFO:tasks.workunit.client.0.smithi063.stdout:4/260: read d7/df/f21 [2092048,112551] 0 2024-05-09T06:38:06.244 INFO:tasks.workunit.client.0.smithi063.stdout:1/219: dread d9/d10/f3e [0,4194304] 0 2024-05-09T06:38:06.245 INFO:tasks.workunit.client.0.smithi063.stdout:3/239: rename d7/d4c/f52 to d7/da/d21/d30/f57 0 2024-05-09T06:38:06.245 INFO:tasks.workunit.client.0.smithi063.stdout:8/282: rmdir d9/da 39 2024-05-09T06:38:06.247 INFO:tasks.workunit.client.0.smithi063.stdout:3/240: read d7/f10 [2123124,41462] 0 2024-05-09T06:38:06.248 INFO:tasks.workunit.client.0.smithi063.stdout:4/261: rename d7/df/d1a/c4c to d7/df/d1f/d28/d31/d32/d34/c4e 0 2024-05-09T06:38:06.249 INFO:tasks.workunit.client.0.smithi063.stdout:9/228: dread d0/f1f [0,4194304] 0 2024-05-09T06:38:06.250 INFO:tasks.workunit.client.0.smithi063.stdout:9/229: fsync d0/d4/d26/f49 0 2024-05-09T06:38:06.251 INFO:tasks.workunit.client.0.smithi063.stdout:9/230: chown d0/d8/d2d 71133 1 2024-05-09T06:38:06.252 INFO:tasks.workunit.client.0.smithi063.stdout:8/283: creat d9/d3b/f5c x:0 0 0 2024-05-09T06:38:06.252 INFO:tasks.workunit.client.0.smithi063.stdout:9/231: chown d0/f24 463423061 1 2024-05-09T06:38:06.255 INFO:tasks.workunit.client.0.smithi063.stdout:4/262: symlink d7/df/d1f/l4f 0 2024-05-09T06:38:06.257 INFO:tasks.workunit.client.0.smithi063.stdout:9/232: mknod d0/d8/d2d/c4c 0 2024-05-09T06:38:06.260 INFO:tasks.workunit.client.0.smithi063.stdout:4/263: chown d7/df/d1f/d28/d31/d32/d34/l4d 2027433436 1 2024-05-09T06:38:06.261 INFO:tasks.workunit.client.0.smithi063.stdout:6/264: sync 2024-05-09T06:38:06.261 INFO:tasks.workunit.client.0.smithi063.stdout:5/194: sync 2024-05-09T06:38:06.262 INFO:tasks.workunit.client.0.smithi063.stdout:9/233: creat d0/d4/f4d x:0 0 0 2024-05-09T06:38:06.262 INFO:tasks.workunit.client.0.smithi063.stdout:5/195: chown d8/f1d 172505 1 2024-05-09T06:38:06.263 INFO:tasks.workunit.client.0.smithi063.stdout:1/220: dread d9/d2a/d35/d1e/f3a [0,4194304] 0 2024-05-09T06:38:06.263 INFO:tasks.workunit.client.0.smithi063.stdout:4/264: mkdir d7/d50 0 2024-05-09T06:38:06.264 INFO:tasks.workunit.client.0.smithi063.stdout:2/293: truncate dc/dd/f13 44188 0 2024-05-09T06:38:06.269 INFO:tasks.workunit.client.0.smithi063.stdout:4/265: creat d7/df/d1f/d28/d31/d32/f51 x:0 0 0 2024-05-09T06:38:06.271 INFO:tasks.workunit.client.0.smithi063.stdout:8/284: link d9/d3b/l35 d9/da/d45/l5d 0 2024-05-09T06:38:06.272 INFO:tasks.workunit.client.0.smithi063.stdout:3/241: write d7/d2a/f33 [91577,120237] 0 2024-05-09T06:38:06.273 INFO:tasks.workunit.client.0.smithi063.stdout:9/234: symlink d0/l4e 0 2024-05-09T06:38:06.274 INFO:tasks.workunit.client.0.smithi063.stdout:1/221: fsync f3 0 2024-05-09T06:38:06.277 INFO:tasks.workunit.client.0.smithi063.stdout:9/235: dread d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:06.278 INFO:tasks.workunit.client.0.smithi063.stdout:9/236: readlink d0/d8/l29 0 2024-05-09T06:38:06.278 INFO:tasks.workunit.client.0.smithi063.stdout:8/285: rename d9/da/d18/d24/d55/f5b to d9/d41/f5e 0 2024-05-09T06:38:06.280 INFO:tasks.workunit.client.0.smithi063.stdout:8/286: truncate d9/da/f2f 4770991 0 2024-05-09T06:38:06.281 INFO:tasks.workunit.client.0.smithi063.stdout:8/287: chown d9/da/d18/d24/f3e 128206413 1 2024-05-09T06:38:06.281 INFO:tasks.workunit.client.0.smithi063.stdout:4/266: symlink d7/d50/l52 0 2024-05-09T06:38:06.282 INFO:tasks.workunit.client.0.smithi063.stdout:5/196: dread d8/d17/f24 [0,4194304] 0 2024-05-09T06:38:06.283 INFO:tasks.workunit.client.0.smithi063.stdout:1/222: fsync d9/f11 0 2024-05-09T06:38:06.283 INFO:tasks.workunit.client.0.smithi063.stdout:9/237: dread d0/d4/d26/f36 [0,4194304] 0 2024-05-09T06:38:06.283 INFO:tasks.workunit.client.0.smithi063.stdout:8/288: write d9/d37/f4d [1543969,21947] 0 2024-05-09T06:38:06.285 INFO:tasks.workunit.client.0.smithi063.stdout:8/289: chown d9/da/l33 19045386 1 2024-05-09T06:38:06.295 INFO:tasks.workunit.client.0.smithi063.stdout:3/242: link f1 d7/d22/d28/f58 0 2024-05-09T06:38:06.306 INFO:tasks.workunit.client.0.smithi063.stdout:4/267: rename d7/df/d1f/f27 to d7/df/d1f/f53 0 2024-05-09T06:38:06.308 INFO:tasks.workunit.client.0.smithi063.stdout:2/294: link dc/dd/d2d/l47 dc/dd/l51 0 2024-05-09T06:38:06.309 INFO:tasks.workunit.client.0.smithi063.stdout:2/295: chown dc/d32 1317993 1 2024-05-09T06:38:06.312 INFO:tasks.workunit.client.0.smithi063.stdout:8/290: mknod d9/da/d34/c5f 0 2024-05-09T06:38:06.320 INFO:tasks.workunit.client.0.smithi063.stdout:5/197: dwrite d8/d19/f3f [0,4194304] 0 2024-05-09T06:38:06.322 INFO:tasks.workunit.client.0.smithi063.stdout:1/223: symlink d9/d2a/d35/l48 0 2024-05-09T06:38:06.323 INFO:tasks.workunit.client.0.smithi063.stdout:1/224: write d9/d10/f21 [5059060,99916] 0 2024-05-09T06:38:06.323 INFO:tasks.workunit.client.0.smithi063.stdout:2/296: fdatasync dc/dd/d2d/f3e 0 2024-05-09T06:38:06.324 INFO:tasks.workunit.client.0.smithi063.stdout:2/297: stat dc/dd/c30 0 2024-05-09T06:38:06.325 INFO:tasks.workunit.client.0.smithi063.stdout:3/243: write d7/f10 [3651502,48093] 0 2024-05-09T06:38:06.327 INFO:tasks.workunit.client.0.smithi063.stdout:8/291: rmdir d9/da/d34 39 2024-05-09T06:38:06.328 INFO:tasks.workunit.client.0.smithi063.stdout:8/292: write d9/d3b/f40 [368958,68192] 0 2024-05-09T06:38:06.333 INFO:tasks.workunit.client.0.smithi063.stdout:5/198: creat d8/d17/d1e/f42 x:0 0 0 2024-05-09T06:38:06.334 INFO:tasks.workunit.client.0.smithi063.stdout:5/199: fdatasync d8/d17/d18/f29 0 2024-05-09T06:38:06.336 INFO:tasks.workunit.client.0.smithi063.stdout:1/225: fdatasync d9/f12 0 2024-05-09T06:38:06.337 INFO:tasks.workunit.client.0.smithi063.stdout:1/226: chown d9/d10/d2b/c32 91778 1 2024-05-09T06:38:06.341 INFO:tasks.workunit.client.0.smithi063.stdout:8/293: symlink d9/da/d45/l60 0 2024-05-09T06:38:06.343 INFO:tasks.workunit.client.0.smithi063.stdout:4/268: link d7/df/d1f/d28/c2c d7/df/d1f/d28/d31/d32/d34/c54 0 2024-05-09T06:38:06.349 INFO:tasks.workunit.client.0.smithi063.stdout:5/200: rename d8/fe to d8/d19/d30/f43 0 2024-05-09T06:38:06.352 INFO:tasks.workunit.client.0.smithi063.stdout:9/238: getdents d0/d8/d2d 0 2024-05-09T06:38:06.356 INFO:tasks.workunit.client.0.smithi063.stdout:4/269: write d7/df/d35/f36 [174630,65956] 0 2024-05-09T06:38:06.357 INFO:tasks.workunit.client.0.smithi063.stdout:1/227: dread f3 [0,4194304] 0 2024-05-09T06:38:06.357 INFO:tasks.workunit.client.0.smithi063.stdout:4/270: chown d7/df/d1f/d28/c40 10 1 2024-05-09T06:38:06.358 INFO:tasks.workunit.client.0.smithi063.stdout:5/201: truncate d8/d17/f1c 2398000 0 2024-05-09T06:38:06.359 INFO:tasks.workunit.client.0.smithi063.stdout:2/298: creat dc/d36/f52 x:0 0 0 2024-05-09T06:38:06.361 INFO:tasks.workunit.client.0.smithi063.stdout:5/202: read d8/f10 [1870135,116630] 0 2024-05-09T06:38:06.361 INFO:tasks.workunit.client.0.smithi063.stdout:8/294: rename d9/da/d34/c4f to d9/d37/c61 0 2024-05-09T06:38:06.364 INFO:tasks.workunit.client.0.smithi063.stdout:4/271: write d7/df/d1f/f53 [2327051,97319] 0 2024-05-09T06:38:06.364 INFO:tasks.workunit.client.0.smithi063.stdout:3/244: getdents d7/da/d21 0 2024-05-09T06:38:06.366 INFO:tasks.workunit.client.0.smithi063.stdout:1/228: dwrite d9/d2a/d35/d1e/f27 [0,4194304] 0 2024-05-09T06:38:06.367 INFO:tasks.workunit.client.0.smithi063.stdout:0/245: sync 2024-05-09T06:38:06.368 INFO:tasks.workunit.client.0.smithi063.stdout:7/261: sync 2024-05-09T06:38:06.368 INFO:tasks.workunit.client.0.smithi063.stdout:2/299: rename dc/d32/d33/f48 to dc/dd/d45/f53 0 2024-05-09T06:38:06.370 INFO:tasks.workunit.client.0.smithi063.stdout:0/246: chown dd/d1a/d22/d3f/d44 0 1 2024-05-09T06:38:06.374 INFO:tasks.workunit.client.0.smithi063.stdout:3/245: unlink d7/d4c/c51 0 2024-05-09T06:38:06.376 INFO:tasks.workunit.client.0.smithi063.stdout:4/272: dread d7/df/d1f/f53 [0,4194304] 0 2024-05-09T06:38:06.378 INFO:tasks.workunit.client.0.smithi063.stdout:5/203: link d8/d17/d18/f1f d8/d19/f44 0 2024-05-09T06:38:06.379 INFO:tasks.workunit.client.0.smithi063.stdout:3/246: chown d7/d2a/d2d/f3e 25003 1 2024-05-09T06:38:06.381 INFO:tasks.workunit.client.0.smithi063.stdout:1/229: dread d9/d2a/d35/d1e/f27 [0,4194304] 0 2024-05-09T06:38:06.382 INFO:tasks.workunit.client.0.smithi063.stdout:7/262: dwrite d0/d15/f2b [0,4194304] 0 2024-05-09T06:38:06.382 INFO:tasks.workunit.client.0.smithi063.stdout:3/247: chown d7/d22/c32 57825 1 2024-05-09T06:38:06.382 INFO:tasks.workunit.client.0.smithi063.stdout:2/300: dwrite dc/dd/f2f [0,4194304] 0 2024-05-09T06:38:06.384 INFO:tasks.workunit.client.0.smithi063.stdout:2/301: chown dc/dd/d2d/f34 198 1 2024-05-09T06:38:06.388 INFO:tasks.workunit.client.0.smithi063.stdout:0/247: write dd/d1a/d22/d23/f2f [4921468,23860] 0 2024-05-09T06:38:06.390 INFO:tasks.workunit.client.0.smithi063.stdout:2/302: write f2 [5056645,25851] 0 2024-05-09T06:38:06.392 INFO:tasks.workunit.client.0.smithi063.stdout:2/303: read dc/dd/d2d/f38 [361470,65058] 0 2024-05-09T06:38:06.395 INFO:tasks.workunit.client.0.smithi063.stdout:1/230: dwrite d9/d10/f15 [0,4194304] 0 2024-05-09T06:38:06.396 INFO:tasks.workunit.client.0.smithi063.stdout:3/248: mknod d7/d4c/d25/c59 0 2024-05-09T06:38:06.400 INFO:tasks.workunit.client.0.smithi063.stdout:7/263: mkdir d0/d15/d49/d54 0 2024-05-09T06:38:06.401 INFO:tasks.workunit.client.0.smithi063.stdout:7/264: chown d0/d15/d4a/d50/f1a 1528 1 2024-05-09T06:38:06.406 INFO:tasks.workunit.client.0.smithi063.stdout:0/248: fdatasync dd/fe 0 2024-05-09T06:38:06.411 INFO:tasks.workunit.client.0.smithi063.stdout:4/273: dread d7/df/d1f/f23 [0,4194304] 0 2024-05-09T06:38:06.412 INFO:tasks.workunit.client.0.smithi063.stdout:2/304: rename dc/l24 to dc/d32/d33/l54 0 2024-05-09T06:38:06.413 INFO:tasks.workunit.client.0.smithi063.stdout:1/231: symlink d9/d2a/d35/d1e/l49 0 2024-05-09T06:38:06.415 INFO:tasks.workunit.client.0.smithi063.stdout:3/249: creat d7/da/d21/d30/f5a x:0 0 0 2024-05-09T06:38:06.417 INFO:tasks.workunit.client.0.smithi063.stdout:7/265: readlink d0/d15/d4a/d50/l2a 0 2024-05-09T06:38:06.417 INFO:tasks.workunit.client.0.smithi063.stdout:7/266: dread - d0/d15/f4e zero size 2024-05-09T06:38:06.418 INFO:tasks.workunit.client.0.smithi063.stdout:7/267: chown d0/l13 106120 1 2024-05-09T06:38:06.422 INFO:tasks.workunit.client.0.smithi063.stdout:4/274: rmdir d7/df/d1f/d28 39 2024-05-09T06:38:06.422 INFO:tasks.workunit.client.0.smithi063.stdout:4/275: truncate f6 4334455 0 2024-05-09T06:38:06.426 INFO:tasks.workunit.client.0.smithi063.stdout:3/250: fsync d7/da/d21/f4d 0 2024-05-09T06:38:06.426 INFO:tasks.workunit.client.0.smithi063.stdout:6/265: sync 2024-05-09T06:38:06.427 INFO:tasks.workunit.client.0.smithi063.stdout:8/295: truncate d9/da/d34/f59 2024524 0 2024-05-09T06:38:06.432 INFO:tasks.workunit.client.0.smithi063.stdout:4/276: creat d7/d3d/f55 x:0 0 0 2024-05-09T06:38:06.434 INFO:tasks.workunit.client.0.smithi063.stdout:2/305: symlink dc/l55 0 2024-05-09T06:38:06.434 INFO:tasks.workunit.client.0.smithi063.stdout:5/204: write d8/f16 [422104,38269] 0 2024-05-09T06:38:06.435 INFO:tasks.workunit.client.0.smithi063.stdout:2/306: fsync dc/d32/f42 0 2024-05-09T06:38:06.437 INFO:tasks.workunit.client.0.smithi063.stdout:3/251: mkdir d7/d2a/d5b 0 2024-05-09T06:38:06.438 INFO:tasks.workunit.client.0.smithi063.stdout:3/252: fsync d7/d22/f27 0 2024-05-09T06:38:06.439 INFO:tasks.workunit.client.0.smithi063.stdout:6/266: creat d2/df/f5d x:0 0 0 2024-05-09T06:38:06.440 INFO:tasks.workunit.client.0.smithi063.stdout:8/296: creat d9/da/d18/d24/f62 x:0 0 0 2024-05-09T06:38:06.443 INFO:tasks.workunit.client.0.smithi063.stdout:0/249: rmdir dd/d1a/d2c/d42/d47 0 2024-05-09T06:38:06.445 INFO:tasks.workunit.client.0.smithi063.stdout:4/277: unlink d7/le 0 2024-05-09T06:38:06.448 INFO:tasks.workunit.client.0.smithi063.stdout:5/205: rmdir d8 39 2024-05-09T06:38:06.451 INFO:tasks.workunit.client.0.smithi063.stdout:2/307: write dc/dd/d2d/f34 [2511048,43272] 0 2024-05-09T06:38:06.454 INFO:tasks.workunit.client.0.smithi063.stdout:3/253: symlink d7/d4c/d31/d35/l5c 0 2024-05-09T06:38:06.454 INFO:tasks.workunit.client.0.smithi063.stdout:3/254: stat d7/d4c/d31 0 2024-05-09T06:38:06.460 INFO:tasks.workunit.client.0.smithi063.stdout:0/250: creat dd/d1a/d2c/d42/f4d x:0 0 0 2024-05-09T06:38:06.461 INFO:tasks.workunit.client.0.smithi063.stdout:4/278: mknod d7/df/c56 0 2024-05-09T06:38:06.463 INFO:tasks.workunit.client.0.smithi063.stdout:0/251: dread dd/d1a/f35 [0,4194304] 0 2024-05-09T06:38:06.465 INFO:tasks.workunit.client.0.smithi063.stdout:1/232: getdents d9/d2a/d35/d24 0 2024-05-09T06:38:06.466 INFO:tasks.workunit.client.0.smithi063.stdout:7/268: dwrite d0/fc [0,4194304] 0 2024-05-09T06:38:06.468 INFO:tasks.workunit.client.0.smithi063.stdout:8/297: dread d9/f13 [4194304,4194304] 0 2024-05-09T06:38:06.468 INFO:tasks.workunit.client.0.smithi063.stdout:7/269: chown d0/d15/d4a/d50/c44 17135646 1 2024-05-09T06:38:06.471 INFO:tasks.workunit.client.0.smithi063.stdout:4/279: write d7/df/d1f/d28/d31/d32/d34/f44 [2050766,70395] 0 2024-05-09T06:38:06.472 INFO:tasks.workunit.client.0.smithi063.stdout:5/206: creat d8/d17/d18/f45 x:0 0 0 2024-05-09T06:38:06.473 INFO:tasks.workunit.client.0.smithi063.stdout:5/207: fsync d8/d19/f1a 0 2024-05-09T06:38:06.473 INFO:tasks.workunit.client.0.smithi063.stdout:4/280: chown d7/df/d1a/f22 1652 1 2024-05-09T06:38:06.479 INFO:tasks.workunit.client.0.smithi063.stdout:5/208: dwrite d8/f21 [0,4194304] 0 2024-05-09T06:38:06.480 INFO:tasks.workunit.client.0.smithi063.stdout:5/209: write d8/f16 [1429164,14709] 0 2024-05-09T06:38:06.482 INFO:tasks.workunit.client.0.smithi063.stdout:8/298: rmdir d9/da/d18/d46 39 2024-05-09T06:38:06.487 INFO:tasks.workunit.client.0.smithi063.stdout:2/308: rename dc/d36/l3b to dc/l56 0 2024-05-09T06:38:06.490 INFO:tasks.workunit.client.0.smithi063.stdout:4/281: mkdir d7/df/d57 0 2024-05-09T06:38:06.498 INFO:tasks.workunit.client.0.smithi063.stdout:8/299: write d9/d41/f5e [821587,95543] 0 2024-05-09T06:38:06.499 INFO:tasks.workunit.client.0.smithi063.stdout:0/252: rename dd/d1a/d22/d23/d31/f41 to dd/d1a/d22/d23/d33/d48/f4e 0 2024-05-09T06:38:06.502 INFO:tasks.workunit.client.0.smithi063.stdout:8/300: dread d9/da/d18/f2d [0,4194304] 0 2024-05-09T06:38:06.503 INFO:tasks.workunit.client.0.smithi063.stdout:4/282: dread d7/df/d35/f36 [0,4194304] 0 2024-05-09T06:38:06.506 INFO:tasks.workunit.client.0.smithi063.stdout:0/253: unlink dd/c19 0 2024-05-09T06:38:06.511 INFO:tasks.workunit.client.0.smithi063.stdout:2/309: creat dc/d36/f57 x:0 0 0 2024-05-09T06:38:06.511 INFO:tasks.workunit.client.0.smithi063.stdout:0/254: creat dd/d1a/d22/d3f/f4f x:0 0 0 2024-05-09T06:38:06.512 INFO:tasks.workunit.client.0.smithi063.stdout:2/310: chown dc/d32/l3a 189992612 1 2024-05-09T06:38:06.513 INFO:tasks.workunit.client.0.smithi063.stdout:0/255: dread - dd/d1a/d22/d23/d33/d48/f4e zero size 2024-05-09T06:38:06.514 INFO:tasks.workunit.client.0.smithi063.stdout:9/239: sync 2024-05-09T06:38:06.514 INFO:tasks.workunit.client.0.smithi063.stdout:8/301: creat d9/da/d18/f63 x:0 0 0 2024-05-09T06:38:06.516 INFO:tasks.workunit.client.0.smithi063.stdout:9/240: mknod d0/d4/c4f 0 2024-05-09T06:38:06.519 INFO:tasks.workunit.client.0.smithi063.stdout:8/302: creat d9/da/d18/d46/f64 x:0 0 0 2024-05-09T06:38:06.522 INFO:tasks.workunit.client.0.smithi063.stdout:8/303: creat d9/da/d18/d56/f65 x:0 0 0 2024-05-09T06:38:06.522 INFO:tasks.workunit.client.0.smithi063.stdout:8/304: fsync d9/da/d18/d24/f27 0 2024-05-09T06:38:06.525 INFO:tasks.workunit.client.0.smithi063.stdout:8/305: mknod d9/d41/c66 0 2024-05-09T06:38:06.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:06 smithi063 ceph-mon[27171]: pgmap v6: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:38:06.606 INFO:tasks.workunit.client.0.smithi063.stdout:2/311: sync 2024-05-09T06:38:06.606 INFO:tasks.workunit.client.0.smithi063.stdout:3/255: sync 2024-05-09T06:38:06.606 INFO:tasks.workunit.client.0.smithi063.stdout:6/267: sync 2024-05-09T06:38:06.606 INFO:tasks.workunit.client.0.smithi063.stdout:9/241: sync 2024-05-09T06:38:06.606 INFO:tasks.workunit.client.0.smithi063.stdout:8/306: sync 2024-05-09T06:38:06.607 INFO:tasks.workunit.client.0.smithi063.stdout:2/312: readlink dc/dd/le 0 2024-05-09T06:38:06.607 INFO:tasks.workunit.client.0.smithi063.stdout:6/268: dread - d2/df/f5d zero size 2024-05-09T06:38:06.607 INFO:tasks.workunit.client.0.smithi063.stdout:9/242: symlink d0/d4/l50 0 2024-05-09T06:38:06.608 INFO:tasks.workunit.client.0.smithi063.stdout:6/269: chown d2/df/d59 149329720 1 2024-05-09T06:38:06.609 INFO:tasks.workunit.client.0.smithi063.stdout:8/307: truncate d9/da/d18/d24/f3e 33896 0 2024-05-09T06:38:06.611 INFO:tasks.workunit.client.0.smithi063.stdout:6/270: read d2/d20/f21 [4150973,59191] 0 2024-05-09T06:38:06.613 INFO:tasks.workunit.client.0.smithi063.stdout:8/308: write d9/da/d18/d56/f65 [781180,126239] 0 2024-05-09T06:38:06.614 INFO:tasks.workunit.client.0.smithi063.stdout:6/271: rmdir d2/d4 39 2024-05-09T06:38:06.617 INFO:tasks.workunit.client.0.smithi063.stdout:3/256: dwrite d7/f11 [0,4194304] 0 2024-05-09T06:38:06.617 INFO:tasks.workunit.client.0.smithi063.stdout:2/313: dread dc/dd/d2d/f3e [0,4194304] 0 2024-05-09T06:38:06.617 INFO:tasks.workunit.client.0.smithi063.stdout:8/309: dread - d9/d37/f5a zero size 2024-05-09T06:38:06.619 INFO:tasks.workunit.client.0.smithi063.stdout:6/272: mknod d2/d4/c5e 0 2024-05-09T06:38:06.619 INFO:tasks.workunit.client.0.smithi063.stdout:9/243: dwrite d0/f32 [0,4194304] 0 2024-05-09T06:38:06.620 INFO:tasks.workunit.client.0.smithi063.stdout:2/314: mkdir dc/dd/d45/d58 0 2024-05-09T06:38:06.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:06 smithi193 ceph-mon[35141]: pgmap v6: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:38:06.623 INFO:tasks.workunit.client.0.smithi063.stdout:9/244: read - d0/d4/d26/f30 zero size 2024-05-09T06:38:06.626 INFO:tasks.workunit.client.0.smithi063.stdout:8/310: dread d9/da/d18/d46/f50 [0,4194304] 0 2024-05-09T06:38:06.628 INFO:tasks.workunit.client.0.smithi063.stdout:6/273: rename d2/d20/d26/l32 to d2/d4/d2d/d42/l5f 0 2024-05-09T06:38:06.628 INFO:tasks.workunit.client.0.smithi063.stdout:6/274: chown d2/d4/l1a 8650 1 2024-05-09T06:38:06.630 INFO:tasks.workunit.client.0.smithi063.stdout:2/315: mknod dc/d36/c59 0 2024-05-09T06:38:06.634 INFO:tasks.workunit.client.0.smithi063.stdout:6/275: rename d2/d4/d2d/d42/d56 to d2/d4/d2d/d42/d60 0 2024-05-09T06:38:06.635 INFO:tasks.workunit.client.0.smithi063.stdout:2/316: unlink dc/dd/d2d/l47 0 2024-05-09T06:38:06.637 INFO:tasks.workunit.client.0.smithi063.stdout:2/317: fsync dc/d32/d33/f4c 0 2024-05-09T06:38:06.637 INFO:tasks.workunit.client.0.smithi063.stdout:6/276: creat d2/d4/d2d/d42/f61 x:0 0 0 2024-05-09T06:38:06.638 INFO:tasks.workunit.client.0.smithi063.stdout:9/245: dwrite d0/d4/d26/f2a [0,4194304] 0 2024-05-09T06:38:06.639 INFO:tasks.workunit.client.0.smithi063.stdout:2/318: truncate f3 144037 0 2024-05-09T06:38:06.642 INFO:tasks.workunit.client.0.smithi063.stdout:9/246: mknod d0/d8/c51 0 2024-05-09T06:38:06.645 INFO:tasks.workunit.client.0.smithi063.stdout:9/247: rmdir d0/d4/d26 39 2024-05-09T06:38:06.646 INFO:tasks.workunit.client.0.smithi063.stdout:6/277: dwrite d2/df/f5d [0,4194304] 0 2024-05-09T06:38:06.648 INFO:tasks.workunit.client.0.smithi063.stdout:6/278: chown d2/d20/c30 25705 1 2024-05-09T06:38:06.650 INFO:tasks.workunit.client.0.smithi063.stdout:9/248: dread d0/f1f [0,4194304] 0 2024-05-09T06:38:06.653 INFO:tasks.workunit.client.0.smithi063.stdout:9/249: creat d0/d8/d2d/f52 x:0 0 0 2024-05-09T06:38:06.654 INFO:tasks.workunit.client.0.smithi063.stdout:8/311: dread d9/f29 [0,4194304] 0 2024-05-09T06:38:06.657 INFO:tasks.workunit.client.0.smithi063.stdout:8/312: dread f2 [0,4194304] 0 2024-05-09T06:38:06.659 INFO:tasks.workunit.client.0.smithi063.stdout:8/313: creat d9/d37/f67 x:0 0 0 2024-05-09T06:38:06.664 INFO:tasks.workunit.client.0.smithi063.stdout:8/314: dread d9/f47 [0,4194304] 0 2024-05-09T06:38:06.665 INFO:tasks.workunit.client.0.smithi063.stdout:1/233: truncate d9/d10/f21 1477532 0 2024-05-09T06:38:06.666 INFO:tasks.workunit.client.0.smithi063.stdout:8/315: creat d9/d3b/f68 x:0 0 0 2024-05-09T06:38:06.667 INFO:tasks.workunit.client.0.smithi063.stdout:1/234: mkdir d9/d2a/d3c/d4a 0 2024-05-09T06:38:06.669 INFO:tasks.workunit.client.0.smithi063.stdout:7/270: truncate d0/d15/f3a 639495 0 2024-05-09T06:38:06.670 INFO:tasks.workunit.client.0.smithi063.stdout:8/316: creat d9/da/d51/f69 x:0 0 0 2024-05-09T06:38:06.671 INFO:tasks.workunit.client.0.smithi063.stdout:8/317: chown d9/da/f4e 1697340 1 2024-05-09T06:38:06.671 INFO:tasks.workunit.client.0.smithi063.stdout:1/235: symlink d9/d2a/d35/d24/l4b 0 2024-05-09T06:38:06.672 INFO:tasks.workunit.client.0.smithi063.stdout:7/271: symlink d0/d15/d4a/d50/l55 0 2024-05-09T06:38:06.674 INFO:tasks.workunit.client.0.smithi063.stdout:5/210: write d8/f1d [1676712,74677] 0 2024-05-09T06:38:06.676 INFO:tasks.workunit.client.0.smithi063.stdout:1/236: rename d9/d2a/d3c/d3d to d9/d2a/d3c/d4c 0 2024-05-09T06:38:06.677 INFO:tasks.workunit.client.0.smithi063.stdout:8/318: symlink d9/l6a 0 2024-05-09T06:38:06.679 INFO:tasks.workunit.client.0.smithi063.stdout:4/283: dread d7/df/f12 [0,4194304] 0 2024-05-09T06:38:06.679 INFO:tasks.workunit.client.0.smithi063.stdout:5/211: mkdir d8/d19/d30/d46 0 2024-05-09T06:38:06.681 INFO:tasks.workunit.client.0.smithi063.stdout:4/284: read d7/df/f2f [1041397,29067] 0 2024-05-09T06:38:06.681 INFO:tasks.workunit.client.0.smithi063.stdout:1/237: symlink d9/l4d 0 2024-05-09T06:38:06.682 INFO:tasks.workunit.client.0.smithi063.stdout:7/272: dwrite d0/fc [0,4194304] 0 2024-05-09T06:38:06.686 INFO:tasks.workunit.client.0.smithi063.stdout:5/212: rmdir d8/d3e 0 2024-05-09T06:38:06.688 INFO:tasks.workunit.client.0.smithi063.stdout:1/238: creat d9/f4e x:0 0 0 2024-05-09T06:38:06.690 INFO:tasks.workunit.client.0.smithi063.stdout:4/285: rename d7/df/f19 to d7/df/d1f/f58 0 2024-05-09T06:38:06.691 INFO:tasks.workunit.client.0.smithi063.stdout:5/213: dread d8/d17/d18/f2d [0,4194304] 0 2024-05-09T06:38:06.691 INFO:tasks.workunit.client.0.smithi063.stdout:4/286: write d7/f39 [5242205,51753] 0 2024-05-09T06:38:06.696 INFO:tasks.workunit.client.0.smithi063.stdout:4/287: creat d7/df/d1f/d28/d31/d32/d34/f59 x:0 0 0 2024-05-09T06:38:06.696 INFO:tasks.workunit.client.0.smithi063.stdout:0/256: getdents dd/d1a/d22/d3f 0 2024-05-09T06:38:06.697 INFO:tasks.workunit.client.0.smithi063.stdout:5/214: truncate d8/d19/f44 1304048 0 2024-05-09T06:38:06.698 INFO:tasks.workunit.client.0.smithi063.stdout:1/239: getdents d9/d2a/d2e 0 2024-05-09T06:38:06.699 INFO:tasks.workunit.client.0.smithi063.stdout:4/288: write d7/df/f21 [3942113,89959] 0 2024-05-09T06:38:06.702 INFO:tasks.workunit.client.0.smithi063.stdout:4/289: readlink d7/df/d1f/l30 0 2024-05-09T06:38:06.702 INFO:tasks.workunit.client.0.smithi063.stdout:0/257: write dd/f15 [5105171,129120] 0 2024-05-09T06:38:06.702 INFO:tasks.workunit.client.0.smithi063.stdout:4/290: chown d7/f42 82 1 2024-05-09T06:38:06.703 INFO:tasks.workunit.client.0.smithi063.stdout:5/215: mknod d8/d17/c47 0 2024-05-09T06:38:06.703 INFO:tasks.workunit.client.0.smithi063.stdout:5/216: read d8/d17/f24 [4189328,63631] 0 2024-05-09T06:38:06.703 INFO:tasks.workunit.client.0.smithi063.stdout:5/217: chown d8/d17/f26 139731 1 2024-05-09T06:38:06.706 INFO:tasks.workunit.client.0.smithi063.stdout:4/291: mkdir d7/df/d35/d5a 0 2024-05-09T06:38:06.708 INFO:tasks.workunit.client.0.smithi063.stdout:4/292: readlink d7/df/d1f/l30 0 2024-05-09T06:38:06.709 INFO:tasks.workunit.client.0.smithi063.stdout:0/258: mkdir dd/d1a/d50 0 2024-05-09T06:38:06.710 INFO:tasks.workunit.client.0.smithi063.stdout:5/218: dwrite d8/d17/d18/f45 [0,4194304] 0 2024-05-09T06:38:06.710 INFO:tasks.workunit.client.0.smithi063.stdout:0/259: fdatasync dd/fe 0 2024-05-09T06:38:06.713 INFO:tasks.workunit.client.0.smithi063.stdout:5/219: creat d8/d19/f48 x:0 0 0 2024-05-09T06:38:06.718 INFO:tasks.workunit.client.0.smithi063.stdout:5/220: symlink d8/d17/l49 0 2024-05-09T06:38:06.720 INFO:tasks.workunit.client.0.smithi063.stdout:0/260: dwrite dd/d1a/d2c/d42/f4d [0,4194304] 0 2024-05-09T06:38:06.722 INFO:tasks.workunit.client.0.smithi063.stdout:5/221: fsync d8/f14 0 2024-05-09T06:38:06.725 INFO:tasks.workunit.client.0.smithi063.stdout:5/222: readlink d8/d17/l1b 0 2024-05-09T06:38:06.725 INFO:tasks.workunit.client.0.smithi063.stdout:0/261: mkdir dd/d1a/d2c/d51 0 2024-05-09T06:38:06.727 INFO:tasks.workunit.client.0.smithi063.stdout:4/293: dread d7/df/d1a/f20 [0,4194304] 0 2024-05-09T06:38:06.730 INFO:tasks.workunit.client.0.smithi063.stdout:0/262: creat dd/d1a/d22/d3f/f52 x:0 0 0 2024-05-09T06:38:06.734 INFO:tasks.workunit.client.0.smithi063.stdout:4/294: symlink d7/df/d35/d5a/l5b 0 2024-05-09T06:38:06.738 INFO:tasks.workunit.client.0.smithi063.stdout:0/263: dread dd/f1d [0,4194304] 0 2024-05-09T06:38:06.739 INFO:tasks.workunit.client.0.smithi063.stdout:0/264: chown dd/f15 44 1 2024-05-09T06:38:06.740 INFO:tasks.workunit.client.0.smithi063.stdout:0/265: chown dd/d1a/c49 30 1 2024-05-09T06:38:06.742 INFO:tasks.workunit.client.0.smithi063.stdout:0/266: write dd/d1a/d22/d3f/f4f [753308,105435] 0 2024-05-09T06:38:06.748 INFO:tasks.workunit.client.0.smithi063.stdout:0/267: creat dd/d1a/d22/d23/d33/d48/f53 x:0 0 0 2024-05-09T06:38:06.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/250: sync 2024-05-09T06:38:06.751 INFO:tasks.workunit.client.0.smithi063.stdout:0/268: mknod dd/d1a/d22/d23/d46/c54 0 2024-05-09T06:38:06.755 INFO:tasks.workunit.client.0.smithi063.stdout:0/269: creat dd/d1a/d22/f55 x:0 0 0 2024-05-09T06:38:06.756 INFO:tasks.workunit.client.0.smithi063.stdout:0/270: dread - dd/d1a/d22/d23/d33/d48/f4e zero size 2024-05-09T06:38:06.759 INFO:tasks.workunit.client.0.smithi063.stdout:0/271: mknod dd/d1a/d22/d23/d33/c56 0 2024-05-09T06:38:06.763 INFO:tasks.workunit.client.0.smithi063.stdout:9/251: dread d0/f24 [0,4194304] 0 2024-05-09T06:38:06.765 INFO:tasks.workunit.client.0.smithi063.stdout:0/272: dwrite dd/d1a/f25 [0,4194304] 0 2024-05-09T06:38:06.767 INFO:tasks.workunit.client.0.smithi063.stdout:0/273: readlink dd/d1a/d22/d23/d31/l37 0 2024-05-09T06:38:06.768 INFO:tasks.workunit.client.0.smithi063.stdout:9/252: rmdir d0/d4/d26/d47 39 2024-05-09T06:38:06.769 INFO:tasks.workunit.client.0.smithi063.stdout:3/257: truncate d7/d2a/f33 157835 0 2024-05-09T06:38:06.769 INFO:tasks.workunit.client.0.smithi063.stdout:9/253: chown d0/d8/c51 232199 1 2024-05-09T06:38:06.774 INFO:tasks.workunit.client.0.smithi063.stdout:9/254: creat d0/d4/d26/d39/f53 x:0 0 0 2024-05-09T06:38:06.777 INFO:tasks.workunit.client.0.smithi063.stdout:0/274: link dd/d1a/d22/f2b dd/d1a/d22/d23/d33/d48/f57 0 2024-05-09T06:38:06.778 INFO:tasks.workunit.client.0.smithi063.stdout:6/279: rmdir d2/d4/d2d/d42/d60 0 2024-05-09T06:38:06.805 INFO:tasks.workunit.client.0.smithi063.stdout:2/319: write fa [1791127,57010] 0 2024-05-09T06:38:06.806 INFO:tasks.workunit.client.0.smithi063.stdout:9/255: dwrite d0/d4/d26/f49 [0,4194304] 0 2024-05-09T06:38:06.807 INFO:tasks.workunit.client.0.smithi063.stdout:8/319: write d9/f29 [669245,47077] 0 2024-05-09T06:38:06.810 INFO:tasks.workunit.client.0.smithi063.stdout:4/295: truncate d7/df/d1f/f53 4093118 0 2024-05-09T06:38:06.810 INFO:tasks.workunit.client.0.smithi063.stdout:8/320: write d9/da/d18/d56/f65 [1875223,92083] 0 2024-05-09T06:38:06.812 INFO:tasks.workunit.client.0.smithi063.stdout:6/280: mkdir d2/d1f/d29/d3f/d62 0 2024-05-09T06:38:06.812 INFO:tasks.workunit.client.0.smithi063.stdout:0/275: getdents dd/d1a/d2c/d51 0 2024-05-09T06:38:06.813 INFO:tasks.workunit.client.0.smithi063.stdout:1/240: write d9/d10/f21 [538413,52731] 0 2024-05-09T06:38:06.816 INFO:tasks.workunit.client.0.smithi063.stdout:4/296: symlink d7/df/l5c 0 2024-05-09T06:38:06.818 INFO:tasks.workunit.client.0.smithi063.stdout:4/297: dread - d7/df/d1f/f3a zero size 2024-05-09T06:38:06.820 INFO:tasks.workunit.client.0.smithi063.stdout:5/223: dwrite d8/d19/f44 [0,4194304] 0 2024-05-09T06:38:06.822 INFO:tasks.workunit.client.0.smithi063.stdout:9/256: creat d0/d4/d26/d39/d3e/f54 x:0 0 0 2024-05-09T06:38:06.823 INFO:tasks.workunit.client.0.smithi063.stdout:9/257: fdatasync d0/d8/d2d/f41 0 2024-05-09T06:38:06.823 INFO:tasks.workunit.client.0.smithi063.stdout:0/276: mkdir dd/d1a/d22/d23/d31/d58 0 2024-05-09T06:38:06.826 INFO:tasks.workunit.client.0.smithi063.stdout:0/277: rename dd/d1a/d22 to dd/d1a/d22/d23/d33/d59 22 2024-05-09T06:38:06.829 INFO:tasks.workunit.client.0.smithi063.stdout:4/298: write d7/df/f12 [4054335,107013] 0 2024-05-09T06:38:06.829 INFO:tasks.workunit.client.0.smithi063.stdout:2/320: mkdir dc/dd/d45/d5a 0 2024-05-09T06:38:06.830 INFO:tasks.workunit.client.0.smithi063.stdout:6/281: write d2/df/f14 [1446242,47805] 0 2024-05-09T06:38:06.830 INFO:tasks.workunit.client.0.smithi063.stdout:0/278: dread dd/fe [0,4194304] 0 2024-05-09T06:38:06.831 INFO:tasks.workunit.client.0.smithi063.stdout:5/224: creat d8/d17/d18/f4a x:0 0 0 2024-05-09T06:38:06.833 INFO:tasks.workunit.client.0.smithi063.stdout:6/282: write d2/d4/f3c [1865524,95201] 0 2024-05-09T06:38:06.837 INFO:tasks.workunit.client.0.smithi063.stdout:2/321: write dc/dd/f2f [3443677,115258] 0 2024-05-09T06:38:06.838 INFO:tasks.workunit.client.0.smithi063.stdout:5/225: rmdir d8/d17 39 2024-05-09T06:38:06.839 INFO:tasks.workunit.client.0.smithi063.stdout:0/279: mknod dd/d1a/d2c/d42/c5a 0 2024-05-09T06:38:06.841 INFO:tasks.workunit.client.0.smithi063.stdout:8/321: link d9/da/d53/l54 d9/da/d18/d24/l6b 0 2024-05-09T06:38:06.842 INFO:tasks.workunit.client.0.smithi063.stdout:6/283: dwrite d2/d1f/d29/d3f/f58 [0,4194304] 0 2024-05-09T06:38:06.844 INFO:tasks.workunit.client.0.smithi063.stdout:9/258: link d0/d43/f4b d0/d4/d26/d39/f55 0 2024-05-09T06:38:06.844 INFO:tasks.workunit.client.0.smithi063.stdout:8/322: readlink d9/da/d45/l60 0 2024-05-09T06:38:06.849 INFO:tasks.workunit.client.0.smithi063.stdout:6/284: creat d2/d1f/d29/f63 x:0 0 0 2024-05-09T06:38:06.850 INFO:tasks.workunit.client.0.smithi063.stdout:2/322: dread dc/dd/f20 [0,4194304] 0 2024-05-09T06:38:06.851 INFO:tasks.workunit.client.0.smithi063.stdout:1/241: dwrite d9/f23 [0,4194304] 0 2024-05-09T06:38:06.852 INFO:tasks.workunit.client.0.smithi063.stdout:8/323: symlink d9/da/d18/d46/l6c 0 2024-05-09T06:38:06.852 INFO:tasks.workunit.client.0.smithi063.stdout:0/280: link dd/d1a/d22/d23/d33/c56 dd/c5b 0 2024-05-09T06:38:06.856 INFO:tasks.workunit.client.0.smithi063.stdout:8/324: symlink d9/da/d34/l6d 0 2024-05-09T06:38:06.857 INFO:tasks.workunit.client.0.smithi063.stdout:1/242: mknod d9/d2a/d35/d24/c4f 0 2024-05-09T06:38:06.859 INFO:tasks.workunit.client.0.smithi063.stdout:6/285: mkdir d2/d1f/d29/d3f/d5c/d64 0 2024-05-09T06:38:06.862 INFO:tasks.workunit.client.0.smithi063.stdout:0/281: rename dd/d1a/d22/f26 to dd/d1a/d2c/f5c 0 2024-05-09T06:38:06.863 INFO:tasks.workunit.client.0.smithi063.stdout:0/282: chown dd/d1a/d22/c24 3272345 1 2024-05-09T06:38:06.864 INFO:tasks.workunit.client.0.smithi063.stdout:0/283: stat dd/d1a/d22/d23/f2f 0 2024-05-09T06:38:06.867 INFO:tasks.workunit.client.0.smithi063.stdout:1/243: creat d9/d2a/d2e/f50 x:0 0 0 2024-05-09T06:38:06.867 INFO:tasks.workunit.client.0.smithi063.stdout:5/226: dread d8/d19/f1a [0,4194304] 0 2024-05-09T06:38:06.867 INFO:tasks.workunit.client.0.smithi063.stdout:9/259: truncate d0/d4/d26/f49 2359083 0 2024-05-09T06:38:06.867 INFO:tasks.workunit.client.0.smithi063.stdout:4/299: dread d7/df/f2b [0,4194304] 0 2024-05-09T06:38:06.868 INFO:tasks.workunit.client.0.smithi063.stdout:8/325: link d9/d3b/f2c d9/da/d34/f6e 0 2024-05-09T06:38:06.868 INFO:tasks.workunit.client.0.smithi063.stdout:2/323: link dc/c15 dc/dd/d45/d5a/c5b 0 2024-05-09T06:38:06.869 INFO:tasks.workunit.client.0.smithi063.stdout:0/284: truncate dd/f2d 396207 0 2024-05-09T06:38:06.871 INFO:tasks.workunit.client.0.smithi063.stdout:9/260: fdatasync d0/d4/d26/d47/f2b 0 2024-05-09T06:38:06.872 INFO:tasks.workunit.client.0.smithi063.stdout:1/244: creat d9/d10/d2b/f51 x:0 0 0 2024-05-09T06:38:06.872 INFO:tasks.workunit.client.0.smithi063.stdout:7/273: sync 2024-05-09T06:38:06.873 INFO:tasks.workunit.client.0.smithi063.stdout:2/324: dread - dc/dd/f4f zero size 2024-05-09T06:38:06.873 INFO:tasks.workunit.client.0.smithi063.stdout:9/261: read - d0/d4/d26/f30 zero size 2024-05-09T06:38:06.874 INFO:tasks.workunit.client.0.smithi063.stdout:0/285: symlink dd/d1a/d22/d3f/l5d 0 2024-05-09T06:38:06.875 INFO:tasks.workunit.client.0.smithi063.stdout:5/227: creat d8/f4b x:0 0 0 2024-05-09T06:38:06.876 INFO:tasks.workunit.client.0.smithi063.stdout:8/326: dread d9/da/d18/f22 [0,4194304] 0 2024-05-09T06:38:06.876 INFO:tasks.workunit.client.0.smithi063.stdout:9/262: read d0/f24 [8333697,3974] 0 2024-05-09T06:38:06.876 INFO:tasks.workunit.client.0.smithi063.stdout:7/274: rmdir d0/d15 39 2024-05-09T06:38:06.876 INFO:tasks.workunit.client.0.smithi063.stdout:2/325: write dc/d32/f42 [1807598,42181] 0 2024-05-09T06:38:06.878 INFO:tasks.workunit.client.0.smithi063.stdout:1/245: mknod d9/c52 0 2024-05-09T06:38:06.879 INFO:tasks.workunit.client.0.smithi063.stdout:0/286: chown dd/d1a/d2c/d42/f4d 655027 1 2024-05-09T06:38:06.879 INFO:tasks.workunit.client.0.smithi063.stdout:4/300: dwrite d7/df/f21 [0,4194304] 0 2024-05-09T06:38:06.881 INFO:tasks.workunit.client.0.smithi063.stdout:1/246: dread - d9/d10/f43 zero size 2024-05-09T06:38:06.883 INFO:tasks.workunit.client.0.smithi063.stdout:2/326: dread dc/f21 [0,4194304] 0 2024-05-09T06:38:06.884 INFO:tasks.workunit.client.0.smithi063.stdout:9/263: creat d0/d3b/f56 x:0 0 0 2024-05-09T06:38:06.885 INFO:tasks.workunit.client.0.smithi063.stdout:1/247: truncate d9/d10/f43 562325 0 2024-05-09T06:38:06.889 INFO:tasks.workunit.client.0.smithi063.stdout:0/287: dread dd/d1a/f1b [0,4194304] 0 2024-05-09T06:38:06.891 INFO:tasks.workunit.client.0.smithi063.stdout:6/286: truncate d2/d12/f24 506670 0 2024-05-09T06:38:06.892 INFO:tasks.workunit.client.0.smithi063.stdout:7/275: symlink d0/d15/d49/l56 0 2024-05-09T06:38:06.894 INFO:tasks.workunit.client.0.smithi063.stdout:2/327: creat dc/d36/d43/f5c x:0 0 0 2024-05-09T06:38:06.894 INFO:tasks.workunit.client.0.smithi063.stdout:9/264: dwrite d0/d8/d2d/f52 [0,4194304] 0 2024-05-09T06:38:06.895 INFO:tasks.workunit.client.0.smithi063.stdout:9/265: chown d0/d8/d2d 327 1 2024-05-09T06:38:06.896 INFO:tasks.workunit.client.0.smithi063.stdout:2/328: dread dc/dd/f20 [0,4194304] 0 2024-05-09T06:38:06.896 INFO:tasks.workunit.client.0.smithi063.stdout:5/228: creat d8/f4c x:0 0 0 2024-05-09T06:38:06.896 INFO:tasks.workunit.client.0.smithi063.stdout:3/258: sync 2024-05-09T06:38:06.898 INFO:tasks.workunit.client.0.smithi063.stdout:3/259: fsync d7/d2a/f4b 0 2024-05-09T06:38:06.901 INFO:tasks.workunit.client.0.smithi063.stdout:4/301: symlink d7/df/d1f/l5d 0 2024-05-09T06:38:06.902 INFO:tasks.workunit.client.0.smithi063.stdout:4/302: stat d7/df/d1f/f23 0 2024-05-09T06:38:06.905 INFO:tasks.workunit.client.0.smithi063.stdout:6/287: truncate d2/f7 2375545 0 2024-05-09T06:38:06.906 INFO:tasks.workunit.client.0.smithi063.stdout:6/288: fsync d2/d4/d2d/d42/f61 0 2024-05-09T06:38:06.906 INFO:tasks.workunit.client.0.smithi063.stdout:8/327: dwrite d9/da/d18/f2d [0,4194304] 0 2024-05-09T06:38:06.906 INFO:tasks.workunit.client.0.smithi063.stdout:8/328: chown d9/da/d34/f6e 106362806 1 2024-05-09T06:38:06.907 INFO:tasks.workunit.client.0.smithi063.stdout:5/229: symlink d8/d35/l4d 0 2024-05-09T06:38:06.914 INFO:tasks.workunit.client.0.smithi063.stdout:9/266: mkdir d0/d4/d57 0 2024-05-09T06:38:06.917 INFO:tasks.workunit.client.0.smithi063.stdout:6/289: dread d2/df/f14 [0,4194304] 0 2024-05-09T06:38:06.920 INFO:tasks.workunit.client.0.smithi063.stdout:1/248: dwrite d9/d2a/d35/d24/f26 [4194304,4194304] 0 2024-05-09T06:38:06.922 INFO:tasks.workunit.client.0.smithi063.stdout:7/276: rename d0/l38 to d0/l57 0 2024-05-09T06:38:06.922 INFO:tasks.workunit.client.0.smithi063.stdout:5/230: chown d8/d17/d18/f45 871725865 1 2024-05-09T06:38:06.925 INFO:tasks.workunit.client.0.smithi063.stdout:2/329: dwrite dc/f1c [0,4194304] 0 2024-05-09T06:38:06.925 INFO:tasks.workunit.client.0.smithi063.stdout:7/277: write d0/f10 [58531,66474] 0 2024-05-09T06:38:06.926 INFO:tasks.workunit.client.0.smithi063.stdout:0/288: dwrite dd/d1a/d22/d23/d33/f3e [0,4194304] 0 2024-05-09T06:38:06.927 INFO:tasks.workunit.client.0.smithi063.stdout:8/329: truncate d9/d3b/f2c 1105555 0 2024-05-09T06:38:06.929 INFO:tasks.workunit.client.0.smithi063.stdout:6/290: mknod d2/d20/c65 0 2024-05-09T06:38:06.929 INFO:tasks.workunit.client.0.smithi063.stdout:1/249: dread f3 [0,4194304] 0 2024-05-09T06:38:06.931 INFO:tasks.workunit.client.0.smithi063.stdout:9/267: mknod d0/d4/d57/c58 0 2024-05-09T06:38:06.932 INFO:tasks.workunit.client.0.smithi063.stdout:2/330: chown dc/dd/l2c 23991 1 2024-05-09T06:38:06.933 INFO:tasks.workunit.client.0.smithi063.stdout:5/231: truncate d8/d17/d18/f38 1529046 0 2024-05-09T06:38:06.935 INFO:tasks.workunit.client.0.smithi063.stdout:7/278: creat d0/d15/f58 x:0 0 0 2024-05-09T06:38:06.938 INFO:tasks.workunit.client.0.smithi063.stdout:0/289: unlink dd/d1a/d2c/f5c 0 2024-05-09T06:38:06.939 INFO:tasks.workunit.client.0.smithi063.stdout:0/290: chown dd/l3b 126 1 2024-05-09T06:38:06.940 INFO:tasks.workunit.client.0.smithi063.stdout:8/330: symlink d9/da/d45/l6f 0 2024-05-09T06:38:06.942 INFO:tasks.workunit.client.0.smithi063.stdout:3/260: getdents d7/d4c/d31 0 2024-05-09T06:38:06.943 INFO:tasks.workunit.client.0.smithi063.stdout:8/331: readlink d9/da/d18/l32 0 2024-05-09T06:38:06.944 INFO:tasks.workunit.client.0.smithi063.stdout:7/279: dwrite d0/d15/d49/f4c [0,4194304] 0 2024-05-09T06:38:06.944 INFO:tasks.workunit.client.0.smithi063.stdout:3/261: truncate f3 5830666 0 2024-05-09T06:38:06.944 INFO:tasks.workunit.client.0.smithi063.stdout:1/250: creat d9/d2a/d35/d24/f53 x:0 0 0 2024-05-09T06:38:06.947 INFO:tasks.workunit.client.0.smithi063.stdout:8/332: write d9/da/d18/d56/f65 [773976,50184] 0 2024-05-09T06:38:06.950 INFO:tasks.workunit.client.0.smithi063.stdout:4/303: link d7/df/d35/c45 d7/df/d1f/c5e 0 2024-05-09T06:38:06.950 INFO:tasks.workunit.client.0.smithi063.stdout:1/251: read d9/d10/f18 [1351517,109147] 0 2024-05-09T06:38:06.951 INFO:tasks.workunit.client.0.smithi063.stdout:0/291: creat dd/d1a/d22/d23/d31/f5e x:0 0 0 2024-05-09T06:38:06.952 INFO:tasks.workunit.client.0.smithi063.stdout:1/252: truncate d9/d2a/d2e/f50 1047716 0 2024-05-09T06:38:06.953 INFO:tasks.workunit.client.0.smithi063.stdout:7/280: symlink d0/d15/d4a/d50/l59 0 2024-05-09T06:38:06.954 INFO:tasks.workunit.client.0.smithi063.stdout:9/268: mknod d0/d4/da/c59 0 2024-05-09T06:38:06.956 INFO:tasks.workunit.client.0.smithi063.stdout:9/269: write d0/d4/f3a [1435676,107248] 0 2024-05-09T06:38:06.957 INFO:tasks.workunit.client.0.smithi063.stdout:2/331: mknod dc/dd/d2d/c5d 0 2024-05-09T06:38:06.958 INFO:tasks.workunit.client.0.smithi063.stdout:9/270: write d0/d4/f4d [778819,107050] 0 2024-05-09T06:38:06.958 INFO:tasks.workunit.client.0.smithi063.stdout:5/232: mkdir d8/d4e 0 2024-05-09T06:38:06.960 INFO:tasks.workunit.client.0.smithi063.stdout:6/291: rmdir d2/d1f/d29/d3f/d62 0 2024-05-09T06:38:06.962 INFO:tasks.workunit.client.0.smithi063.stdout:6/292: chown d2/f4d 117938442 1 2024-05-09T06:38:06.963 INFO:tasks.workunit.client.0.smithi063.stdout:4/304: creat d7/df/d1f/d28/d31/f5f x:0 0 0 2024-05-09T06:38:06.965 INFO:tasks.workunit.client.0.smithi063.stdout:5/233: dread d8/d17/d18/f22 [0,4194304] 0 2024-05-09T06:38:06.965 INFO:tasks.workunit.client.0.smithi063.stdout:4/305: chown d7/df/f12 24 1 2024-05-09T06:38:06.967 INFO:tasks.workunit.client.0.smithi063.stdout:9/271: dwrite d0/d4/d26/d39/f53 [0,4194304] 0 2024-05-09T06:38:06.971 INFO:tasks.workunit.client.0.smithi063.stdout:3/262: rename d7/d2a/f33 to d7/d2a/d2d/f5d 0 2024-05-09T06:38:06.971 INFO:tasks.workunit.client.0.smithi063.stdout:1/253: rmdir d9/d2a/d35 39 2024-05-09T06:38:06.973 INFO:tasks.workunit.client.0.smithi063.stdout:2/332: truncate dc/dd/d2d/f38 2992208 0 2024-05-09T06:38:06.978 INFO:tasks.workunit.client.0.smithi063.stdout:2/333: dwrite dc/d32/f42 [0,4194304] 0 2024-05-09T06:38:06.980 INFO:tasks.workunit.client.0.smithi063.stdout:2/334: chown dc/d36/d43/d46/l4b 109770 1 2024-05-09T06:38:06.980 INFO:tasks.workunit.client.0.smithi063.stdout:5/234: creat d8/d19/f4f x:0 0 0 2024-05-09T06:38:06.985 INFO:tasks.workunit.client.0.smithi063.stdout:1/254: creat d9/d46/f54 x:0 0 0 2024-05-09T06:38:06.987 INFO:tasks.workunit.client.0.smithi063.stdout:1/255: truncate f8 3016595 0 2024-05-09T06:38:06.988 INFO:tasks.workunit.client.0.smithi063.stdout:1/256: dread - d9/d10/d2b/f51 zero size 2024-05-09T06:38:06.989 INFO:tasks.workunit.client.0.smithi063.stdout:0/292: dwrite dd/d1a/d22/f2b [0,4194304] 0 2024-05-09T06:38:06.991 INFO:tasks.workunit.client.0.smithi063.stdout:0/293: read dd/d1a/d22/d3f/f4f [346356,39805] 0 2024-05-09T06:38:06.992 INFO:tasks.workunit.client.0.smithi063.stdout:8/333: truncate d9/da/f1a 1305180 0 2024-05-09T06:38:06.993 INFO:tasks.workunit.client.0.smithi063.stdout:9/272: creat d0/d4/d26/d39/d48/f5a x:0 0 0 2024-05-09T06:38:06.994 INFO:tasks.workunit.client.0.smithi063.stdout:9/273: write d0/d4/f3a [2655529,80945] 0 2024-05-09T06:38:06.999 INFO:tasks.workunit.client.0.smithi063.stdout:8/334: dwrite d9/d3b/f40 [0,4194304] 0 2024-05-09T06:38:06.999 INFO:tasks.workunit.client.0.smithi063.stdout:1/257: rmdir d9/d10 39 2024-05-09T06:38:07.000 INFO:tasks.workunit.client.0.smithi063.stdout:8/335: fsync d9/da/d18/f23 0 2024-05-09T06:38:07.002 INFO:tasks.workunit.client.0.smithi063.stdout:7/281: getdents d0/d15/d4a/d50 0 2024-05-09T06:38:07.003 INFO:tasks.workunit.client.0.smithi063.stdout:6/293: write d2/fd [910600,79229] 0 2024-05-09T06:38:07.004 INFO:tasks.workunit.client.0.smithi063.stdout:7/282: chown d0/d15/f26 9 1 2024-05-09T06:38:07.004 INFO:tasks.workunit.client.0.smithi063.stdout:6/294: read - d2/d1f/d29/f63 zero size 2024-05-09T06:38:07.011 INFO:tasks.workunit.client.0.smithi063.stdout:1/258: stat d9/d2a/d35/d1e 0 2024-05-09T06:38:07.012 INFO:tasks.workunit.client.0.smithi063.stdout:3/263: truncate d7/d22/d28/f58 1773760 0 2024-05-09T06:38:07.014 INFO:tasks.workunit.client.0.smithi063.stdout:0/294: creat dd/d1a/d50/f5f x:0 0 0 2024-05-09T06:38:07.016 INFO:tasks.workunit.client.0.smithi063.stdout:5/235: truncate d8/d17/d18/f1f 1406027 0 2024-05-09T06:38:07.018 INFO:tasks.workunit.client.0.smithi063.stdout:3/264: mknod d7/da/d21/d30/c5e 0 2024-05-09T06:38:07.018 INFO:tasks.workunit.client.0.smithi063.stdout:5/236: write f6 [1013046,41693] 0 2024-05-09T06:38:07.019 INFO:tasks.workunit.client.0.smithi063.stdout:0/295: fdatasync f2 0 2024-05-09T06:38:07.020 INFO:tasks.workunit.client.0.smithi063.stdout:1/259: dwrite d9/d2a/d35/d24/f53 [0,4194304] 0 2024-05-09T06:38:07.022 INFO:tasks.workunit.client.0.smithi063.stdout:7/283: truncate d0/d15/f20 571450 0 2024-05-09T06:38:07.022 INFO:tasks.workunit.client.0.smithi063.stdout:5/237: read - d8/d17/d1e/f42 zero size 2024-05-09T06:38:07.023 INFO:tasks.workunit.client.0.smithi063.stdout:0/296: dread - dd/d1a/d22/d3f/f52 zero size 2024-05-09T06:38:07.023 INFO:tasks.workunit.client.0.smithi063.stdout:9/274: creat d0/f5b x:0 0 0 2024-05-09T06:38:07.024 INFO:tasks.workunit.client.0.smithi063.stdout:9/275: stat d0/d4/l50 0 2024-05-09T06:38:07.025 INFO:tasks.workunit.client.0.smithi063.stdout:8/336: dread d9/da/d34/f59 [0,4194304] 0 2024-05-09T06:38:07.027 INFO:tasks.workunit.client.0.smithi063.stdout:3/265: dread d7/f16 [0,4194304] 0 2024-05-09T06:38:07.027 INFO:tasks.workunit.client.0.smithi063.stdout:1/260: creat d9/d2a/d3c/d4c/f55 x:0 0 0 2024-05-09T06:38:07.032 INFO:tasks.workunit.client.0.smithi063.stdout:9/276: creat d0/d4/d26/d39/d3e/f5c x:0 0 0 2024-05-09T06:38:07.034 INFO:tasks.workunit.client.0.smithi063.stdout:3/266: symlink d7/d4c/l5f 0 2024-05-09T06:38:07.037 INFO:tasks.workunit.client.0.smithi063.stdout:7/284: symlink d0/l5a 0 2024-05-09T06:38:07.037 INFO:tasks.workunit.client.0.smithi063.stdout:6/295: truncate d2/d20/f44 1680269 0 2024-05-09T06:38:07.038 INFO:tasks.workunit.client.0.smithi063.stdout:5/238: mknod d8/c50 0 2024-05-09T06:38:07.039 INFO:tasks.workunit.client.0.smithi063.stdout:1/261: creat d9/d2a/d3c/d4c/f56 x:0 0 0 2024-05-09T06:38:07.039 INFO:tasks.workunit.client.0.smithi063.stdout:5/239: dread - d8/d17/d18/f2f zero size 2024-05-09T06:38:07.043 INFO:tasks.workunit.client.0.smithi063.stdout:3/267: mknod d7/da/d21/d30/c60 0 2024-05-09T06:38:07.044 INFO:tasks.workunit.client.0.smithi063.stdout:7/285: creat d0/d15/d49/f5b x:0 0 0 2024-05-09T06:38:07.046 INFO:tasks.workunit.client.0.smithi063.stdout:5/240: dwrite d8/f1d [0,4194304] 0 2024-05-09T06:38:07.047 INFO:tasks.workunit.client.0.smithi063.stdout:1/262: mkdir d9/d46/d57 0 2024-05-09T06:38:07.049 INFO:tasks.workunit.client.0.smithi063.stdout:8/337: rename f2 to d9/f70 0 2024-05-09T06:38:07.051 INFO:tasks.workunit.client.0.smithi063.stdout:7/286: creat d0/d15/d49/f5c x:0 0 0 2024-05-09T06:38:07.056 INFO:tasks.workunit.client.0.smithi063.stdout:5/241: creat d8/d17/d18/f51 x:0 0 0 2024-05-09T06:38:07.056 INFO:tasks.workunit.client.0.smithi063.stdout:6/296: dread d2/d1f/f4c [0,4194304] 0 2024-05-09T06:38:07.058 INFO:tasks.workunit.client.0.smithi063.stdout:1/263: symlink d9/d2a/d2e/l58 0 2024-05-09T06:38:07.058 INFO:tasks.workunit.client.0.smithi063.stdout:8/338: dwrite d9/da/d34/f4b [0,4194304] 0 2024-05-09T06:38:07.060 INFO:tasks.workunit.client.0.smithi063.stdout:8/339: read d9/da/d18/d46/f50 [1550872,45397] 0 2024-05-09T06:38:07.063 INFO:tasks.workunit.client.0.smithi063.stdout:9/277: rename d0/d4/f3a to d0/d4/d26/d47/f5d 0 2024-05-09T06:38:07.063 INFO:tasks.workunit.client.0.smithi063.stdout:0/297: write dd/d1a/d2c/f43 [1400039,66512] 0 2024-05-09T06:38:07.063 INFO:tasks.workunit.client.0.smithi063.stdout:4/306: sync 2024-05-09T06:38:07.071 INFO:tasks.workunit.client.0.smithi063.stdout:9/278: dwrite d0/f28 [0,4194304] 0 2024-05-09T06:38:07.071 INFO:tasks.workunit.client.0.smithi063.stdout:7/287: mknod d0/d15/d49/d4f/c5d 0 2024-05-09T06:38:07.073 INFO:tasks.workunit.client.0.smithi063.stdout:8/340: rename d9/da/d18/d24/d55 to d9/da/d18/d24/d71 0 2024-05-09T06:38:07.075 INFO:tasks.workunit.client.0.smithi063.stdout:4/307: dwrite d7/df/f21 [4194304,4194304] 0 2024-05-09T06:38:07.076 INFO:tasks.workunit.client.0.smithi063.stdout:7/288: truncate d0/f2f 1979643 0 2024-05-09T06:38:07.077 INFO:tasks.workunit.client.0.smithi063.stdout:3/268: getdents d7/d22 0 2024-05-09T06:38:07.084 INFO:tasks.workunit.client.0.smithi063.stdout:4/308: mkdir d7/df/d35/d5a/d60 0 2024-05-09T06:38:07.084 INFO:tasks.workunit.client.0.smithi063.stdout:7/289: creat d0/d15/d49/f5e x:0 0 0 2024-05-09T06:38:07.086 INFO:tasks.workunit.client.0.smithi063.stdout:6/297: link d2/d20/c65 d2/d1f/d29/d3f/c66 0 2024-05-09T06:38:07.087 INFO:tasks.workunit.client.0.smithi063.stdout:9/279: mknod d0/d4/da/c5e 0 2024-05-09T06:38:07.087 INFO:tasks.workunit.client.0.smithi063.stdout:5/242: dread d8/d17/f1c [0,4194304] 0 2024-05-09T06:38:07.089 INFO:tasks.workunit.client.0.smithi063.stdout:7/290: creat d0/d15/d49/f5f x:0 0 0 2024-05-09T06:38:07.089 INFO:tasks.workunit.client.0.smithi063.stdout:6/298: chown d2/df/c48 34 1 2024-05-09T06:38:07.090 INFO:tasks.workunit.client.0.smithi063.stdout:0/298: getdents dd/d1a/d22/d23/d33/d48 0 2024-05-09T06:38:07.090 INFO:tasks.workunit.client.0.smithi063.stdout:1/264: dwrite d9/d10/f3e [0,4194304] 0 2024-05-09T06:38:07.091 INFO:tasks.workunit.client.0.smithi063.stdout:9/280: chown d0/d4/d26/d47/f2b 1447407522 1 2024-05-09T06:38:07.093 INFO:tasks.workunit.client.0.smithi063.stdout:5/243: symlink d8/d35/l52 0 2024-05-09T06:38:07.093 INFO:tasks.workunit.client.0.smithi063.stdout:3/269: dread d7/d22/f50 [0,4194304] 0 2024-05-09T06:38:07.096 INFO:tasks.workunit.client.0.smithi063.stdout:9/281: dread - d0/d4/d26/d47/f2b zero size 2024-05-09T06:38:07.097 INFO:tasks.workunit.client.0.smithi063.stdout:6/299: truncate d2/d1f/d29/d3f/f25 900746 0 2024-05-09T06:38:07.099 INFO:tasks.workunit.client.0.smithi063.stdout:3/270: rmdir d7/da/d21/d30 39 2024-05-09T06:38:07.099 INFO:tasks.workunit.client.0.smithi063.stdout:9/282: write d0/d4/d26/d47/f2b [582219,58802] 0 2024-05-09T06:38:07.101 INFO:tasks.workunit.client.0.smithi063.stdout:3/271: stat d7/d22/d28/f3c 0 2024-05-09T06:38:07.101 INFO:tasks.workunit.client.0.smithi063.stdout:7/291: dwrite d0/d15/f2b [0,4194304] 0 2024-05-09T06:38:07.102 INFO:tasks.workunit.client.0.smithi063.stdout:5/244: mkdir d8/d53 0 2024-05-09T06:38:07.104 INFO:tasks.workunit.client.0.smithi063.stdout:9/283: write d0/d4/d26/f30 [1030990,43585] 0 2024-05-09T06:38:07.110 INFO:tasks.workunit.client.0.smithi063.stdout:9/284: chown d0/d4/d26/c3d 2973510 1 2024-05-09T06:38:07.110 INFO:tasks.workunit.client.0.smithi063.stdout:7/292: write d0/d15/d4a/d50/f37 [171672,118068] 0 2024-05-09T06:38:07.110 INFO:tasks.workunit.client.0.smithi063.stdout:6/300: write d2/d1f/f4c [5503647,129052] 0 2024-05-09T06:38:07.110 INFO:tasks.workunit.client.0.smithi063.stdout:0/299: getdents dd/d1a/d22/d3f 0 2024-05-09T06:38:07.110 INFO:tasks.workunit.client.0.smithi063.stdout:3/272: fsync d7/da/d21/d30/d44/d48/f53 0 2024-05-09T06:38:07.113 INFO:tasks.workunit.client.0.smithi063.stdout:2/335: sync 2024-05-09T06:38:07.113 INFO:tasks.workunit.client.0.smithi063.stdout:9/285: readlink d0/d8/le 0 2024-05-09T06:38:07.117 INFO:tasks.workunit.client.0.smithi063.stdout:0/300: rename dd/d1a/d22/d3f to dd/d60 0 2024-05-09T06:38:07.118 INFO:tasks.workunit.client.0.smithi063.stdout:3/273: mknod d7/d4c/d31/c61 0 2024-05-09T06:38:07.118 INFO:tasks.workunit.client.0.smithi063.stdout:7/293: truncate d0/d15/f48 199574 0 2024-05-09T06:38:07.121 INFO:tasks.workunit.client.0.smithi063.stdout:6/301: creat d2/d1f/d29/d3b/d4e/f67 x:0 0 0 2024-05-09T06:38:07.123 INFO:tasks.workunit.client.0.smithi063.stdout:0/301: write dd/d1a/d22/f55 [772856,38653] 0 2024-05-09T06:38:07.124 INFO:tasks.workunit.client.0.smithi063.stdout:8/341: dwrite d9/da/d34/f59 [0,4194304] 0 2024-05-09T06:38:07.125 INFO:tasks.workunit.client.0.smithi063.stdout:7/294: dread - d0/d15/d49/f5b zero size 2024-05-09T06:38:07.127 INFO:tasks.workunit.client.0.smithi063.stdout:2/336: getdents dc/d36/d43 0 2024-05-09T06:38:07.128 INFO:tasks.workunit.client.0.smithi063.stdout:7/295: fdatasync d0/f10 0 2024-05-09T06:38:07.131 INFO:tasks.workunit.client.0.smithi063.stdout:3/274: dread d7/f2f [0,4194304] 0 2024-05-09T06:38:07.134 INFO:tasks.workunit.client.0.smithi063.stdout:1/265: dread d9/d10/d2b/f33 [0,4194304] 0 2024-05-09T06:38:07.134 INFO:tasks.workunit.client.0.smithi063.stdout:4/309: dread d7/df/d1a/f3b [0,4194304] 0 2024-05-09T06:38:07.135 INFO:tasks.workunit.client.0.smithi063.stdout:6/302: mknod d2/d1f/d29/d3f/d5c/d64/c68 0 2024-05-09T06:38:07.135 INFO:tasks.workunit.client.0.smithi063.stdout:8/342: chown d9/c17 1 1 2024-05-09T06:38:07.137 INFO:tasks.workunit.client.0.smithi063.stdout:2/337: creat dc/dd/d45/d5a/f5e x:0 0 0 2024-05-09T06:38:07.137 INFO:tasks.workunit.client.0.smithi063.stdout:7/296: creat d0/d15/d49/d54/f60 x:0 0 0 2024-05-09T06:38:07.138 INFO:tasks.workunit.client.0.smithi063.stdout:5/245: dread f6 [0,4194304] 0 2024-05-09T06:38:07.140 INFO:tasks.workunit.client.0.smithi063.stdout:3/275: truncate d7/da/f29 865361 0 2024-05-09T06:38:07.140 INFO:tasks.workunit.client.0.smithi063.stdout:0/302: truncate dd/d1a/d22/d23/d33/d48/f53 102533 0 2024-05-09T06:38:07.141 INFO:tasks.workunit.client.0.smithi063.stdout:9/286: dread d0/d4/da/f15 [0,4194304] 0 2024-05-09T06:38:07.142 INFO:tasks.workunit.client.0.smithi063.stdout:8/343: write d9/da/d18/d24/f3c [4421937,78178] 0 2024-05-09T06:38:07.143 INFO:tasks.workunit.client.0.smithi063.stdout:2/338: symlink dc/d36/l5f 0 2024-05-09T06:38:07.144 INFO:tasks.workunit.client.0.smithi063.stdout:0/303: write dd/d1a/d22/f2b [2386752,55366] 0 2024-05-09T06:38:07.147 INFO:tasks.workunit.client.0.smithi063.stdout:9/287: creat d0/d43/f5f x:0 0 0 2024-05-09T06:38:07.148 INFO:tasks.workunit.client.0.smithi063.stdout:3/276: truncate d7/da/f18 777591 0 2024-05-09T06:38:07.149 INFO:tasks.workunit.client.0.smithi063.stdout:2/339: mkdir dc/dd/d2d/d60 0 2024-05-09T06:38:07.151 INFO:tasks.workunit.client.0.smithi063.stdout:1/266: truncate d9/d10/f21 1325536 0 2024-05-09T06:38:07.152 INFO:tasks.workunit.client.0.smithi063.stdout:0/304: creat dd/d60/d44/f61 x:0 0 0 2024-05-09T06:38:07.153 INFO:tasks.workunit.client.0.smithi063.stdout:8/344: mknod d9/da/d18/d56/d58/c72 0 2024-05-09T06:38:07.154 INFO:tasks.workunit.client.0.smithi063.stdout:2/340: creat dc/d32/d33/f61 x:0 0 0 2024-05-09T06:38:07.155 INFO:tasks.workunit.client.0.smithi063.stdout:0/305: readlink dd/d1a/l20 0 2024-05-09T06:38:07.156 INFO:tasks.workunit.client.0.smithi063.stdout:3/277: symlink d7/da/d21/d30/d44/d48/l62 0 2024-05-09T06:38:07.157 INFO:tasks.workunit.client.0.smithi063.stdout:6/303: dwrite d2/d12/f24 [0,4194304] 0 2024-05-09T06:38:07.158 INFO:tasks.workunit.client.0.smithi063.stdout:1/267: write d9/d2a/d35/f29 [1449923,74334] 0 2024-05-09T06:38:07.161 INFO:tasks.workunit.client.0.smithi063.stdout:9/288: dread d0/d4/d26/f33 [0,4194304] 0 2024-05-09T06:38:07.162 INFO:tasks.workunit.client.0.smithi063.stdout:2/341: creat dc/d32/f62 x:0 0 0 2024-05-09T06:38:07.163 INFO:tasks.workunit.client.0.smithi063.stdout:0/306: symlink dd/l62 0 2024-05-09T06:38:07.164 INFO:tasks.workunit.client.0.smithi063.stdout:6/304: creat d2/d1f/f69 x:0 0 0 2024-05-09T06:38:07.167 INFO:tasks.workunit.client.0.smithi063.stdout:7/297: dread d0/f18 [0,4194304] 0 2024-05-09T06:38:07.169 INFO:tasks.workunit.client.0.smithi063.stdout:1/268: write d9/d10/f1c [5623539,15466] 0 2024-05-09T06:38:07.169 INFO:tasks.workunit.client.0.smithi063.stdout:2/342: mkdir dc/d36/d43/d46/d63 0 2024-05-09T06:38:07.171 INFO:tasks.workunit.client.0.smithi063.stdout:0/307: rename dd/d1a/d22 to dd/d60/d63 0 2024-05-09T06:38:07.173 INFO:tasks.workunit.client.0.smithi063.stdout:3/278: dread d7/da/f13 [0,4194304] 0 2024-05-09T06:38:07.174 INFO:tasks.workunit.client.0.smithi063.stdout:8/345: dwrite f4 [4194304,4194304] 0 2024-05-09T06:38:07.176 INFO:tasks.workunit.client.0.smithi063.stdout:3/279: write f5 [2182137,78934] 0 2024-05-09T06:38:07.180 INFO:tasks.workunit.client.0.smithi063.stdout:8/346: mkdir d9/d37/d73 0 2024-05-09T06:38:07.183 INFO:tasks.workunit.client.0.smithi063.stdout:8/347: write d9/d37/f4d [3347092,80901] 0 2024-05-09T06:38:07.183 INFO:tasks.workunit.client.0.smithi063.stdout:9/289: write d0/f1f [1948090,107846] 0 2024-05-09T06:38:07.187 INFO:tasks.workunit.client.0.smithi063.stdout:7/298: getdents d0/d15 0 2024-05-09T06:38:07.189 INFO:tasks.workunit.client.0.smithi063.stdout:0/308: creat dd/d60/d63/f64 x:0 0 0 2024-05-09T06:38:07.192 INFO:tasks.workunit.client.0.smithi063.stdout:9/290: mkdir d0/d4/d57/d60 0 2024-05-09T06:38:07.193 INFO:tasks.workunit.client.0.smithi063.stdout:0/309: write f5 [1709453,100648] 0 2024-05-09T06:38:07.193 INFO:tasks.workunit.client.0.smithi063.stdout:8/348: rename d9/da/d18/l21 to d9/da/d18/d24/d71/l74 0 2024-05-09T06:38:07.193 INFO:tasks.workunit.client.0.smithi063.stdout:3/280: dread d7/da/d21/f2c [0,4194304] 0 2024-05-09T06:38:07.201 INFO:tasks.workunit.client.0.smithi063.stdout:6/305: dread d2/d4/f1c [0,4194304] 0 2024-05-09T06:38:07.202 INFO:tasks.workunit.client.0.smithi063.stdout:0/310: write dd/d1a/d2c/d42/f4d [2869021,45519] 0 2024-05-09T06:38:07.202 INFO:tasks.workunit.client.0.smithi063.stdout:8/349: write d9/da/d34/f4b [3634857,119830] 0 2024-05-09T06:38:07.207 INFO:tasks.workunit.client.0.smithi063.stdout:1/269: dread d9/d10/f22 [0,4194304] 0 2024-05-09T06:38:07.208 INFO:tasks.workunit.client.0.smithi063.stdout:7/299: dwrite d0/d15/f3a [0,4194304] 0 2024-05-09T06:38:07.209 INFO:tasks.workunit.client.0.smithi063.stdout:0/311: unlink dd/c21 0 2024-05-09T06:38:07.210 INFO:tasks.workunit.client.0.smithi063.stdout:2/343: dread dc/f25 [0,4194304] 0 2024-05-09T06:38:07.213 INFO:tasks.workunit.client.0.smithi063.stdout:9/291: dwrite d0/f1f [0,4194304] 0 2024-05-09T06:38:07.213 INFO:tasks.workunit.client.0.smithi063.stdout:8/350: rename d9/da/d18/d56/c57 to d9/d3b/c75 0 2024-05-09T06:38:07.218 INFO:tasks.workunit.client.0.smithi063.stdout:1/270: mkdir d9/d10/d2b/d3f/d59 0 2024-05-09T06:38:07.221 INFO:tasks.workunit.client.0.smithi063.stdout:1/271: write f8 [1344419,32805] 0 2024-05-09T06:38:07.222 INFO:tasks.workunit.client.0.smithi063.stdout:8/351: creat d9/da/d18/d56/f76 x:0 0 0 2024-05-09T06:38:07.224 INFO:tasks.workunit.client.0.smithi063.stdout:8/352: read d9/d3b/f40 [1763158,7149] 0 2024-05-09T06:38:07.225 INFO:tasks.workunit.client.0.smithi063.stdout:3/281: dwrite d7/da/fb [0,4194304] 0 2024-05-09T06:38:07.228 INFO:tasks.workunit.client.0.smithi063.stdout:0/312: getdents dd/d60/d63 0 2024-05-09T06:38:07.233 INFO:tasks.workunit.client.0.smithi063.stdout:8/353: write d9/da/d18/f1d [504532,617] 0 2024-05-09T06:38:07.234 INFO:tasks.workunit.client.0.smithi063.stdout:0/313: write f5 [3719984,69034] 0 2024-05-09T06:38:07.236 INFO:tasks.workunit.client.0.smithi063.stdout:9/292: dwrite d0/d4/d26/d39/f53 [0,4194304] 0 2024-05-09T06:38:07.236 INFO:tasks.workunit.client.0.smithi063.stdout:3/282: mknod d7/d4c/d31/c63 0 2024-05-09T06:38:07.241 INFO:tasks.workunit.client.0.smithi063.stdout:8/354: truncate d9/da/f4e 825447 0 2024-05-09T06:38:07.245 INFO:tasks.workunit.client.0.smithi063.stdout:1/272: dread d9/d10/f3e [0,4194304] 0 2024-05-09T06:38:07.246 INFO:tasks.workunit.client.0.smithi063.stdout:7/300: dwrite d0/d15/d4a/d50/f3c [0,4194304] 0 2024-05-09T06:38:07.250 INFO:tasks.workunit.client.0.smithi063.stdout:7/301: write d0/f18 [2455600,100893] 0 2024-05-09T06:38:07.252 INFO:tasks.workunit.client.0.smithi063.stdout:8/355: creat d9/f77 x:0 0 0 2024-05-09T06:38:07.254 INFO:tasks.workunit.client.0.smithi063.stdout:0/314: write dd/d1a/f1b [752111,17493] 0 2024-05-09T06:38:07.254 INFO:tasks.workunit.client.0.smithi063.stdout:9/293: dwrite d0/d4/d26/d39/d3e/f54 [0,4194304] 0 2024-05-09T06:38:07.254 INFO:tasks.workunit.client.0.smithi063.stdout:1/273: symlink d9/l5a 0 2024-05-09T06:38:07.258 INFO:tasks.workunit.client.0.smithi063.stdout:2/344: dread dc/dd/f2f [0,4194304] 0 2024-05-09T06:38:07.258 INFO:tasks.workunit.client.0.smithi063.stdout:9/294: dread - d0/d4/f37 zero size 2024-05-09T06:38:07.260 INFO:tasks.workunit.client.0.smithi063.stdout:0/315: unlink dd/l28 0 2024-05-09T06:38:07.261 INFO:tasks.workunit.client.0.smithi063.stdout:7/302: creat d0/d15/d4a/d50/f61 x:0 0 0 2024-05-09T06:38:07.265 INFO:tasks.workunit.client.0.smithi063.stdout:1/274: chown d9/cd 5327 1 2024-05-09T06:38:07.267 INFO:tasks.workunit.client.0.smithi063.stdout:7/303: chown d0/c39 1263 1 2024-05-09T06:38:07.268 INFO:tasks.workunit.client.0.smithi063.stdout:9/295: mknod d0/c61 0 2024-05-09T06:38:07.269 INFO:tasks.workunit.client.0.smithi063.stdout:8/356: dwrite d9/da/d18/f2d [0,4194304] 0 2024-05-09T06:38:07.270 INFO:tasks.workunit.client.0.smithi063.stdout:1/275: mknod d9/d46/c5b 0 2024-05-09T06:38:07.270 INFO:tasks.workunit.client.0.smithi063.stdout:0/316: read f2 [2889793,51337] 0 2024-05-09T06:38:07.273 INFO:tasks.workunit.client.0.smithi063.stdout:8/357: stat d9/d3b/f5c 0 2024-05-09T06:38:07.273 INFO:tasks.workunit.client.0.smithi063.stdout:3/283: dread d7/f10 [0,4194304] 0 2024-05-09T06:38:07.275 INFO:tasks.workunit.client.0.smithi063.stdout:9/296: dread - d0/d4/d26/d39/d48/f5a zero size 2024-05-09T06:38:07.276 INFO:tasks.workunit.client.0.smithi063.stdout:1/276: symlink d9/d46/l5c 0 2024-05-09T06:38:07.277 INFO:tasks.workunit.client.0.smithi063.stdout:8/358: fdatasync d9/da/d18/d46/f64 0 2024-05-09T06:38:07.281 INFO:tasks.workunit.client.0.smithi063.stdout:0/317: creat dd/d1a/f65 x:0 0 0 2024-05-09T06:38:07.282 INFO:tasks.workunit.client.0.smithi063.stdout:2/345: dwrite f6 [0,4194304] 0 2024-05-09T06:38:07.284 INFO:tasks.workunit.client.0.smithi063.stdout:3/284: creat d7/da/d2b/f64 x:0 0 0 2024-05-09T06:38:07.288 INFO:tasks.workunit.client.0.smithi063.stdout:7/304: link d0/d15/d49/d4f/c5d d0/d15/d49/d54/c62 0 2024-05-09T06:38:07.288 INFO:tasks.workunit.client.0.smithi063.stdout:4/310: sync 2024-05-09T06:38:07.288 INFO:tasks.workunit.client.0.smithi063.stdout:8/359: chown d9/d37/c61 64758310 1 2024-05-09T06:38:07.290 INFO:tasks.workunit.client.0.smithi063.stdout:9/297: dread d0/f1f [0,4194304] 0 2024-05-09T06:38:07.291 INFO:tasks.workunit.client.0.smithi063.stdout:2/346: chown dc/dd/l28 11 1 2024-05-09T06:38:07.292 INFO:tasks.workunit.client.0.smithi063.stdout:7/305: symlink d0/d15/l63 0 2024-05-09T06:38:07.292 INFO:tasks.workunit.client.0.smithi063.stdout:1/277: dread d9/d2a/d35/d24/f53 [0,4194304] 0 2024-05-09T06:38:07.293 INFO:tasks.workunit.client.0.smithi063.stdout:8/360: write d9/da/d18/f63 [535008,52243] 0 2024-05-09T06:38:07.293 INFO:tasks.workunit.client.0.smithi063.stdout:3/285: rmdir d7/da/d2b 39 2024-05-09T06:38:07.293 INFO:tasks.workunit.client.0.smithi063.stdout:4/311: mkdir d7/df/d1f/d28/d31/d61 0 2024-05-09T06:38:07.295 INFO:tasks.workunit.client.0.smithi063.stdout:9/298: mknod d0/d4/d26/d39/d42/c62 0 2024-05-09T06:38:07.296 INFO:tasks.workunit.client.0.smithi063.stdout:4/312: write d7/d3d/f55 [878246,127846] 0 2024-05-09T06:38:07.297 INFO:tasks.workunit.client.0.smithi063.stdout:1/278: write d9/d2a/d2e/f50 [1937578,37755] 0 2024-05-09T06:38:07.297 INFO:tasks.workunit.client.0.smithi063.stdout:0/318: link dd/d60/d63/d23/c3a dd/d60/d63/d23/d33/d48/c66 0 2024-05-09T06:38:07.298 INFO:tasks.workunit.client.0.smithi063.stdout:7/306: mknod d0/d15/d49/d4f/c64 0 2024-05-09T06:38:07.298 INFO:tasks.workunit.client.0.smithi063.stdout:4/313: chown d7/df/d1a/f20 237 1 2024-05-09T06:38:07.299 INFO:tasks.workunit.client.0.smithi063.stdout:0/319: fsync dd/d1a/f1b 0 2024-05-09T06:38:07.300 INFO:tasks.workunit.client.0.smithi063.stdout:3/286: creat d7/d2a/d2d/f65 x:0 0 0 2024-05-09T06:38:07.301 INFO:tasks.workunit.client.0.smithi063.stdout:4/314: read - d7/df/d1f/f3a zero size 2024-05-09T06:38:07.301 INFO:tasks.workunit.client.0.smithi063.stdout:7/307: write d0/d15/d4a/d50/f3c [2109440,3176] 0 2024-05-09T06:38:07.302 INFO:tasks.workunit.client.0.smithi063.stdout:2/347: creat dc/dd/f64 x:0 0 0 2024-05-09T06:38:07.302 INFO:tasks.workunit.client.0.smithi063.stdout:7/308: fsync d0/d15/f1b 0 2024-05-09T06:38:07.303 INFO:tasks.workunit.client.0.smithi063.stdout:8/361: creat d9/da/d18/d56/f78 x:0 0 0 2024-05-09T06:38:07.304 INFO:tasks.workunit.client.0.smithi063.stdout:9/299: mkdir d0/d3b/d63 0 2024-05-09T06:38:07.305 INFO:tasks.workunit.client.0.smithi063.stdout:9/300: chown d0/d4/d26/f36 274806 1 2024-05-09T06:38:07.305 INFO:tasks.workunit.client.0.smithi063.stdout:1/279: mkdir d9/d2a/d39/d5d 0 2024-05-09T06:38:07.309 INFO:tasks.workunit.client.0.smithi063.stdout:0/320: creat dd/d60/d63/f67 x:0 0 0 2024-05-09T06:38:07.313 INFO:tasks.workunit.client.0.smithi063.stdout:4/315: chown d7/df/d1f/d28/d31/d32/d34/c54 902 1 2024-05-09T06:38:07.313 INFO:tasks.workunit.client.0.smithi063.stdout:2/348: chown dc/dd/d2d/f38 12275816 1 2024-05-09T06:38:07.314 INFO:tasks.workunit.client.0.smithi063.stdout:6/306: sync 2024-05-09T06:38:07.314 INFO:tasks.workunit.client.0.smithi063.stdout:5/246: sync 2024-05-09T06:38:07.315 INFO:tasks.workunit.client.0.smithi063.stdout:5/247: readlink d8/d35/l4d 0 2024-05-09T06:38:07.315 INFO:tasks.workunit.client.0.smithi063.stdout:7/309: creat d0/d15/d49/f65 x:0 0 0 2024-05-09T06:38:07.316 INFO:tasks.workunit.client.0.smithi063.stdout:7/310: write d0/f32 [2444648,71020] 0 2024-05-09T06:38:07.317 INFO:tasks.workunit.client.0.smithi063.stdout:7/311: fsync d0/f1c 0 2024-05-09T06:38:07.319 INFO:tasks.workunit.client.0.smithi063.stdout:9/301: creat d0/d4/d57/f64 x:0 0 0 2024-05-09T06:38:07.321 INFO:tasks.workunit.client.0.smithi063.stdout:1/280: creat d9/d2a/d39/f5e x:0 0 0 2024-05-09T06:38:07.323 INFO:tasks.workunit.client.0.smithi063.stdout:3/287: mkdir d7/da/d2b/d66 0 2024-05-09T06:38:07.324 INFO:tasks.workunit.client.0.smithi063.stdout:3/288: dread - d7/da/d21/d30/f5a zero size 2024-05-09T06:38:07.328 INFO:tasks.workunit.client.0.smithi063.stdout:4/316: creat d7/f62 x:0 0 0 2024-05-09T06:38:07.332 INFO:tasks.workunit.client.0.smithi063.stdout:5/248: symlink d8/d19/d30/l54 0 2024-05-09T06:38:07.338 INFO:tasks.workunit.client.0.smithi063.stdout:7/312: unlink d0/d15/d49/l56 0 2024-05-09T06:38:07.339 INFO:tasks.workunit.client.0.smithi063.stdout:7/313: write d0/d15/f3a [1409227,68071] 0 2024-05-09T06:38:07.343 INFO:tasks.workunit.client.0.smithi063.stdout:1/281: rename d9/c52 to d9/d2a/d39/c5f 0 2024-05-09T06:38:07.343 INFO:tasks.workunit.client.0.smithi063.stdout:3/289: rename d7/d2a/d2d to d7/d2a/d2d/d67 22 2024-05-09T06:38:07.344 INFO:tasks.workunit.client.0.smithi063.stdout:3/290: fdatasync d7/da/d21/d30/f40 0 2024-05-09T06:38:07.345 INFO:tasks.workunit.client.0.smithi063.stdout:3/291: write d7/f17 [4465163,43299] 0 2024-05-09T06:38:07.346 INFO:tasks.workunit.client.0.smithi063.stdout:2/349: mkdir dc/dd/d2d/d60/d65 0 2024-05-09T06:38:07.347 INFO:tasks.workunit.client.0.smithi063.stdout:2/350: chown f3 18568140 1 2024-05-09T06:38:07.350 INFO:tasks.workunit.client.0.smithi063.stdout:1/282: dwrite d9/d10/f15 [0,4194304] 0 2024-05-09T06:38:07.350 INFO:tasks.workunit.client.0.smithi063.stdout:5/249: rmdir d8/d17 39 2024-05-09T06:38:07.352 INFO:tasks.workunit.client.0.smithi063.stdout:8/362: write d9/d3b/f2c [1238355,111647] 0 2024-05-09T06:38:07.362 INFO:tasks.workunit.client.0.smithi063.stdout:4/317: creat d7/df/d35/d5a/d60/f63 x:0 0 0 2024-05-09T06:38:07.365 INFO:tasks.workunit.client.0.smithi063.stdout:9/302: dwrite d0/d4/d26/d39/f55 [0,4194304] 0 2024-05-09T06:38:07.369 INFO:tasks.workunit.client.0.smithi063.stdout:0/321: mknod dd/d60/d63/c68 0 2024-05-09T06:38:07.371 INFO:tasks.workunit.client.0.smithi063.stdout:1/283: stat d9/d10/f21 0 2024-05-09T06:38:07.378 INFO:tasks.workunit.client.0.smithi063.stdout:6/307: getdents d2/d20/d26 0 2024-05-09T06:38:07.382 INFO:tasks.workunit.client.0.smithi063.stdout:3/292: truncate d7/da/d21/f4d 569143 0 2024-05-09T06:38:07.385 INFO:tasks.workunit.client.0.smithi063.stdout:0/322: creat dd/d60/d44/f69 x:0 0 0 2024-05-09T06:38:07.390 INFO:tasks.workunit.client.0.smithi063.stdout:8/363: truncate d9/da/f1a 1653632 0 2024-05-09T06:38:07.392 INFO:tasks.workunit.client.0.smithi063.stdout:4/318: mknod d7/df/d57/c64 0 2024-05-09T06:38:07.393 INFO:tasks.workunit.client.0.smithi063.stdout:2/351: dread dc/dd/f23 [0,4194304] 0 2024-05-09T06:38:07.393 INFO:tasks.workunit.client.0.smithi063.stdout:2/352: chown dc/f1c 3 1 2024-05-09T06:38:07.398 INFO:tasks.workunit.client.0.smithi063.stdout:0/323: chown dd/f2d 11230607 1 2024-05-09T06:38:07.405 INFO:tasks.workunit.client.0.smithi063.stdout:8/364: mkdir d9/da/d18/d56/d58/d79 0 2024-05-09T06:38:07.408 INFO:tasks.workunit.client.0.smithi063.stdout:6/308: mknod d2/c6a 0 2024-05-09T06:38:07.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/353: write dc/dd/f4f [1009167,19160] 0 2024-05-09T06:38:07.412 INFO:tasks.workunit.client.0.smithi063.stdout:3/293: unlink d7/d22/d28/f58 0 2024-05-09T06:38:07.413 INFO:tasks.workunit.client.0.smithi063.stdout:3/294: write d7/d22/d28/f3f [3761686,47970] 0 2024-05-09T06:38:07.413 INFO:tasks.workunit.client.0.smithi063.stdout:9/303: creat d0/f65 x:0 0 0 2024-05-09T06:38:07.414 INFO:tasks.workunit.client.0.smithi063.stdout:3/295: chown d7/da/d21/l4a 350 1 2024-05-09T06:38:07.416 INFO:tasks.workunit.client.0.smithi063.stdout:0/324: mknod dd/d1a/d2c/c6a 0 2024-05-09T06:38:07.418 INFO:tasks.workunit.client.0.smithi063.stdout:5/250: truncate d8/d17/d18/f45 1965523 0 2024-05-09T06:38:07.418 INFO:tasks.workunit.client.0.smithi063.stdout:5/251: chown d8/d17/d18/f22 6089480 1 2024-05-09T06:38:07.423 INFO:tasks.workunit.client.0.smithi063.stdout:8/365: rename d9/d41/c66 to d9/da/d18/d56/c7a 0 2024-05-09T06:38:07.429 INFO:tasks.workunit.client.0.smithi063.stdout:2/354: chown dc/f12 42 1 2024-05-09T06:38:07.434 INFO:tasks.workunit.client.0.smithi063.stdout:9/304: mkdir d0/d4/d26/d39/d3e/d66 0 2024-05-09T06:38:07.436 INFO:tasks.workunit.client.0.smithi063.stdout:0/325: creat dd/f6b x:0 0 0 2024-05-09T06:38:07.436 INFO:tasks.workunit.client.0.smithi063.stdout:9/305: dread d0/d4/da/f1c [4194304,4194304] 0 2024-05-09T06:38:07.437 INFO:tasks.workunit.client.0.smithi063.stdout:1/284: link d9/d2a/d35/l1b d9/d10/d2b/d3f/d59/l60 0 2024-05-09T06:38:07.439 INFO:tasks.workunit.client.0.smithi063.stdout:1/285: dread - d9/d2a/d39/f5e zero size 2024-05-09T06:38:07.439 INFO:tasks.workunit.client.0.smithi063.stdout:1/286: readlink d9/d2a/d35/d1e/l49 0 2024-05-09T06:38:07.440 INFO:tasks.workunit.client.0.smithi063.stdout:5/252: truncate d8/d17/d18/f22 955465 0 2024-05-09T06:38:07.444 INFO:tasks.workunit.client.0.smithi063.stdout:8/366: rename d9/da/d34/c5f to d9/d37/c7b 0 2024-05-09T06:38:07.446 INFO:tasks.workunit.client.0.smithi063.stdout:4/319: creat d7/df/d1f/f65 x:0 0 0 2024-05-09T06:38:07.451 INFO:tasks.workunit.client.0.smithi063.stdout:1/287: dread d9/d2a/d35/d24/f26 [0,4194304] 0 2024-05-09T06:38:07.453 INFO:tasks.workunit.client.0.smithi063.stdout:1/288: readlink d9/d2a/d2e/l58 0 2024-05-09T06:38:07.453 INFO:tasks.workunit.client.0.smithi063.stdout:3/296: symlink d7/da/l68 0 2024-05-09T06:38:07.455 INFO:tasks.workunit.client.0.smithi063.stdout:4/320: dwrite d7/df/d1f/d28/d31/d32/f51 [0,4194304] 0 2024-05-09T06:38:07.463 INFO:tasks.workunit.client.0.smithi063.stdout:5/253: mknod d8/d17/d1e/c55 0 2024-05-09T06:38:07.465 INFO:tasks.workunit.client.0.smithi063.stdout:8/367: mknod d9/d37/c7c 0 2024-05-09T06:38:07.470 INFO:tasks.workunit.client.0.smithi063.stdout:3/297: creat d7/d4c/f69 x:0 0 0 2024-05-09T06:38:07.471 INFO:tasks.workunit.client.0.smithi063.stdout:4/321: mknod d7/df/d35/c66 0 2024-05-09T06:38:07.473 INFO:tasks.workunit.client.0.smithi063.stdout:9/306: symlink d0/l67 0 2024-05-09T06:38:07.477 INFO:tasks.workunit.client.0.smithi063.stdout:0/326: rename dd/d60/d63/d23/d33/d48/f53 to dd/d60/d63/d23/d31/d58/f6c 0 2024-05-09T06:38:07.478 INFO:tasks.workunit.client.0.smithi063.stdout:5/254: creat d8/d17/f56 x:0 0 0 2024-05-09T06:38:07.479 INFO:tasks.workunit.client.0.smithi063.stdout:5/255: readlink d8/d17/l1b 0 2024-05-09T06:38:07.480 INFO:tasks.workunit.client.0.smithi063.stdout:1/289: symlink d9/d46/d57/l61 0 2024-05-09T06:38:07.480 INFO:tasks.workunit.client.0.smithi063.stdout:4/322: dread d7/df/d1f/d28/d31/d32/d34/f44 [0,4194304] 0 2024-05-09T06:38:07.480 INFO:tasks.workunit.client.0.smithi063.stdout:3/298: mkdir d7/da/d2b/d6a 0 2024-05-09T06:38:07.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/290: write d9/d46/f54 [310171,63136] 0 2024-05-09T06:38:07.482 INFO:tasks.workunit.client.0.smithi063.stdout:3/299: fdatasync d7/d2a/f4b 0 2024-05-09T06:38:07.483 INFO:tasks.workunit.client.0.smithi063.stdout:9/307: rmdir d0/d8/d2d 39 2024-05-09T06:38:07.484 INFO:tasks.workunit.client.0.smithi063.stdout:9/308: read - d0/f5b zero size 2024-05-09T06:38:07.484 INFO:tasks.workunit.client.0.smithi063.stdout:6/309: write d2/d20/f44 [2726848,118235] 0 2024-05-09T06:38:07.485 INFO:tasks.workunit.client.0.smithi063.stdout:9/309: readlink d0/d8/le 0 2024-05-09T06:38:07.486 INFO:tasks.workunit.client.0.smithi063.stdout:6/310: fsync d2/d1f/d29/f63 0 2024-05-09T06:38:07.487 INFO:tasks.workunit.client.0.smithi063.stdout:9/310: chown d0/d4/d26/f2a 5 1 2024-05-09T06:38:07.488 INFO:tasks.workunit.client.0.smithi063.stdout:3/300: dread d7/da/f14 [0,4194304] 0 2024-05-09T06:38:07.488 INFO:tasks.workunit.client.0.smithi063.stdout:0/327: mknod dd/d1a/d2c/d42/c6d 0 2024-05-09T06:38:07.489 INFO:tasks.workunit.client.0.smithi063.stdout:7/314: sync 2024-05-09T06:38:07.489 INFO:tasks.workunit.client.0.smithi063.stdout:7/315: stat d0/d15 0 2024-05-09T06:38:07.490 INFO:tasks.workunit.client.0.smithi063.stdout:3/301: chown d7/d22/d28/f3c 326255944 1 2024-05-09T06:38:07.490 INFO:tasks.workunit.client.0.smithi063.stdout:4/323: symlink d7/df/d1f/d28/d31/l67 0 2024-05-09T06:38:07.495 INFO:tasks.workunit.client.0.smithi063.stdout:6/311: creat d2/d1f/d29/d3b/d4e/f6b x:0 0 0 2024-05-09T06:38:07.496 INFO:tasks.workunit.client.0.smithi063.stdout:3/302: dwrite d7/d22/d28/f3f [0,4194304] 0 2024-05-09T06:38:07.497 INFO:tasks.workunit.client.0.smithi063.stdout:5/256: symlink d8/l57 0 2024-05-09T06:38:07.499 INFO:tasks.workunit.client.0.smithi063.stdout:7/316: rmdir d0/d15/d4a/d50 39 2024-05-09T06:38:07.499 INFO:tasks.workunit.client.0.smithi063.stdout:5/257: read - d8/d35/f41 zero size 2024-05-09T06:38:07.501 INFO:tasks.workunit.client.0.smithi063.stdout:4/324: symlink d7/df/d35/d5a/l68 0 2024-05-09T06:38:07.502 INFO:tasks.workunit.client.0.smithi063.stdout:1/291: mkdir d9/d10/d2b/d3f/d59/d62 0 2024-05-09T06:38:07.504 INFO:tasks.workunit.client.0.smithi063.stdout:8/368: rename d9/d41/f5e to d9/f7d 0 2024-05-09T06:38:07.504 INFO:tasks.workunit.client.0.smithi063.stdout:6/312: dwrite d2/d1f/d29/d3f/f58 [4194304,4194304] 0 2024-05-09T06:38:07.504 INFO:tasks.workunit.client.0.smithi063.stdout:4/325: chown d7/df/d57/c64 173144 1 2024-05-09T06:38:07.506 INFO:tasks.workunit.client.0.smithi063.stdout:7/317: truncate d0/d15/d49/d54/f60 555646 0 2024-05-09T06:38:07.508 INFO:tasks.workunit.client.0.smithi063.stdout:3/303: stat d7/da/f18 0 2024-05-09T06:38:07.509 INFO:tasks.workunit.client.0.smithi063.stdout:6/313: chown d2/d20/f21 1 1 2024-05-09T06:38:07.509 INFO:tasks.workunit.client.0.smithi063.stdout:5/258: creat d8/d17/d18/f58 x:0 0 0 2024-05-09T06:38:07.517 INFO:tasks.workunit.client.0.smithi063.stdout:6/314: readlink d2/df/l49 0 2024-05-09T06:38:07.518 INFO:tasks.workunit.client.0.smithi063.stdout:0/328: rename dd/d1a/d50/f5f to dd/d60/d63/d23/d31/d58/f6e 0 2024-05-09T06:38:07.518 INFO:tasks.workunit.client.0.smithi063.stdout:6/315: readlink d2/d20/d26/l43 0 2024-05-09T06:38:07.520 INFO:tasks.workunit.client.0.smithi063.stdout:8/369: dwrite d9/da/d34/f4b [0,4194304] 0 2024-05-09T06:38:07.521 INFO:tasks.workunit.client.0.smithi063.stdout:6/316: write d2/d1f/d29/f2a [1379231,24024] 0 2024-05-09T06:38:07.522 INFO:tasks.workunit.client.0.smithi063.stdout:9/311: getdents d0/d43 0 2024-05-09T06:38:07.522 INFO:tasks.workunit.client.0.smithi063.stdout:8/370: chown d9/d3b 7845637 1 2024-05-09T06:38:07.525 INFO:tasks.workunit.client.0.smithi063.stdout:9/312: readlink d0/l4e 0 2024-05-09T06:38:07.526 INFO:tasks.workunit.client.0.smithi063.stdout:0/329: creat dd/d1a/d50/f6f x:0 0 0 2024-05-09T06:38:07.526 INFO:tasks.workunit.client.0.smithi063.stdout:5/259: creat d8/d19/d30/d46/f59 x:0 0 0 2024-05-09T06:38:07.527 INFO:tasks.workunit.client.0.smithi063.stdout:8/371: symlink d9/da/d18/d56/l7e 0 2024-05-09T06:38:07.527 INFO:tasks.workunit.client.0.smithi063.stdout:6/317: fdatasync d2/f38 0 2024-05-09T06:38:07.529 INFO:tasks.workunit.client.0.smithi063.stdout:7/318: getdents d0/d15/d4a 0 2024-05-09T06:38:07.530 INFO:tasks.workunit.client.0.smithi063.stdout:5/260: unlink d8/c2e 0 2024-05-09T06:38:07.533 INFO:tasks.workunit.client.0.smithi063.stdout:6/318: write d2/d1f/f5b [877118,17735] 0 2024-05-09T06:38:07.533 INFO:tasks.workunit.client.0.smithi063.stdout:9/313: truncate d0/d4/d26/d47/f2b 1211261 0 2024-05-09T06:38:07.533 INFO:tasks.workunit.client.0.smithi063.stdout:8/372: symlink d9/da/d18/d24/l7f 0 2024-05-09T06:38:07.534 INFO:tasks.workunit.client.0.smithi063.stdout:7/319: truncate d0/d15/f58 178991 0 2024-05-09T06:38:07.535 INFO:tasks.workunit.client.0.smithi063.stdout:0/330: rename dd/d1a/d2c/d42/c6d to dd/d1a/d2c/c70 0 2024-05-09T06:38:07.537 INFO:tasks.workunit.client.0.smithi063.stdout:6/319: mkdir d2/d1f/d6c 0 2024-05-09T06:38:07.537 INFO:tasks.workunit.client.0.smithi063.stdout:7/320: chown d0/f1c 26 1 2024-05-09T06:38:07.539 INFO:tasks.workunit.client.0.smithi063.stdout:7/321: readlink d0/d15/l41 0 2024-05-09T06:38:07.539 INFO:tasks.workunit.client.0.smithi063.stdout:6/320: stat d2/d1f/d29/f63 0 2024-05-09T06:38:07.540 INFO:tasks.workunit.client.0.smithi063.stdout:9/314: symlink d0/d4/d26/d39/d3e/d66/l68 0 2024-05-09T06:38:07.540 INFO:tasks.workunit.client.0.smithi063.stdout:0/331: chown dd/d60/d63/d23/d33/c56 864927 1 2024-05-09T06:38:07.543 INFO:tasks.workunit.client.0.smithi063.stdout:0/332: dread - dd/d60/d44/f69 zero size 2024-05-09T06:38:07.544 INFO:tasks.workunit.client.0.smithi063.stdout:9/315: write d0/d43/f4b [4425515,97735] 0 2024-05-09T06:38:07.544 INFO:tasks.workunit.client.0.smithi063.stdout:1/292: dread d9/d10/f18 [0,4194304] 0 2024-05-09T06:38:07.545 INFO:tasks.workunit.client.0.smithi063.stdout:8/373: creat d9/da/d18/f80 x:0 0 0 2024-05-09T06:38:07.546 INFO:tasks.workunit.client.0.smithi063.stdout:0/333: chown dd/d1a/f25 31370184 1 2024-05-09T06:38:07.546 INFO:tasks.workunit.client.0.smithi063.stdout:4/326: dread d7/f2d [0,4194304] 0 2024-05-09T06:38:07.547 INFO:tasks.workunit.client.0.smithi063.stdout:1/293: readlink d9/d46/l47 0 2024-05-09T06:38:07.547 INFO:tasks.workunit.client.0.smithi063.stdout:7/322: dread d0/d15/d4a/d50/f3f [0,4194304] 0 2024-05-09T06:38:07.548 INFO:tasks.workunit.client.0.smithi063.stdout:0/334: stat dd 0 2024-05-09T06:38:07.548 INFO:tasks.workunit.client.0.smithi063.stdout:9/316: truncate d0/d4/f4d 1664299 0 2024-05-09T06:38:07.550 INFO:tasks.workunit.client.0.smithi063.stdout:8/374: unlink d9/da/d34/f38 0 2024-05-09T06:38:07.550 INFO:tasks.workunit.client.0.smithi063.stdout:6/321: dread d2/d12/f24 [0,4194304] 0 2024-05-09T06:38:07.552 INFO:tasks.workunit.client.0.smithi063.stdout:1/294: creat d9/d46/f63 x:0 0 0 2024-05-09T06:38:07.554 INFO:tasks.workunit.client.0.smithi063.stdout:7/323: mkdir d0/d15/d49/d4f/d66 0 2024-05-09T06:38:07.555 INFO:tasks.workunit.client.0.smithi063.stdout:0/335: symlink dd/d1a/d50/l71 0 2024-05-09T06:38:07.557 INFO:tasks.workunit.client.0.smithi063.stdout:6/322: rename d2/df/l2f to d2/df/d59/l6d 0 2024-05-09T06:38:07.560 INFO:tasks.workunit.client.0.smithi063.stdout:8/375: symlink d9/da/d18/d56/d58/d79/l81 0 2024-05-09T06:38:07.561 INFO:tasks.workunit.client.0.smithi063.stdout:9/317: creat d0/d8/f69 x:0 0 0 2024-05-09T06:38:07.562 INFO:tasks.workunit.client.0.smithi063.stdout:5/261: dwrite d8/d17/d18/f38 [0,4194304] 0 2024-05-09T06:38:07.562 INFO:tasks.workunit.client.0.smithi063.stdout:0/336: mkdir dd/d1a/d2c/d51/d72 0 2024-05-09T06:38:07.564 INFO:tasks.workunit.client.0.smithi063.stdout:7/324: mkdir d0/d15/d49/d4f/d66/d67 0 2024-05-09T06:38:07.564 INFO:tasks.workunit.client.0.smithi063.stdout:2/355: sync 2024-05-09T06:38:07.569 INFO:tasks.workunit.client.0.smithi063.stdout:4/327: dwrite d7/df/d35/f36 [0,4194304] 0 2024-05-09T06:38:07.570 INFO:tasks.workunit.client.0.smithi063.stdout:9/318: fsync d0/d4/d26/d39/d48/f5a 0 2024-05-09T06:38:07.571 INFO:tasks.workunit.client.0.smithi063.stdout:8/376: dread - d9/da/d18/d24/f42 zero size 2024-05-09T06:38:07.572 INFO:tasks.workunit.client.0.smithi063.stdout:6/323: creat d2/d1f/d29/d3f/d5c/d64/f6e x:0 0 0 2024-05-09T06:38:07.573 INFO:tasks.workunit.client.0.smithi063.stdout:2/356: unlink dc/l10 0 2024-05-09T06:38:07.574 INFO:tasks.workunit.client.0.smithi063.stdout:1/295: dread d9/d2a/d35/d1e/f27 [0,4194304] 0 2024-05-09T06:38:07.575 INFO:tasks.workunit.client.0.smithi063.stdout:5/262: truncate d8/d17/d18/f58 125437 0 2024-05-09T06:38:07.575 INFO:tasks.workunit.client.0.smithi063.stdout:6/324: read - d2/d12/f53 zero size 2024-05-09T06:38:07.578 INFO:tasks.workunit.client.0.smithi063.stdout:1/296: write d9/d2a/d35/f29 [4927880,61339] 0 2024-05-09T06:38:07.580 INFO:tasks.workunit.client.0.smithi063.stdout:6/325: chown d2/d1f/f69 226235 1 2024-05-09T06:38:07.582 INFO:tasks.workunit.client.0.smithi063.stdout:1/297: fsync d9/d46/f63 0 2024-05-09T06:38:07.582 INFO:tasks.workunit.client.0.smithi063.stdout:9/319: creat d0/d4/f6a x:0 0 0 2024-05-09T06:38:07.586 INFO:tasks.workunit.client.0.smithi063.stdout:2/357: write dc/f12 [3486523,94841] 0 2024-05-09T06:38:07.586 INFO:tasks.workunit.client.0.smithi063.stdout:6/326: chown d2/d1f/d29/d3f/l16 16262641 1 2024-05-09T06:38:07.588 INFO:tasks.workunit.client.0.smithi063.stdout:1/298: write d9/d10/f15 [3011351,93401] 0 2024-05-09T06:38:07.588 INFO:tasks.workunit.client.0.smithi063.stdout:7/325: dwrite d0/d15/f4e [0,4194304] 0 2024-05-09T06:38:07.588 INFO:tasks.workunit.client.0.smithi063.stdout:5/263: rmdir d8/d4e 0 2024-05-09T06:38:07.591 INFO:tasks.workunit.client.0.smithi063.stdout:0/337: link dd/d60/d63/d23/d46/c54 dd/d60/d63/d23/c73 0 2024-05-09T06:38:07.593 INFO:tasks.workunit.client.0.smithi063.stdout:4/328: truncate d7/df/d1f/f58 434958 0 2024-05-09T06:38:07.595 INFO:tasks.workunit.client.0.smithi063.stdout:2/358: creat dc/dd/d45/d5a/f66 x:0 0 0 2024-05-09T06:38:07.595 INFO:tasks.workunit.client.0.smithi063.stdout:8/377: link d9/d37/c7b d9/da/d18/c82 0 2024-05-09T06:38:07.598 INFO:tasks.workunit.client.0.smithi063.stdout:1/299: mknod d9/d10/d2b/d3f/c64 0 2024-05-09T06:38:07.598 INFO:tasks.workunit.client.0.smithi063.stdout:4/329: stat d7/d3d/f49 0 2024-05-09T06:38:07.598 INFO:tasks.workunit.client.0.smithi063.stdout:5/264: symlink d8/d53/l5a 0 2024-05-09T06:38:07.603 INFO:tasks.workunit.client.0.smithi063.stdout:5/265: chown d8/c40 184872 1 2024-05-09T06:38:07.603 INFO:tasks.workunit.client.0.smithi063.stdout:8/378: creat d9/d37/f83 x:0 0 0 2024-05-09T06:38:07.604 INFO:tasks.workunit.client.0.smithi063.stdout:9/320: dwrite d0/d8/f69 [0,4194304] 0 2024-05-09T06:38:07.604 INFO:tasks.workunit.client.0.smithi063.stdout:1/300: creat d9/d46/d57/f65 x:0 0 0 2024-05-09T06:38:07.606 INFO:tasks.workunit.client.0.smithi063.stdout:4/330: symlink d7/df/d1f/d28/l69 0 2024-05-09T06:38:07.609 INFO:tasks.workunit.client.0.smithi063.stdout:6/327: dwrite d2/d12/f34 [0,4194304] 0 2024-05-09T06:38:07.610 INFO:tasks.workunit.client.0.smithi063.stdout:4/331: fsync d7/f39 0 2024-05-09T06:38:07.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/359: truncate dc/dd/d2d/f38 2334993 0 2024-05-09T06:38:07.616 INFO:tasks.workunit.client.0.smithi063.stdout:1/301: chown d9/d2a/d35/c1f 14 1 2024-05-09T06:38:07.616 INFO:tasks.workunit.client.0.smithi063.stdout:5/266: write d8/f27 [259115,38424] 0 2024-05-09T06:38:07.618 INFO:tasks.workunit.client.0.smithi063.stdout:4/332: readlink d7/df/d1f/d28/l69 0 2024-05-09T06:38:07.619 INFO:tasks.workunit.client.0.smithi063.stdout:7/326: dwrite d0/d15/d4a/f53 [0,4194304] 0 2024-05-09T06:38:07.622 INFO:tasks.workunit.client.0.smithi063.stdout:7/327: stat d0/d15/d4a/d50/f61 0 2024-05-09T06:38:07.622 INFO:tasks.workunit.client.0.smithi063.stdout:1/302: truncate d9/f4e 752624 0 2024-05-09T06:38:07.626 INFO:tasks.workunit.client.0.smithi063.stdout:9/321: dwrite d0/d43/f5f [0,4194304] 0 2024-05-09T06:38:07.632 INFO:tasks.workunit.client.0.smithi063.stdout:4/333: truncate d7/df/d1f/d28/d31/d32/d34/f44 831050 0 2024-05-09T06:38:07.632 INFO:tasks.workunit.client.0.smithi063.stdout:9/322: truncate d0/d4/d26/d47/f2b 1613918 0 2024-05-09T06:38:07.633 INFO:tasks.workunit.client.0.smithi063.stdout:6/328: truncate d2/d4/f5 1672644 0 2024-05-09T06:38:07.635 INFO:tasks.workunit.client.0.smithi063.stdout:5/267: rename d8/d17/d1e/f37 to d8/f5b 0 2024-05-09T06:38:07.636 INFO:tasks.workunit.client.0.smithi063.stdout:6/329: stat d2/d20/d26/l43 0 2024-05-09T06:38:07.636 INFO:tasks.workunit.client.0.smithi063.stdout:5/268: readlink d8/d35/l52 0 2024-05-09T06:38:07.636 INFO:tasks.workunit.client.0.smithi063.stdout:9/323: symlink d0/d4/da/l6b 0 2024-05-09T06:38:07.637 INFO:tasks.workunit.client.0.smithi063.stdout:1/303: getdents d9/d46/d57 0 2024-05-09T06:38:07.638 INFO:tasks.workunit.client.0.smithi063.stdout:9/324: readlink d0/l67 0 2024-05-09T06:38:07.639 INFO:tasks.workunit.client.0.smithi063.stdout:7/328: dwrite d0/d15/f48 [0,4194304] 0 2024-05-09T06:38:07.639 INFO:tasks.workunit.client.0.smithi063.stdout:4/334: truncate d7/df/d1f/f53 1953238 0 2024-05-09T06:38:07.640 INFO:tasks.workunit.client.0.smithi063.stdout:6/330: write d2/d12/f53 [214821,62577] 0 2024-05-09T06:38:07.642 INFO:tasks.workunit.client.0.smithi063.stdout:5/269: mknod d8/d17/d1e/c5c 0 2024-05-09T06:38:07.647 INFO:tasks.workunit.client.0.smithi063.stdout:6/331: dread d2/d1f/f5b [0,4194304] 0 2024-05-09T06:38:07.648 INFO:tasks.workunit.client.0.smithi063.stdout:7/329: mkdir d0/d15/d4a/d68 0 2024-05-09T06:38:07.650 INFO:tasks.workunit.client.0.smithi063.stdout:7/330: truncate d0/d15/d49/f65 289993 0 2024-05-09T06:38:07.650 INFO:tasks.workunit.client.0.smithi063.stdout:7/331: readlink d0/d15/l63 0 2024-05-09T06:38:07.654 INFO:tasks.workunit.client.0.smithi063.stdout:6/332: creat d2/d1f/d29/d3f/d5c/d64/f6f x:0 0 0 2024-05-09T06:38:07.657 INFO:tasks.workunit.client.0.smithi063.stdout:6/333: fsync d2/d1f/d29/f63 0 2024-05-09T06:38:07.659 INFO:tasks.workunit.client.0.smithi063.stdout:3/304: sync 2024-05-09T06:38:07.659 INFO:tasks.workunit.client.0.smithi063.stdout:9/325: getdents d0/d4/d26 0 2024-05-09T06:38:07.660 INFO:tasks.workunit.client.0.smithi063.stdout:1/304: dwrite d9/d2a/d35/d1e/f25 [4194304,4194304] 0 2024-05-09T06:38:07.662 INFO:tasks.workunit.client.0.smithi063.stdout:7/332: creat d0/f69 x:0 0 0 2024-05-09T06:38:07.666 INFO:tasks.workunit.client.0.smithi063.stdout:9/326: rename d0/d4/d26/d39/d3e/f54 to d0/d4/d26/d39/f6c 0 2024-05-09T06:38:07.666 INFO:tasks.workunit.client.0.smithi063.stdout:4/335: read d7/df/d1f/d28/d31/d32/d34/f44 [589818,54166] 0 2024-05-09T06:38:07.668 INFO:tasks.workunit.client.0.smithi063.stdout:4/336: chown d7/df/d35/d5a/d60 478 1 2024-05-09T06:38:07.671 INFO:tasks.workunit.client.0.smithi063.stdout:7/333: dread d0/d15/d4a/d50/f3f [0,4194304] 0 2024-05-09T06:38:07.671 INFO:tasks.workunit.client.0.smithi063.stdout:3/305: creat d7/da/d21/f6b x:0 0 0 2024-05-09T06:38:07.672 INFO:tasks.workunit.client.0.smithi063.stdout:3/306: chown d7/l42 206690749 1 2024-05-09T06:38:07.672 INFO:tasks.workunit.client.0.smithi063.stdout:7/334: chown d0/d15/d4a/d50/l59 12970143 1 2024-05-09T06:38:07.673 INFO:tasks.workunit.client.0.smithi063.stdout:7/335: read - d0/d15/d49/f5f zero size 2024-05-09T06:38:07.674 INFO:tasks.workunit.client.0.smithi063.stdout:9/327: symlink d0/d3b/d63/l6d 0 2024-05-09T06:38:07.676 INFO:tasks.workunit.client.0.smithi063.stdout:3/307: creat d7/d4c/d25/f6c x:0 0 0 2024-05-09T06:38:07.676 INFO:tasks.workunit.client.0.smithi063.stdout:3/308: readlink d7/d4c/d31/d35/l37 0 2024-05-09T06:38:07.676 INFO:tasks.workunit.client.0.smithi063.stdout:4/337: truncate d7/df/f2f 2537382 0 2024-05-09T06:38:07.677 INFO:tasks.workunit.client.0.smithi063.stdout:4/338: chown d7/df/f12 273 1 2024-05-09T06:38:07.678 INFO:tasks.workunit.client.0.smithi063.stdout:7/336: unlink d0/d15/c1d 0 2024-05-09T06:38:07.678 INFO:tasks.workunit.client.0.smithi063.stdout:9/328: creat d0/d3b/f6e x:0 0 0 2024-05-09T06:38:07.680 INFO:tasks.workunit.client.0.smithi063.stdout:4/339: mknod d7/df/c6a 0 2024-05-09T06:38:07.681 INFO:tasks.workunit.client.0.smithi063.stdout:4/340: write d7/fb [1359423,80597] 0 2024-05-09T06:38:07.683 INFO:tasks.workunit.client.0.smithi063.stdout:9/329: creat d0/d4/d26/d39/f6f x:0 0 0 2024-05-09T06:38:07.685 INFO:tasks.workunit.client.0.smithi063.stdout:4/341: creat d7/df/d57/f6b x:0 0 0 2024-05-09T06:38:07.686 INFO:tasks.workunit.client.0.smithi063.stdout:7/337: dwrite d0/d15/d4a/d50/f37 [0,4194304] 0 2024-05-09T06:38:07.688 INFO:tasks.workunit.client.0.smithi063.stdout:7/338: write d0/d15/f35 [887589,77088] 0 2024-05-09T06:38:07.693 INFO:tasks.workunit.client.0.smithi063.stdout:7/339: write d0/d15/f26 [2636241,115471] 0 2024-05-09T06:38:07.694 INFO:tasks.workunit.client.0.smithi063.stdout:4/342: link d7/df/l37 d7/df/l6c 0 2024-05-09T06:38:07.699 INFO:tasks.workunit.client.0.smithi063.stdout:9/330: dwrite d0/d4/d26/d47/f2b [0,4194304] 0 2024-05-09T06:38:07.700 INFO:tasks.workunit.client.0.smithi063.stdout:7/340: write d0/d15/d4a/d50/f2e [214998,3906] 0 2024-05-09T06:38:07.701 INFO:tasks.workunit.client.0.smithi063.stdout:3/309: dwrite d7/d22/f50 [0,4194304] 0 2024-05-09T06:38:07.707 INFO:tasks.workunit.client.0.smithi063.stdout:4/343: creat d7/df/d1f/d28/d31/d61/f6d x:0 0 0 2024-05-09T06:38:07.709 INFO:tasks.workunit.client.0.smithi063.stdout:7/341: mknod d0/c6a 0 2024-05-09T06:38:07.709 INFO:tasks.workunit.client.0.smithi063.stdout:9/331: creat d0/d4/d57/d60/f70 x:0 0 0 2024-05-09T06:38:07.710 INFO:tasks.workunit.client.0.smithi063.stdout:3/310: chown d7/da/d2b/f43 229381461 1 2024-05-09T06:38:07.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/311: symlink d7/da/d21/l6d 0 2024-05-09T06:38:07.713 INFO:tasks.workunit.client.0.smithi063.stdout:9/332: creat d0/f71 x:0 0 0 2024-05-09T06:38:07.714 INFO:tasks.workunit.client.0.smithi063.stdout:3/312: truncate d7/d22/d28/f3c 314768 0 2024-05-09T06:38:07.714 INFO:tasks.workunit.client.0.smithi063.stdout:9/333: fdatasync d0/d4/d26/d39/f6f 0 2024-05-09T06:38:07.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/313: rmdir d7/da/d2b/d6a 0 2024-05-09T06:38:07.721 INFO:tasks.workunit.client.0.smithi063.stdout:3/314: rmdir d7/da/d21/d30/d44/d48 39 2024-05-09T06:38:07.722 INFO:tasks.workunit.client.0.smithi063.stdout:9/334: truncate d0/f1f 1094351 0 2024-05-09T06:38:07.723 INFO:tasks.workunit.client.0.smithi063.stdout:3/315: creat d7/d4c/d31/f6e x:0 0 0 2024-05-09T06:38:07.725 INFO:tasks.workunit.client.0.smithi063.stdout:3/316: chown d7/d22/f27 6406 1 2024-05-09T06:38:07.726 INFO:tasks.workunit.client.0.smithi063.stdout:9/335: dread d0/d4/d26/d39/f6c [0,4194304] 0 2024-05-09T06:38:07.730 INFO:tasks.workunit.client.0.smithi063.stdout:0/338: sync 2024-05-09T06:38:07.735 INFO:tasks.workunit.client.0.smithi063.stdout:9/336: dread d0/d4/d26/f49 [0,4194304] 0 2024-05-09T06:38:07.742 INFO:tasks.workunit.client.0.smithi063.stdout:0/339: dwrite dd/f2d [0,4194304] 0 2024-05-09T06:38:07.757 INFO:tasks.workunit.client.0.smithi063.stdout:0/340: truncate dd/d60/d63/d23/f2f 1201122 0 2024-05-09T06:38:07.758 INFO:tasks.workunit.client.0.smithi063.stdout:9/337: dread d0/d4/da/f1c [0,4194304] 0 2024-05-09T06:38:07.758 INFO:tasks.workunit.client.0.smithi063.stdout:0/341: dread - dd/d60/d63/d23/d31/d58/f6e zero size 2024-05-09T06:38:07.759 INFO:tasks.workunit.client.0.smithi063.stdout:9/338: write d0/d43/f5f [1877539,47347] 0 2024-05-09T06:38:07.784 INFO:tasks.workunit.client.0.smithi063.stdout:9/339: link d0/l67 d0/d4/d26/d39/d48/l72 0 2024-05-09T06:38:07.785 INFO:tasks.workunit.client.0.smithi063.stdout:0/342: dread f6 [0,4194304] 0 2024-05-09T06:38:07.786 INFO:tasks.workunit.client.0.smithi063.stdout:0/343: dread - dd/d1a/d50/f6f zero size 2024-05-09T06:38:07.788 INFO:tasks.workunit.client.0.smithi063.stdout:0/344: mknod dd/d60/c74 0 2024-05-09T06:38:07.789 INFO:tasks.workunit.client.0.smithi063.stdout:0/345: chown dd 124 1 2024-05-09T06:38:07.792 INFO:tasks.workunit.client.0.smithi063.stdout:9/340: dwrite d0/d4/d26/d39/f53 [0,4194304] 0 2024-05-09T06:38:07.793 INFO:tasks.workunit.client.0.smithi063.stdout:0/346: mkdir dd/d60/d63/d23/d33/d48/d75 0 2024-05-09T06:38:07.797 INFO:tasks.workunit.client.0.smithi063.stdout:9/341: mknod d0/d4/d26/d39/d3e/d66/c73 0 2024-05-09T06:38:07.803 INFO:tasks.workunit.client.0.smithi063.stdout:9/342: creat d0/d4/d26/d39/d42/f74 x:0 0 0 2024-05-09T06:38:07.805 INFO:tasks.workunit.client.0.smithi063.stdout:9/343: stat d0/d4/d26/c3d 0 2024-05-09T06:38:07.806 INFO:tasks.workunit.client.0.smithi063.stdout:0/347: dwrite dd/d60/d63/f55 [0,4194304] 0 2024-05-09T06:38:07.809 INFO:tasks.workunit.client.0.smithi063.stdout:8/379: sync 2024-05-09T06:38:07.809 INFO:tasks.workunit.client.0.smithi063.stdout:0/348: mkdir dd/d1a/d2c/d51/d76 0 2024-05-09T06:38:07.809 INFO:tasks.workunit.client.0.smithi063.stdout:0/349: stat dd/d1a 0 2024-05-09T06:38:07.813 INFO:tasks.workunit.client.0.smithi063.stdout:0/350: unlink dd/d1a/f4b 0 2024-05-09T06:38:07.818 INFO:tasks.workunit.client.0.smithi063.stdout:0/351: mknod dd/d1a/d2c/d51/d72/c77 0 2024-05-09T06:38:07.823 INFO:tasks.workunit.client.0.smithi063.stdout:0/352: dread dd/d60/d63/f55 [0,4194304] 0 2024-05-09T06:38:07.826 INFO:tasks.workunit.client.0.smithi063.stdout:8/380: dread d9/da/fb [0,4194304] 0 2024-05-09T06:38:07.828 INFO:tasks.workunit.client.0.smithi063.stdout:8/381: write d9/da/d18/d24/f3e [876726,114980] 0 2024-05-09T06:38:07.834 INFO:tasks.workunit.client.0.smithi063.stdout:0/353: dwrite dd/d60/f52 [0,4194304] 0 2024-05-09T06:38:07.835 INFO:tasks.workunit.client.0.smithi063.stdout:8/382: write d9/da/d18/d46/f50 [4863493,35552] 0 2024-05-09T06:38:07.837 INFO:tasks.workunit.client.0.smithi063.stdout:8/383: fdatasync d9/da/d34/f59 0 2024-05-09T06:38:07.840 INFO:tasks.workunit.client.0.smithi063.stdout:9/344: dread d0/f24 [4194304,4194304] 0 2024-05-09T06:38:07.840 INFO:tasks.workunit.client.0.smithi063.stdout:8/384: read d9/da/d18/d24/f3e [976617,22750] 0 2024-05-09T06:38:07.840 INFO:tasks.workunit.client.0.smithi063.stdout:2/360: sync 2024-05-09T06:38:07.840 INFO:tasks.workunit.client.0.smithi063.stdout:5/270: sync 2024-05-09T06:38:07.847 INFO:tasks.workunit.client.0.smithi063.stdout:8/385: creat d9/da/d18/d24/f84 x:0 0 0 2024-05-09T06:38:07.850 INFO:tasks.workunit.client.0.smithi063.stdout:0/354: dwrite dd/f12 [4194304,4194304] 0 2024-05-09T06:38:07.851 INFO:tasks.workunit.client.0.smithi063.stdout:5/271: dwrite d8/d17/d18/f29 [0,4194304] 0 2024-05-09T06:38:07.856 INFO:tasks.workunit.client.0.smithi063.stdout:5/272: creat d8/d19/d30/d46/f5d x:0 0 0 2024-05-09T06:38:07.861 INFO:tasks.workunit.client.0.smithi063.stdout:0/355: dread - dd/d60/d44/f61 zero size 2024-05-09T06:38:07.861 INFO:tasks.workunit.client.0.smithi063.stdout:8/386: truncate d9/f31 167169 0 2024-05-09T06:38:07.861 INFO:tasks.workunit.client.0.smithi063.stdout:5/273: mknod d8/d35/c5e 0 2024-05-09T06:38:07.865 INFO:tasks.workunit.client.0.smithi063.stdout:9/345: dwrite d0/d8/d2d/f41 [0,4194304] 0 2024-05-09T06:38:07.865 INFO:tasks.workunit.client.0.smithi063.stdout:8/387: creat d9/d3b/f85 x:0 0 0 2024-05-09T06:38:07.867 INFO:tasks.workunit.client.0.smithi063.stdout:5/274: chown d8/d19/d30/d46/f59 232 1 2024-05-09T06:38:07.869 INFO:tasks.workunit.client.0.smithi063.stdout:8/388: symlink d9/d3b/l86 0 2024-05-09T06:38:07.869 INFO:tasks.workunit.client.0.smithi063.stdout:9/346: write d0/d43/f4b [1379406,125047] 0 2024-05-09T06:38:07.874 INFO:tasks.workunit.client.0.smithi063.stdout:5/275: creat d8/d17/d1e/f5f x:0 0 0 2024-05-09T06:38:07.875 INFO:tasks.workunit.client.0.smithi063.stdout:8/389: read d9/da/fb [3716808,62550] 0 2024-05-09T06:38:07.879 INFO:tasks.workunit.client.0.smithi063.stdout:8/390: fdatasync d9/da/d18/d24/f42 0 2024-05-09T06:38:07.879 INFO:tasks.workunit.client.0.smithi063.stdout:5/276: rmdir d8 39 2024-05-09T06:38:07.880 INFO:tasks.workunit.client.0.smithi063.stdout:0/356: dwrite dd/d1a/f65 [0,4194304] 0 2024-05-09T06:38:07.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/277: stat d8/d19/d30/d46/f59 0 2024-05-09T06:38:07.883 INFO:tasks.workunit.client.0.smithi063.stdout:8/391: rmdir d9/da/d18 39 2024-05-09T06:38:07.884 INFO:tasks.workunit.client.0.smithi063.stdout:0/357: stat dd/d1a/d2c/d42 0 2024-05-09T06:38:07.885 INFO:tasks.workunit.client.0.smithi063.stdout:9/347: dread d0/d8/f69 [0,4194304] 0 2024-05-09T06:38:07.886 INFO:tasks.workunit.client.0.smithi063.stdout:8/392: dread - d9/da/d18/d56/f78 zero size 2024-05-09T06:38:07.887 INFO:tasks.workunit.client.0.smithi063.stdout:5/278: symlink d8/d53/l60 0 2024-05-09T06:38:07.890 INFO:tasks.workunit.client.0.smithi063.stdout:9/348: symlink d0/d4/d26/d39/d48/l75 0 2024-05-09T06:38:07.892 INFO:tasks.workunit.client.0.smithi063.stdout:0/358: unlink dd/d60/d63/d23/d46/c54 0 2024-05-09T06:38:07.893 INFO:tasks.workunit.client.0.smithi063.stdout:5/279: creat d8/d19/d30/f61 x:0 0 0 2024-05-09T06:38:07.894 INFO:tasks.workunit.client.0.smithi063.stdout:9/349: mkdir d0/d4/d57/d60/d76 0 2024-05-09T06:38:07.896 INFO:tasks.workunit.client.0.smithi063.stdout:8/393: dwrite d9/d3b/f68 [0,4194304] 0 2024-05-09T06:38:07.897 INFO:tasks.workunit.client.0.smithi063.stdout:9/350: creat d0/d43/f77 x:0 0 0 2024-05-09T06:38:07.900 INFO:tasks.workunit.client.0.smithi063.stdout:9/351: write d0/d43/f77 [841056,32852] 0 2024-05-09T06:38:07.904 INFO:tasks.workunit.client.0.smithi063.stdout:8/394: creat d9/da/d34/f87 x:0 0 0 2024-05-09T06:38:07.905 INFO:tasks.workunit.client.0.smithi063.stdout:0/359: dwrite dd/d60/d63/d23/d31/d58/f6c [0,4194304] 0 2024-05-09T06:38:07.906 INFO:tasks.workunit.client.0.smithi063.stdout:9/352: creat d0/d3b/f78 x:0 0 0 2024-05-09T06:38:07.907 INFO:tasks.workunit.client.0.smithi063.stdout:9/353: read - d0/d3b/f56 zero size 2024-05-09T06:38:07.908 INFO:tasks.workunit.client.0.smithi063.stdout:9/354: chown d0/d4/d57/c58 164584 1 2024-05-09T06:38:07.910 INFO:tasks.workunit.client.0.smithi063.stdout:8/395: creat d9/da/d51/f88 x:0 0 0 2024-05-09T06:38:07.910 INFO:tasks.workunit.client.0.smithi063.stdout:9/355: write d0/d4/f6a [293531,125674] 0 2024-05-09T06:38:07.914 INFO:tasks.workunit.client.0.smithi063.stdout:9/356: read - d0/f65 zero size 2024-05-09T06:38:07.914 INFO:tasks.workunit.client.0.smithi063.stdout:0/360: rename f5 to dd/d1a/f78 0 2024-05-09T06:38:07.917 INFO:tasks.workunit.client.0.smithi063.stdout:9/357: read d0/f1f [190383,119676] 0 2024-05-09T06:38:07.918 INFO:tasks.workunit.client.0.smithi063.stdout:5/280: dwrite d8/d17/f24 [0,4194304] 0 2024-05-09T06:38:07.920 INFO:tasks.workunit.client.0.smithi063.stdout:0/361: truncate dd/d60/d63/f64 672084 0 2024-05-09T06:38:07.920 INFO:tasks.workunit.client.0.smithi063.stdout:8/396: getdents d9/da 0 2024-05-09T06:38:07.923 INFO:tasks.workunit.client.0.smithi063.stdout:5/281: truncate d8/d17/d18/f51 56610 0 2024-05-09T06:38:07.926 INFO:tasks.workunit.client.0.smithi063.stdout:8/397: rename d9/da/d51/f69 to d9/d37/d73/f89 0 2024-05-09T06:38:07.928 INFO:tasks.workunit.client.0.smithi063.stdout:5/282: unlink d8/d17/d18/f2f 0 2024-05-09T06:38:07.933 INFO:tasks.workunit.client.0.smithi063.stdout:5/283: creat d8/d19/d30/d46/f62 x:0 0 0 2024-05-09T06:38:07.933 INFO:tasks.workunit.client.0.smithi063.stdout:9/358: link d0/d4/da/l20 d0/d4/d57/d60/l79 0 2024-05-09T06:38:07.936 INFO:tasks.workunit.client.0.smithi063.stdout:5/284: fsync d8/d17/d18/f58 0 2024-05-09T06:38:07.941 INFO:tasks.workunit.client.0.smithi063.stdout:5/285: mknod d8/d53/c63 0 2024-05-09T06:38:07.943 INFO:tasks.workunit.client.0.smithi063.stdout:0/362: dwrite dd/f1d [0,4194304] 0 2024-05-09T06:38:07.944 INFO:tasks.workunit.client.0.smithi063.stdout:8/398: dwrite d9/f7d [0,4194304] 0 2024-05-09T06:38:07.946 INFO:tasks.workunit.client.0.smithi063.stdout:8/399: stat d9/da 0 2024-05-09T06:38:07.951 INFO:tasks.workunit.client.0.smithi063.stdout:8/400: write d9/da/d18/f63 [1247126,36980] 0 2024-05-09T06:38:07.951 INFO:tasks.workunit.client.0.smithi063.stdout:9/359: dwrite d0/d3b/f6e [0,4194304] 0 2024-05-09T06:38:07.953 INFO:tasks.workunit.client.0.smithi063.stdout:8/401: write d9/d37/f4d [746745,100980] 0 2024-05-09T06:38:07.955 INFO:tasks.workunit.client.0.smithi063.stdout:0/363: chown dd/d60/d63/d23/d31/c4c 25138 1 2024-05-09T06:38:07.957 INFO:tasks.workunit.client.0.smithi063.stdout:9/360: creat d0/d4/d26/d47/f7a x:0 0 0 2024-05-09T06:38:07.957 INFO:tasks.workunit.client.0.smithi063.stdout:1/305: sync 2024-05-09T06:38:07.957 INFO:tasks.workunit.client.0.smithi063.stdout:4/344: sync 2024-05-09T06:38:07.958 INFO:tasks.workunit.client.0.smithi063.stdout:7/342: sync 2024-05-09T06:38:07.958 INFO:tasks.workunit.client.0.smithi063.stdout:3/317: sync 2024-05-09T06:38:07.958 INFO:tasks.workunit.client.0.smithi063.stdout:6/334: sync 2024-05-09T06:38:07.958 INFO:tasks.workunit.client.0.smithi063.stdout:9/361: write d0/d4/d26/f30 [1931727,42905] 0 2024-05-09T06:38:07.958 INFO:tasks.workunit.client.0.smithi063.stdout:8/402: symlink d9/da/d18/d56/d58/d79/l8a 0 2024-05-09T06:38:07.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/286: getdents d8/d53 0 2024-05-09T06:38:07.962 INFO:tasks.workunit.client.0.smithi063.stdout:7/343: chown d0/f1c 776788 1 2024-05-09T06:38:07.962 INFO:tasks.workunit.client.0.smithi063.stdout:1/306: rmdir d9/d2a/d2e 39 2024-05-09T06:38:07.963 INFO:tasks.workunit.client.0.smithi063.stdout:5/287: stat d8/d19/f3f 0 2024-05-09T06:38:07.963 INFO:tasks.workunit.client.0.smithi063.stdout:4/345: fdatasync d7/f2d 0 2024-05-09T06:38:07.964 INFO:tasks.workunit.client.0.smithi063.stdout:6/335: chown d2/d4/d2d/d42/l5f 11595 1 2024-05-09T06:38:07.969 INFO:tasks.workunit.client.0.smithi063.stdout:8/403: link d9/d3b/f2c d9/da/d51/f8b 0 2024-05-09T06:38:07.970 INFO:tasks.workunit.client.0.smithi063.stdout:4/346: creat d7/df/d35/d5a/d60/f6e x:0 0 0 2024-05-09T06:38:07.970 INFO:tasks.workunit.client.0.smithi063.stdout:0/364: write dd/d60/f4f [1714123,116198] 0 2024-05-09T06:38:07.970 INFO:tasks.workunit.client.0.smithi063.stdout:8/404: chown d9/da/d18/d24/d71 2005779 1 2024-05-09T06:38:07.971 INFO:tasks.workunit.client.0.smithi063.stdout:1/307: creat d9/d2a/d2e/f66 x:0 0 0 2024-05-09T06:38:07.974 INFO:tasks.workunit.client.0.smithi063.stdout:3/318: creat d7/da/d21/d30/d44/d48/f6f x:0 0 0 2024-05-09T06:38:07.975 INFO:tasks.workunit.client.0.smithi063.stdout:4/347: readlink d7/df/l29 0 2024-05-09T06:38:07.975 INFO:tasks.workunit.client.0.smithi063.stdout:5/288: dwrite d8/d17/d18/f4a [0,4194304] 0 2024-05-09T06:38:07.976 INFO:tasks.workunit.client.0.smithi063.stdout:3/319: dread - d7/d4c/f69 zero size 2024-05-09T06:38:07.979 INFO:tasks.workunit.client.0.smithi063.stdout:0/365: mkdir dd/d1a/d2c/d51/d79 0 2024-05-09T06:38:07.980 INFO:tasks.workunit.client.0.smithi063.stdout:6/336: truncate d2/d4/f5 468389 0 2024-05-09T06:38:07.983 INFO:tasks.workunit.client.0.smithi063.stdout:4/348: creat d7/df/d35/d5a/f6f x:0 0 0 2024-05-09T06:38:07.983 INFO:tasks.workunit.client.0.smithi063.stdout:0/366: dread - dd/d60/d63/d23/d31/f5e zero size 2024-05-09T06:38:07.985 INFO:tasks.workunit.client.0.smithi063.stdout:1/308: write d9/f12 [5364737,84533] 0 2024-05-09T06:38:07.985 INFO:tasks.workunit.client.0.smithi063.stdout:8/405: dread d9/da/d34/f6e [0,4194304] 0 2024-05-09T06:38:07.986 INFO:tasks.workunit.client.0.smithi063.stdout:3/320: dread d7/da/d21/f2c [0,4194304] 0 2024-05-09T06:38:07.987 INFO:tasks.workunit.client.0.smithi063.stdout:0/367: stat dd/d60/d44/f69 0 2024-05-09T06:38:07.988 INFO:tasks.workunit.client.0.smithi063.stdout:1/309: dread - d9/d10/d2b/f51 zero size 2024-05-09T06:38:07.991 INFO:tasks.workunit.client.0.smithi063.stdout:3/321: dread - d7/d2a/d2d/f65 zero size 2024-05-09T06:38:07.991 INFO:tasks.workunit.client.0.smithi063.stdout:7/344: dread d0/d15/f2b [0,4194304] 0 2024-05-09T06:38:07.992 INFO:tasks.workunit.client.0.smithi063.stdout:0/368: truncate dd/f6b 1045714 0 2024-05-09T06:38:07.992 INFO:tasks.workunit.client.0.smithi063.stdout:8/406: fdatasync d9/da/d18/d46/f52 0 2024-05-09T06:38:07.994 INFO:tasks.workunit.client.0.smithi063.stdout:1/310: dread - d9/d2a/d35/d1e/f42 zero size 2024-05-09T06:38:07.995 INFO:tasks.workunit.client.0.smithi063.stdout:6/337: dwrite d2/df/f28 [0,4194304] 0 2024-05-09T06:38:07.996 INFO:tasks.workunit.client.0.smithi063.stdout:8/407: chown d9/da/d18/d56/l7e 516518 1 2024-05-09T06:38:08.002 INFO:tasks.workunit.client.0.smithi063.stdout:7/345: dwrite d0/d15/d49/f5f [0,4194304] 0 2024-05-09T06:38:08.031 INFO:tasks.workunit.client.0.smithi063.stdout:2/361: sync 2024-05-09T06:38:08.046 INFO:tasks.workunit.client.0.smithi063.stdout:4/349: mkdir d7/df/d35/d5a/d60/d70 0 2024-05-09T06:38:08.048 INFO:tasks.workunit.client.0.smithi063.stdout:3/322: chown l6 13502 1 2024-05-09T06:38:08.052 INFO:tasks.workunit.client.0.smithi063.stdout:0/369: creat dd/d1a/d50/f7a x:0 0 0 2024-05-09T06:38:08.052 INFO:tasks.workunit.client.0.smithi063.stdout:0/370: fdatasync dd/f1d 0 2024-05-09T06:38:08.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/408: symlink d9/d41/l8c 0 2024-05-09T06:38:08.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/409: read - d9/d37/f83 zero size 2024-05-09T06:38:08.059 INFO:tasks.workunit.client.0.smithi063.stdout:8/410: dread d9/da/fb [4194304,4194304] 0 2024-05-09T06:38:08.063 INFO:tasks.workunit.client.0.smithi063.stdout:4/350: mkdir d7/df/d1f/d28/d31/d71 0 2024-05-09T06:38:08.068 INFO:tasks.workunit.client.0.smithi063.stdout:0/371: readlink dd/d60/d63/d23/d33/l38 0 2024-05-09T06:38:08.075 INFO:tasks.workunit.client.0.smithi063.stdout:0/372: dwrite dd/d1a/f1b [0,4194304] 0 2024-05-09T06:38:08.081 INFO:tasks.workunit.client.0.smithi063.stdout:5/289: getdents d8/d17/d1e 0 2024-05-09T06:38:08.082 INFO:tasks.workunit.client.0.smithi063.stdout:5/290: truncate d8/f3a 900399 0 2024-05-09T06:38:08.083 INFO:tasks.workunit.client.0.smithi063.stdout:1/311: getdents d9/d2a/d39 0 2024-05-09T06:38:08.085 INFO:tasks.workunit.client.0.smithi063.stdout:6/338: getdents d2/d4/d2d/d42 0 2024-05-09T06:38:08.089 INFO:tasks.workunit.client.0.smithi063.stdout:5/291: creat d8/d35/f64 x:0 0 0 2024-05-09T06:38:08.090 INFO:tasks.workunit.client.0.smithi063.stdout:0/373: dwrite dd/d60/d63/d23/d31/d58/f6e [0,4194304] 0 2024-05-09T06:38:08.091 INFO:tasks.workunit.client.0.smithi063.stdout:0/374: write dd/f6b [2060023,74589] 0 2024-05-09T06:38:08.095 INFO:tasks.workunit.client.0.smithi063.stdout:6/339: creat d2/d1f/d29/d3f/d5c/f70 x:0 0 0 2024-05-09T06:38:08.096 INFO:tasks.workunit.client.0.smithi063.stdout:1/312: creat d9/d2a/d3c/d4a/f67 x:0 0 0 2024-05-09T06:38:08.099 INFO:tasks.workunit.client.0.smithi063.stdout:0/375: fsync dd/d1a/f35 0 2024-05-09T06:38:08.101 INFO:tasks.workunit.client.0.smithi063.stdout:0/376: readlink dd/d60/d63/d23/d31/l37 0 2024-05-09T06:38:08.102 INFO:tasks.workunit.client.0.smithi063.stdout:1/313: mkdir d9/d2a/d35/d24/d68 0 2024-05-09T06:38:08.103 INFO:tasks.workunit.client.0.smithi063.stdout:8/411: dread d9/f36 [0,4194304] 0 2024-05-09T06:38:08.104 INFO:tasks.workunit.client.0.smithi063.stdout:6/340: creat d2/d1f/d6c/f71 x:0 0 0 2024-05-09T06:38:08.105 INFO:tasks.workunit.client.0.smithi063.stdout:6/341: chown d2/d1f/d29/d3f/f58 23576 1 2024-05-09T06:38:08.108 INFO:tasks.workunit.client.0.smithi063.stdout:8/412: creat d9/da/d45/f8d x:0 0 0 2024-05-09T06:38:08.109 INFO:tasks.workunit.client.0.smithi063.stdout:8/413: truncate d9/f77 446488 0 2024-05-09T06:38:08.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/414: fsync d9/f77 0 2024-05-09T06:38:08.111 INFO:tasks.workunit.client.0.smithi063.stdout:6/342: fdatasync d2/d12/f24 0 2024-05-09T06:38:08.112 INFO:tasks.workunit.client.0.smithi063.stdout:1/314: dwrite d9/d10/d2b/f33 [0,4194304] 0 2024-05-09T06:38:08.113 INFO:tasks.workunit.client.0.smithi063.stdout:1/315: chown d9/d10 7772703 1 2024-05-09T06:38:08.114 INFO:tasks.workunit.client.0.smithi063.stdout:6/343: fdatasync d2/d1f/d29/d3f/d5c/f70 0 2024-05-09T06:38:08.115 INFO:tasks.workunit.client.0.smithi063.stdout:3/323: mkdir d7/d70 0 2024-05-09T06:38:08.115 INFO:tasks.workunit.client.0.smithi063.stdout:1/316: write d9/d2a/d35/d1e/f25 [2714256,69473] 0 2024-05-09T06:38:08.116 INFO:tasks.workunit.client.0.smithi063.stdout:6/344: read - d2/d1f/d29/d3f/d5c/f70 zero size 2024-05-09T06:38:08.117 INFO:tasks.workunit.client.0.smithi063.stdout:6/345: stat d2/d1f/d29/d3b/d4e/f67 0 2024-05-09T06:38:08.117 INFO:tasks.workunit.client.0.smithi063.stdout:3/324: rmdir d7/da/d21 39 2024-05-09T06:38:08.120 INFO:tasks.workunit.client.0.smithi063.stdout:1/317: symlink d9/d2a/d39/l69 0 2024-05-09T06:38:08.123 INFO:tasks.workunit.client.0.smithi063.stdout:9/362: sync 2024-05-09T06:38:08.127 INFO:tasks.workunit.client.0.smithi063.stdout:1/318: mknod d9/d10/d2b/d3f/c6a 0 2024-05-09T06:38:08.131 INFO:tasks.workunit.client.0.smithi063.stdout:7/346: unlink d0/f2f 0 2024-05-09T06:38:08.133 INFO:tasks.workunit.client.0.smithi063.stdout:1/319: symlink d9/d10/d2b/l6b 0 2024-05-09T06:38:08.139 INFO:tasks.workunit.client.0.smithi063.stdout:1/320: creat d9/d10/d2b/d3f/d59/d62/f6c x:0 0 0 2024-05-09T06:38:08.139 INFO:tasks.workunit.client.0.smithi063.stdout:6/346: dread d2/df/f5d [0,4194304] 0 2024-05-09T06:38:08.140 INFO:tasks.workunit.client.0.smithi063.stdout:3/325: dread d7/f17 [0,4194304] 0 2024-05-09T06:38:08.142 INFO:tasks.workunit.client.0.smithi063.stdout:3/326: read - d7/d4c/d31/f6e zero size 2024-05-09T06:38:08.148 INFO:tasks.workunit.client.0.smithi063.stdout:6/347: link d2/d20/c30 d2/d1f/d29/d3b/d4e/c72 0 2024-05-09T06:38:08.149 INFO:tasks.workunit.client.0.smithi063.stdout:6/348: mkdir d2/d1f/d73 0 2024-05-09T06:38:08.150 INFO:tasks.workunit.client.0.smithi063.stdout:6/349: fsync d2/d1f/d29/d3f/f25 0 2024-05-09T06:38:08.151 INFO:tasks.workunit.client.0.smithi063.stdout:6/350: write d2/d1f/d29/d3f/d5c/f70 [317828,105965] 0 2024-05-09T06:38:08.153 INFO:tasks.workunit.client.0.smithi063.stdout:7/347: dread d0/ff [4194304,4194304] 0 2024-05-09T06:38:08.156 INFO:tasks.workunit.client.0.smithi063.stdout:7/348: rmdir d0/d15 39 2024-05-09T06:38:08.158 INFO:tasks.workunit.client.0.smithi063.stdout:5/292: rmdir d8/d35 39 2024-05-09T06:38:08.160 INFO:tasks.workunit.client.0.smithi063.stdout:5/293: unlink d8/d17/d1e/c34 0 2024-05-09T06:38:08.161 INFO:tasks.workunit.client.0.smithi063.stdout:9/363: unlink d0/f28 0 2024-05-09T06:38:08.163 INFO:tasks.workunit.client.0.smithi063.stdout:7/349: dwrite d0/d15/d49/f65 [0,4194304] 0 2024-05-09T06:38:08.165 INFO:tasks.workunit.client.0.smithi063.stdout:5/294: dread d8/d17/d18/f2d [0,4194304] 0 2024-05-09T06:38:08.165 INFO:tasks.workunit.client.0.smithi063.stdout:5/295: stat d8/d17/d1e 0 2024-05-09T06:38:08.168 INFO:tasks.workunit.client.0.smithi063.stdout:5/296: fsync d8/d17/d18/f2d 0 2024-05-09T06:38:08.170 INFO:tasks.workunit.client.0.smithi063.stdout:0/377: truncate dd/d1a/f2e 280177 0 2024-05-09T06:38:08.174 INFO:tasks.workunit.client.0.smithi063.stdout:2/362: rename dc/f12 to dc/dd/d2d/d60/f67 0 2024-05-09T06:38:08.176 INFO:tasks.workunit.client.0.smithi063.stdout:0/378: unlink dd/d1a/d2c/d42/f4d 0 2024-05-09T06:38:08.180 INFO:tasks.workunit.client.0.smithi063.stdout:8/415: rename d9/da/d34/f6e to d9/da/d18/d56/d58/f8e 0 2024-05-09T06:38:08.182 INFO:tasks.workunit.client.0.smithi063.stdout:5/297: getdents d8 0 2024-05-09T06:38:08.183 INFO:tasks.workunit.client.0.smithi063.stdout:2/363: mknod dc/dd/c68 0 2024-05-09T06:38:08.187 INFO:tasks.workunit.client.0.smithi063.stdout:0/379: creat dd/d60/d63/d23/d33/d48/d75/f7b x:0 0 0 2024-05-09T06:38:08.189 INFO:tasks.workunit.client.0.smithi063.stdout:6/351: rename d2/d12/c5a to d2/d1f/d29/c74 0 2024-05-09T06:38:08.189 INFO:tasks.workunit.client.0.smithi063.stdout:2/364: dwrite dc/d32/d33/f41 [0,4194304] 0 2024-05-09T06:38:08.190 INFO:tasks.workunit.client.0.smithi063.stdout:8/416: fsync d9/f31 0 2024-05-09T06:38:08.190 INFO:tasks.workunit.client.0.smithi063.stdout:2/365: fsync dc/f1f 0 2024-05-09T06:38:08.191 INFO:tasks.workunit.client.0.smithi063.stdout:6/352: write d2/d1f/d6c/f71 [898899,35254] 0 2024-05-09T06:38:08.194 INFO:tasks.workunit.client.0.smithi063.stdout:0/380: creat dd/d1a/d50/f7c x:0 0 0 2024-05-09T06:38:08.196 INFO:tasks.workunit.client.0.smithi063.stdout:9/364: rename d0/d4/d26/d39/f6f to d0/d8/d2d/f7b 0 2024-05-09T06:38:08.196 INFO:tasks.workunit.client.0.smithi063.stdout:1/321: write f3 [2901594,109010] 0 2024-05-09T06:38:08.198 INFO:tasks.workunit.client.0.smithi063.stdout:3/327: write d7/d2a/d2d/f5d [723443,96520] 0 2024-05-09T06:38:08.201 INFO:tasks.workunit.client.0.smithi063.stdout:8/417: rename d9/d37/d73 to d9/da/d18/d56/d8f 0 2024-05-09T06:38:08.202 INFO:tasks.workunit.client.0.smithi063.stdout:9/365: creat d0/d4/d57/d60/f7c x:0 0 0 2024-05-09T06:38:08.203 INFO:tasks.workunit.client.0.smithi063.stdout:3/328: stat f3 0 2024-05-09T06:38:08.205 INFO:tasks.workunit.client.0.smithi063.stdout:8/418: creat d9/da/d18/d56/d8f/f90 x:0 0 0 2024-05-09T06:38:08.208 INFO:tasks.workunit.client.0.smithi063.stdout:8/419: dread - d9/da/d18/d56/f76 zero size 2024-05-09T06:38:08.208 INFO:tasks.workunit.client.0.smithi063.stdout:9/366: mkdir d0/d4/d7d 0 2024-05-09T06:38:08.210 INFO:tasks.workunit.client.0.smithi063.stdout:3/329: dread f1 [0,4194304] 0 2024-05-09T06:38:08.211 INFO:tasks.workunit.client.0.smithi063.stdout:9/367: readlink d0/l25 0 2024-05-09T06:38:08.212 INFO:tasks.workunit.client.0.smithi063.stdout:7/350: dwrite d0/f16 [0,4194304] 0 2024-05-09T06:38:08.216 INFO:tasks.workunit.client.0.smithi063.stdout:2/366: dread dc/dd/d2d/f34 [0,4194304] 0 2024-05-09T06:38:08.219 INFO:tasks.workunit.client.0.smithi063.stdout:6/353: rename d2/d1f/d29/c74 to d2/c75 0 2024-05-09T06:38:08.220 INFO:tasks.workunit.client.0.smithi063.stdout:2/367: write dc/f1f [129121,110590] 0 2024-05-09T06:38:08.224 INFO:tasks.workunit.client.0.smithi063.stdout:0/381: rename dd/c17 to dd/d60/d63/d23/d33/c7d 0 2024-05-09T06:38:08.225 INFO:tasks.workunit.client.0.smithi063.stdout:5/298: dwrite d8/f16 [0,4194304] 0 2024-05-09T06:38:08.228 INFO:tasks.workunit.client.0.smithi063.stdout:1/322: rename d9/d2a/d35/d24/l4b to d9/d2a/d35/d1e/l6d 0 2024-05-09T06:38:08.230 INFO:tasks.workunit.client.0.smithi063.stdout:5/299: mkdir d8/d65 0 2024-05-09T06:38:08.230 INFO:tasks.workunit.client.0.smithi063.stdout:0/382: write dd/d60/d63/d23/d31/d58/f6e [5124975,121733] 0 2024-05-09T06:38:08.234 INFO:tasks.workunit.client.0.smithi063.stdout:2/368: dwrite dc/dd/d45/d5a/f5e [0,4194304] 0 2024-05-09T06:38:08.235 INFO:tasks.workunit.client.0.smithi063.stdout:1/323: creat d9/d46/f6e x:0 0 0 2024-05-09T06:38:08.236 INFO:tasks.workunit.client.0.smithi063.stdout:2/369: readlink dc/dd/l28 0 2024-05-09T06:38:08.237 INFO:tasks.workunit.client.0.smithi063.stdout:0/383: stat dd/c13 0 2024-05-09T06:38:08.237 INFO:tasks.workunit.client.0.smithi063.stdout:1/324: fdatasync d9/d2a/d2e/f50 0 2024-05-09T06:38:08.237 INFO:tasks.workunit.client.0.smithi063.stdout:8/420: dread d9/da/d18/d24/f27 [0,4194304] 0 2024-05-09T06:38:08.238 INFO:tasks.workunit.client.0.smithi063.stdout:1/325: readlink d9/l13 0 2024-05-09T06:38:08.239 INFO:tasks.workunit.client.0.smithi063.stdout:8/421: dread - d9/d37/f83 zero size 2024-05-09T06:38:08.239 INFO:tasks.workunit.client.0.smithi063.stdout:0/384: write dd/d1a/f1b [981502,12617] 0 2024-05-09T06:38:08.244 INFO:tasks.workunit.client.0.smithi063.stdout:3/330: dread d7/da/fb [0,4194304] 0 2024-05-09T06:38:08.247 INFO:tasks.workunit.client.0.smithi063.stdout:8/422: dwrite d9/da/d34/f59 [0,4194304] 0 2024-05-09T06:38:08.277 INFO:tasks.workunit.client.0.smithi063.stdout:2/370: fsync dc/dd/d2d/f34 0 2024-05-09T06:38:08.278 INFO:tasks.workunit.client.0.smithi063.stdout:1/326: creat d9/d46/f6f x:0 0 0 2024-05-09T06:38:08.281 INFO:tasks.workunit.client.0.smithi063.stdout:8/423: creat d9/da/d18/d56/f91 x:0 0 0 2024-05-09T06:38:08.282 INFO:tasks.workunit.client.0.smithi063.stdout:2/371: symlink dc/dd/d2d/d60/l69 0 2024-05-09T06:38:08.282 INFO:tasks.workunit.client.0.smithi063.stdout:0/385: mknod dd/d60/d63/d23/c7e 0 2024-05-09T06:38:08.284 INFO:tasks.workunit.client.0.smithi063.stdout:5/300: dread f6 [0,4194304] 0 2024-05-09T06:38:08.285 INFO:tasks.workunit.client.0.smithi063.stdout:3/331: chown d7/da/d21/d30/d44/d48/f6f 22 1 2024-05-09T06:38:08.286 INFO:tasks.workunit.client.0.smithi063.stdout:0/386: stat dd/d1a/d2c/d42 0 2024-05-09T06:38:08.286 INFO:tasks.workunit.client.0.smithi063.stdout:8/424: write d9/da/d18/f63 [983647,31281] 0 2024-05-09T06:38:08.289 INFO:tasks.workunit.client.0.smithi063.stdout:1/327: dwrite d9/d2a/d35/d1e/f42 [0,4194304] 0 2024-05-09T06:38:08.289 INFO:tasks.workunit.client.0.smithi063.stdout:5/301: fsync d8/d17/d18/f1f 0 2024-05-09T06:38:08.293 INFO:tasks.workunit.client.0.smithi063.stdout:8/425: truncate d9/da/d18/f22 188222 0 2024-05-09T06:38:08.295 INFO:tasks.workunit.client.0.smithi063.stdout:5/302: creat d8/f66 x:0 0 0 2024-05-09T06:38:08.297 INFO:tasks.workunit.client.0.smithi063.stdout:3/332: dwrite d7/f1b [0,4194304] 0 2024-05-09T06:38:08.298 INFO:tasks.workunit.client.0.smithi063.stdout:8/426: write d9/da/d34/f59 [1532881,33444] 0 2024-05-09T06:38:08.303 INFO:tasks.workunit.client.0.smithi063.stdout:3/333: fdatasync d7/d22/d28/f3c 0 2024-05-09T06:38:08.304 INFO:tasks.workunit.client.0.smithi063.stdout:8/427: stat d9/d3b/f85 0 2024-05-09T06:38:08.306 INFO:tasks.workunit.client.0.smithi063.stdout:0/387: dwrite dd/d60/d63/d23/d31/d58/f6e [0,4194304] 0 2024-05-09T06:38:08.310 INFO:tasks.workunit.client.0.smithi063.stdout:2/372: dwrite dc/f27 [4194304,4194304] 0 2024-05-09T06:38:08.314 INFO:tasks.workunit.client.0.smithi063.stdout:8/428: readlink d9/da/d45/l6f 0 2024-05-09T06:38:08.318 INFO:tasks.workunit.client.0.smithi063.stdout:5/303: dwrite d8/f4b [0,4194304] 0 2024-05-09T06:38:08.321 INFO:tasks.workunit.client.0.smithi063.stdout:1/328: dwrite d9/d2a/d35/d1e/f27 [0,4194304] 0 2024-05-09T06:38:08.322 INFO:tasks.workunit.client.0.smithi063.stdout:8/429: creat d9/da/d18/d24/f92 x:0 0 0 2024-05-09T06:38:08.327 INFO:tasks.workunit.client.0.smithi063.stdout:3/334: dwrite d7/f10 [0,4194304] 0 2024-05-09T06:38:08.328 INFO:tasks.workunit.client.0.smithi063.stdout:1/329: read d9/d10/f3e [2079849,76349] 0 2024-05-09T06:38:08.328 INFO:tasks.workunit.client.0.smithi063.stdout:0/388: dread dd/d60/d63/d23/d31/d58/f6e [0,4194304] 0 2024-05-09T06:38:08.329 INFO:tasks.workunit.client.0.smithi063.stdout:0/389: write dd/d60/f4f [1766989,4379] 0 2024-05-09T06:38:08.333 INFO:tasks.workunit.client.0.smithi063.stdout:3/335: symlink d7/d2a/d5b/l71 0 2024-05-09T06:38:08.335 INFO:tasks.workunit.client.0.smithi063.stdout:1/330: link d9/d10/d2b/d3f/d59/d62/f6c d9/d2a/d35/d1e/f70 0 2024-05-09T06:38:08.342 INFO:tasks.workunit.client.0.smithi063.stdout:1/331: dread d9/d10/f18 [0,4194304] 0 2024-05-09T06:38:08.343 INFO:tasks.workunit.client.0.smithi063.stdout:1/332: fsync d9/d2a/d3c/d4c/f56 0 2024-05-09T06:38:08.345 INFO:tasks.workunit.client.0.smithi063.stdout:4/351: sync 2024-05-09T06:38:08.345 INFO:tasks.workunit.client.0.smithi063.stdout:4/352: stat d7/f2d 0 2024-05-09T06:38:08.346 INFO:tasks.workunit.client.0.smithi063.stdout:8/430: truncate d9/f47 65409 0 2024-05-09T06:38:08.348 INFO:tasks.workunit.client.0.smithi063.stdout:4/353: mknod d7/df/d35/d5a/d60/c72 0 2024-05-09T06:38:08.348 INFO:tasks.workunit.client.0.smithi063.stdout:5/304: dread d8/d17/d18/f45 [0,4194304] 0 2024-05-09T06:38:08.350 INFO:tasks.workunit.client.0.smithi063.stdout:8/431: mknod d9/da/d51/c93 0 2024-05-09T06:38:08.352 INFO:tasks.workunit.client.0.smithi063.stdout:3/336: dread d7/da/f14 [0,4194304] 0 2024-05-09T06:38:08.352 INFO:tasks.workunit.client.0.smithi063.stdout:0/390: dread dd/f15 [0,4194304] 0 2024-05-09T06:38:08.352 INFO:tasks.workunit.client.0.smithi063.stdout:5/305: creat d8/d17/d1e/f67 x:0 0 0 2024-05-09T06:38:08.354 INFO:tasks.workunit.client.0.smithi063.stdout:8/432: write d9/da/d18/d46/f64 [639689,41521] 0 2024-05-09T06:38:08.356 INFO:tasks.workunit.client.0.smithi063.stdout:5/306: write d8/fc [3034582,61288] 0 2024-05-09T06:38:08.357 INFO:tasks.workunit.client.0.smithi063.stdout:0/391: symlink dd/d60/d63/d23/l7f 0 2024-05-09T06:38:08.359 INFO:tasks.workunit.client.0.smithi063.stdout:3/337: rename d7/d2a/f4b to d7/d4c/d25/f72 0 2024-05-09T06:38:08.361 INFO:tasks.workunit.client.0.smithi063.stdout:4/354: dwrite d7/df/d1f/f65 [0,4194304] 0 2024-05-09T06:38:08.363 INFO:tasks.workunit.client.0.smithi063.stdout:3/338: chown d7/d4c/d31/c61 114173 1 2024-05-09T06:38:08.364 INFO:tasks.workunit.client.0.smithi063.stdout:4/355: readlink d7/df/l37 0 2024-05-09T06:38:08.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/307: dwrite d8/f14 [0,4194304] 0 2024-05-09T06:38:08.367 INFO:tasks.workunit.client.0.smithi063.stdout:0/392: dread dd/d60/d63/d23/d31/d58/f6e [0,4194304] 0 2024-05-09T06:38:08.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/308: readlink d8/l31 0 2024-05-09T06:38:08.368 INFO:tasks.workunit.client.0.smithi063.stdout:3/339: rename d7/da/f13 to d7/d70/f73 0 2024-05-09T06:38:08.370 INFO:tasks.workunit.client.0.smithi063.stdout:5/309: truncate d8/d17/f56 734832 0 2024-05-09T06:38:08.371 INFO:tasks.workunit.client.0.smithi063.stdout:3/340: write d7/d22/d28/f3f [827849,5668] 0 2024-05-09T06:38:08.372 INFO:tasks.workunit.client.0.smithi063.stdout:4/356: rename d7/df/d35/d5a/d60/d70 to d7/d3d/d73 0 2024-05-09T06:38:08.374 INFO:tasks.workunit.client.0.smithi063.stdout:5/310: creat d8/f68 x:0 0 0 2024-05-09T06:38:08.378 INFO:tasks.workunit.client.0.smithi063.stdout:4/357: dwrite d7/f2d [0,4194304] 0 2024-05-09T06:38:08.379 INFO:tasks.workunit.client.0.smithi063.stdout:5/311: rename d8/fc to d8/d17/d18/f69 0 2024-05-09T06:38:08.382 INFO:tasks.workunit.client.0.smithi063.stdout:5/312: rename d8/d19/d30/d46/f59 to d8/d17/f6a 0 2024-05-09T06:38:08.383 INFO:tasks.workunit.client.0.smithi063.stdout:4/358: link d7/c25 d7/df/c74 0 2024-05-09T06:38:08.385 INFO:tasks.workunit.client.0.smithi063.stdout:5/313: mknod d8/d17/d18/c6b 0 2024-05-09T06:38:08.386 INFO:tasks.workunit.client.0.smithi063.stdout:4/359: mkdir d7/df/d35/d5a/d60/d75 0 2024-05-09T06:38:08.389 INFO:tasks.workunit.client.0.smithi063.stdout:4/360: mknod d7/df/d35/d5a/d60/c76 0 2024-05-09T06:38:08.390 INFO:tasks.workunit.client.0.smithi063.stdout:5/314: dread d8/d17/d18/f22 [0,4194304] 0 2024-05-09T06:38:08.391 INFO:tasks.workunit.client.0.smithi063.stdout:5/315: write d8/d19/f4f [246396,31772] 0 2024-05-09T06:38:08.392 INFO:tasks.workunit.client.0.smithi063.stdout:3/341: unlink d7/d70/f73 0 2024-05-09T06:38:08.393 INFO:tasks.workunit.client.0.smithi063.stdout:4/361: read d7/df/d1a/f22 [937757,4504] 0 2024-05-09T06:38:08.396 INFO:tasks.workunit.client.0.smithi063.stdout:0/393: truncate dd/d1a/f25 1412247 0 2024-05-09T06:38:08.396 INFO:tasks.workunit.client.0.smithi063.stdout:4/362: rmdir d7/d50 39 2024-05-09T06:38:08.397 INFO:tasks.workunit.client.0.smithi063.stdout:4/363: write d7/d3d/f55 [1411513,24752] 0 2024-05-09T06:38:08.398 INFO:tasks.workunit.client.0.smithi063.stdout:3/342: mkdir d7/da/d2b/d66/d74 0 2024-05-09T06:38:08.400 INFO:tasks.workunit.client.0.smithi063.stdout:0/394: unlink dd/d60/f4f 0 2024-05-09T06:38:08.403 INFO:tasks.workunit.client.0.smithi063.stdout:4/364: dwrite d7/f62 [0,4194304] 0 2024-05-09T06:38:08.406 INFO:tasks.workunit.client.0.smithi063.stdout:3/343: symlink d7/da/d21/d30/d44/l75 0 2024-05-09T06:38:08.408 INFO:tasks.workunit.client.0.smithi063.stdout:0/395: creat dd/d60/d63/f80 x:0 0 0 2024-05-09T06:38:08.409 INFO:tasks.workunit.client.0.smithi063.stdout:4/365: dwrite d7/df/d1a/f20 [0,4194304] 0 2024-05-09T06:38:08.411 INFO:tasks.workunit.client.0.smithi063.stdout:3/344: rename l6 to d7/d4c/d31/l76 0 2024-05-09T06:38:08.411 INFO:tasks.workunit.client.0.smithi063.stdout:4/366: write d7/df/d1a/f20 [3698708,70694] 0 2024-05-09T06:38:08.416 INFO:tasks.workunit.client.0.smithi063.stdout:5/316: dread d8/d19/d30/f43 [0,4194304] 0 2024-05-09T06:38:08.416 INFO:tasks.workunit.client.0.smithi063.stdout:4/367: read d7/f39 [2996482,36986] 0 2024-05-09T06:38:08.418 INFO:tasks.workunit.client.0.smithi063.stdout:5/317: write d8/d35/f41 [918843,72936] 0 2024-05-09T06:38:08.420 INFO:tasks.workunit.client.0.smithi063.stdout:0/396: link dd/d1a/l20 dd/d1a/d2c/d42/l81 0 2024-05-09T06:38:08.421 INFO:tasks.workunit.client.0.smithi063.stdout:3/345: dwrite d7/d22/f27 [0,4194304] 0 2024-05-09T06:38:08.421 INFO:tasks.workunit.client.0.smithi063.stdout:4/368: mkdir d7/df/d77 0 2024-05-09T06:38:08.429 INFO:tasks.workunit.client.0.smithi063.stdout:5/318: dwrite d8/d17/f3d [0,4194304] 0 2024-05-09T06:38:08.430 INFO:tasks.workunit.client.0.smithi063.stdout:4/369: write d7/df/d1f/f53 [134082,31284] 0 2024-05-09T06:38:08.435 INFO:tasks.workunit.client.0.smithi063.stdout:4/370: mkdir d7/df/d35/d78 0 2024-05-09T06:38:08.436 INFO:tasks.workunit.client.0.smithi063.stdout:5/319: creat d8/d19/d30/d46/f6c x:0 0 0 2024-05-09T06:38:08.437 INFO:tasks.workunit.client.0.smithi063.stdout:3/346: dwrite d7/d4c/d25/f72 [0,4194304] 0 2024-05-09T06:38:08.442 INFO:tasks.workunit.client.0.smithi063.stdout:4/371: dread d7/f62 [0,4194304] 0 2024-05-09T06:38:08.443 INFO:tasks.workunit.client.0.smithi063.stdout:5/320: unlink d8/d19/d30/l54 0 2024-05-09T06:38:08.443 INFO:tasks.workunit.client.0.smithi063.stdout:3/347: mkdir d7/da/d21/d30/d77 0 2024-05-09T06:38:08.444 INFO:tasks.workunit.client.0.smithi063.stdout:5/321: stat d8/d17/d18/f69 0 2024-05-09T06:38:08.445 INFO:tasks.workunit.client.0.smithi063.stdout:4/372: write d7/f62 [1198506,32542] 0 2024-05-09T06:38:08.446 INFO:tasks.workunit.client.0.smithi063.stdout:4/373: write f6 [696505,80888] 0 2024-05-09T06:38:08.446 INFO:tasks.workunit.client.0.smithi063.stdout:3/348: creat d7/d2a/d5b/f78 x:0 0 0 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/0: sync 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/1: dwrite - no filename 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/2: dread - no filename 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:2/5: sync 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/3: chown . 675328093 1 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/4: dread - no filename 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/5: rename - no filename 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/6: chown . 21 1 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/7: fdatasync - no filename 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/8: rename - no filename 2024-05-09T06:38:08.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/9: rmdir - no directory 2024-05-09T06:38:08.448 INFO:tasks.workunit.client.0.smithi063.stdout:5/322: creat d8/d17/f6d x:0 0 0 2024-05-09T06:38:08.448 INFO:tasks.workunit.client.0.smithi063.stdout:5/323: fsync d8/d17/d18/f29 0 2024-05-09T06:38:08.449 INFO:tasks.workunit.client.0.smithi063.stdout:5/324: write d8/d17/d1e/f42 [54695,100403] 0 2024-05-09T06:38:08.452 INFO:tasks.workunit.client.0.smithi063.stdout:0/397: dread dd/d60/d63/d23/d33/f3e [0,4194304] 0 2024-05-09T06:38:08.453 INFO:tasks.workunit.client.1.smithi193.stdout:1/10: creat f0 x:0 0 0 2024-05-09T06:38:08.453 INFO:tasks.workunit.client.0.smithi063.stdout:0/398: fdatasync dd/d60/d63/d23/d33/d48/f57 0 2024-05-09T06:38:08.454 INFO:tasks.workunit.client.1.smithi193.stdout:1/11: write f0 [744726,9974] 0 2024-05-09T06:38:08.454 INFO:tasks.workunit.client.0.smithi063.stdout:0/399: write dd/d1a/f65 [3485355,60267] 0 2024-05-09T06:38:08.455 INFO:tasks.workunit.client.0.smithi063.stdout:4/374: rename d7/df/d1f/f58 to d7/df/d1f/d28/f79 0 2024-05-09T06:38:08.455 INFO:tasks.workunit.client.1.smithi193.stdout:2/6: link f1 f2 0 2024-05-09T06:38:08.456 INFO:tasks.workunit.client.0.smithi063.stdout:5/325: mknod d8/d35/c6e 0 2024-05-09T06:38:08.457 INFO:tasks.workunit.client.1.smithi193.stdout:1/12: symlink l1 0 2024-05-09T06:38:08.457 INFO:tasks.workunit.client.1.smithi193.stdout:1/13: chown l1 2714 1 2024-05-09T06:38:08.458 INFO:tasks.workunit.client.0.smithi063.stdout:5/326: write d8/d35/f64 [257869,115665] 0 2024-05-09T06:38:08.458 INFO:tasks.workunit.client.0.smithi063.stdout:3/349: creat d7/da/f79 x:0 0 0 2024-05-09T06:38:08.458 INFO:tasks.workunit.client.0.smithi063.stdout:5/327: chown d8/d17/d18 3 1 2024-05-09T06:38:08.461 INFO:tasks.workunit.client.1.smithi193.stdout:1/14: dread f0 [0,4194304] 0 2024-05-09T06:38:08.467 INFO:tasks.workunit.client.0.smithi063.stdout:3/350: symlink d7/da/d21/d30/d44/d48/l7a 0 2024-05-09T06:38:08.468 INFO:tasks.workunit.client.0.smithi063.stdout:3/351: stat d7/d22/d28/f3f 0 2024-05-09T06:38:08.468 INFO:tasks.workunit.client.0.smithi063.stdout:9/368: sync 2024-05-09T06:38:08.468 INFO:tasks.workunit.client.0.smithi063.stdout:7/351: sync 2024-05-09T06:38:08.469 INFO:tasks.workunit.client.0.smithi063.stdout:0/400: creat dd/d60/d63/d23/d46/f82 x:0 0 0 2024-05-09T06:38:08.470 INFO:tasks.workunit.client.0.smithi063.stdout:9/369: write d0/d4/d26/d39/f53 [4449982,103411] 0 2024-05-09T06:38:08.471 INFO:tasks.workunit.client.0.smithi063.stdout:5/328: creat d8/d65/f6f x:0 0 0 2024-05-09T06:38:08.474 INFO:tasks.workunit.client.0.smithi063.stdout:0/401: symlink dd/d60/d44/l83 0 2024-05-09T06:38:08.474 INFO:tasks.workunit.client.0.smithi063.stdout:0/402: truncate dd/f15 6050034 0 2024-05-09T06:38:08.475 INFO:tasks.workunit.client.0.smithi063.stdout:0/403: fsync dd/d60/f52 0 2024-05-09T06:38:08.476 INFO:tasks.workunit.client.0.smithi063.stdout:7/352: mkdir d0/d15/d49/d4f/d66/d67/d6b 0 2024-05-09T06:38:08.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/7: write f1 [303599,90379] 0 2024-05-09T06:38:08.477 INFO:tasks.workunit.client.0.smithi063.stdout:9/370: unlink d0/d4/da/ld 0 2024-05-09T06:38:08.478 INFO:tasks.workunit.client.0.smithi063.stdout:4/375: write d7/df/d1f/d28/d31/d32/d34/f44 [1486182,36087] 0 2024-05-09T06:38:08.480 INFO:tasks.workunit.client.1.smithi193.stdout:2/8: mknod c3 0 2024-05-09T06:38:08.480 INFO:tasks.workunit.client.1.smithi193.stdout:2/9: truncate f2 873601 0 2024-05-09T06:38:08.481 INFO:tasks.workunit.client.1.smithi193.stdout:2/10: write f1 [1903805,61279] 0 2024-05-09T06:38:08.481 INFO:tasks.workunit.client.1.smithi193.stdout:2/11: chown c3 162894 1 2024-05-09T06:38:08.482 INFO:tasks.workunit.client.0.smithi063.stdout:3/352: rename d7/da/lf to d7/l7b 0 2024-05-09T06:38:08.482 INFO:tasks.workunit.client.1.smithi193.stdout:1/15: truncate f0 195123 0 2024-05-09T06:38:08.483 INFO:tasks.workunit.client.0.smithi063.stdout:0/404: mknod dd/c84 0 2024-05-09T06:38:08.484 INFO:tasks.workunit.client.0.smithi063.stdout:0/405: chown dd/c16 77977 1 2024-05-09T06:38:08.485 INFO:tasks.workunit.client.0.smithi063.stdout:7/353: creat d0/d15/d49/d4f/d66/f6c x:0 0 0 2024-05-09T06:38:08.486 INFO:tasks.workunit.client.1.smithi193.stdout:2/12: creat f4 x:0 0 0 2024-05-09T06:38:08.486 INFO:tasks.workunit.client.0.smithi063.stdout:7/354: chown d0/l13 8061 1 2024-05-09T06:38:08.487 INFO:tasks.workunit.client.1.smithi193.stdout:2/13: read - f4 zero size 2024-05-09T06:38:08.487 INFO:tasks.workunit.client.1.smithi193.stdout:2/14: read f1 [1832135,56019] 0 2024-05-09T06:38:08.488 INFO:tasks.workunit.client.0.smithi063.stdout:3/353: dwrite d7/f10 [4194304,4194304] 0 2024-05-09T06:38:08.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/354: read - d7/d4c/d31/d35/f45 zero size 2024-05-09T06:38:08.493 INFO:tasks.workunit.client.1.smithi193.stdout:2/15: unlink c3 0 2024-05-09T06:38:08.493 INFO:tasks.workunit.client.0.smithi063.stdout:5/329: rename d8/d17/d18/c6b to d8/d17/c70 0 2024-05-09T06:38:08.495 INFO:tasks.workunit.client.0.smithi063.stdout:0/406: dwrite dd/d1a/f65 [4194304,4194304] 0 2024-05-09T06:38:08.499 INFO:tasks.workunit.client.0.smithi063.stdout:7/355: symlink d0/d15/d49/d4f/l6d 0 2024-05-09T06:38:08.499 INFO:tasks.workunit.client.1.smithi193.stdout:2/16: fsync f2 0 2024-05-09T06:38:08.500 INFO:tasks.workunit.client.1.smithi193.stdout:2/17: chown f1 10212647 1 2024-05-09T06:38:08.500 INFO:tasks.workunit.client.1.smithi193.stdout:2/18: chown f4 4001 1 2024-05-09T06:38:08.501 INFO:tasks.workunit.client.1.smithi193.stdout:2/19: write f2 [70073,112019] 0 2024-05-09T06:38:08.503 INFO:tasks.workunit.client.1.smithi193.stdout:2/20: read - f4 zero size 2024-05-09T06:38:08.506 INFO:tasks.workunit.client.0.smithi063.stdout:3/355: dread f1 [0,4194304] 0 2024-05-09T06:38:08.507 INFO:tasks.workunit.client.0.smithi063.stdout:9/371: rename d0/d4/d26/d39/d48/l75 to d0/d4/d26/d39/l7e 0 2024-05-09T06:38:08.508 INFO:tasks.workunit.client.1.smithi193.stdout:2/21: creat f5 x:0 0 0 2024-05-09T06:38:08.509 INFO:tasks.workunit.client.1.smithi193.stdout:1/16: write f0 [165215,108146] 0 2024-05-09T06:38:08.510 INFO:tasks.workunit.client.1.smithi193.stdout:2/22: write f4 [902098,82949] 0 2024-05-09T06:38:08.510 INFO:tasks.workunit.client.0.smithi063.stdout:5/330: symlink d8/d19/d30/l71 0 2024-05-09T06:38:08.510 INFO:tasks.workunit.client.1.smithi193.stdout:1/17: read f0 [185850,23409] 0 2024-05-09T06:38:08.517 INFO:tasks.workunit.client.1.smithi193.stdout:2/23: dwrite f5 [0,4194304] 0 2024-05-09T06:38:08.519 INFO:tasks.workunit.client.0.smithi063.stdout:7/356: rename d0/d15/f3a to d0/d15/d49/d4f/d66/d67/d6b/f6e 0 2024-05-09T06:38:08.519 INFO:tasks.workunit.client.0.smithi063.stdout:9/372: symlink d0/d4/d57/l7f 0 2024-05-09T06:38:08.522 INFO:tasks.workunit.client.0.smithi063.stdout:4/376: getdents d7/df 0 2024-05-09T06:38:08.522 INFO:tasks.workunit.client.1.smithi193.stdout:2/24: rename f2 to f6 0 2024-05-09T06:38:08.524 INFO:tasks.workunit.client.1.smithi193.stdout:1/18: getdents . 0 2024-05-09T06:38:08.526 INFO:tasks.workunit.client.1.smithi193.stdout:2/25: dread f5 [0,4194304] 0 2024-05-09T06:38:08.528 INFO:tasks.workunit.client.0.smithi063.stdout:7/357: dwrite d0/d15/d4a/d50/f2e [0,4194304] 0 2024-05-09T06:38:08.529 INFO:tasks.workunit.client.0.smithi063.stdout:0/407: truncate dd/f12 6010834 0 2024-05-09T06:38:08.534 INFO:tasks.workunit.client.0.smithi063.stdout:5/331: rename d8/d17/d1e/c5c to d8/d17/d18/c72 0 2024-05-09T06:38:08.534 INFO:tasks.workunit.client.0.smithi063.stdout:0/408: rename dd/d1a to dd/d1a/d2c/d42/d85 22 2024-05-09T06:38:08.537 INFO:tasks.workunit.client.0.smithi063.stdout:9/373: mkdir d0/d4/d26/d39/d48/d80 0 2024-05-09T06:38:08.539 INFO:tasks.workunit.client.0.smithi063.stdout:9/374: write d0/d4/d26/f30 [330202,128796] 0 2024-05-09T06:38:08.540 INFO:tasks.workunit.client.1.smithi193.stdout:1/19: write f0 [155439,96365] 0 2024-05-09T06:38:08.540 INFO:tasks.workunit.client.0.smithi063.stdout:9/375: stat d0/d4/da/l27 0 2024-05-09T06:38:08.541 INFO:tasks.workunit.client.1.smithi193.stdout:1/20: dread f0 [0,4194304] 0 2024-05-09T06:38:08.544 INFO:tasks.workunit.client.0.smithi063.stdout:5/332: symlink d8/d17/l73 0 2024-05-09T06:38:08.544 INFO:tasks.workunit.client.0.smithi063.stdout:5/333: stat d8/d19/f44 0 2024-05-09T06:38:08.547 INFO:tasks.workunit.client.1.smithi193.stdout:9/75: sync 2024-05-09T06:38:08.547 INFO:tasks.workunit.client.1.smithi193.stdout:8/28: sync 2024-05-09T06:38:08.547 INFO:tasks.workunit.client.1.smithi193.stdout:4/18: sync 2024-05-09T06:38:08.547 INFO:tasks.workunit.client.1.smithi193.stdout:3/3: sync 2024-05-09T06:38:08.547 INFO:tasks.workunit.client.1.smithi193.stdout:6/70: sync 2024-05-09T06:38:08.548 INFO:tasks.workunit.client.1.smithi193.stdout:0/126: sync 2024-05-09T06:38:08.548 INFO:tasks.workunit.client.1.smithi193.stdout:7/72: sync 2024-05-09T06:38:08.548 INFO:tasks.workunit.client.1.smithi193.stdout:5/11: sync 2024-05-09T06:38:08.548 INFO:tasks.workunit.client.0.smithi063.stdout:0/409: unlink dd/d60/f52 0 2024-05-09T06:38:08.548 INFO:tasks.workunit.client.0.smithi063.stdout:0/410: chown dd/d60/d63/f67 5 1 2024-05-09T06:38:08.549 INFO:tasks.workunit.client.1.smithi193.stdout:6/71: fdatasync d1/f3 0 2024-05-09T06:38:08.549 INFO:tasks.workunit.client.1.smithi193.stdout:0/127: chown f13 3 1 2024-05-09T06:38:08.549 INFO:tasks.workunit.client.1.smithi193.stdout:9/76: write dd/f18 [108554,66060] 0 2024-05-09T06:38:08.549 INFO:tasks.workunit.client.1.smithi193.stdout:8/29: write f0 [2139977,122430] 0 2024-05-09T06:38:08.551 INFO:tasks.workunit.client.1.smithi193.stdout:2/26: getdents . 0 2024-05-09T06:38:08.552 INFO:tasks.workunit.client.0.smithi063.stdout:3/356: getdents d7/d4c 0 2024-05-09T06:38:08.553 INFO:tasks.workunit.client.1.smithi193.stdout:6/72: write d1/f5 [1390829,49137] 0 2024-05-09T06:38:08.555 INFO:tasks.workunit.client.1.smithi193.stdout:1/21: rename f0 to f2 0 2024-05-09T06:38:08.556 INFO:tasks.workunit.client.0.smithi063.stdout:9/376: mknod d0/d4/d26/d39/d48/c81 0 2024-05-09T06:38:08.556 INFO:tasks.workunit.client.0.smithi063.stdout:6/354: sync 2024-05-09T06:38:08.558 INFO:tasks.workunit.client.0.smithi063.stdout:7/358: creat d0/d15/d4a/d68/f6f x:0 0 0 2024-05-09T06:38:08.559 INFO:tasks.workunit.client.1.smithi193.stdout:8/30: dread d6/f7 [0,4194304] 0 2024-05-09T06:38:08.559 INFO:tasks.workunit.client.1.smithi193.stdout:3/4: dread f0 [0,4194304] 0 2024-05-09T06:38:08.560 INFO:tasks.workunit.client.0.smithi063.stdout:5/334: mkdir d8/d19/d30/d74 0 2024-05-09T06:38:08.563 INFO:tasks.workunit.client.0.smithi063.stdout:6/355: dwrite d2/d4/d2d/d42/f61 [0,4194304] 0 2024-05-09T06:38:08.563 INFO:tasks.workunit.client.1.smithi193.stdout:4/19: dwrite f6 [0,4194304] 0 2024-05-09T06:38:08.563 INFO:tasks.workunit.client.1.smithi193.stdout:4/20: rmdir - no directory 2024-05-09T06:38:08.564 INFO:tasks.workunit.client.0.smithi063.stdout:6/356: truncate d2/d1f/d29/f63 95127 0 2024-05-09T06:38:08.565 INFO:tasks.workunit.client.0.smithi063.stdout:4/377: truncate d7/df/d1a/f22 1509648 0 2024-05-09T06:38:08.567 INFO:tasks.workunit.client.0.smithi063.stdout:5/335: dread d8/d19/f3f [0,4194304] 0 2024-05-09T06:38:08.567 INFO:tasks.workunit.client.0.smithi063.stdout:4/378: fdatasync d7/df/d35/d5a/d60/f63 0 2024-05-09T06:38:08.568 INFO:tasks.workunit.client.1.smithi193.stdout:2/27: mknod c7 0 2024-05-09T06:38:08.568 INFO:tasks.workunit.client.1.smithi193.stdout:6/73: dwrite d1/f2 [0,4194304] 0 2024-05-09T06:38:08.574 INFO:tasks.workunit.client.0.smithi063.stdout:7/359: creat d0/d15/d49/f70 x:0 0 0 2024-05-09T06:38:08.575 INFO:tasks.workunit.client.1.smithi193.stdout:8/31: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:08.575 INFO:tasks.workunit.client.1.smithi193.stdout:1/22: symlink l3 0 2024-05-09T06:38:08.577 INFO:tasks.workunit.client.0.smithi063.stdout:6/357: dwrite d2/d4/f3c [4194304,4194304] 0 2024-05-09T06:38:08.582 INFO:tasks.workunit.client.0.smithi063.stdout:0/411: link dd/d60/d44/f61 dd/d1a/d50/f86 0 2024-05-09T06:38:08.582 INFO:tasks.workunit.client.1.smithi193.stdout:8/32: dread d6/f7 [0,4194304] 0 2024-05-09T06:38:08.582 INFO:tasks.workunit.client.1.smithi193.stdout:6/74: dread d1/f2 [0,4194304] 0 2024-05-09T06:38:08.583 INFO:tasks.workunit.client.0.smithi063.stdout:5/336: mkdir d8/d17/d75 0 2024-05-09T06:38:08.583 INFO:tasks.workunit.client.1.smithi193.stdout:0/128: dwrite f9 [0,4194304] 0 2024-05-09T06:38:08.584 INFO:tasks.workunit.client.1.smithi193.stdout:3/5: mkdir d1 0 2024-05-09T06:38:08.585 INFO:tasks.workunit.client.1.smithi193.stdout:8/33: dread d6/f7 [0,4194304] 0 2024-05-09T06:38:08.586 INFO:tasks.workunit.client.0.smithi063.stdout:4/379: rmdir d7/df/d1f/d28/d31/d32 39 2024-05-09T06:38:08.586 INFO:tasks.workunit.client.1.smithi193.stdout:6/75: truncate d1/f9 962535 0 2024-05-09T06:38:08.587 INFO:tasks.workunit.client.1.smithi193.stdout:6/76: chown d1/lc 7 1 2024-05-09T06:38:08.590 INFO:tasks.workunit.client.1.smithi193.stdout:4/21: fdatasync f5 0 2024-05-09T06:38:08.591 INFO:tasks.workunit.client.1.smithi193.stdout:4/22: write f6 [1525023,79271] 0 2024-05-09T06:38:08.592 INFO:tasks.workunit.client.0.smithi063.stdout:3/357: symlink d7/da/d21/d30/d44/l7c 0 2024-05-09T06:38:08.592 INFO:tasks.workunit.client.1.smithi193.stdout:4/23: write f5 [2923772,76951] 0 2024-05-09T06:38:08.593 INFO:tasks.workunit.client.1.smithi193.stdout:4/24: read f5 [2859121,131069] 0 2024-05-09T06:38:08.594 INFO:tasks.workunit.client.0.smithi063.stdout:9/377: creat d0/d4/d7d/f82 x:0 0 0 2024-05-09T06:38:08.596 INFO:tasks.workunit.client.1.smithi193.stdout:2/28: rename f4 to f8 0 2024-05-09T06:38:08.596 INFO:tasks.workunit.client.0.smithi063.stdout:9/378: write d0/d4/d57/f64 [1014824,73175] 0 2024-05-09T06:38:08.599 INFO:tasks.workunit.client.0.smithi063.stdout:7/360: rename d0/f1c to d0/d15/d49/d4f/d66/f71 0 2024-05-09T06:38:08.600 INFO:tasks.workunit.client.0.smithi063.stdout:7/361: readlink d0/l2d 0 2024-05-09T06:38:08.600 INFO:tasks.workunit.client.1.smithi193.stdout:7/73: write d6/dd/fe [330691,98932] 0 2024-05-09T06:38:08.600 INFO:tasks.workunit.client.0.smithi063.stdout:7/362: fdatasync d0/f10 0 2024-05-09T06:38:08.604 INFO:tasks.workunit.client.1.smithi193.stdout:9/77: mknod dd/df/d15/d16/c1b 0 2024-05-09T06:38:08.606 INFO:tasks.workunit.client.0.smithi063.stdout:0/412: mknod dd/d1a/d2c/d42/c87 0 2024-05-09T06:38:08.606 INFO:tasks.workunit.client.0.smithi063.stdout:6/358: dwrite d2/df/f5d [0,4194304] 0 2024-05-09T06:38:08.606 INFO:tasks.workunit.client.1.smithi193.stdout:1/23: fsync f2 0 2024-05-09T06:38:08.607 INFO:tasks.workunit.client.0.smithi063.stdout:0/413: readlink dd/d1a/d2c/l30 0 2024-05-09T06:38:08.607 INFO:tasks.workunit.client.1.smithi193.stdout:1/24: write f2 [596247,26718] 0 2024-05-09T06:38:08.608 INFO:tasks.workunit.client.0.smithi063.stdout:5/337: creat d8/d17/d1e/f76 x:0 0 0 2024-05-09T06:38:08.611 INFO:tasks.workunit.client.0.smithi063.stdout:9/379: symlink d0/d4/d7d/l83 0 2024-05-09T06:38:08.611 INFO:tasks.workunit.client.1.smithi193.stdout:3/6: truncate f0 208712 0 2024-05-09T06:38:08.613 INFO:tasks.workunit.client.1.smithi193.stdout:6/77: creat d1/de/f13 x:0 0 0 2024-05-09T06:38:08.616 INFO:tasks.workunit.client.0.smithi063.stdout:7/363: rename d0/f16 to d0/d15/f72 0 2024-05-09T06:38:08.617 INFO:tasks.workunit.client.0.smithi063.stdout:6/359: creat d2/d12/f76 x:0 0 0 2024-05-09T06:38:08.617 INFO:tasks.workunit.client.0.smithi063.stdout:2/373: sync 2024-05-09T06:38:08.618 INFO:tasks.workunit.client.0.smithi063.stdout:8/433: sync 2024-05-09T06:38:08.618 INFO:tasks.workunit.client.0.smithi063.stdout:1/333: sync 2024-05-09T06:38:08.619 INFO:tasks.workunit.client.0.smithi063.stdout:8/434: chown d9/da/d18/d46/f52 908966326 1 2024-05-09T06:38:08.619 INFO:tasks.workunit.client.0.smithi063.stdout:0/414: creat dd/d60/d63/d23/d33/f88 x:0 0 0 2024-05-09T06:38:08.620 INFO:tasks.workunit.client.1.smithi193.stdout:1/25: dread f2 [0,4194304] 0 2024-05-09T06:38:08.620 INFO:tasks.workunit.client.0.smithi063.stdout:6/360: write d2/d1f/d29/d3f/f1e [991184,81105] 0 2024-05-09T06:38:08.621 INFO:tasks.workunit.client.0.smithi063.stdout:0/415: stat dd/d1a/d50/f7c 0 2024-05-09T06:38:08.621 INFO:tasks.workunit.client.1.smithi193.stdout:4/25: creat f7 x:0 0 0 2024-05-09T06:38:08.622 INFO:tasks.workunit.client.1.smithi193.stdout:4/26: truncate f7 571489 0 2024-05-09T06:38:08.626 INFO:tasks.workunit.client.0.smithi063.stdout:9/380: fsync d0/d4/d26/f36 0 2024-05-09T06:38:08.626 INFO:tasks.workunit.client.0.smithi063.stdout:8/435: dwrite d9/da/d18/d56/f65 [0,4194304] 0 2024-05-09T06:38:08.628 INFO:tasks.workunit.client.0.smithi063.stdout:8/436: write d9/da/d18/d46/f52 [207601,6945] 0 2024-05-09T06:38:08.628 INFO:tasks.workunit.client.1.smithi193.stdout:7/74: rename d6/d8/db to d6/d17 0 2024-05-09T06:38:08.629 INFO:tasks.workunit.client.1.smithi193.stdout:2/29: dwrite f6 [0,4194304] 0 2024-05-09T06:38:08.630 INFO:tasks.workunit.client.1.smithi193.stdout:7/75: write d6/dd/fe [357640,31425] 0 2024-05-09T06:38:08.631 INFO:tasks.workunit.client.1.smithi193.stdout:7/76: fsync d6/d8/fa 0 2024-05-09T06:38:08.632 INFO:tasks.workunit.client.1.smithi193.stdout:7/77: dread - d6/dd/f14 zero size 2024-05-09T06:38:08.633 INFO:tasks.workunit.client.0.smithi063.stdout:9/381: dwrite d0/d3b/f6e [0,4194304] 0 2024-05-09T06:38:08.639 INFO:tasks.workunit.client.0.smithi063.stdout:1/334: mknod d9/d2a/d35/d1e/c71 0 2024-05-09T06:38:08.644 INFO:tasks.workunit.client.1.smithi193.stdout:1/26: mkdir d4 0 2024-05-09T06:38:08.647 INFO:tasks.workunit.client.1.smithi193.stdout:4/27: symlink l8 0 2024-05-09T06:38:08.649 INFO:tasks.workunit.client.1.smithi193.stdout:6/78: dwrite d1/f9 [0,4194304] 0 2024-05-09T06:38:08.650 INFO:tasks.workunit.client.1.smithi193.stdout:6/79: fdatasync d1/fd 0 2024-05-09T06:38:08.652 INFO:tasks.workunit.client.1.smithi193.stdout:6/80: write d1/de/f13 [789006,89096] 0 2024-05-09T06:38:08.654 INFO:tasks.workunit.client.1.smithi193.stdout:2/30: unlink c7 0 2024-05-09T06:38:08.654 INFO:tasks.workunit.client.1.smithi193.stdout:2/31: chown f6 44491371 1 2024-05-09T06:38:08.656 INFO:tasks.workunit.client.1.smithi193.stdout:7/78: creat d6/f18 x:0 0 0 2024-05-09T06:38:08.660 INFO:tasks.workunit.client.1.smithi193.stdout:4/28: unlink l3 0 2024-05-09T06:38:08.662 INFO:tasks.workunit.client.1.smithi193.stdout:2/32: rename f5 to f9 0 2024-05-09T06:38:08.663 INFO:tasks.workunit.client.1.smithi193.stdout:9/78: link lc dd/df/d15/d16/l1c 0 2024-05-09T06:38:08.666 INFO:tasks.workunit.client.1.smithi193.stdout:1/27: symlink d4/l5 0 2024-05-09T06:38:08.668 INFO:tasks.workunit.client.1.smithi193.stdout:4/29: symlink l9 0 2024-05-09T06:38:08.668 INFO:tasks.workunit.client.1.smithi193.stdout:2/33: dread f9 [0,4194304] 0 2024-05-09T06:38:08.669 INFO:tasks.workunit.client.1.smithi193.stdout:6/81: getdents d1/de/d11 0 2024-05-09T06:38:08.672 INFO:tasks.workunit.client.1.smithi193.stdout:9/79: dwrite dd/fe [0,4194304] 0 2024-05-09T06:38:08.675 INFO:tasks.workunit.client.1.smithi193.stdout:1/28: rename l1 to d4/l6 0 2024-05-09T06:38:08.677 INFO:tasks.workunit.client.1.smithi193.stdout:2/34: dread f6 [0,4194304] 0 2024-05-09T06:38:08.677 INFO:tasks.workunit.client.1.smithi193.stdout:4/30: creat fa x:0 0 0 2024-05-09T06:38:08.677 INFO:tasks.workunit.client.1.smithi193.stdout:4/31: rmdir - no directory 2024-05-09T06:38:08.679 INFO:tasks.workunit.client.1.smithi193.stdout:3/7: dread f0 [0,4194304] 0 2024-05-09T06:38:08.682 INFO:tasks.workunit.client.1.smithi193.stdout:9/80: dwrite dd/f13 [0,4194304] 0 2024-05-09T06:38:08.685 INFO:tasks.workunit.client.0.smithi063.stdout:5/338: mkdir d8/d19/d30/d74/d77 0 2024-05-09T06:38:08.685 INFO:tasks.workunit.client.1.smithi193.stdout:1/29: mknod d4/c7 0 2024-05-09T06:38:08.687 INFO:tasks.workunit.client.0.smithi063.stdout:7/364: symlink d0/l73 0 2024-05-09T06:38:08.688 INFO:tasks.workunit.client.0.smithi063.stdout:6/361: fsync d2/d12/f76 0 2024-05-09T06:38:08.689 INFO:tasks.workunit.client.1.smithi193.stdout:2/35: mknod ca 0 2024-05-09T06:38:08.689 INFO:tasks.workunit.client.0.smithi063.stdout:4/380: dwrite f5 [4194304,4194304] 0 2024-05-09T06:38:08.691 INFO:tasks.workunit.client.1.smithi193.stdout:9/81: dwrite dd/f11 [0,4194304] 0 2024-05-09T06:38:08.693 INFO:tasks.workunit.client.0.smithi063.stdout:3/358: dwrite d7/f2f [0,4194304] 0 2024-05-09T06:38:08.694 INFO:tasks.workunit.client.0.smithi063.stdout:6/362: write d2/f38 [2538922,16752] 0 2024-05-09T06:38:08.697 INFO:tasks.workunit.client.0.smithi063.stdout:9/382: mkdir d0/d4/d26/d39/d48/d80/d84 0 2024-05-09T06:38:08.697 INFO:tasks.workunit.client.0.smithi063.stdout:1/335: readlink d9/d2a/d35/l20 0 2024-05-09T06:38:08.699 INFO:tasks.workunit.client.0.smithi063.stdout:5/339: symlink d8/d53/l78 0 2024-05-09T06:38:08.703 INFO:tasks.workunit.client.0.smithi063.stdout:9/383: dread - d0/d4/d26/d39/d3e/f5c zero size 2024-05-09T06:38:08.706 INFO:tasks.workunit.client.0.smithi063.stdout:0/416: dwrite dd/d60/d63/f2b [0,4194304] 0 2024-05-09T06:38:08.708 INFO:tasks.workunit.client.1.smithi193.stdout:7/79: write f5 [318660,115404] 0 2024-05-09T06:38:08.709 INFO:tasks.workunit.client.1.smithi193.stdout:4/32: chown l2 97 1 2024-05-09T06:38:08.710 INFO:tasks.workunit.client.0.smithi063.stdout:5/340: dread d8/d17/d18/f51 [0,4194304] 0 2024-05-09T06:38:08.711 INFO:tasks.workunit.client.0.smithi063.stdout:2/374: dwrite dc/dd/d2d/d60/f67 [0,4194304] 0 2024-05-09T06:38:08.714 INFO:tasks.workunit.client.1.smithi193.stdout:9/82: creat dd/df/f1d x:0 0 0 2024-05-09T06:38:08.715 INFO:tasks.workunit.client.1.smithi193.stdout:6/82: write d1/f7 [4197648,113797] 0 2024-05-09T06:38:08.716 INFO:tasks.workunit.client.1.smithi193.stdout:3/8: mknod d1/c2 0 2024-05-09T06:38:08.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/359: dwrite d7/d2a/d2d/f3e [0,4194304] 0 2024-05-09T06:38:08.717 INFO:tasks.workunit.client.1.smithi193.stdout:3/9: stat d1/c2 0 2024-05-09T06:38:08.717 INFO:tasks.workunit.client.1.smithi193.stdout:6/83: chown d1/f2 7 1 2024-05-09T06:38:08.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/360: dread - d7/da/d21/d30/d44/d48/f53 zero size 2024-05-09T06:38:08.719 INFO:tasks.workunit.client.0.smithi063.stdout:6/363: dwrite d2/d1f/d29/f63 [0,4194304] 0 2024-05-09T06:38:08.720 INFO:tasks.workunit.client.1.smithi193.stdout:6/84: chown d1/f2 43 1 2024-05-09T06:38:08.721 INFO:tasks.workunit.client.0.smithi063.stdout:3/361: write d7/f10 [3742265,71929] 0 2024-05-09T06:38:08.722 INFO:tasks.workunit.client.0.smithi063.stdout:3/362: readlink d7/d4c/l5f 0 2024-05-09T06:38:08.723 INFO:tasks.workunit.client.1.smithi193.stdout:2/36: dwrite f1 [0,4194304] 0 2024-05-09T06:38:08.724 INFO:tasks.workunit.client.0.smithi063.stdout:3/363: dread - d7/d2a/d2d/f65 zero size 2024-05-09T06:38:08.724 INFO:tasks.workunit.client.1.smithi193.stdout:6/85: truncate d1/de/f10 1407621 0 2024-05-09T06:38:08.727 INFO:tasks.workunit.client.0.smithi063.stdout:4/381: symlink d7/df/d77/l7a 0 2024-05-09T06:38:08.731 INFO:tasks.workunit.client.0.smithi063.stdout:9/384: symlink d0/d4/d26/d39/d48/l85 0 2024-05-09T06:38:08.731 INFO:tasks.workunit.client.0.smithi063.stdout:0/417: dwrite f6 [8388608,4194304] 0 2024-05-09T06:38:08.732 INFO:tasks.workunit.client.0.smithi063.stdout:5/341: unlink d8/d17/f26 0 2024-05-09T06:38:08.732 INFO:tasks.workunit.client.0.smithi063.stdout:5/342: chown d8/l36 186497 1 2024-05-09T06:38:08.736 INFO:tasks.workunit.client.0.smithi063.stdout:2/375: mkdir dc/dd/d45/d5a/d6a 0 2024-05-09T06:38:08.738 INFO:tasks.workunit.client.0.smithi063.stdout:2/376: stat dc/d32/d33/l3c 0 2024-05-09T06:38:08.743 INFO:tasks.workunit.client.0.smithi063.stdout:5/343: dwrite d8/d17/d18/f38 [4194304,4194304] 0 2024-05-09T06:38:08.744 INFO:tasks.workunit.client.0.smithi063.stdout:9/385: mknod d0/d43/c86 0 2024-05-09T06:38:08.747 INFO:tasks.workunit.client.0.smithi063.stdout:2/377: creat dc/d36/d43/d46/f6b x:0 0 0 2024-05-09T06:38:08.747 INFO:tasks.workunit.client.0.smithi063.stdout:1/336: link d9/d46/c5b d9/d2a/d35/d1e/c72 0 2024-05-09T06:38:08.751 INFO:tasks.workunit.client.0.smithi063.stdout:9/386: creat d0/d4/d7d/f87 x:0 0 0 2024-05-09T06:38:08.752 INFO:tasks.workunit.client.0.smithi063.stdout:2/378: write dc/f25 [4253614,51654] 0 2024-05-09T06:38:08.753 INFO:tasks.workunit.client.0.smithi063.stdout:3/364: dwrite d7/da/d21/d30/f57 [0,4194304] 0 2024-05-09T06:38:08.757 INFO:tasks.workunit.client.0.smithi063.stdout:5/344: mknod d8/d19/d30/d74/d77/c79 0 2024-05-09T06:38:08.760 INFO:tasks.workunit.client.0.smithi063.stdout:9/387: read - d0/d4/d26/d47/f7a zero size 2024-05-09T06:38:08.760 INFO:tasks.workunit.client.0.smithi063.stdout:1/337: symlink d9/d2a/l73 0 2024-05-09T06:38:08.762 INFO:tasks.workunit.client.0.smithi063.stdout:2/379: mkdir dc/dd/d45/d5a/d6c 0 2024-05-09T06:38:08.763 INFO:tasks.workunit.client.0.smithi063.stdout:7/365: dread d0/d15/d4a/d50/f23 [0,4194304] 0 2024-05-09T06:38:08.764 INFO:tasks.workunit.client.0.smithi063.stdout:7/366: chown d0/d15/d49/d4f/d66/d67 36176642 1 2024-05-09T06:38:08.766 INFO:tasks.workunit.client.0.smithi063.stdout:1/338: dread - d9/d46/f6e zero size 2024-05-09T06:38:08.766 INFO:tasks.workunit.client.0.smithi063.stdout:7/367: rename d0/f14 to d0/d15/d49/f74 0 2024-05-09T06:38:08.767 INFO:tasks.workunit.client.0.smithi063.stdout:2/380: rename dc/dd/d2d/d60 to dc/dd/d2d/d60/d65/d6d 22 2024-05-09T06:38:08.767 INFO:tasks.workunit.client.0.smithi063.stdout:2/381: fsync dc/f1f 0 2024-05-09T06:38:08.768 INFO:tasks.workunit.client.0.smithi063.stdout:5/345: unlink d8/f5b 0 2024-05-09T06:38:08.769 INFO:tasks.workunit.client.0.smithi063.stdout:9/388: mknod d0/d8/d2d/c88 0 2024-05-09T06:38:08.770 INFO:tasks.workunit.client.0.smithi063.stdout:1/339: chown d9/d10/d2b/d3f/d59/l60 18 1 2024-05-09T06:38:08.772 INFO:tasks.workunit.client.0.smithi063.stdout:0/418: dread f2 [0,4194304] 0 2024-05-09T06:38:08.772 INFO:tasks.workunit.client.0.smithi063.stdout:9/389: write d0/f24 [7504404,9412] 0 2024-05-09T06:38:08.774 INFO:tasks.workunit.client.0.smithi063.stdout:9/390: fsync d0/d3b/f78 0 2024-05-09T06:38:08.776 INFO:tasks.workunit.client.0.smithi063.stdout:3/365: getdents d7/da/d21/d30/d44 0 2024-05-09T06:38:08.776 INFO:tasks.workunit.client.0.smithi063.stdout:2/382: dread dc/dd/d45/d5a/f5e [0,4194304] 0 2024-05-09T06:38:08.776 INFO:tasks.workunit.client.0.smithi063.stdout:0/419: mkdir dd/d60/d63/d23/d89 0 2024-05-09T06:38:08.776 INFO:tasks.workunit.client.0.smithi063.stdout:1/340: creat d9/d2a/f74 x:0 0 0 2024-05-09T06:38:08.777 INFO:tasks.workunit.client.0.smithi063.stdout:8/437: sync 2024-05-09T06:38:08.778 INFO:tasks.workunit.client.0.smithi063.stdout:2/383: fdatasync dc/d36/d43/d46/f6b 0 2024-05-09T06:38:08.779 INFO:tasks.workunit.client.0.smithi063.stdout:2/384: chown dc 1569400 1 2024-05-09T06:38:08.780 INFO:tasks.workunit.client.0.smithi063.stdout:9/391: link d0/d43/l4a d0/d4/l89 0 2024-05-09T06:38:08.781 INFO:tasks.workunit.client.0.smithi063.stdout:0/420: stat dd/d60/d63/d23/d33/d48/c66 0 2024-05-09T06:38:08.782 INFO:tasks.workunit.client.0.smithi063.stdout:8/438: mknod d9/c94 0 2024-05-09T06:38:08.783 INFO:tasks.workunit.client.0.smithi063.stdout:3/366: dread d7/d2a/d2d/f5d [0,4194304] 0 2024-05-09T06:38:08.783 INFO:tasks.workunit.client.1.smithi193.stdout:9/83: mkdir dd/d1e 0 2024-05-09T06:38:08.785 INFO:tasks.workunit.client.1.smithi193.stdout:2/37: mkdir db 0 2024-05-09T06:38:08.785 INFO:tasks.workunit.client.1.smithi193.stdout:4/33: link f5 fb 0 2024-05-09T06:38:08.786 INFO:tasks.workunit.client.0.smithi063.stdout:3/367: write d7/da/f14 [4971647,116686] 0 2024-05-09T06:38:08.787 INFO:tasks.workunit.client.0.smithi063.stdout:9/392: mknod d0/c8a 0 2024-05-09T06:38:08.788 INFO:tasks.workunit.client.0.smithi063.stdout:9/393: chown d0/d43/f5f 22305 1 2024-05-09T06:38:08.789 INFO:tasks.workunit.client.1.smithi193.stdout:6/86: dread d1/f6 [0,4194304] 0 2024-05-09T06:38:08.789 INFO:tasks.workunit.client.0.smithi063.stdout:5/346: dread d8/d17/d1e/f23 [0,4194304] 0 2024-05-09T06:38:08.790 INFO:tasks.workunit.client.0.smithi063.stdout:8/439: mknod d9/da/d18/d24/d71/c95 0 2024-05-09T06:38:08.791 INFO:tasks.workunit.client.0.smithi063.stdout:2/385: dwrite dc/d32/f62 [0,4194304] 0 2024-05-09T06:38:08.792 INFO:tasks.workunit.client.1.smithi193.stdout:9/84: dwrite dd/f18 [0,4194304] 0 2024-05-09T06:38:08.792 INFO:tasks.workunit.client.0.smithi063.stdout:8/440: write d9/d37/f5a [201427,82268] 0 2024-05-09T06:38:08.794 INFO:tasks.workunit.client.1.smithi193.stdout:2/38: rename ca to db/cc 0 2024-05-09T06:38:08.795 INFO:tasks.workunit.client.0.smithi063.stdout:8/441: dread - d9/da/d18/d24/f42 zero size 2024-05-09T06:38:08.796 INFO:tasks.workunit.client.0.smithi063.stdout:0/421: creat dd/d60/d63/d23/d89/f8a x:0 0 0 2024-05-09T06:38:08.796 INFO:tasks.workunit.client.1.smithi193.stdout:1/30: write f2 [223040,46266] 0 2024-05-09T06:38:08.796 INFO:tasks.workunit.client.1.smithi193.stdout:7/80: getdents d6/d8 0 2024-05-09T06:38:08.799 INFO:tasks.workunit.client.1.smithi193.stdout:9/85: dread dd/fe [0,4194304] 0 2024-05-09T06:38:08.799 INFO:tasks.workunit.client.1.smithi193.stdout:4/34: read fb [3187548,76032] 0 2024-05-09T06:38:08.800 INFO:tasks.workunit.client.0.smithi063.stdout:5/347: creat d8/d65/f7a x:0 0 0 2024-05-09T06:38:08.800 INFO:tasks.workunit.client.1.smithi193.stdout:4/35: stat l4 0 2024-05-09T06:38:08.801 INFO:tasks.workunit.client.0.smithi063.stdout:2/386: fsync dc/dd/f20 0 2024-05-09T06:38:08.802 INFO:tasks.workunit.client.0.smithi063.stdout:5/348: fsync d8/d35/f64 0 2024-05-09T06:38:08.802 INFO:tasks.workunit.client.0.smithi063.stdout:9/394: dwrite d0/d4/d26/f34 [0,4194304] 0 2024-05-09T06:38:08.806 INFO:tasks.workunit.client.0.smithi063.stdout:2/387: write f2 [5209564,26815] 0 2024-05-09T06:38:08.806 INFO:tasks.workunit.client.0.smithi063.stdout:9/395: fsync d0/d4/f37 0 2024-05-09T06:38:08.807 INFO:tasks.workunit.client.1.smithi193.stdout:2/39: mknod db/cd 0 2024-05-09T06:38:08.808 INFO:tasks.workunit.client.0.smithi063.stdout:4/382: getdents d7/df/d77 0 2024-05-09T06:38:08.809 INFO:tasks.workunit.client.1.smithi193.stdout:3/10: write f0 [438035,76913] 0 2024-05-09T06:38:08.810 INFO:tasks.workunit.client.0.smithi063.stdout:9/396: readlink d0/d4/d7d/l83 0 2024-05-09T06:38:08.810 INFO:tasks.workunit.client.0.smithi063.stdout:4/383: readlink d7/df/l29 0 2024-05-09T06:38:08.811 INFO:tasks.workunit.client.0.smithi063.stdout:2/388: dread - dc/d36/d43/d46/f6b zero size 2024-05-09T06:38:08.811 INFO:tasks.workunit.client.0.smithi063.stdout:8/442: dwrite d9/da/d18/f1d [0,4194304] 0 2024-05-09T06:38:08.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/422: dwrite f2 [0,4194304] 0 2024-05-09T06:38:08.812 INFO:tasks.workunit.client.1.smithi193.stdout:1/31: unlink l3 0 2024-05-09T06:38:08.814 INFO:tasks.workunit.client.1.smithi193.stdout:7/81: rmdir d6/d8 39 2024-05-09T06:38:08.815 INFO:tasks.workunit.client.0.smithi063.stdout:2/389: fdatasync f6 0 2024-05-09T06:38:08.816 INFO:tasks.workunit.client.0.smithi063.stdout:5/349: creat d8/d35/f7b x:0 0 0 2024-05-09T06:38:08.819 INFO:tasks.workunit.client.1.smithi193.stdout:4/36: symlink lc 0 2024-05-09T06:38:08.819 INFO:tasks.workunit.client.1.smithi193.stdout:1/32: dwrite f2 [0,4194304] 0 2024-05-09T06:38:08.820 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:08 smithi063 ceph-mon[27171]: pgmap v7: 65 pgs: 65 active+clean; 750 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 120 MiB/s wr, 256 op/s 2024-05-09T06:38:08.821 INFO:tasks.workunit.client.1.smithi193.stdout:4/37: chown fa 1372705 1 2024-05-09T06:38:08.828 INFO:tasks.workunit.client.1.smithi193.stdout:2/40: mknod db/ce 0 2024-05-09T06:38:08.830 INFO:tasks.workunit.client.0.smithi063.stdout:0/423: creat dd/d60/d63/d23/d89/f8b x:0 0 0 2024-05-09T06:38:08.830 INFO:tasks.workunit.client.0.smithi063.stdout:6/364: truncate d2/d1f/d29/d3f/f58 1326945 0 2024-05-09T06:38:08.831 INFO:tasks.workunit.client.1.smithi193.stdout:3/11: symlink d1/l3 0 2024-05-09T06:38:08.833 INFO:tasks.workunit.client.0.smithi063.stdout:4/384: mkdir d7/df/d1f/d28/d31/d7b 0 2024-05-09T06:38:08.834 INFO:tasks.workunit.client.1.smithi193.stdout:7/82: rmdir d6 39 2024-05-09T06:38:08.836 INFO:tasks.workunit.client.1.smithi193.stdout:6/87: rename d1/lc to d1/l14 0 2024-05-09T06:38:08.837 INFO:tasks.workunit.client.1.smithi193.stdout:1/33: mknod d4/c8 0 2024-05-09T06:38:08.839 INFO:tasks.workunit.client.0.smithi063.stdout:2/390: mkdir dc/d6e 0 2024-05-09T06:38:08.840 INFO:tasks.workunit.client.1.smithi193.stdout:4/38: creat fd x:0 0 0 2024-05-09T06:38:08.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/368: rename d0/d15/d49/f74 to d0/d15/d4a/d68/f75 0 2024-05-09T06:38:08.843 INFO:tasks.workunit.client.1.smithi193.stdout:1/34: fsync f2 0 2024-05-09T06:38:08.845 INFO:tasks.workunit.client.1.smithi193.stdout:3/12: dread f0 [0,4194304] 0 2024-05-09T06:38:08.845 INFO:tasks.workunit.client.1.smithi193.stdout:1/35: chown d4/l5 729 1 2024-05-09T06:38:08.846 INFO:tasks.workunit.client.1.smithi193.stdout:9/86: mkdir dd/d1e/d1f 0 2024-05-09T06:38:08.846 INFO:tasks.workunit.client.1.smithi193.stdout:3/13: read f0 [198768,40933] 0 2024-05-09T06:38:08.847 INFO:tasks.workunit.client.1.smithi193.stdout:9/87: fsync f0 0 2024-05-09T06:38:08.847 INFO:tasks.workunit.client.1.smithi193.stdout:1/36: write f2 [3918843,130048] 0 2024-05-09T06:38:08.849 INFO:tasks.workunit.client.1.smithi193.stdout:2/41: mkdir db/df 0 2024-05-09T06:38:08.849 INFO:tasks.workunit.client.1.smithi193.stdout:9/88: fsync dd/f13 0 2024-05-09T06:38:08.850 INFO:tasks.workunit.client.1.smithi193.stdout:3/14: dread f0 [0,4194304] 0 2024-05-09T06:38:08.850 INFO:tasks.workunit.client.1.smithi193.stdout:4/39: dwrite fa [0,4194304] 0 2024-05-09T06:38:08.855 INFO:tasks.workunit.client.0.smithi063.stdout:0/424: rename dd/d1a/d2c/d42/c5a to dd/c8c 0 2024-05-09T06:38:08.856 INFO:tasks.workunit.client.1.smithi193.stdout:2/42: dread f1 [0,4194304] 0 2024-05-09T06:38:08.856 INFO:tasks.workunit.client.1.smithi193.stdout:1/37: dread f2 [0,4194304] 0 2024-05-09T06:38:08.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/369: creat d0/d15/f76 x:0 0 0 2024-05-09T06:38:08.861 INFO:tasks.workunit.client.0.smithi063.stdout:8/443: getdents d9/d3b 0 2024-05-09T06:38:08.862 INFO:tasks.workunit.client.0.smithi063.stdout:1/341: truncate d9/f23 3887638 0 2024-05-09T06:38:08.862 INFO:tasks.workunit.client.0.smithi063.stdout:9/397: getdents d0/d4/d26/d39/d3e 0 2024-05-09T06:38:08.866 INFO:tasks.workunit.client.0.smithi063.stdout:7/370: dwrite d0/d15/d49/f70 [0,4194304] 0 2024-05-09T06:38:08.868 INFO:tasks.workunit.client.0.smithi063.stdout:6/365: rename d2/d12/l19 to d2/d20/d4a/l77 0 2024-05-09T06:38:08.868 INFO:tasks.workunit.client.0.smithi063.stdout:7/371: write d0/d15/d4a/d50/f3c [2266247,32344] 0 2024-05-09T06:38:08.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:08 smithi193 ceph-mon[35141]: pgmap v7: 65 pgs: 65 active+clean; 750 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 120 MiB/s wr, 256 op/s 2024-05-09T06:38:08.873 INFO:tasks.workunit.client.0.smithi063.stdout:6/366: write d2/d12/f53 [380742,43224] 0 2024-05-09T06:38:08.873 INFO:tasks.workunit.client.0.smithi063.stdout:7/372: read d0/d15/d49/f5f [3316809,94638] 0 2024-05-09T06:38:08.873 INFO:tasks.workunit.client.0.smithi063.stdout:6/367: fdatasync d2/d12/f34 0 2024-05-09T06:38:08.873 INFO:tasks.workunit.client.0.smithi063.stdout:7/373: chown d0/l5a 9663625 1 2024-05-09T06:38:08.874 INFO:tasks.workunit.client.0.smithi063.stdout:6/368: stat d2/d12/f76 0 2024-05-09T06:38:08.874 INFO:tasks.workunit.client.0.smithi063.stdout:8/444: creat d9/da/d18/d56/d58/d79/f96 x:0 0 0 2024-05-09T06:38:08.875 INFO:tasks.workunit.client.0.smithi063.stdout:9/398: mkdir d0/d4/d7d/d8b 0 2024-05-09T06:38:08.877 INFO:tasks.workunit.client.1.smithi193.stdout:4/40: mknod ce 0 2024-05-09T06:38:08.878 INFO:tasks.workunit.client.1.smithi193.stdout:4/41: chown f6 139984 1 2024-05-09T06:38:08.879 INFO:tasks.workunit.client.1.smithi193.stdout:4/42: fsync f6 0 2024-05-09T06:38:08.879 INFO:tasks.workunit.client.1.smithi193.stdout:4/43: chown ce 5457928 1 2024-05-09T06:38:08.879 INFO:tasks.workunit.client.1.smithi193.stdout:9/89: symlink dd/d1e/l20 0 2024-05-09T06:38:08.882 INFO:tasks.workunit.client.1.smithi193.stdout:2/43: rmdir db 39 2024-05-09T06:38:08.882 INFO:tasks.workunit.client.1.smithi193.stdout:2/44: stat f1 0 2024-05-09T06:38:08.885 INFO:tasks.workunit.client.1.smithi193.stdout:1/38: creat d4/f9 x:0 0 0 2024-05-09T06:38:08.887 INFO:tasks.workunit.client.1.smithi193.stdout:7/83: chown d6/d8/fa 411903 1 2024-05-09T06:38:08.887 INFO:tasks.workunit.client.1.smithi193.stdout:7/84: fsync d6/f18 0 2024-05-09T06:38:08.890 INFO:tasks.workunit.client.1.smithi193.stdout:6/88: symlink d1/l15 0 2024-05-09T06:38:08.892 INFO:tasks.workunit.client.0.smithi063.stdout:7/374: mknod d0/d15/d49/d4f/d66/c77 0 2024-05-09T06:38:08.897 INFO:tasks.workunit.client.1.smithi193.stdout:3/15: link f0 d1/f4 0 2024-05-09T06:38:08.898 INFO:tasks.workunit.client.1.smithi193.stdout:3/16: dread d1/f4 [0,4194304] 0 2024-05-09T06:38:08.899 INFO:tasks.workunit.client.0.smithi063.stdout:9/399: unlink d0/d4/da/f3f 0 2024-05-09T06:38:08.899 INFO:tasks.workunit.client.0.smithi063.stdout:8/445: dwrite d9/f13 [4194304,4194304] 0 2024-05-09T06:38:08.900 INFO:tasks.workunit.client.0.smithi063.stdout:1/342: mkdir d9/d2a/d39/d5d/d75 0 2024-05-09T06:38:08.904 INFO:tasks.workunit.client.1.smithi193.stdout:4/44: dread fb [0,4194304] 0 2024-05-09T06:38:08.905 INFO:tasks.workunit.client.1.smithi193.stdout:2/45: chown db/df 733259358 1 2024-05-09T06:38:08.907 INFO:tasks.workunit.client.1.smithi193.stdout:1/39: stat d4/l6 0 2024-05-09T06:38:08.908 INFO:tasks.workunit.client.0.smithi063.stdout:2/391: write dc/dd/d2d/f38 [2690719,78229] 0 2024-05-09T06:38:08.909 INFO:tasks.workunit.client.0.smithi063.stdout:8/446: creat d9/da/d18/d46/f97 x:0 0 0 2024-05-09T06:38:08.910 INFO:tasks.workunit.client.1.smithi193.stdout:7/85: rename d6/d17/f12 to d6/d17/f19 0 2024-05-09T06:38:08.915 INFO:tasks.workunit.client.0.smithi063.stdout:0/425: rename dd/cf to dd/d60/d63/c8d 0 2024-05-09T06:38:08.916 INFO:tasks.workunit.client.0.smithi063.stdout:8/447: read d9/d37/f3a [744199,105046] 0 2024-05-09T06:38:08.917 INFO:tasks.workunit.client.0.smithi063.stdout:2/392: mknod dc/d32/c6f 0 2024-05-09T06:38:08.918 INFO:tasks.workunit.client.0.smithi063.stdout:8/448: fsync d9/da/d18/d56/f76 0 2024-05-09T06:38:08.918 INFO:tasks.workunit.client.1.smithi193.stdout:7/86: dwrite d6/f16 [0,4194304] 0 2024-05-09T06:38:08.918 INFO:tasks.workunit.client.1.smithi193.stdout:5/12: sync 2024-05-09T06:38:08.923 INFO:tasks.workunit.client.0.smithi063.stdout:1/343: truncate d9/d2a/d2e/f50 1839629 0 2024-05-09T06:38:08.923 INFO:tasks.workunit.client.1.smithi193.stdout:9/90: creat dd/d1e/d1f/f21 x:0 0 0 2024-05-09T06:38:08.925 INFO:tasks.workunit.client.0.smithi063.stdout:0/426: symlink dd/l8e 0 2024-05-09T06:38:08.926 INFO:tasks.workunit.client.1.smithi193.stdout:4/45: rename ce to cf 0 2024-05-09T06:38:08.928 INFO:tasks.workunit.client.0.smithi063.stdout:2/393: creat dc/d36/d43/d46/f70 x:0 0 0 2024-05-09T06:38:08.929 INFO:tasks.workunit.client.0.smithi063.stdout:8/449: mknod d9/da/d53/c98 0 2024-05-09T06:38:08.930 INFO:tasks.workunit.client.1.smithi193.stdout:1/40: mkdir d4/da 0 2024-05-09T06:38:08.931 INFO:tasks.workunit.client.1.smithi193.stdout:1/41: write d4/f9 [153514,21667] 0 2024-05-09T06:38:08.932 INFO:tasks.workunit.client.1.smithi193.stdout:1/42: chown d4/l6 2340 1 2024-05-09T06:38:08.932 INFO:tasks.workunit.client.1.smithi193.stdout:1/43: chown d4/l5 56265 1 2024-05-09T06:38:08.935 INFO:tasks.workunit.client.0.smithi063.stdout:0/427: fdatasync dd/d1a/f35 0 2024-05-09T06:38:08.936 INFO:tasks.workunit.client.1.smithi193.stdout:7/87: mkdir d6/d8/d1a 0 2024-05-09T06:38:08.936 INFO:tasks.workunit.client.1.smithi193.stdout:7/88: stat f5 0 2024-05-09T06:38:08.939 INFO:tasks.workunit.client.1.smithi193.stdout:5/13: creat d0/d1/f3 x:0 0 0 2024-05-09T06:38:08.942 INFO:tasks.workunit.client.1.smithi193.stdout:5/14: truncate d0/d1/f3 581575 0 2024-05-09T06:38:08.942 INFO:tasks.workunit.client.1.smithi193.stdout:5/15: stat d0 0 2024-05-09T06:38:08.943 INFO:tasks.workunit.client.1.smithi193.stdout:9/91: creat dd/d1e/d1f/f22 x:0 0 0 2024-05-09T06:38:08.944 INFO:tasks.workunit.client.1.smithi193.stdout:9/92: readlink l4 0 2024-05-09T06:38:08.944 INFO:tasks.workunit.client.1.smithi193.stdout:7/89: dwrite d6/dd/ff [4194304,4194304] 0 2024-05-09T06:38:08.945 INFO:tasks.workunit.client.0.smithi063.stdout:8/450: mkdir d9/d3b/d99 0 2024-05-09T06:38:08.948 INFO:tasks.workunit.client.1.smithi193.stdout:4/46: rename fd to f10 0 2024-05-09T06:38:08.950 INFO:tasks.workunit.client.1.smithi193.stdout:4/47: write f7 [625354,27607] 0 2024-05-09T06:38:08.953 INFO:tasks.workunit.client.0.smithi063.stdout:1/344: mkdir d9/d76 0 2024-05-09T06:38:08.954 INFO:tasks.workunit.client.1.smithi193.stdout:2/46: creat db/df/f10 x:0 0 0 2024-05-09T06:38:08.954 INFO:tasks.workunit.client.1.smithi193.stdout:5/16: dwrite d0/d1/f3 [0,4194304] 0 2024-05-09T06:38:08.954 INFO:tasks.workunit.client.1.smithi193.stdout:2/47: truncate f8 1630202 0 2024-05-09T06:38:08.957 INFO:tasks.workunit.client.1.smithi193.stdout:6/89: creat d1/f16 x:0 0 0 2024-05-09T06:38:08.966 INFO:tasks.workunit.client.1.smithi193.stdout:7/90: creat d6/d8/f1b x:0 0 0 2024-05-09T06:38:08.970 INFO:tasks.workunit.client.1.smithi193.stdout:7/91: dread d6/d8/f9 [0,4194304] 0 2024-05-09T06:38:08.970 INFO:tasks.workunit.client.1.smithi193.stdout:4/48: creat f11 x:0 0 0 2024-05-09T06:38:08.971 INFO:tasks.workunit.client.1.smithi193.stdout:1/44: creat d4/da/fb x:0 0 0 2024-05-09T06:38:08.972 INFO:tasks.workunit.client.1.smithi193.stdout:1/45: fdatasync d4/da/fb 0 2024-05-09T06:38:08.973 INFO:tasks.workunit.client.1.smithi193.stdout:8/34: sync 2024-05-09T06:38:08.973 INFO:tasks.workunit.client.1.smithi193.stdout:0/129: sync 2024-05-09T06:38:08.976 INFO:tasks.workunit.client.1.smithi193.stdout:0/130: read - d18/f28 zero size 2024-05-09T06:38:08.976 INFO:tasks.workunit.client.1.smithi193.stdout:3/17: getdents d1 0 2024-05-09T06:38:08.976 INFO:tasks.workunit.client.0.smithi063.stdout:0/428: mkdir dd/d60/d63/d23/d33/d8f 0 2024-05-09T06:38:08.976 INFO:tasks.workunit.client.0.smithi063.stdout:2/394: symlink dc/dd/d45/d58/l71 0 2024-05-09T06:38:08.979 INFO:tasks.workunit.client.0.smithi063.stdout:0/429: dread dd/d60/d63/f64 [0,4194304] 0 2024-05-09T06:38:08.979 INFO:tasks.workunit.client.1.smithi193.stdout:4/49: dwrite f7 [0,4194304] 0 2024-05-09T06:38:08.981 INFO:tasks.workunit.client.1.smithi193.stdout:7/92: mkdir d6/d8/d1c 0 2024-05-09T06:38:08.988 INFO:tasks.workunit.client.0.smithi063.stdout:6/369: dread d2/d1f/d29/d3f/f58 [0,4194304] 0 2024-05-09T06:38:08.990 INFO:tasks.workunit.client.1.smithi193.stdout:6/90: getdents d1/de/d11 0 2024-05-09T06:38:08.991 INFO:tasks.workunit.client.0.smithi063.stdout:0/430: mkdir dd/d60/d44/d90 0 2024-05-09T06:38:08.992 INFO:tasks.workunit.client.1.smithi193.stdout:8/35: mknod d6/ca 0 2024-05-09T06:38:08.994 INFO:tasks.workunit.client.0.smithi063.stdout:2/395: mkdir dc/dd/d2d/d60/d65/d72 0 2024-05-09T06:38:08.995 INFO:tasks.workunit.client.0.smithi063.stdout:2/396: write dc/dd/f23 [728267,80050] 0 2024-05-09T06:38:08.995 INFO:tasks.workunit.client.1.smithi193.stdout:8/36: dread f0 [4194304,4194304] 0 2024-05-09T06:38:08.996 INFO:tasks.workunit.client.1.smithi193.stdout:3/18: mknod d1/c5 0 2024-05-09T06:38:08.996 INFO:tasks.workunit.client.0.smithi063.stdout:2/397: write dc/d32/d33/f61 [1030900,76516] 0 2024-05-09T06:38:08.997 INFO:tasks.workunit.client.0.smithi063.stdout:6/370: mknod d2/d1f/d29/d3b/d4e/c78 0 2024-05-09T06:38:08.998 INFO:tasks.workunit.client.0.smithi063.stdout:6/371: write d2/d1f/d29/f2a [1207694,21294] 0 2024-05-09T06:38:08.999 INFO:tasks.workunit.client.0.smithi063.stdout:6/372: chown d2/d1f/d29/d3b/d4e/c78 61045203 1 2024-05-09T06:38:08.999 INFO:tasks.workunit.client.0.smithi063.stdout:0/431: mkdir dd/d1a/d2c/d51/d72/d91 0 2024-05-09T06:38:09.002 INFO:tasks.workunit.client.0.smithi063.stdout:2/398: mknod dc/dd/d45/c73 0 2024-05-09T06:38:09.004 INFO:tasks.workunit.client.0.smithi063.stdout:8/451: write d9/f14 [1570063,12147] 0 2024-05-09T06:38:09.005 INFO:tasks.workunit.client.1.smithi193.stdout:2/48: truncate f9 320628 0 2024-05-09T06:38:09.005 INFO:tasks.workunit.client.1.smithi193.stdout:5/17: truncate d0/d1/f3 2756282 0 2024-05-09T06:38:09.005 INFO:tasks.workunit.client.0.smithi063.stdout:8/452: fdatasync d9/da/d18/d56/f65 0 2024-05-09T06:38:09.006 INFO:tasks.workunit.client.0.smithi063.stdout:6/373: creat d2/d4/d2d/f79 x:0 0 0 2024-05-09T06:38:09.007 INFO:tasks.workunit.client.0.smithi063.stdout:8/453: chown d9/da/d18/f1d 60301176 1 2024-05-09T06:38:09.008 INFO:tasks.workunit.client.0.smithi063.stdout:0/432: creat dd/d1a/d2c/f92 x:0 0 0 2024-05-09T06:38:09.009 INFO:tasks.workunit.client.0.smithi063.stdout:2/399: dwrite dc/f2a [4194304,4194304] 0 2024-05-09T06:38:09.010 INFO:tasks.workunit.client.0.smithi063.stdout:2/400: stat dc/dd/d45/d58/l71 0 2024-05-09T06:38:09.011 INFO:tasks.workunit.client.1.smithi193.stdout:9/93: getdents dd/df 0 2024-05-09T06:38:09.012 INFO:tasks.workunit.client.1.smithi193.stdout:1/46: dwrite f2 [4194304,4194304] 0 2024-05-09T06:38:09.013 INFO:tasks.workunit.client.1.smithi193.stdout:1/47: chown d4 43 1 2024-05-09T06:38:09.014 INFO:tasks.workunit.client.1.smithi193.stdout:0/131: mknod d18/c2a 0 2024-05-09T06:38:09.014 INFO:tasks.workunit.client.1.smithi193.stdout:1/48: write f2 [1083233,114843] 0 2024-05-09T06:38:09.015 INFO:tasks.workunit.client.0.smithi063.stdout:6/374: chown d2/c75 5301351 1 2024-05-09T06:38:09.017 INFO:tasks.workunit.client.0.smithi063.stdout:2/401: dwrite dc/f1f [0,4194304] 0 2024-05-09T06:38:09.017 INFO:tasks.workunit.client.1.smithi193.stdout:8/37: mkdir d6/db 0 2024-05-09T06:38:09.019 INFO:tasks.workunit.client.0.smithi063.stdout:8/454: rmdir d9/da/d18/d24/d71 39 2024-05-09T06:38:09.019 INFO:tasks.workunit.client.0.smithi063.stdout:2/402: read dc/f27 [1257303,41661] 0 2024-05-09T06:38:09.021 INFO:tasks.workunit.client.1.smithi193.stdout:3/19: rename d1/c5 to d1/c6 0 2024-05-09T06:38:09.023 INFO:tasks.workunit.client.0.smithi063.stdout:0/433: readlink dd/d60/d63/d23/l45 0 2024-05-09T06:38:09.025 INFO:tasks.workunit.client.0.smithi063.stdout:6/375: symlink d2/d20/d26/l7a 0 2024-05-09T06:38:09.030 INFO:tasks.workunit.client.1.smithi193.stdout:7/93: symlink d6/l1d 0 2024-05-09T06:38:09.030 INFO:tasks.workunit.client.1.smithi193.stdout:0/132: dwrite f9 [0,4194304] 0 2024-05-09T06:38:09.030 INFO:tasks.workunit.client.1.smithi193.stdout:4/50: dwrite f5 [4194304,4194304] 0 2024-05-09T06:38:09.031 INFO:tasks.workunit.client.1.smithi193.stdout:2/49: symlink db/df/l11 0 2024-05-09T06:38:09.037 INFO:tasks.workunit.client.0.smithi063.stdout:6/376: stat d2/d20/c65 0 2024-05-09T06:38:09.039 INFO:tasks.workunit.client.0.smithi063.stdout:3/368: sync 2024-05-09T06:38:09.041 INFO:tasks.workunit.client.1.smithi193.stdout:9/94: mknod dd/d1e/d1f/c23 0 2024-05-09T06:38:09.042 INFO:tasks.workunit.client.1.smithi193.stdout:9/95: write dd/f13 [1964970,67598] 0 2024-05-09T06:38:09.045 INFO:tasks.workunit.client.0.smithi063.stdout:0/434: truncate dd/d60/d63/d23/f2f 695741 0 2024-05-09T06:38:09.046 INFO:tasks.workunit.client.1.smithi193.stdout:1/49: creat d4/fc x:0 0 0 2024-05-09T06:38:09.047 INFO:tasks.workunit.client.1.smithi193.stdout:1/50: chown d4/c7 304 1 2024-05-09T06:38:09.048 INFO:tasks.workunit.client.1.smithi193.stdout:8/38: fsync d6/f7 0 2024-05-09T06:38:09.049 INFO:tasks.workunit.client.0.smithi063.stdout:3/369: symlink d7/d4c/d31/d35/l7d 0 2024-05-09T06:38:09.050 INFO:tasks.workunit.client.0.smithi063.stdout:8/455: rmdir d9/d3b/d99 0 2024-05-09T06:38:09.052 INFO:tasks.workunit.client.0.smithi063.stdout:8/456: write d9/da/d34/f59 [5627202,80842] 0 2024-05-09T06:38:09.053 INFO:tasks.workunit.client.0.smithi063.stdout:8/457: write d9/d3b/f85 [109081,122964] 0 2024-05-09T06:38:09.053 INFO:tasks.workunit.client.0.smithi063.stdout:8/458: readlink d9/da/l3d 0 2024-05-09T06:38:09.054 INFO:tasks.workunit.client.0.smithi063.stdout:8/459: write d9/f7d [3908518,26589] 0 2024-05-09T06:38:09.054 INFO:tasks.workunit.client.1.smithi193.stdout:7/94: unlink f5 0 2024-05-09T06:38:09.056 INFO:tasks.workunit.client.0.smithi063.stdout:0/435: rename dd/d1a/f1b to dd/d60/d63/d23/d33/d8f/f93 0 2024-05-09T06:38:09.065 INFO:tasks.workunit.client.1.smithi193.stdout:2/50: mkdir db/df/d12 0 2024-05-09T06:38:09.065 INFO:tasks.workunit.client.1.smithi193.stdout:5/18: unlink d0/d1/f3 0 2024-05-09T06:38:09.068 INFO:tasks.workunit.client.0.smithi063.stdout:8/460: creat d9/da/f9a x:0 0 0 2024-05-09T06:38:09.069 INFO:tasks.workunit.client.0.smithi063.stdout:6/377: truncate d2/d1f/d29/f63 1460525 0 2024-05-09T06:38:09.071 INFO:tasks.workunit.client.0.smithi063.stdout:3/370: rename d7/d4c/d31/d35 to d7/da/d2b/d66/d74/d7e 0 2024-05-09T06:38:09.073 INFO:tasks.workunit.client.0.smithi063.stdout:0/436: readlink dd/d1a/l20 0 2024-05-09T06:38:09.075 INFO:tasks.workunit.client.1.smithi193.stdout:9/96: dread dd/fe [0,4194304] 0 2024-05-09T06:38:09.076 INFO:tasks.workunit.client.1.smithi193.stdout:3/20: dwrite f0 [0,4194304] 0 2024-05-09T06:38:09.078 INFO:tasks.workunit.client.0.smithi063.stdout:6/378: creat d2/d20/f7b x:0 0 0 2024-05-09T06:38:09.082 INFO:tasks.workunit.client.0.smithi063.stdout:3/371: creat d7/da/d2b/d66/d74/d7e/f7f x:0 0 0 2024-05-09T06:38:09.082 INFO:tasks.workunit.client.1.smithi193.stdout:0/133: creat d18/d29/f2b x:0 0 0 2024-05-09T06:38:09.082 INFO:tasks.workunit.client.0.smithi063.stdout:3/372: stat d7/da/d21/d30/d44/l7c 0 2024-05-09T06:38:09.083 INFO:tasks.workunit.client.1.smithi193.stdout:0/134: fsync d18/d1d/f23 0 2024-05-09T06:38:09.083 INFO:tasks.workunit.client.0.smithi063.stdout:0/437: mknod dd/d60/d63/d23/d31/c94 0 2024-05-09T06:38:09.084 INFO:tasks.workunit.client.0.smithi063.stdout:0/438: stat dd/d60/l5d 0 2024-05-09T06:38:09.085 INFO:tasks.workunit.client.0.smithi063.stdout:3/373: truncate d7/da/d2b/f64 939577 0 2024-05-09T06:38:09.086 INFO:tasks.workunit.client.0.smithi063.stdout:3/374: readlink d7/da/l19 0 2024-05-09T06:38:09.086 INFO:tasks.workunit.client.0.smithi063.stdout:3/375: write d7/da/fe [1947877,102018] 0 2024-05-09T06:38:09.089 INFO:tasks.workunit.client.0.smithi063.stdout:3/376: dread d7/f16 [0,4194304] 0 2024-05-09T06:38:09.090 INFO:tasks.workunit.client.0.smithi063.stdout:6/379: rename d2/d4/d2d to d2/d4/d7c 0 2024-05-09T06:38:09.091 INFO:tasks.workunit.client.0.smithi063.stdout:3/377: dread - d7/da/d21/d30/d44/d48/f6f zero size 2024-05-09T06:38:09.091 INFO:tasks.workunit.client.0.smithi063.stdout:6/380: write d2/d12/f76 [329431,21527] 0 2024-05-09T06:38:09.092 INFO:tasks.workunit.client.1.smithi193.stdout:5/19: symlink d0/d1/l4 0 2024-05-09T06:38:09.092 INFO:tasks.workunit.client.0.smithi063.stdout:3/378: chown d7/da/d2b/f64 629699623 1 2024-05-09T06:38:09.093 INFO:tasks.workunit.client.0.smithi063.stdout:3/379: stat d7/f17 0 2024-05-09T06:38:09.101 INFO:tasks.workunit.client.1.smithi193.stdout:9/97: fdatasync dd/fe 0 2024-05-09T06:38:09.102 INFO:tasks.workunit.client.1.smithi193.stdout:9/98: chown dd/df/d15 245205609 1 2024-05-09T06:38:09.103 INFO:tasks.workunit.client.0.smithi063.stdout:0/439: dread dd/d60/d63/f55 [0,4194304] 0 2024-05-09T06:38:09.106 INFO:tasks.workunit.client.1.smithi193.stdout:4/51: truncate f5 6979199 0 2024-05-09T06:38:09.106 INFO:tasks.workunit.client.0.smithi063.stdout:3/380: link d7/da/d21/f6b d7/d4c/f80 0 2024-05-09T06:38:09.107 INFO:tasks.workunit.client.1.smithi193.stdout:4/52: write f11 [830750,98045] 0 2024-05-09T06:38:09.107 INFO:tasks.workunit.client.1.smithi193.stdout:7/95: mknod d6/d8/d1a/c1e 0 2024-05-09T06:38:09.113 INFO:tasks.workunit.client.0.smithi063.stdout:8/461: write d9/da/d51/f8b [21677,69476] 0 2024-05-09T06:38:09.113 INFO:tasks.workunit.client.1.smithi193.stdout:5/20: rmdir d0 39 2024-05-09T06:38:09.114 INFO:tasks.workunit.client.1.smithi193.stdout:9/99: dwrite dd/df/f1d [0,4194304] 0 2024-05-09T06:38:09.115 INFO:tasks.workunit.client.0.smithi063.stdout:6/381: getdents d2/d20/d26 0 2024-05-09T06:38:09.116 INFO:tasks.workunit.client.1.smithi193.stdout:1/51: link d4/l6 d4/da/ld 0 2024-05-09T06:38:09.116 INFO:tasks.workunit.client.0.smithi063.stdout:6/382: fsync d2/d12/f34 0 2024-05-09T06:38:09.117 INFO:tasks.workunit.client.1.smithi193.stdout:8/39: truncate d6/f7 1798155 0 2024-05-09T06:38:09.117 INFO:tasks.workunit.client.1.smithi193.stdout:9/100: dread - dd/d1e/d1f/f21 zero size 2024-05-09T06:38:09.118 INFO:tasks.workunit.client.0.smithi063.stdout:0/440: truncate dd/d1a/f2e 867759 0 2024-05-09T06:38:09.119 INFO:tasks.workunit.client.1.smithi193.stdout:2/51: dwrite f1 [4194304,4194304] 0 2024-05-09T06:38:09.119 INFO:tasks.workunit.client.0.smithi063.stdout:3/381: dwrite d7/da/d21/f2c [0,4194304] 0 2024-05-09T06:38:09.119 INFO:tasks.workunit.client.0.smithi063.stdout:8/462: fdatasync d9/da/f4e 0 2024-05-09T06:38:09.120 INFO:tasks.workunit.client.0.smithi063.stdout:0/441: dread - dd/d1a/d50/f7a zero size 2024-05-09T06:38:09.123 INFO:tasks.workunit.client.1.smithi193.stdout:9/101: dread dd/f13 [0,4194304] 0 2024-05-09T06:38:09.125 INFO:tasks.workunit.client.0.smithi063.stdout:3/382: write f5 [5320923,64625] 0 2024-05-09T06:38:09.125 INFO:tasks.workunit.client.1.smithi193.stdout:0/135: symlink d18/l2c 0 2024-05-09T06:38:09.127 INFO:tasks.workunit.client.1.smithi193.stdout:0/136: dread d18/d1d/f24 [0,4194304] 0 2024-05-09T06:38:09.128 INFO:tasks.workunit.client.1.smithi193.stdout:0/137: write f9 [2208200,90245] 0 2024-05-09T06:38:09.130 INFO:tasks.workunit.client.0.smithi063.stdout:0/442: rename dd/c13 to dd/d1a/d2c/d42/c95 0 2024-05-09T06:38:09.137 INFO:tasks.workunit.client.0.smithi063.stdout:8/463: creat d9/da/d18/d46/f9b x:0 0 0 2024-05-09T06:38:09.138 INFO:tasks.workunit.client.1.smithi193.stdout:1/52: mkdir d4/da/de 0 2024-05-09T06:38:09.139 INFO:tasks.workunit.client.0.smithi063.stdout:3/383: creat d7/da/d21/d30/d44/d48/f81 x:0 0 0 2024-05-09T06:38:09.141 INFO:tasks.workunit.client.0.smithi063.stdout:6/383: dread d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:09.141 INFO:tasks.workunit.client.0.smithi063.stdout:0/443: mknod dd/d60/d63/d23/d89/c96 0 2024-05-09T06:38:09.142 INFO:tasks.workunit.client.0.smithi063.stdout:0/444: read dd/d60/d63/f2b [1634435,49566] 0 2024-05-09T06:38:09.144 INFO:tasks.workunit.client.0.smithi063.stdout:3/384: symlink d7/da/d21/d30/d44/d48/l82 0 2024-05-09T06:38:09.145 INFO:tasks.workunit.client.0.smithi063.stdout:3/385: chown d7/d22/d28 77 1 2024-05-09T06:38:09.145 INFO:tasks.workunit.client.1.smithi193.stdout:7/96: write d6/d8/f1b [870433,16148] 0 2024-05-09T06:38:09.146 INFO:tasks.workunit.client.1.smithi193.stdout:7/97: chown d6/f18 41 1 2024-05-09T06:38:09.147 INFO:tasks.workunit.client.1.smithi193.stdout:4/53: link f6 f12 0 2024-05-09T06:38:09.147 INFO:tasks.workunit.client.0.smithi063.stdout:3/386: fdatasync d7/da/d21/f2c 0 2024-05-09T06:38:09.148 INFO:tasks.workunit.client.0.smithi063.stdout:6/384: dwrite d2/df/f5d [0,4194304] 0 2024-05-09T06:38:09.148 INFO:tasks.workunit.client.1.smithi193.stdout:7/98: write d6/d8/f1b [1415136,1212] 0 2024-05-09T06:38:09.149 INFO:tasks.workunit.client.0.smithi063.stdout:3/387: chown d7/d22/f50 24232846 1 2024-05-09T06:38:09.150 INFO:tasks.workunit.client.0.smithi063.stdout:3/388: dread - d7/da/d21/f6b zero size 2024-05-09T06:38:09.150 INFO:tasks.workunit.client.0.smithi063.stdout:8/464: fdatasync d9/da/fb 0 2024-05-09T06:38:09.150 INFO:tasks.workunit.client.1.smithi193.stdout:7/99: write d6/dd/ff [1398552,92290] 0 2024-05-09T06:38:09.151 INFO:tasks.workunit.client.0.smithi063.stdout:5/350: sync 2024-05-09T06:38:09.152 INFO:tasks.workunit.client.0.smithi063.stdout:0/445: stat dd/d1a/d2c/c70 0 2024-05-09T06:38:09.152 INFO:tasks.workunit.client.1.smithi193.stdout:5/21: mknod d0/c5 0 2024-05-09T06:38:09.154 INFO:tasks.workunit.client.0.smithi063.stdout:8/465: truncate d9/da/d18/d56/d8f/f89 618394 0 2024-05-09T06:38:09.158 INFO:tasks.workunit.client.1.smithi193.stdout:0/138: creat d18/d1d/f2d x:0 0 0 2024-05-09T06:38:09.158 INFO:tasks.workunit.client.0.smithi063.stdout:6/385: creat d2/d1f/d6c/f7d x:0 0 0 2024-05-09T06:38:09.158 INFO:tasks.workunit.client.1.smithi193.stdout:0/139: write f17 [701659,94354] 0 2024-05-09T06:38:09.159 INFO:tasks.workunit.client.1.smithi193.stdout:8/40: fdatasync d6/f7 0 2024-05-09T06:38:09.159 INFO:tasks.workunit.client.1.smithi193.stdout:0/140: chown f13 96251 1 2024-05-09T06:38:09.160 INFO:tasks.workunit.client.1.smithi193.stdout:0/141: write f13 [1038613,21300] 0 2024-05-09T06:38:09.164 INFO:tasks.workunit.client.1.smithi193.stdout:1/53: creat d4/ff x:0 0 0 2024-05-09T06:38:09.165 INFO:tasks.workunit.client.0.smithi063.stdout:5/351: mknod d8/d19/c7c 0 2024-05-09T06:38:09.167 INFO:tasks.workunit.client.1.smithi193.stdout:3/21: getdents d1 0 2024-05-09T06:38:09.169 INFO:tasks.workunit.client.0.smithi063.stdout:3/389: rename d7/f10 to d7/d2a/d5b/f83 0 2024-05-09T06:38:09.170 INFO:tasks.workunit.client.1.smithi193.stdout:2/52: creat db/df/d12/f13 x:0 0 0 2024-05-09T06:38:09.171 INFO:tasks.workunit.client.0.smithi063.stdout:3/390: dread f1 [0,4194304] 0 2024-05-09T06:38:09.171 INFO:tasks.workunit.client.1.smithi193.stdout:4/54: creat f13 x:0 0 0 2024-05-09T06:38:09.172 INFO:tasks.workunit.client.0.smithi063.stdout:6/386: read d2/df/f1d [119770,116939] 0 2024-05-09T06:38:09.174 INFO:tasks.workunit.client.0.smithi063.stdout:5/352: chown d8/d17/c70 1 1 2024-05-09T06:38:09.174 INFO:tasks.workunit.client.0.smithi063.stdout:6/387: chown d2/d1f/d29/f47 15 1 2024-05-09T06:38:09.175 INFO:tasks.workunit.client.1.smithi193.stdout:9/102: rename c7 to dd/d1e/c24 0 2024-05-09T06:38:09.177 INFO:tasks.workunit.client.0.smithi063.stdout:5/353: dread d8/d17/f56 [0,4194304] 0 2024-05-09T06:38:09.178 INFO:tasks.workunit.client.0.smithi063.stdout:3/391: dwrite d7/f1b [4194304,4194304] 0 2024-05-09T06:38:09.180 INFO:tasks.workunit.client.0.smithi063.stdout:5/354: write d8/d19/d30/d46/f62 [361095,5434] 0 2024-05-09T06:38:09.181 INFO:tasks.workunit.client.1.smithi193.stdout:5/22: creat d0/d1/f6 x:0 0 0 2024-05-09T06:38:09.183 INFO:tasks.workunit.client.1.smithi193.stdout:9/103: dwrite dd/df/f10 [0,4194304] 0 2024-05-09T06:38:09.183 INFO:tasks.workunit.client.1.smithi193.stdout:6/91: sync 2024-05-09T06:38:09.184 INFO:tasks.workunit.client.1.smithi193.stdout:8/41: write f0 [1112905,5312] 0 2024-05-09T06:38:09.185 INFO:tasks.workunit.client.1.smithi193.stdout:5/23: write d0/d1/f6 [487791,125813] 0 2024-05-09T06:38:09.185 INFO:tasks.workunit.client.0.smithi063.stdout:6/388: dwrite d2/d1f/d29/d3f/d5c/f70 [0,4194304] 0 2024-05-09T06:38:09.186 INFO:tasks.workunit.client.1.smithi193.stdout:8/42: chown d6/db 391 1 2024-05-09T06:38:09.187 INFO:tasks.workunit.client.0.smithi063.stdout:6/389: write d2/d4/d7c/f79 [65409,63241] 0 2024-05-09T06:38:09.188 INFO:tasks.workunit.client.0.smithi063.stdout:6/390: write d2/d12/f53 [1390891,55510] 0 2024-05-09T06:38:09.194 INFO:tasks.workunit.client.1.smithi193.stdout:1/54: creat d4/da/f10 x:0 0 0 2024-05-09T06:38:09.199 INFO:tasks.workunit.client.1.smithi193.stdout:3/22: symlink d1/l7 0 2024-05-09T06:38:09.203 INFO:tasks.workunit.client.1.smithi193.stdout:5/24: dwrite d0/d1/f6 [0,4194304] 0 2024-05-09T06:38:09.203 INFO:tasks.workunit.client.1.smithi193.stdout:2/53: chown f9 139087462 1 2024-05-09T06:38:09.203 INFO:tasks.workunit.client.1.smithi193.stdout:8/43: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:09.203 INFO:tasks.workunit.client.1.smithi193.stdout:4/55: symlink l14 0 2024-05-09T06:38:09.204 INFO:tasks.workunit.client.0.smithi063.stdout:0/446: dwrite dd/d60/d63/d23/f2f [0,4194304] 0 2024-05-09T06:38:09.204 INFO:tasks.workunit.client.1.smithi193.stdout:8/44: fdatasync d6/f7 0 2024-05-09T06:38:09.205 INFO:tasks.workunit.client.1.smithi193.stdout:5/25: write d0/d1/f6 [1142298,63308] 0 2024-05-09T06:38:09.206 INFO:tasks.workunit.client.1.smithi193.stdout:4/56: chown l2 136841402 1 2024-05-09T06:38:09.208 INFO:tasks.workunit.client.0.smithi063.stdout:6/391: mkdir d2/d4/d7c/d7e 0 2024-05-09T06:38:09.210 INFO:tasks.workunit.client.0.smithi063.stdout:8/466: link d9/f47 d9/da/d18/d46/f9c 0 2024-05-09T06:38:09.212 INFO:tasks.workunit.client.0.smithi063.stdout:4/385: sync 2024-05-09T06:38:09.212 INFO:tasks.workunit.client.0.smithi063.stdout:7/375: sync 2024-05-09T06:38:09.212 INFO:tasks.workunit.client.0.smithi063.stdout:9/400: sync 2024-05-09T06:38:09.212 INFO:tasks.workunit.client.0.smithi063.stdout:1/345: sync 2024-05-09T06:38:09.212 INFO:tasks.workunit.client.0.smithi063.stdout:2/403: sync 2024-05-09T06:38:09.213 INFO:tasks.workunit.client.1.smithi193.stdout:5/26: dread d0/d1/f6 [0,4194304] 0 2024-05-09T06:38:09.213 INFO:tasks.workunit.client.1.smithi193.stdout:6/92: fdatasync d1/f2 0 2024-05-09T06:38:09.213 INFO:tasks.workunit.client.1.smithi193.stdout:6/93: chown d1/de/d11 43 1 2024-05-09T06:38:09.213 INFO:tasks.workunit.client.1.smithi193.stdout:5/27: readlink d0/d1/l4 0 2024-05-09T06:38:09.214 INFO:tasks.workunit.client.0.smithi063.stdout:0/447: dwrite dd/d60/d63/d23/f2f [0,4194304] 0 2024-05-09T06:38:09.215 INFO:tasks.workunit.client.0.smithi063.stdout:5/355: link d8/d19/d30/d46/f5d d8/d65/f7d 0 2024-05-09T06:38:09.215 INFO:tasks.workunit.client.1.smithi193.stdout:9/104: dread dd/f13 [0,4194304] 0 2024-05-09T06:38:09.216 INFO:tasks.workunit.client.1.smithi193.stdout:9/105: chown c9 23935 1 2024-05-09T06:38:09.217 INFO:tasks.workunit.client.1.smithi193.stdout:0/142: symlink d18/d27/l2e 0 2024-05-09T06:38:09.217 INFO:tasks.workunit.client.1.smithi193.stdout:6/94: dread d1/f3 [4194304,4194304] 0 2024-05-09T06:38:09.218 INFO:tasks.workunit.client.0.smithi063.stdout:4/386: write d7/df/d35/d5a/d60/f63 [123960,61815] 0 2024-05-09T06:38:09.219 INFO:tasks.workunit.client.0.smithi063.stdout:6/392: mkdir d2/d20/d26/d7f 0 2024-05-09T06:38:09.219 INFO:tasks.workunit.client.1.smithi193.stdout:1/55: creat d4/f11 x:0 0 0 2024-05-09T06:38:09.220 INFO:tasks.workunit.client.0.smithi063.stdout:6/393: chown d2/d1f/d29/d3f/d5c 3560 1 2024-05-09T06:38:09.220 INFO:tasks.workunit.client.0.smithi063.stdout:0/448: write dd/d60/d63/d23/d31/d58/f6c [4709186,99193] 0 2024-05-09T06:38:09.221 INFO:tasks.workunit.client.1.smithi193.stdout:2/54: creat db/df/d12/f14 x:0 0 0 2024-05-09T06:38:09.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/346: rmdir d9/d46/d57 39 2024-05-09T06:38:09.225 INFO:tasks.workunit.client.0.smithi063.stdout:2/404: symlink dc/d36/d43/d46/l74 0 2024-05-09T06:38:09.225 INFO:tasks.workunit.client.0.smithi063.stdout:6/394: fsync d2/d1f/d29/d3f/d5c/f70 0 2024-05-09T06:38:09.226 INFO:tasks.workunit.client.0.smithi063.stdout:0/449: chown dd/d1a/d2c/d42/l81 288021334 1 2024-05-09T06:38:09.226 INFO:tasks.workunit.client.0.smithi063.stdout:1/347: read - d9/d10/d2b/d3f/d59/d62/f6c zero size 2024-05-09T06:38:09.226 INFO:tasks.workunit.client.1.smithi193.stdout:7/100: rename d6/d8/d1c to d6/d8/d1a/d1f 0 2024-05-09T06:38:09.228 INFO:tasks.workunit.client.0.smithi063.stdout:7/376: rename d0/d15/l63 to d0/d15/d49/d54/l78 0 2024-05-09T06:38:09.228 INFO:tasks.workunit.client.0.smithi063.stdout:7/377: chown d0/d15/d49/d4f/d66/c77 28943 1 2024-05-09T06:38:09.229 INFO:tasks.workunit.client.1.smithi193.stdout:7/101: write d6/dd/ff [9257127,107850] 0 2024-05-09T06:38:09.229 INFO:tasks.workunit.client.0.smithi063.stdout:8/467: symlink d9/da/d18/d24/l9d 0 2024-05-09T06:38:09.230 INFO:tasks.workunit.client.1.smithi193.stdout:2/55: dwrite db/df/f10 [0,4194304] 0 2024-05-09T06:38:09.230 INFO:tasks.workunit.client.1.smithi193.stdout:7/102: readlink d6/l1d 0 2024-05-09T06:38:09.231 INFO:tasks.workunit.client.1.smithi193.stdout:7/103: readlink d6/l11 0 2024-05-09T06:38:09.231 INFO:tasks.workunit.client.0.smithi063.stdout:9/401: symlink d0/d4/d26/d39/d48/d80/d84/l8c 0 2024-05-09T06:38:09.232 INFO:tasks.workunit.client.1.smithi193.stdout:0/143: rmdir d18/d27 39 2024-05-09T06:38:09.232 INFO:tasks.workunit.client.1.smithi193.stdout:0/144: chown f16 47241265 1 2024-05-09T06:38:09.234 INFO:tasks.workunit.client.0.smithi063.stdout:4/387: unlink d7/df/d35/d5a/l5b 0 2024-05-09T06:38:09.236 INFO:tasks.workunit.client.1.smithi193.stdout:6/95: unlink d1/f16 0 2024-05-09T06:38:09.237 INFO:tasks.workunit.client.1.smithi193.stdout:6/96: chown d1/f7 40015191 1 2024-05-09T06:38:09.238 INFO:tasks.workunit.client.1.smithi193.stdout:6/97: read d1/f2 [6238571,46833] 0 2024-05-09T06:38:09.242 INFO:tasks.workunit.client.0.smithi063.stdout:1/348: symlink d9/d10/d2b/d3f/l77 0 2024-05-09T06:38:09.243 INFO:tasks.workunit.client.0.smithi063.stdout:7/378: mknod d0/d15/d49/c79 0 2024-05-09T06:38:09.245 INFO:tasks.workunit.client.0.smithi063.stdout:8/468: creat d9/d3b/f9e x:0 0 0 2024-05-09T06:38:09.245 INFO:tasks.workunit.client.0.smithi063.stdout:7/379: read d0/d15/f2b [1080925,86773] 0 2024-05-09T06:38:09.246 INFO:tasks.workunit.client.0.smithi063.stdout:9/402: creat d0/d4/d26/d39/d3e/f8d x:0 0 0 2024-05-09T06:38:09.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/56: mknod db/df/c15 0 2024-05-09T06:38:09.248 INFO:tasks.workunit.client.1.smithi193.stdout:7/104: symlink d6/d8/d1a/l20 0 2024-05-09T06:38:09.252 INFO:tasks.workunit.client.0.smithi063.stdout:9/403: dwrite d0/d4/f37 [0,4194304] 0 2024-05-09T06:38:09.252 INFO:tasks.workunit.client.0.smithi063.stdout:4/388: mknod d7/df/d1f/d28/d31/d7b/c7c 0 2024-05-09T06:38:09.253 INFO:tasks.workunit.client.1.smithi193.stdout:1/56: creat d4/da/de/f12 x:0 0 0 2024-05-09T06:38:09.253 INFO:tasks.workunit.client.1.smithi193.stdout:1/57: dread - d4/ff zero size 2024-05-09T06:38:09.254 INFO:tasks.workunit.client.1.smithi193.stdout:1/58: fsync d4/ff 0 2024-05-09T06:38:09.254 INFO:tasks.workunit.client.0.smithi063.stdout:7/380: symlink d0/d15/d49/d4f/d66/d67/l7a 0 2024-05-09T06:38:09.255 INFO:tasks.workunit.client.1.smithi193.stdout:1/59: chown d4/f9 2 1 2024-05-09T06:38:09.255 INFO:tasks.workunit.client.1.smithi193.stdout:1/60: chown d4 68293 1 2024-05-09T06:38:09.255 INFO:tasks.workunit.client.1.smithi193.stdout:1/61: dread - d4/fc zero size 2024-05-09T06:38:09.256 INFO:tasks.workunit.client.0.smithi063.stdout:7/381: truncate d0/d15/f35 1190822 0 2024-05-09T06:38:09.256 INFO:tasks.workunit.client.1.smithi193.stdout:1/62: chown d4/da/fb 1026153604 1 2024-05-09T06:38:09.257 INFO:tasks.workunit.client.1.smithi193.stdout:1/63: chown d4/f11 2611830 1 2024-05-09T06:38:09.259 INFO:tasks.workunit.client.0.smithi063.stdout:5/356: dread d8/d17/d18/f58 [0,4194304] 0 2024-05-09T06:38:09.259 INFO:tasks.workunit.client.0.smithi063.stdout:5/357: readlink d8/d35/l52 0 2024-05-09T06:38:09.260 INFO:tasks.workunit.client.1.smithi193.stdout:2/57: mknod db/df/d12/c16 0 2024-05-09T06:38:09.267 INFO:tasks.workunit.client.0.smithi063.stdout:7/382: rename d0/d15/c33 to d0/d15/d49/d4f/d66/c7b 0 2024-05-09T06:38:09.268 INFO:tasks.workunit.client.1.smithi193.stdout:0/145: mknod d18/d27/c2f 0 2024-05-09T06:38:09.269 INFO:tasks.workunit.client.1.smithi193.stdout:0/146: fsync d18/d1d/f23 0 2024-05-09T06:38:09.270 INFO:tasks.workunit.client.0.smithi063.stdout:5/358: truncate d8/d19/f3f 1582291 0 2024-05-09T06:38:09.272 INFO:tasks.workunit.client.1.smithi193.stdout:5/28: getdents d0/d1 0 2024-05-09T06:38:09.273 INFO:tasks.workunit.client.0.smithi063.stdout:9/404: mknod d0/d4/d7d/d8b/c8e 0 2024-05-09T06:38:09.275 INFO:tasks.workunit.client.0.smithi063.stdout:7/383: write d0/d15/d4a/d50/f3f [54332,70478] 0 2024-05-09T06:38:09.276 INFO:tasks.workunit.client.0.smithi063.stdout:7/384: stat d0/d15/d49/d4f/d66/d67 0 2024-05-09T06:38:09.277 INFO:tasks.workunit.client.0.smithi063.stdout:5/359: mkdir d8/d7e 0 2024-05-09T06:38:09.278 INFO:tasks.workunit.client.1.smithi193.stdout:0/147: creat d18/d1d/f30 x:0 0 0 2024-05-09T06:38:09.280 INFO:tasks.workunit.client.0.smithi063.stdout:3/392: truncate d7/d2a/d5b/f83 7529085 0 2024-05-09T06:38:09.282 INFO:tasks.workunit.client.0.smithi063.stdout:3/393: dread - d7/da/d21/d30/d44/d48/f53 zero size 2024-05-09T06:38:09.285 INFO:tasks.workunit.client.1.smithi193.stdout:5/29: creat d0/f7 x:0 0 0 2024-05-09T06:38:09.287 INFO:tasks.workunit.client.1.smithi193.stdout:7/105: truncate d6/d8/fa 3337487 0 2024-05-09T06:38:09.289 INFO:tasks.workunit.client.0.smithi063.stdout:9/405: dwrite d0/d4/d26/d47/f5d [0,4194304] 0 2024-05-09T06:38:09.290 INFO:tasks.workunit.client.1.smithi193.stdout:5/30: creat d0/f8 x:0 0 0 2024-05-09T06:38:09.290 INFO:tasks.workunit.client.1.smithi193.stdout:1/64: fsync d4/f11 0 2024-05-09T06:38:09.291 INFO:tasks.workunit.client.1.smithi193.stdout:0/148: dwrite f9 [4194304,4194304] 0 2024-05-09T06:38:09.292 INFO:tasks.workunit.client.1.smithi193.stdout:1/65: write f2 [6582120,106175] 0 2024-05-09T06:38:09.292 INFO:tasks.workunit.client.1.smithi193.stdout:5/31: truncate d0/f8 305370 0 2024-05-09T06:38:09.292 INFO:tasks.workunit.client.1.smithi193.stdout:7/106: rmdir d6/d8/d1a 39 2024-05-09T06:38:09.293 INFO:tasks.workunit.client.0.smithi063.stdout:9/406: chown d0/l4e 210606 1 2024-05-09T06:38:09.294 INFO:tasks.workunit.client.1.smithi193.stdout:2/58: fsync db/df/d12/f14 0 2024-05-09T06:38:09.294 INFO:tasks.workunit.client.0.smithi063.stdout:9/407: chown d0/d3b/f78 969273193 1 2024-05-09T06:38:09.294 INFO:tasks.workunit.client.1.smithi193.stdout:2/59: chown c0 280722 1 2024-05-09T06:38:09.296 INFO:tasks.workunit.client.1.smithi193.stdout:2/60: fsync f1 0 2024-05-09T06:38:09.296 INFO:tasks.workunit.client.0.smithi063.stdout:5/360: creat d8/d17/d75/f7f x:0 0 0 2024-05-09T06:38:09.297 INFO:tasks.workunit.client.0.smithi063.stdout:3/394: dwrite d7/da/fe [0,4194304] 0 2024-05-09T06:38:09.297 INFO:tasks.workunit.client.1.smithi193.stdout:0/149: chown c14 1523 1 2024-05-09T06:38:09.299 INFO:tasks.workunit.client.1.smithi193.stdout:5/32: creat d0/f9 x:0 0 0 2024-05-09T06:38:09.300 INFO:tasks.workunit.client.1.smithi193.stdout:5/33: readlink d0/d1/l4 0 2024-05-09T06:38:09.301 INFO:tasks.workunit.client.0.smithi063.stdout:7/385: dread d0/d15/f48 [0,4194304] 0 2024-05-09T06:38:09.303 INFO:tasks.workunit.client.1.smithi193.stdout:5/34: write d0/f8 [657201,26939] 0 2024-05-09T06:38:09.304 INFO:tasks.workunit.client.1.smithi193.stdout:1/66: dwrite d4/da/fb [0,4194304] 0 2024-05-09T06:38:09.304 INFO:tasks.workunit.client.0.smithi063.stdout:8/469: dread d9/f7d [0,4194304] 0 2024-05-09T06:38:09.305 INFO:tasks.workunit.client.0.smithi063.stdout:4/389: dread d7/df/f2b [0,4194304] 0 2024-05-09T06:38:09.309 INFO:tasks.workunit.client.1.smithi193.stdout:9/106: write dd/df/d15/f17 [237679,77817] 0 2024-05-09T06:38:09.311 INFO:tasks.workunit.client.0.smithi063.stdout:8/470: dread d9/d3b/f85 [0,4194304] 0 2024-05-09T06:38:09.313 INFO:tasks.workunit.client.0.smithi063.stdout:9/408: unlink d0/d4/d26/d39/d42/f74 0 2024-05-09T06:38:09.315 INFO:tasks.workunit.client.0.smithi063.stdout:9/409: dread - d0/d4/d57/d60/f7c zero size 2024-05-09T06:38:09.316 INFO:tasks.workunit.client.1.smithi193.stdout:6/98: rmdir d1 39 2024-05-09T06:38:09.317 INFO:tasks.workunit.client.1.smithi193.stdout:7/107: creat d6/d8/d1a/d1f/f21 x:0 0 0 2024-05-09T06:38:09.317 INFO:tasks.workunit.client.1.smithi193.stdout:0/150: dwrite d18/d1d/f2d [0,4194304] 0 2024-05-09T06:38:09.317 INFO:tasks.workunit.client.1.smithi193.stdout:7/108: fsync f4 0 2024-05-09T06:38:09.320 INFO:tasks.workunit.client.0.smithi063.stdout:7/386: chown d0/d15/d4a/d50/l4b 0 1 2024-05-09T06:38:09.320 INFO:tasks.workunit.client.1.smithi193.stdout:4/57: write f5 [7314073,39487] 0 2024-05-09T06:38:09.322 INFO:tasks.workunit.client.1.smithi193.stdout:4/58: truncate f13 809306 0 2024-05-09T06:38:09.323 INFO:tasks.workunit.client.0.smithi063.stdout:2/405: write dc/f21 [245118,26673] 0 2024-05-09T06:38:09.323 INFO:tasks.workunit.client.1.smithi193.stdout:1/67: dwrite d4/da/de/f12 [0,4194304] 0 2024-05-09T06:38:09.324 INFO:tasks.workunit.client.0.smithi063.stdout:2/406: readlink dc/d36/d43/d46/l74 0 2024-05-09T06:38:09.325 INFO:tasks.workunit.client.0.smithi063.stdout:9/410: dread d0/d4/f6a [0,4194304] 0 2024-05-09T06:38:09.325 INFO:tasks.workunit.client.1.smithi193.stdout:1/68: read d4/da/fb [332373,77512] 0 2024-05-09T06:38:09.326 INFO:tasks.workunit.client.0.smithi063.stdout:2/407: chown dc/dd/d45/c73 53793842 1 2024-05-09T06:38:09.328 INFO:tasks.workunit.client.1.smithi193.stdout:4/59: dread fa [0,4194304] 0 2024-05-09T06:38:09.329 INFO:tasks.workunit.client.0.smithi063.stdout:4/390: creat d7/df/d1f/d28/f7d x:0 0 0 2024-05-09T06:38:09.332 INFO:tasks.workunit.client.0.smithi063.stdout:4/391: fdatasync d7/df/d1f/d28/f7d 0 2024-05-09T06:38:09.333 INFO:tasks.workunit.client.1.smithi193.stdout:4/60: dread f13 [0,4194304] 0 2024-05-09T06:38:09.335 INFO:tasks.workunit.client.0.smithi063.stdout:2/408: dwrite dc/f21 [0,4194304] 0 2024-05-09T06:38:09.335 INFO:tasks.workunit.client.0.smithi063.stdout:3/395: dread d7/f1b [0,4194304] 0 2024-05-09T06:38:09.337 INFO:tasks.workunit.client.0.smithi063.stdout:4/392: readlink d7/df/d1f/d28/d31/l67 0 2024-05-09T06:38:09.337 INFO:tasks.workunit.client.0.smithi063.stdout:2/409: fdatasync dc/d32/d33/f41 0 2024-05-09T06:38:09.338 INFO:tasks.workunit.client.0.smithi063.stdout:3/396: dread - d7/da/d21/d30/d44/d48/f6f zero size 2024-05-09T06:38:09.340 INFO:tasks.workunit.client.0.smithi063.stdout:2/410: dread - dc/d36/f57 zero size 2024-05-09T06:38:09.340 INFO:tasks.workunit.client.0.smithi063.stdout:7/387: rename d0/l2d to d0/d15/d4a/d50/l7c 0 2024-05-09T06:38:09.341 INFO:tasks.workunit.client.0.smithi063.stdout:2/411: dread - dc/d36/f52 zero size 2024-05-09T06:38:09.342 INFO:tasks.workunit.client.0.smithi063.stdout:3/397: dread d7/da/f29 [0,4194304] 0 2024-05-09T06:38:09.343 INFO:tasks.workunit.client.0.smithi063.stdout:3/398: fdatasync d7/f11 0 2024-05-09T06:38:09.349 INFO:tasks.workunit.client.0.smithi063.stdout:1/349: dwrite d9/fe [4194304,4194304] 0 2024-05-09T06:38:09.350 INFO:tasks.workunit.client.0.smithi063.stdout:1/350: dread - d9/d10/d2b/d3f/d59/d62/f6c zero size 2024-05-09T06:38:09.351 INFO:tasks.workunit.client.1.smithi193.stdout:5/35: mkdir d0/d1/da 0 2024-05-09T06:38:09.358 INFO:tasks.workunit.client.0.smithi063.stdout:4/393: rename d7/df/d1f/d28/d31/d7b/c7c to d7/df/d1f/d28/c7e 0 2024-05-09T06:38:09.361 INFO:tasks.workunit.client.0.smithi063.stdout:2/412: symlink dc/dd/d2d/l75 0 2024-05-09T06:38:09.362 INFO:tasks.workunit.client.1.smithi193.stdout:9/107: read f6 [934314,105041] 0 2024-05-09T06:38:09.362 INFO:tasks.workunit.client.0.smithi063.stdout:2/413: truncate dc/dd/f64 135021 0 2024-05-09T06:38:09.366 INFO:tasks.workunit.client.0.smithi063.stdout:5/361: link d8/f21 d8/d65/f80 0 2024-05-09T06:38:09.368 INFO:tasks.workunit.client.1.smithi193.stdout:0/151: write d18/d1d/f24 [477175,10571] 0 2024-05-09T06:38:09.373 INFO:tasks.workunit.client.1.smithi193.stdout:6/99: dwrite d1/f5 [0,4194304] 0 2024-05-09T06:38:09.373 INFO:tasks.workunit.client.0.smithi063.stdout:9/411: creat d0/d4/d57/d60/d76/f8f x:0 0 0 2024-05-09T06:38:09.376 INFO:tasks.workunit.client.1.smithi193.stdout:2/61: dwrite f9 [0,4194304] 0 2024-05-09T06:38:09.377 INFO:tasks.workunit.client.1.smithi193.stdout:2/62: chown db/df 5675 1 2024-05-09T06:38:09.378 INFO:tasks.workunit.client.0.smithi063.stdout:1/351: dread d9/f23 [0,4194304] 0 2024-05-09T06:38:09.385 INFO:tasks.workunit.client.0.smithi063.stdout:4/394: creat d7/df/d35/f7f x:0 0 0 2024-05-09T06:38:09.387 INFO:tasks.workunit.client.1.smithi193.stdout:4/61: symlink l15 0 2024-05-09T06:38:09.388 INFO:tasks.workunit.client.1.smithi193.stdout:4/62: readlink l9 0 2024-05-09T06:38:09.388 INFO:tasks.workunit.client.1.smithi193.stdout:4/63: stat f7 0 2024-05-09T06:38:09.388 INFO:tasks.workunit.client.1.smithi193.stdout:4/64: read f12 [444377,20867] 0 2024-05-09T06:38:09.389 INFO:tasks.workunit.client.1.smithi193.stdout:5/36: mknod d0/d1/cb 0 2024-05-09T06:38:09.390 INFO:tasks.workunit.client.1.smithi193.stdout:5/37: write d0/f9 [639388,96481] 0 2024-05-09T06:38:09.391 INFO:tasks.workunit.client.1.smithi193.stdout:5/38: fsync d0/f7 0 2024-05-09T06:38:09.393 INFO:tasks.workunit.client.0.smithi063.stdout:4/395: read d7/df/f12 [1250699,129928] 0 2024-05-09T06:38:09.393 INFO:tasks.workunit.client.0.smithi063.stdout:5/362: rmdir d8/d35 39 2024-05-09T06:38:09.397 INFO:tasks.workunit.client.1.smithi193.stdout:9/108: mknod dd/df/d15/d16/c25 0 2024-05-09T06:38:09.398 INFO:tasks.workunit.client.0.smithi063.stdout:4/396: dwrite d7/df/f2f [0,4194304] 0 2024-05-09T06:38:09.403 INFO:tasks.workunit.client.0.smithi063.stdout:9/412: rmdir d0/d4/d26/d39 39 2024-05-09T06:38:09.406 INFO:tasks.workunit.client.1.smithi193.stdout:7/109: link d6/d8/f1b d6/dd/f22 0 2024-05-09T06:38:09.408 INFO:tasks.workunit.client.0.smithi063.stdout:1/352: unlink f8 0 2024-05-09T06:38:09.421 INFO:tasks.workunit.client.0.smithi063.stdout:5/363: creat d8/d17/d1e/f81 x:0 0 0 2024-05-09T06:38:09.427 INFO:tasks.workunit.client.1.smithi193.stdout:2/63: dwrite f8 [0,4194304] 0 2024-05-09T06:38:09.430 INFO:tasks.workunit.client.1.smithi193.stdout:2/64: dread db/df/f10 [0,4194304] 0 2024-05-09T06:38:09.431 INFO:tasks.workunit.client.1.smithi193.stdout:2/65: write f8 [353981,31617] 0 2024-05-09T06:38:09.432 INFO:tasks.workunit.client.1.smithi193.stdout:2/66: write f9 [154336,33641] 0 2024-05-09T06:38:09.450 INFO:tasks.workunit.client.0.smithi063.stdout:6/395: sync 2024-05-09T06:38:09.478 INFO:tasks.workunit.client.1.smithi193.stdout:1/69: link d4/fc d4/f13 0 2024-05-09T06:38:09.486 INFO:tasks.workunit.client.0.smithi063.stdout:1/353: unlink d9/d10/f18 0 2024-05-09T06:38:09.486 INFO:tasks.workunit.client.1.smithi193.stdout:1/70: dwrite d4/da/de/f12 [0,4194304] 0 2024-05-09T06:38:09.487 INFO:tasks.workunit.client.1.smithi193.stdout:1/71: truncate d4/fc 861909 0 2024-05-09T06:38:09.488 INFO:tasks.workunit.client.1.smithi193.stdout:1/72: fsync d4/da/f10 0 2024-05-09T06:38:09.489 INFO:tasks.workunit.client.1.smithi193.stdout:5/39: creat d0/fc x:0 0 0 2024-05-09T06:38:09.489 INFO:tasks.workunit.client.1.smithi193.stdout:5/40: fsync d0/f9 0 2024-05-09T06:38:09.490 INFO:tasks.workunit.client.1.smithi193.stdout:5/41: stat d0 0 2024-05-09T06:38:09.495 INFO:tasks.workunit.client.1.smithi193.stdout:9/109: truncate dd/f13 1829493 0 2024-05-09T06:38:09.495 INFO:tasks.workunit.client.1.smithi193.stdout:9/110: chown dd/df/d15 0 1 2024-05-09T06:38:09.497 INFO:tasks.workunit.client.0.smithi063.stdout:6/396: mknod d2/d1f/c80 0 2024-05-09T06:38:09.498 INFO:tasks.workunit.client.1.smithi193.stdout:7/110: creat d6/d17/f23 x:0 0 0 2024-05-09T06:38:09.499 INFO:tasks.workunit.client.0.smithi063.stdout:3/399: truncate d7/d4c/d25/f72 451029 0 2024-05-09T06:38:09.499 INFO:tasks.workunit.client.0.smithi063.stdout:2/414: dwrite dc/f21 [4194304,4194304] 0 2024-05-09T06:38:09.500 INFO:tasks.workunit.client.0.smithi063.stdout:4/397: creat d7/df/d1f/d28/d31/d32/d34/f80 x:0 0 0 2024-05-09T06:38:09.500 INFO:tasks.workunit.client.0.smithi063.stdout:3/400: chown d7/da/fb 61789 1 2024-05-09T06:38:09.501 INFO:tasks.workunit.client.0.smithi063.stdout:2/415: fdatasync dc/f1c 0 2024-05-09T06:38:09.502 INFO:tasks.workunit.client.0.smithi063.stdout:2/416: chown dc/f1b 0 1 2024-05-09T06:38:09.502 INFO:tasks.workunit.client.0.smithi063.stdout:9/413: write d0/d4/d26/d47/f1d [522439,94685] 0 2024-05-09T06:38:09.503 INFO:tasks.workunit.client.0.smithi063.stdout:2/417: write dc/f21 [5725023,18281] 0 2024-05-09T06:38:09.504 INFO:tasks.workunit.client.0.smithi063.stdout:1/354: creat d9/d2a/d3c/d4c/f78 x:0 0 0 2024-05-09T06:38:09.508 INFO:tasks.workunit.client.0.smithi063.stdout:6/397: mknod d2/d1f/d29/d3f/d5c/d64/c81 0 2024-05-09T06:38:09.508 INFO:tasks.workunit.client.1.smithi193.stdout:6/100: write d1/fa [508137,70725] 0 2024-05-09T06:38:09.510 INFO:tasks.workunit.client.1.smithi193.stdout:6/101: write d1/fa [41679,61254] 0 2024-05-09T06:38:09.515 INFO:tasks.workunit.client.0.smithi063.stdout:4/398: mkdir d7/df/d1f/d28/d31/d7b/d81 0 2024-05-09T06:38:09.515 INFO:tasks.workunit.client.0.smithi063.stdout:5/364: rmdir d8 39 2024-05-09T06:38:09.521 INFO:tasks.workunit.client.1.smithi193.stdout:1/73: mkdir d4/d14 0 2024-05-09T06:38:09.522 INFO:tasks.workunit.client.0.smithi063.stdout:3/401: rename d7/da/l68 to d7/d22/d28/l84 0 2024-05-09T06:38:09.523 INFO:tasks.workunit.client.0.smithi063.stdout:9/414: rmdir d0/d43 39 2024-05-09T06:38:09.524 INFO:tasks.workunit.client.0.smithi063.stdout:4/399: dwrite d7/df/d35/f7f [0,4194304] 0 2024-05-09T06:38:09.526 INFO:tasks.workunit.client.0.smithi063.stdout:2/418: readlink dc/l56 0 2024-05-09T06:38:09.529 INFO:tasks.workunit.client.0.smithi063.stdout:1/355: creat d9/d10/d2b/d3f/d59/f79 x:0 0 0 2024-05-09T06:38:09.530 INFO:tasks.workunit.client.0.smithi063.stdout:8/471: sync 2024-05-09T06:38:09.530 INFO:tasks.workunit.client.0.smithi063.stdout:7/388: sync 2024-05-09T06:38:09.530 INFO:tasks.workunit.client.0.smithi063.stdout:0/450: sync 2024-05-09T06:38:09.531 INFO:tasks.workunit.client.0.smithi063.stdout:0/451: read - dd/d1a/d50/f6f zero size 2024-05-09T06:38:09.533 INFO:tasks.workunit.client.0.smithi063.stdout:6/398: creat d2/d1f/d29/d3f/f82 x:0 0 0 2024-05-09T06:38:09.539 INFO:tasks.workunit.client.1.smithi193.stdout:6/102: rename d1/f7 to d1/de/d11/f17 0 2024-05-09T06:38:09.542 INFO:tasks.workunit.client.0.smithi063.stdout:3/402: mknod d7/da/d2b/d66/d74/c85 0 2024-05-09T06:38:09.543 INFO:tasks.workunit.client.1.smithi193.stdout:6/103: dread d1/f2 [0,4194304] 0 2024-05-09T06:38:09.543 INFO:tasks.workunit.client.0.smithi063.stdout:3/403: read d7/d2a/d2d/f3e [3315867,78892] 0 2024-05-09T06:38:09.543 INFO:tasks.workunit.client.1.smithi193.stdout:5/42: mkdir d0/d1/da/dd 0 2024-05-09T06:38:09.544 INFO:tasks.workunit.client.0.smithi063.stdout:3/404: dread - d7/da/d21/d30/d44/d48/f81 zero size 2024-05-09T06:38:09.545 INFO:tasks.workunit.client.0.smithi063.stdout:9/415: write d0/d4/d26/d39/f53 [2124496,117028] 0 2024-05-09T06:38:09.548 INFO:tasks.workunit.client.0.smithi063.stdout:4/400: rename d7/df/d1f/d28/l69 to d7/d3d/d73/l82 0 2024-05-09T06:38:09.549 INFO:tasks.workunit.client.0.smithi063.stdout:4/401: chown d7/df/d1f/f23 502 1 2024-05-09T06:38:09.550 INFO:tasks.workunit.client.0.smithi063.stdout:5/365: dread d8/d17/d18/f1f [0,4194304] 0 2024-05-09T06:38:09.550 INFO:tasks.workunit.client.1.smithi193.stdout:4/65: truncate fb 1935776 0 2024-05-09T06:38:09.550 INFO:tasks.workunit.client.1.smithi193.stdout:6/104: dwrite d1/de/f13 [0,4194304] 0 2024-05-09T06:38:09.551 INFO:tasks.workunit.client.0.smithi063.stdout:3/405: dwrite d7/d2a/d2d/f3e [0,4194304] 0 2024-05-09T06:38:09.552 INFO:tasks.workunit.client.1.smithi193.stdout:7/111: truncate f4 190229 0 2024-05-09T06:38:09.554 INFO:tasks.workunit.client.0.smithi063.stdout:1/356: rmdir d9/d2a/d39/d5d 39 2024-05-09T06:38:09.559 INFO:tasks.workunit.client.1.smithi193.stdout:1/74: mkdir d4/d14/d15 0 2024-05-09T06:38:09.561 INFO:tasks.workunit.client.0.smithi063.stdout:6/399: creat d2/d1f/d29/d3f/d5c/d64/f83 x:0 0 0 2024-05-09T06:38:09.563 INFO:tasks.workunit.client.0.smithi063.stdout:0/452: dwrite dd/d60/d63/d23/d31/d58/f6e [0,4194304] 0 2024-05-09T06:38:09.564 INFO:tasks.workunit.client.1.smithi193.stdout:5/43: symlink d0/le 0 2024-05-09T06:38:09.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/416: mknod d0/d4/d7d/c90 0 2024-05-09T06:38:09.572 INFO:tasks.workunit.client.0.smithi063.stdout:6/400: dwrite d2/d1f/d29/d3f/f1e [0,4194304] 0 2024-05-09T06:38:09.573 INFO:tasks.workunit.client.0.smithi063.stdout:9/417: write d0/d4/d57/d60/f7c [196291,35287] 0 2024-05-09T06:38:09.575 INFO:tasks.workunit.client.0.smithi063.stdout:2/419: truncate f3 2503195 0 2024-05-09T06:38:09.575 INFO:tasks.workunit.client.0.smithi063.stdout:9/418: dread - d0/d4/d7d/f82 zero size 2024-05-09T06:38:09.576 INFO:tasks.workunit.client.0.smithi063.stdout:4/402: dwrite d7/df/d35/f7f [4194304,4194304] 0 2024-05-09T06:38:09.583 INFO:tasks.workunit.client.0.smithi063.stdout:3/406: mkdir d7/d2a/d86 0 2024-05-09T06:38:09.585 INFO:tasks.workunit.client.1.smithi193.stdout:1/75: unlink d4/l5 0 2024-05-09T06:38:09.586 INFO:tasks.workunit.client.1.smithi193.stdout:1/76: read d4/f9 [153461,115432] 0 2024-05-09T06:38:09.588 INFO:tasks.workunit.client.0.smithi063.stdout:5/366: rename d8/d65/f7d to d8/d17/d18/f82 0 2024-05-09T06:38:09.589 INFO:tasks.workunit.client.1.smithi193.stdout:5/44: mknod d0/d1/da/cf 0 2024-05-09T06:38:09.591 INFO:tasks.workunit.client.1.smithi193.stdout:9/111: dread dd/f13 [0,4194304] 0 2024-05-09T06:38:09.592 INFO:tasks.workunit.client.0.smithi063.stdout:1/357: fdatasync d9/d46/d57/f65 0 2024-05-09T06:38:09.593 INFO:tasks.workunit.client.0.smithi063.stdout:8/472: rmdir d9/da/d45 39 2024-05-09T06:38:09.596 INFO:tasks.workunit.client.1.smithi193.stdout:6/105: truncate d1/f2 8163883 0 2024-05-09T06:38:09.598 INFO:tasks.workunit.client.0.smithi063.stdout:1/358: dwrite d9/d10/f43 [0,4194304] 0 2024-05-09T06:38:09.602 INFO:tasks.workunit.client.1.smithi193.stdout:5/45: symlink d0/d1/l10 0 2024-05-09T06:38:09.602 INFO:tasks.workunit.client.0.smithi063.stdout:6/401: creat d2/d1f/d29/f84 x:0 0 0 2024-05-09T06:38:09.603 INFO:tasks.workunit.client.0.smithi063.stdout:6/402: write d2/d1f/d6c/f71 [1839523,79997] 0 2024-05-09T06:38:09.603 INFO:tasks.workunit.client.0.smithi063.stdout:2/420: unlink dc/d36/d43/d46/l74 0 2024-05-09T06:38:09.604 INFO:tasks.workunit.client.0.smithi063.stdout:6/403: dread - d2/d20/f7b zero size 2024-05-09T06:38:09.604 INFO:tasks.workunit.client.0.smithi063.stdout:6/404: dread - d2/df/f50 zero size 2024-05-09T06:38:09.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:09 smithi063 ceph-mon[27171]: Standby manager daemon smithi063.egfguy started 2024-05-09T06:38:09.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:09 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.63:0/2440379023' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.egfguy/crt"}]: dispatch 2024-05-09T06:38:09.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:09 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.63:0/2440379023' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:38:09.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:09 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.63:0/2440379023' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.egfguy/key"}]: dispatch 2024-05-09T06:38:09.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:09 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.63:0/2440379023' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:38:09.606 INFO:tasks.workunit.client.0.smithi063.stdout:9/419: rmdir d0/d4/d26/d39 39 2024-05-09T06:38:09.607 INFO:tasks.workunit.client.0.smithi063.stdout:3/407: mknod d7/da/d21/d30/d44/c87 0 2024-05-09T06:38:09.610 INFO:tasks.workunit.client.0.smithi063.stdout:3/408: write d7/d2a/f4e [815089,34639] 0 2024-05-09T06:38:09.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/421: dwrite dc/d36/f57 [0,4194304] 0 2024-05-09T06:38:09.612 INFO:tasks.workunit.client.0.smithi063.stdout:0/453: mkdir dd/d1a/d2c/d51/d76/d97 0 2024-05-09T06:38:09.612 INFO:tasks.workunit.client.0.smithi063.stdout:2/422: chown dc/dd/f2f 3466 1 2024-05-09T06:38:09.616 INFO:tasks.workunit.client.0.smithi063.stdout:3/409: creat d7/d4c/d25/f88 x:0 0 0 2024-05-09T06:38:09.619 INFO:tasks.workunit.client.0.smithi063.stdout:8/473: getdents d9/da/d53 0 2024-05-09T06:38:09.621 INFO:tasks.workunit.client.0.smithi063.stdout:0/454: dwrite f6 [4194304,4194304] 0 2024-05-09T06:38:09.621 INFO:tasks.workunit.client.0.smithi063.stdout:3/410: symlink d7/l89 0 2024-05-09T06:38:09.622 INFO:tasks.workunit.client.0.smithi063.stdout:3/411: fsync d7/d2a/d2d/f5d 0 2024-05-09T06:38:09.624 INFO:tasks.workunit.client.0.smithi063.stdout:8/474: chown d9/da/d18/d46/f64 267794793 1 2024-05-09T06:38:09.627 INFO:tasks.workunit.client.0.smithi063.stdout:8/475: write d9/da/d18/d46/f64 [693305,67799] 0 2024-05-09T06:38:09.631 INFO:tasks.workunit.client.0.smithi063.stdout:8/476: rmdir d9/da/d51 39 2024-05-09T06:38:09.632 INFO:tasks.workunit.client.0.smithi063.stdout:3/412: dwrite d7/d4c/d31/f6e [0,4194304] 0 2024-05-09T06:38:09.634 INFO:tasks.workunit.client.0.smithi063.stdout:3/413: write d7/d2a/d5b/f78 [539354,47259] 0 2024-05-09T06:38:09.635 INFO:tasks.workunit.client.0.smithi063.stdout:3/414: chown f2 208653 1 2024-05-09T06:38:09.640 INFO:tasks.workunit.client.1.smithi193.stdout:5/46: dread d0/d1/f6 [0,4194304] 0 2024-05-09T06:38:09.640 INFO:tasks.workunit.client.0.smithi063.stdout:2/423: dread dc/dd/f17 [0,4194304] 0 2024-05-09T06:38:09.641 INFO:tasks.workunit.client.1.smithi193.stdout:9/112: mkdir dd/df/d15/d16/d26 0 2024-05-09T06:38:09.642 INFO:tasks.workunit.client.1.smithi193.stdout:7/112: rename d6/d8/f1b to d6/f24 0 2024-05-09T06:38:09.643 INFO:tasks.workunit.client.1.smithi193.stdout:9/113: stat lb 0 2024-05-09T06:38:09.644 INFO:tasks.workunit.client.0.smithi063.stdout:2/424: dread dc/f21 [4194304,4194304] 0 2024-05-09T06:38:09.644 INFO:tasks.workunit.client.1.smithi193.stdout:4/66: getdents . 0 2024-05-09T06:38:09.644 INFO:tasks.workunit.client.0.smithi063.stdout:8/477: creat d9/da/f9f x:0 0 0 2024-05-09T06:38:09.645 INFO:tasks.workunit.client.0.smithi063.stdout:2/425: chown dc/dd/d45/d58 16611 1 2024-05-09T06:38:09.645 INFO:tasks.workunit.client.0.smithi063.stdout:8/478: chown d9/d37/f5a 31376 1 2024-05-09T06:38:09.649 INFO:tasks.workunit.client.0.smithi063.stdout:2/426: dread dc/d36/f57 [0,4194304] 0 2024-05-09T06:38:09.651 INFO:tasks.workunit.client.1.smithi193.stdout:7/113: dwrite d6/dd/ff [4194304,4194304] 0 2024-05-09T06:38:09.652 INFO:tasks.workunit.client.1.smithi193.stdout:4/67: symlink l16 0 2024-05-09T06:38:09.652 INFO:tasks.workunit.client.1.smithi193.stdout:7/114: chown d6/f18 1006840779 1 2024-05-09T06:38:09.654 INFO:tasks.workunit.client.1.smithi193.stdout:5/47: creat d0/d1/da/dd/f11 x:0 0 0 2024-05-09T06:38:09.655 INFO:tasks.workunit.client.0.smithi063.stdout:2/427: write dc/dd/f2f [5036123,116752] 0 2024-05-09T06:38:09.660 INFO:tasks.workunit.client.0.smithi063.stdout:8/479: mknod d9/da/d18/ca0 0 2024-05-09T06:38:09.661 INFO:tasks.workunit.client.0.smithi063.stdout:8/480: truncate d9/da/d18/d46/f52 352545 0 2024-05-09T06:38:09.662 INFO:tasks.workunit.client.1.smithi193.stdout:7/115: unlink d6/f18 0 2024-05-09T06:38:09.662 INFO:tasks.workunit.client.0.smithi063.stdout:8/481: truncate d9/da/f9f 140784 0 2024-05-09T06:38:09.663 INFO:tasks.workunit.client.0.smithi063.stdout:2/428: truncate dc/dd/f20 1005371 0 2024-05-09T06:38:09.664 INFO:tasks.workunit.client.1.smithi193.stdout:5/48: dwrite d0/d1/da/dd/f11 [0,4194304] 0 2024-05-09T06:38:09.672 INFO:tasks.workunit.client.1.smithi193.stdout:7/116: symlink d6/d8/d1a/l25 0 2024-05-09T06:38:09.673 INFO:tasks.workunit.client.1.smithi193.stdout:7/117: chown d6/d8/d1a 659826 1 2024-05-09T06:38:09.673 INFO:tasks.workunit.client.0.smithi063.stdout:2/429: mknod dc/d6e/c76 0 2024-05-09T06:38:09.677 INFO:tasks.workunit.client.1.smithi193.stdout:6/106: dread d1/f3 [0,4194304] 0 2024-05-09T06:38:09.678 INFO:tasks.workunit.client.1.smithi193.stdout:7/118: symlink d6/d8/d1a/l26 0 2024-05-09T06:38:09.679 INFO:tasks.workunit.client.0.smithi063.stdout:2/430: fdatasync dc/dd/f20 0 2024-05-09T06:38:09.683 INFO:tasks.workunit.client.1.smithi193.stdout:6/107: mkdir d1/de/d11/d18 0 2024-05-09T06:38:09.686 INFO:tasks.workunit.client.1.smithi193.stdout:6/108: write d1/de/f13 [1369683,89075] 0 2024-05-09T06:38:09.687 INFO:tasks.workunit.client.1.smithi193.stdout:1/77: dwrite d4/fc [0,4194304] 0 2024-05-09T06:38:09.687 INFO:tasks.workunit.client.1.smithi193.stdout:7/119: dread d6/dd/f22 [0,4194304] 0 2024-05-09T06:38:09.689 INFO:tasks.workunit.client.0.smithi063.stdout:3/415: chown d7/da/d21/d30/d44/d48/f53 2062665 1 2024-05-09T06:38:09.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/416: creat d7/da/d21/d30/f8a x:0 0 0 2024-05-09T06:38:09.695 INFO:tasks.workunit.client.0.smithi063.stdout:3/417: dread - d7/d4c/d25/f6c zero size 2024-05-09T06:38:09.695 INFO:tasks.workunit.client.1.smithi193.stdout:6/109: creat d1/de/f19 x:0 0 0 2024-05-09T06:38:09.696 INFO:tasks.workunit.client.0.smithi063.stdout:3/418: truncate d7/d4c/f54 478915 0 2024-05-09T06:38:09.699 INFO:tasks.workunit.client.1.smithi193.stdout:1/78: rename d4/da to d4/d16 0 2024-05-09T06:38:09.703 INFO:tasks.workunit.client.1.smithi193.stdout:6/110: fdatasync d1/de/ff 0 2024-05-09T06:38:09.703 INFO:tasks.workunit.client.1.smithi193.stdout:1/79: dread f2 [4194304,4194304] 0 2024-05-09T06:38:09.704 INFO:tasks.workunit.client.1.smithi193.stdout:1/80: read d4/f13 [4106432,68539] 0 2024-05-09T06:38:09.709 INFO:tasks.workunit.client.1.smithi193.stdout:7/120: rename d6/dd/fe to d6/d8/d1a/f27 0 2024-05-09T06:38:09.712 INFO:tasks.workunit.client.1.smithi193.stdout:6/111: dwrite d1/de/f19 [0,4194304] 0 2024-05-09T06:38:09.715 INFO:tasks.workunit.client.1.smithi193.stdout:6/112: rename d1/de/d11 to d1/de/d11/d18/d1a 22 2024-05-09T06:38:09.715 INFO:tasks.workunit.client.1.smithi193.stdout:1/81: dwrite d4/fc [0,4194304] 0 2024-05-09T06:38:09.716 INFO:tasks.workunit.client.1.smithi193.stdout:7/121: mknod d6/d17/c28 0 2024-05-09T06:38:09.716 INFO:tasks.workunit.client.1.smithi193.stdout:9/114: write fa [1502224,59455] 0 2024-05-09T06:38:09.720 INFO:tasks.workunit.client.1.smithi193.stdout:1/82: mkdir d4/d16/d17 0 2024-05-09T06:38:09.721 INFO:tasks.workunit.client.1.smithi193.stdout:9/115: rmdir dd/df 39 2024-05-09T06:38:09.722 INFO:tasks.workunit.client.1.smithi193.stdout:6/113: rename d1/de/d11/f17 to d1/de/d11/d18/f1b 0 2024-05-09T06:38:09.724 INFO:tasks.workunit.client.1.smithi193.stdout:7/122: mkdir d6/d8/d29 0 2024-05-09T06:38:09.725 INFO:tasks.workunit.client.1.smithi193.stdout:9/116: creat dd/d1e/f27 x:0 0 0 2024-05-09T06:38:09.727 INFO:tasks.workunit.client.1.smithi193.stdout:6/114: dread d1/de/f19 [0,4194304] 0 2024-05-09T06:38:09.729 INFO:tasks.workunit.client.1.smithi193.stdout:1/83: rename f2 to d4/d14/d15/f18 0 2024-05-09T06:38:09.731 INFO:tasks.workunit.client.1.smithi193.stdout:4/68: write f6 [921455,4392] 0 2024-05-09T06:38:09.732 INFO:tasks.workunit.client.1.smithi193.stdout:4/69: write f6 [3423922,27150] 0 2024-05-09T06:38:09.734 INFO:tasks.workunit.client.1.smithi193.stdout:5/49: write d0/d1/da/dd/f11 [4943239,107611] 0 2024-05-09T06:38:09.738 INFO:tasks.workunit.client.1.smithi193.stdout:5/50: truncate d0/f7 642943 0 2024-05-09T06:38:09.741 INFO:tasks.workunit.client.1.smithi193.stdout:1/84: dwrite d4/f11 [0,4194304] 0 2024-05-09T06:38:09.742 INFO:tasks.workunit.client.1.smithi193.stdout:9/117: dread dd/fe [0,4194304] 0 2024-05-09T06:38:09.742 INFO:tasks.workunit.client.1.smithi193.stdout:4/70: dread fb [0,4194304] 0 2024-05-09T06:38:09.744 INFO:tasks.workunit.client.1.smithi193.stdout:6/115: rename d1/de/f19 to d1/de/d11/d18/f1c 0 2024-05-09T06:38:09.744 INFO:tasks.workunit.client.1.smithi193.stdout:1/85: fdatasync d4/f13 0 2024-05-09T06:38:09.745 INFO:tasks.workunit.client.1.smithi193.stdout:1/86: readlink d4/l6 0 2024-05-09T06:38:09.747 INFO:tasks.workunit.client.1.smithi193.stdout:5/51: creat d0/d1/da/f12 x:0 0 0 2024-05-09T06:38:09.747 INFO:tasks.workunit.client.0.smithi063.stdout:2/431: getdents dc/d36/d43/d46 0 2024-05-09T06:38:09.748 INFO:tasks.workunit.client.1.smithi193.stdout:7/123: read f4 [123860,108108] 0 2024-05-09T06:38:09.748 INFO:tasks.workunit.client.1.smithi193.stdout:5/52: truncate d0/f7 682099 0 2024-05-09T06:38:09.749 INFO:tasks.workunit.client.0.smithi063.stdout:3/419: creat d7/da/d2b/d66/d74/d7e/f8b x:0 0 0 2024-05-09T06:38:09.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/420: truncate d0/d43/f5f 3768228 0 2024-05-09T06:38:09.750 INFO:tasks.workunit.client.0.smithi063.stdout:0/455: truncate dd/d1a/f65 3246419 0 2024-05-09T06:38:09.750 INFO:tasks.workunit.client.1.smithi193.stdout:4/71: read - f10 zero size 2024-05-09T06:38:09.750 INFO:tasks.workunit.client.1.smithi193.stdout:5/53: write d0/d1/da/dd/f11 [1480577,101871] 0 2024-05-09T06:38:09.751 INFO:tasks.workunit.client.1.smithi193.stdout:9/118: creat dd/d1e/f28 x:0 0 0 2024-05-09T06:38:09.752 INFO:tasks.workunit.client.0.smithi063.stdout:0/456: write f6 [3165036,22241] 0 2024-05-09T06:38:09.753 INFO:tasks.workunit.client.0.smithi063.stdout:6/405: dwrite d2/df/f1d [0,4194304] 0 2024-05-09T06:38:09.754 INFO:tasks.workunit.client.0.smithi063.stdout:3/420: truncate d7/da/d21/d30/d44/d48/f53 762265 0 2024-05-09T06:38:09.756 INFO:tasks.workunit.client.1.smithi193.stdout:6/116: creat d1/de/d11/f1d x:0 0 0 2024-05-09T06:38:09.758 INFO:tasks.workunit.client.0.smithi063.stdout:2/432: creat dc/dd/d2d/f77 x:0 0 0 2024-05-09T06:38:09.760 INFO:tasks.workunit.client.1.smithi193.stdout:1/87: mknod d4/d14/c19 0 2024-05-09T06:38:09.770 INFO:tasks.workunit.client.0.smithi063.stdout:3/421: dread d7/d22/f27 [4194304,4194304] 0 2024-05-09T06:38:09.771 INFO:tasks.workunit.client.1.smithi193.stdout:1/88: dwrite d4/f13 [0,4194304] 0 2024-05-09T06:38:09.775 INFO:tasks.workunit.client.1.smithi193.stdout:1/89: dread d4/d14/d15/f18 [0,4194304] 0 2024-05-09T06:38:09.775 INFO:tasks.workunit.client.0.smithi063.stdout:0/457: chown dd/d1a/d2c/d42/c95 4419333 1 2024-05-09T06:38:09.779 INFO:tasks.workunit.client.1.smithi193.stdout:9/119: chown dd/df/d15 3 1 2024-05-09T06:38:09.783 INFO:tasks.workunit.client.0.smithi063.stdout:2/433: symlink dc/dd/d45/d58/l78 0 2024-05-09T06:38:09.784 INFO:tasks.workunit.client.1.smithi193.stdout:7/124: symlink d6/d8/d29/l2a 0 2024-05-09T06:38:09.786 INFO:tasks.workunit.client.0.smithi063.stdout:9/421: mkdir d0/d4/d26/d39/d91 0 2024-05-09T06:38:09.790 INFO:tasks.workunit.client.0.smithi063.stdout:0/458: mknod dd/d60/d63/d23/d46/c98 0 2024-05-09T06:38:09.790 INFO:tasks.workunit.client.1.smithi193.stdout:1/90: write d4/f11 [4801599,50398] 0 2024-05-09T06:38:09.792 INFO:tasks.workunit.client.0.smithi063.stdout:9/422: dwrite d0/d4/d26/d47/f5d [4194304,4194304] 0 2024-05-09T06:38:09.793 INFO:tasks.workunit.client.1.smithi193.stdout:3/23: sync 2024-05-09T06:38:09.796 INFO:tasks.workunit.client.1.smithi193.stdout:5/54: truncate d0/f9 151058 0 2024-05-09T06:38:09.797 INFO:tasks.workunit.client.0.smithi063.stdout:0/459: dwrite dd/d1a/d2c/f43 [0,4194304] 0 2024-05-09T06:38:09.802 INFO:tasks.workunit.client.1.smithi193.stdout:6/117: link d1/de/d11/d18/f1b d1/de/d11/d18/f1e 0 2024-05-09T06:38:09.804 INFO:tasks.workunit.client.0.smithi063.stdout:6/406: creat d2/f85 x:0 0 0 2024-05-09T06:38:09.809 INFO:tasks.workunit.client.0.smithi063.stdout:2/434: mkdir dc/d36/d43/d46/d63/d79 0 2024-05-09T06:38:09.809 INFO:tasks.workunit.client.0.smithi063.stdout:6/407: read d2/d12/f24 [200614,55336] 0 2024-05-09T06:38:09.810 INFO:tasks.workunit.client.1.smithi193.stdout:3/24: creat d1/f8 x:0 0 0 2024-05-09T06:38:09.812 INFO:tasks.workunit.client.0.smithi063.stdout:3/422: rmdir d7/da/d21/d30/d77 0 2024-05-09T06:38:09.815 INFO:tasks.workunit.client.1.smithi193.stdout:6/118: creat d1/de/d11/f1f x:0 0 0 2024-05-09T06:38:09.816 INFO:tasks.workunit.client.0.smithi063.stdout:5/367: sync 2024-05-09T06:38:09.816 INFO:tasks.workunit.client.0.smithi063.stdout:4/403: sync 2024-05-09T06:38:09.816 INFO:tasks.workunit.client.0.smithi063.stdout:7/389: sync 2024-05-09T06:38:09.817 INFO:tasks.workunit.client.0.smithi063.stdout:1/359: sync 2024-05-09T06:38:09.817 INFO:tasks.workunit.client.0.smithi063.stdout:9/423: mknod d0/d4/c92 0 2024-05-09T06:38:09.817 INFO:tasks.workunit.client.1.smithi193.stdout:6/119: read d1/de/d11/d18/f1e [1230594,26182] 0 2024-05-09T06:38:09.817 INFO:tasks.workunit.client.1.smithi193.stdout:6/120: chown d1/de/d11/f1d 11 1 2024-05-09T06:38:09.818 INFO:tasks.workunit.client.0.smithi063.stdout:1/360: fdatasync d9/d46/f6e 0 2024-05-09T06:38:09.818 INFO:tasks.workunit.client.0.smithi063.stdout:6/408: symlink d2/d1f/d29/d3b/l86 0 2024-05-09T06:38:09.819 INFO:tasks.workunit.client.1.smithi193.stdout:4/72: link fb f17 0 2024-05-09T06:38:09.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/361: dread - d9/d2a/d35/d1e/f70 zero size 2024-05-09T06:38:09.820 INFO:tasks.workunit.client.0.smithi063.stdout:6/409: truncate d2/d12/f76 748516 0 2024-05-09T06:38:09.821 INFO:tasks.workunit.client.0.smithi063.stdout:2/435: creat dc/dd/d2d/d60/d65/f7a x:0 0 0 2024-05-09T06:38:09.824 INFO:tasks.workunit.client.1.smithi193.stdout:3/25: creat d1/f9 x:0 0 0 2024-05-09T06:38:09.826 INFO:tasks.workunit.client.0.smithi063.stdout:3/423: creat d7/d22/d28/f8c x:0 0 0 2024-05-09T06:38:09.828 INFO:tasks.workunit.client.1.smithi193.stdout:9/120: write dd/fe [231113,51349] 0 2024-05-09T06:38:09.828 INFO:tasks.workunit.client.0.smithi063.stdout:3/424: chown d7/da/d21/f6b 0 1 2024-05-09T06:38:09.829 INFO:tasks.workunit.client.1.smithi193.stdout:9/121: write dd/d1e/d1f/f21 [790247,94033] 0 2024-05-09T06:38:09.830 INFO:tasks.workunit.client.0.smithi063.stdout:3/425: dread - d7/da/d21/d30/f40 zero size 2024-05-09T06:38:09.832 INFO:tasks.workunit.client.1.smithi193.stdout:9/122: dread dd/df/d15/f17 [0,4194304] 0 2024-05-09T06:38:09.833 INFO:tasks.workunit.client.1.smithi193.stdout:7/125: rename d6/d17/c15 to d6/d8/c2b 0 2024-05-09T06:38:09.834 INFO:tasks.workunit.client.0.smithi063.stdout:2/436: dread f2 [0,4194304] 0 2024-05-09T06:38:09.834 INFO:tasks.workunit.client.1.smithi193.stdout:3/26: rename d1 to d1/da 22 2024-05-09T06:38:09.836 INFO:tasks.workunit.client.1.smithi193.stdout:7/126: write d6/f16 [6469937,88398] 0 2024-05-09T06:38:09.837 INFO:tasks.workunit.client.1.smithi193.stdout:3/27: truncate d1/f9 188422 0 2024-05-09T06:38:09.837 INFO:tasks.workunit.client.1.smithi193.stdout:6/121: mkdir d1/de/d11/d20 0 2024-05-09T06:38:09.837 INFO:tasks.workunit.client.1.smithi193.stdout:3/28: fsync d1/f4 0 2024-05-09T06:38:09.837 INFO:tasks.workunit.client.1.smithi193.stdout:7/127: read d6/d17/f19 [5370907,111547] 0 2024-05-09T06:38:09.838 INFO:tasks.workunit.client.0.smithi063.stdout:7/390: creat d0/d15/d4a/d68/f7d x:0 0 0 2024-05-09T06:38:09.839 INFO:tasks.workunit.client.1.smithi193.stdout:7/128: write d6/f16 [6994291,18295] 0 2024-05-09T06:38:09.840 INFO:tasks.workunit.client.0.smithi063.stdout:9/424: dread d0/f32 [0,4194304] 0 2024-05-09T06:38:09.841 INFO:tasks.workunit.client.0.smithi063.stdout:5/368: dread d8/f4b [0,4194304] 0 2024-05-09T06:38:09.842 INFO:tasks.workunit.client.0.smithi063.stdout:9/425: chown d0/d8/d2d/c4c 22373 1 2024-05-09T06:38:09.842 INFO:tasks.workunit.client.0.smithi063.stdout:5/369: chown d8/d65/f7a 258074355 1 2024-05-09T06:38:09.843 INFO:tasks.workunit.client.0.smithi063.stdout:3/426: dread d7/f17 [0,4194304] 0 2024-05-09T06:38:09.843 INFO:tasks.workunit.client.1.smithi193.stdout:1/91: dwrite d4/d14/d15/f18 [0,4194304] 0 2024-05-09T06:38:09.843 INFO:tasks.workunit.client.0.smithi063.stdout:9/426: write d0/d43/f77 [570247,53042] 0 2024-05-09T06:38:09.844 INFO:tasks.workunit.client.1.smithi193.stdout:1/92: dread - d4/ff zero size 2024-05-09T06:38:09.845 INFO:tasks.workunit.client.0.smithi063.stdout:3/427: write d7/da/d21/f4d [3999856,111508] 0 2024-05-09T06:38:09.845 INFO:tasks.workunit.client.1.smithi193.stdout:4/73: dread f7 [0,4194304] 0 2024-05-09T06:38:09.845 INFO:tasks.workunit.client.0.smithi063.stdout:3/428: dread - d7/da/d21/f6b zero size 2024-05-09T06:38:09.846 INFO:tasks.workunit.client.0.smithi063.stdout:3/429: chown d7/da/d21/d30 4 1 2024-05-09T06:38:09.847 INFO:tasks.workunit.client.1.smithi193.stdout:2/67: sync 2024-05-09T06:38:09.847 INFO:tasks.workunit.client.1.smithi193.stdout:8/45: sync 2024-05-09T06:38:09.847 INFO:tasks.workunit.client.1.smithi193.stdout:0/152: sync 2024-05-09T06:38:09.847 INFO:tasks.workunit.client.1.smithi193.stdout:2/68: readlink db/df/l11 0 2024-05-09T06:38:09.848 INFO:tasks.workunit.client.1.smithi193.stdout:2/69: stat c0 0 2024-05-09T06:38:09.849 INFO:tasks.workunit.client.1.smithi193.stdout:0/153: chown d18/c1c 2 1 2024-05-09T06:38:09.850 INFO:tasks.workunit.client.1.smithi193.stdout:0/154: chown d18/d27/l2e 1 1 2024-05-09T06:38:09.850 INFO:tasks.workunit.client.1.smithi193.stdout:0/155: chown d18/d1d/c22 74873219 1 2024-05-09T06:38:09.853 INFO:tasks.workunit.client.0.smithi063.stdout:0/460: truncate dd/d1a/f78 981130 0 2024-05-09T06:38:09.855 INFO:tasks.workunit.client.0.smithi063.stdout:4/404: symlink d7/df/d1f/l83 0 2024-05-09T06:38:09.855 INFO:tasks.workunit.client.0.smithi063.stdout:4/405: chown d7/df/d35/d5a/d60/d75 9252 1 2024-05-09T06:38:09.857 INFO:tasks.workunit.client.0.smithi063.stdout:2/437: unlink dc/d36/d43/f5c 0 2024-05-09T06:38:09.859 INFO:tasks.workunit.client.0.smithi063.stdout:7/391: truncate d0/f32 216571 0 2024-05-09T06:38:09.859 INFO:tasks.workunit.client.0.smithi063.stdout:7/392: stat d0/d15/l3e 0 2024-05-09T06:38:09.860 INFO:tasks.workunit.client.1.smithi193.stdout:3/29: mkdir d1/db 0 2024-05-09T06:38:09.861 INFO:tasks.workunit.client.0.smithi063.stdout:1/362: symlink d9/l7a 0 2024-05-09T06:38:09.865 INFO:tasks.workunit.client.0.smithi063.stdout:6/410: creat d2/d4/d7c/d7e/f87 x:0 0 0 2024-05-09T06:38:09.870 INFO:tasks.workunit.client.1.smithi193.stdout:5/55: getdents d0 0 2024-05-09T06:38:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:09 smithi193 ceph-mon[35141]: Standby manager daemon smithi063.egfguy started 2024-05-09T06:38:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:09 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.63:0/2440379023' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.egfguy/crt"}]: dispatch 2024-05-09T06:38:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:09 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.63:0/2440379023' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:38:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:09 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.63:0/2440379023' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.egfguy/key"}]: dispatch 2024-05-09T06:38:09.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:09 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.63:0/2440379023' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:38:09.872 INFO:tasks.workunit.client.1.smithi193.stdout:5/56: write d0/f7 [878346,77481] 0 2024-05-09T06:38:09.872 INFO:tasks.workunit.client.0.smithi063.stdout:9/427: write d0/d4/d26/d39/f6c [3438969,127822] 0 2024-05-09T06:38:09.876 INFO:tasks.workunit.client.1.smithi193.stdout:4/74: dread fa [0,4194304] 0 2024-05-09T06:38:09.878 INFO:tasks.workunit.client.1.smithi193.stdout:8/46: unlink d6/f7 0 2024-05-09T06:38:09.879 INFO:tasks.workunit.client.1.smithi193.stdout:8/47: write f0 [6203583,107095] 0 2024-05-09T06:38:09.880 INFO:tasks.workunit.client.0.smithi063.stdout:0/461: symlink dd/d60/d63/d23/d33/d48/d75/l99 0 2024-05-09T06:38:09.880 INFO:tasks.workunit.client.1.smithi193.stdout:8/48: write f0 [6822153,37958] 0 2024-05-09T06:38:09.881 INFO:tasks.workunit.client.1.smithi193.stdout:2/70: creat db/df/d12/f17 x:0 0 0 2024-05-09T06:38:09.883 INFO:tasks.workunit.client.1.smithi193.stdout:2/71: fdatasync db/df/d12/f17 0 2024-05-09T06:38:09.884 INFO:tasks.workunit.client.0.smithi063.stdout:4/406: unlink d7/df/d57/f6b 0 2024-05-09T06:38:09.884 INFO:tasks.workunit.client.0.smithi063.stdout:5/370: write d8/d17/d18/f2d [763405,110288] 0 2024-05-09T06:38:09.884 INFO:tasks.workunit.client.0.smithi063.stdout:0/462: dwrite dd/f1d [0,4194304] 0 2024-05-09T06:38:09.887 INFO:tasks.workunit.client.1.smithi193.stdout:8/49: dwrite f0 [0,4194304] 0 2024-05-09T06:38:09.888 INFO:tasks.workunit.client.1.smithi193.stdout:3/30: unlink d1/l7 0 2024-05-09T06:38:09.890 INFO:tasks.workunit.client.1.smithi193.stdout:3/31: write d1/f9 [132043,37252] 0 2024-05-09T06:38:09.891 INFO:tasks.workunit.client.0.smithi063.stdout:1/363: rename d9/d2a/d2e/f66 to d9/d2a/d3c/d4a/f7b 0 2024-05-09T06:38:09.897 INFO:tasks.workunit.client.1.smithi193.stdout:3/32: dread f0 [0,4194304] 0 2024-05-09T06:38:09.897 INFO:tasks.workunit.client.1.smithi193.stdout:6/122: link d1/de/f13 d1/de/d11/d20/f21 0 2024-05-09T06:38:09.898 INFO:tasks.workunit.client.1.smithi193.stdout:7/129: mknod d6/d8/c2c 0 2024-05-09T06:38:09.898 INFO:tasks.workunit.client.1.smithi193.stdout:3/33: chown d1/f9 87645873 1 2024-05-09T06:38:09.899 INFO:tasks.workunit.client.0.smithi063.stdout:3/430: symlink d7/da/d21/d30/d44/l8d 0 2024-05-09T06:38:09.899 INFO:tasks.workunit.client.1.smithi193.stdout:3/34: chown d1/c6 31005531 1 2024-05-09T06:38:09.900 INFO:tasks.workunit.client.0.smithi063.stdout:3/431: fsync d7/da/d21/d30/f57 0 2024-05-09T06:38:09.902 INFO:tasks.workunit.client.1.smithi193.stdout:1/93: mkdir d4/d16/d17/d1a 0 2024-05-09T06:38:09.903 INFO:tasks.workunit.client.0.smithi063.stdout:2/438: mkdir dc/dd/d2d/d60/d65/d72/d7b 0 2024-05-09T06:38:09.904 INFO:tasks.workunit.client.0.smithi063.stdout:2/439: write dc/dd/f64 [1179972,31501] 0 2024-05-09T06:38:09.907 INFO:tasks.workunit.client.1.smithi193.stdout:2/72: mknod db/c18 0 2024-05-09T06:38:09.909 INFO:tasks.workunit.client.0.smithi063.stdout:4/407: rename d7/df/d1f/l30 to d7/df/d35/l84 0 2024-05-09T06:38:09.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/123: creat dd/f29 x:0 0 0 2024-05-09T06:38:09.909 INFO:tasks.workunit.client.0.smithi063.stdout:4/408: chown d7/f39 58023 1 2024-05-09T06:38:09.912 INFO:tasks.workunit.client.1.smithi193.stdout:8/50: fsync f0 0 2024-05-09T06:38:09.914 INFO:tasks.workunit.client.0.smithi063.stdout:4/409: dwrite d7/df/d35/d5a/d60/f63 [0,4194304] 0 2024-05-09T06:38:09.915 INFO:tasks.workunit.client.0.smithi063.stdout:4/410: fdatasync f6 0 2024-05-09T06:38:09.915 INFO:tasks.workunit.client.1.smithi193.stdout:7/130: dread d6/d8/fa [0,4194304] 0 2024-05-09T06:38:09.916 INFO:tasks.workunit.client.0.smithi063.stdout:4/411: write d7/df/d1f/d28/d31/f5f [87071,3265] 0 2024-05-09T06:38:09.917 INFO:tasks.workunit.client.0.smithi063.stdout:4/412: readlink d7/df/d1f/l5d 0 2024-05-09T06:38:09.921 INFO:tasks.workunit.client.1.smithi193.stdout:1/94: read d4/d16/fb [3819730,15300] 0 2024-05-09T06:38:09.922 INFO:tasks.workunit.client.0.smithi063.stdout:2/440: creat dc/d32/f7c x:0 0 0 2024-05-09T06:38:09.923 INFO:tasks.workunit.client.0.smithi063.stdout:2/441: fdatasync dc/dd/d2d/f3e 0 2024-05-09T06:38:09.924 INFO:tasks.workunit.client.1.smithi193.stdout:4/75: mknod c18 0 2024-05-09T06:38:09.925 INFO:tasks.workunit.client.0.smithi063.stdout:0/463: fdatasync dd/d1a/f25 0 2024-05-09T06:38:09.928 INFO:tasks.workunit.client.0.smithi063.stdout:7/393: creat d0/d15/d4a/f7e x:0 0 0 2024-05-09T06:38:09.930 INFO:tasks.workunit.client.1.smithi193.stdout:9/124: unlink dd/df/c14 0 2024-05-09T06:38:09.933 INFO:tasks.workunit.client.1.smithi193.stdout:8/51: creat d6/fc x:0 0 0 2024-05-09T06:38:09.934 INFO:tasks.workunit.client.1.smithi193.stdout:8/52: write d6/fc [317992,82896] 0 2024-05-09T06:38:09.935 INFO:tasks.workunit.client.1.smithi193.stdout:8/53: fsync f0 0 2024-05-09T06:38:09.935 INFO:tasks.workunit.client.1.smithi193.stdout:3/35: mknod d1/db/cc 0 2024-05-09T06:38:09.936 INFO:tasks.workunit.client.1.smithi193.stdout:8/54: chown d6/db 289700914 1 2024-05-09T06:38:09.936 INFO:tasks.workunit.client.1.smithi193.stdout:3/36: chown d1/c2 38 1 2024-05-09T06:38:09.936 INFO:tasks.workunit.client.1.smithi193.stdout:8/55: truncate f0 8398635 0 2024-05-09T06:38:09.938 INFO:tasks.workunit.client.1.smithi193.stdout:7/131: fdatasync d6/d8/fa 0 2024-05-09T06:38:09.939 INFO:tasks.workunit.client.0.smithi063.stdout:8/482: sync 2024-05-09T06:38:09.939 INFO:tasks.workunit.client.0.smithi063.stdout:5/371: dwrite d8/f10 [0,4194304] 0 2024-05-09T06:38:09.940 INFO:tasks.workunit.client.0.smithi063.stdout:3/432: mkdir d7/d8e 0 2024-05-09T06:38:09.945 INFO:tasks.workunit.client.1.smithi193.stdout:3/37: dwrite d1/f4 [0,4194304] 0 2024-05-09T06:38:09.946 INFO:tasks.workunit.client.1.smithi193.stdout:3/38: fsync d1/f8 0 2024-05-09T06:38:09.949 INFO:tasks.workunit.client.1.smithi193.stdout:1/95: rmdir d4/d16/d17 39 2024-05-09T06:38:09.952 INFO:tasks.workunit.client.0.smithi063.stdout:6/411: truncate d2/d1f/f4c 2532442 0 2024-05-09T06:38:09.956 INFO:tasks.workunit.client.0.smithi063.stdout:0/464: mknod dd/d60/d63/d23/d46/c9a 0 2024-05-09T06:38:09.959 INFO:tasks.workunit.client.1.smithi193.stdout:9/125: rename dd/fe to dd/df/d15/d16/f2a 0 2024-05-09T06:38:09.960 INFO:tasks.workunit.client.1.smithi193.stdout:9/126: write fa [1121778,40581] 0 2024-05-09T06:38:09.961 INFO:tasks.workunit.client.1.smithi193.stdout:8/56: creat d6/fd x:0 0 0 2024-05-09T06:38:09.964 INFO:tasks.workunit.client.0.smithi063.stdout:5/372: creat d8/d65/f83 x:0 0 0 2024-05-09T06:38:09.965 INFO:tasks.workunit.client.1.smithi193.stdout:7/132: unlink d6/l11 0 2024-05-09T06:38:09.967 INFO:tasks.workunit.client.0.smithi063.stdout:8/483: mkdir d9/da/d18/d56/d58/d79/da1 0 2024-05-09T06:38:09.967 INFO:tasks.workunit.client.1.smithi193.stdout:5/57: getdents d0/d1/da 0 2024-05-09T06:38:09.969 INFO:tasks.workunit.client.1.smithi193.stdout:3/39: symlink d1/db/ld 0 2024-05-09T06:38:09.970 INFO:tasks.workunit.client.1.smithi193.stdout:3/40: write f0 [1458178,97813] 0 2024-05-09T06:38:09.971 INFO:tasks.workunit.client.0.smithi063.stdout:4/413: mkdir d7/d50/d85 0 2024-05-09T06:38:09.974 INFO:tasks.workunit.client.0.smithi063.stdout:8/484: dwrite d9/da/d18/f80 [0,4194304] 0 2024-05-09T06:38:09.974 INFO:tasks.workunit.client.1.smithi193.stdout:3/41: dread - d1/f8 zero size 2024-05-09T06:38:09.974 INFO:tasks.workunit.client.1.smithi193.stdout:6/123: truncate d1/de/d11/d20/f21 1235163 0 2024-05-09T06:38:09.974 INFO:tasks.workunit.client.1.smithi193.stdout:1/96: rmdir d4/d14 39 2024-05-09T06:38:09.975 INFO:tasks.workunit.client.1.smithi193.stdout:1/97: chown d4/d16/fb 593 1 2024-05-09T06:38:09.975 INFO:tasks.workunit.client.0.smithi063.stdout:2/442: write f2 [6181399,114143] 0 2024-05-09T06:38:09.975 INFO:tasks.workunit.client.0.smithi063.stdout:2/443: write dc/d32/d33/f4c [203709,54652] 0 2024-05-09T06:38:09.976 INFO:tasks.workunit.client.1.smithi193.stdout:3/42: dread d1/f4 [0,4194304] 0 2024-05-09T06:38:09.977 INFO:tasks.workunit.client.1.smithi193.stdout:4/76: mknod c19 0 2024-05-09T06:38:09.977 INFO:tasks.workunit.client.1.smithi193.stdout:4/77: rmdir - no directory 2024-05-09T06:38:09.982 INFO:tasks.workunit.client.0.smithi063.stdout:1/364: getdents d9/d2a/d3c 0 2024-05-09T06:38:09.982 INFO:tasks.workunit.client.1.smithi193.stdout:9/127: symlink dd/d1e/l2b 0 2024-05-09T06:38:09.984 INFO:tasks.workunit.client.0.smithi063.stdout:1/365: chown d9/d10/f21 14342842 1 2024-05-09T06:38:09.984 INFO:tasks.workunit.client.0.smithi063.stdout:0/465: dwrite dd/d60/d63/d23/d33/d48/f4e [0,4194304] 0 2024-05-09T06:38:09.985 INFO:tasks.workunit.client.1.smithi193.stdout:7/133: truncate d6/f24 2085096 0 2024-05-09T06:38:09.987 INFO:tasks.workunit.client.1.smithi193.stdout:5/58: write d0/d1/f6 [4292590,32522] 0 2024-05-09T06:38:09.988 INFO:tasks.workunit.client.1.smithi193.stdout:5/59: truncate d0/f7 1522169 0 2024-05-09T06:38:09.991 INFO:tasks.workunit.client.0.smithi063.stdout:9/428: sync 2024-05-09T06:38:09.994 INFO:tasks.workunit.client.0.smithi063.stdout:8/485: dread - d9/da/d45/f8d zero size 2024-05-09T06:38:09.994 INFO:tasks.workunit.client.0.smithi063.stdout:6/412: mkdir d2/d1f/d73/d88 0 2024-05-09T06:38:09.995 INFO:tasks.workunit.client.0.smithi063.stdout:6/413: chown d2/l4f 710 1 2024-05-09T06:38:09.999 INFO:tasks.workunit.client.0.smithi063.stdout:7/394: link d0/l52 d0/d15/d49/d4f/l7f 0 2024-05-09T06:38:10.000 INFO:tasks.workunit.client.0.smithi063.stdout:7/395: dread - d0/d15/d49/f5b zero size 2024-05-09T06:38:10.000 INFO:tasks.workunit.client.1.smithi193.stdout:2/73: link db/cc db/df/c19 0 2024-05-09T06:38:10.001 INFO:tasks.workunit.client.1.smithi193.stdout:2/74: fsync db/df/d12/f13 0 2024-05-09T06:38:10.008 INFO:tasks.workunit.client.0.smithi063.stdout:1/366: mknod d9/d10/d2b/c7c 0 2024-05-09T06:38:10.010 INFO:tasks.workunit.client.0.smithi063.stdout:0/466: creat dd/d60/d63/d23/d33/d48/d75/f9b x:0 0 0 2024-05-09T06:38:10.015 INFO:tasks.workunit.client.0.smithi063.stdout:8/486: creat d9/da/d53/fa2 x:0 0 0 2024-05-09T06:38:10.017 INFO:tasks.workunit.client.0.smithi063.stdout:9/429: rename d0/d4/d26/d39/d3e/d66/c73 to d0/d4/d7d/d8b/c93 0 2024-05-09T06:38:10.018 INFO:tasks.workunit.client.0.smithi063.stdout:9/430: chown d0/f5b 291 1 2024-05-09T06:38:10.021 INFO:tasks.workunit.client.0.smithi063.stdout:2/444: mkdir dc/d36/d43/d46/d63/d79/d7d 0 2024-05-09T06:38:10.022 INFO:tasks.workunit.client.0.smithi063.stdout:2/445: write dc/d36/d43/d46/f70 [131337,83008] 0 2024-05-09T06:38:10.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/446: write dc/d32/d33/f4c [1225001,28061] 0 2024-05-09T06:38:10.024 INFO:tasks.workunit.client.0.smithi063.stdout:3/433: getdents d7/d4c 0 2024-05-09T06:38:10.025 INFO:tasks.workunit.client.0.smithi063.stdout:1/367: creat d9/d2a/d2e/f7d x:0 0 0 2024-05-09T06:38:10.028 INFO:tasks.workunit.client.0.smithi063.stdout:8/487: mkdir d9/da/d53/da3 0 2024-05-09T06:38:10.030 INFO:tasks.workunit.client.0.smithi063.stdout:9/431: rmdir d0/d8/d2d 39 2024-05-09T06:38:10.031 INFO:tasks.workunit.client.0.smithi063.stdout:6/414: mkdir d2/d89 0 2024-05-09T06:38:10.033 INFO:tasks.workunit.client.0.smithi063.stdout:5/373: getdents d8/d19/d30/d74 0 2024-05-09T06:38:10.033 INFO:tasks.workunit.client.0.smithi063.stdout:6/415: fsync d2/df/f50 0 2024-05-09T06:38:10.036 INFO:tasks.workunit.client.0.smithi063.stdout:0/467: getdents dd/d1a/d2c/d51/d72/d91 0 2024-05-09T06:38:10.036 INFO:tasks.workunit.client.0.smithi063.stdout:2/447: dwrite dc/dd/f4f [0,4194304] 0 2024-05-09T06:38:10.037 INFO:tasks.workunit.client.0.smithi063.stdout:7/396: creat d0/f80 x:0 0 0 2024-05-09T06:38:10.039 INFO:tasks.workunit.client.0.smithi063.stdout:4/414: dwrite d7/df/d1a/f22 [0,4194304] 0 2024-05-09T06:38:10.039 INFO:tasks.workunit.client.0.smithi063.stdout:9/432: mkdir d0/d4/d7d/d94 0 2024-05-09T06:38:10.040 INFO:tasks.workunit.client.0.smithi063.stdout:8/488: chown d9/da/d34/l6d 0 1 2024-05-09T06:38:10.042 INFO:tasks.workunit.client.0.smithi063.stdout:5/374: fdatasync d8/d19/f44 0 2024-05-09T06:38:10.043 INFO:tasks.workunit.client.0.smithi063.stdout:3/434: dread d7/da/d21/f4d [0,4194304] 0 2024-05-09T06:38:10.046 INFO:tasks.workunit.client.0.smithi063.stdout:6/416: rename d2/d1f/d29/f2a to d2/d20/f8a 0 2024-05-09T06:38:10.048 INFO:tasks.workunit.client.0.smithi063.stdout:0/468: rmdir dd/d60/d63/d23/d46 39 2024-05-09T06:38:10.048 INFO:tasks.workunit.client.0.smithi063.stdout:6/417: chown d2/d20/d26/d7f 1058846627 1 2024-05-09T06:38:10.050 INFO:tasks.workunit.client.0.smithi063.stdout:0/469: chown dd/d60/l5d 15970516 1 2024-05-09T06:38:10.051 INFO:tasks.workunit.client.0.smithi063.stdout:2/448: mknod dc/d36/d43/c7e 0 2024-05-09T06:38:10.053 INFO:tasks.workunit.client.0.smithi063.stdout:4/415: dread d7/df/d1f/f23 [0,4194304] 0 2024-05-09T06:38:10.053 INFO:tasks.workunit.client.0.smithi063.stdout:7/397: dwrite d0/d15/f72 [0,4194304] 0 2024-05-09T06:38:10.053 INFO:tasks.workunit.client.0.smithi063.stdout:2/449: chown dc/d36/d43/d46/d63 31234 1 2024-05-09T06:38:10.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/489: creat d9/da/d45/fa4 x:0 0 0 2024-05-09T06:38:10.056 INFO:tasks.workunit.client.0.smithi063.stdout:7/398: readlink d0/d15/d4a/d50/l4b 0 2024-05-09T06:38:10.059 INFO:tasks.workunit.client.0.smithi063.stdout:1/368: write d9/f11 [4140108,116236] 0 2024-05-09T06:38:10.060 INFO:tasks.workunit.client.0.smithi063.stdout:0/470: dread dd/d1a/f35 [0,4194304] 0 2024-05-09T06:38:10.061 INFO:tasks.workunit.client.0.smithi063.stdout:2/450: rmdir dc/dd/d2d/d60 39 2024-05-09T06:38:10.061 INFO:tasks.workunit.client.0.smithi063.stdout:1/369: chown d9/d10/d2b/d3f/d59 0 1 2024-05-09T06:38:10.063 INFO:tasks.workunit.client.0.smithi063.stdout:3/435: dread f2 [0,4194304] 0 2024-05-09T06:38:10.063 INFO:tasks.workunit.client.0.smithi063.stdout:1/370: fdatasync d9/d10/d2b/f33 0 2024-05-09T06:38:10.066 INFO:tasks.workunit.client.0.smithi063.stdout:8/490: chown d9/da/d18/d46/f9c 165 1 2024-05-09T06:38:10.068 INFO:tasks.workunit.client.0.smithi063.stdout:5/375: write d8/d19/d30/d46/f5d [307365,5182] 0 2024-05-09T06:38:10.071 INFO:tasks.workunit.client.0.smithi063.stdout:6/418: truncate d2/df/f5d 1013490 0 2024-05-09T06:38:10.073 INFO:tasks.workunit.client.0.smithi063.stdout:3/436: rename d7/da/d2b/f64 to d7/d2a/d5b/f8f 0 2024-05-09T06:38:10.073 INFO:tasks.workunit.client.0.smithi063.stdout:3/437: stat d7/da/f29 0 2024-05-09T06:38:10.073 INFO:tasks.workunit.client.0.smithi063.stdout:1/371: mkdir d9/d10/d2b/d3f/d59/d62/d7e 0 2024-05-09T06:38:10.075 INFO:tasks.workunit.client.0.smithi063.stdout:0/471: unlink dd/d60/d63/c39 0 2024-05-09T06:38:10.076 INFO:tasks.workunit.client.1.smithi193.stdout:4/78: readlink l4 0 2024-05-09T06:38:10.076 INFO:tasks.workunit.client.1.smithi193.stdout:8/57: unlink d6/fc 0 2024-05-09T06:38:10.076 INFO:tasks.workunit.client.1.smithi193.stdout:5/60: rmdir d0/d1 39 2024-05-09T06:38:10.077 INFO:tasks.workunit.client.1.smithi193.stdout:2/75: mknod db/df/c1a 0 2024-05-09T06:38:10.077 INFO:tasks.workunit.client.0.smithi063.stdout:7/399: creat d0/f81 x:0 0 0 2024-05-09T06:38:10.081 INFO:tasks.workunit.client.0.smithi063.stdout:6/419: mknod d2/df/d59/c8b 0 2024-05-09T06:38:10.082 INFO:tasks.workunit.client.0.smithi063.stdout:5/376: dread d8/d35/f64 [0,4194304] 0 2024-05-09T06:38:10.083 INFO:tasks.workunit.client.0.smithi063.stdout:4/416: dread d7/df/d1f/d28/d31/d32/f51 [0,4194304] 0 2024-05-09T06:38:10.084 INFO:tasks.workunit.client.0.smithi063.stdout:5/377: stat d8/d17/d18/f45 0 2024-05-09T06:38:10.085 INFO:tasks.workunit.client.1.smithi193.stdout:6/124: dwrite d1/f3 [8388608,4194304] 0 2024-05-09T06:38:10.089 INFO:tasks.workunit.client.0.smithi063.stdout:0/472: mknod dd/d1a/d2c/d42/c9c 0 2024-05-09T06:38:10.092 INFO:tasks.workunit.client.1.smithi193.stdout:2/76: creat db/f1b x:0 0 0 2024-05-09T06:38:10.093 INFO:tasks.workunit.client.0.smithi063.stdout:8/491: dread d9/d37/f4d [0,4194304] 0 2024-05-09T06:38:10.093 INFO:tasks.workunit.client.1.smithi193.stdout:6/125: dwrite d1/f9 [0,4194304] 0 2024-05-09T06:38:10.094 INFO:tasks.workunit.client.0.smithi063.stdout:7/400: mkdir d0/d15/d82 0 2024-05-09T06:38:10.096 INFO:tasks.workunit.client.0.smithi063.stdout:2/451: dread dc/dd/d2d/d60/f67 [0,4194304] 0 2024-05-09T06:38:10.097 INFO:tasks.workunit.client.0.smithi063.stdout:6/420: dread d2/f7 [0,4194304] 0 2024-05-09T06:38:10.098 INFO:tasks.workunit.client.0.smithi063.stdout:2/452: write dc/dd/d2d/f38 [1740060,35696] 0 2024-05-09T06:38:10.100 INFO:tasks.workunit.client.1.smithi193.stdout:4/79: mkdir d1a 0 2024-05-09T06:38:10.100 INFO:tasks.workunit.client.0.smithi063.stdout:2/453: write dc/f1b [3379530,88327] 0 2024-05-09T06:38:10.100 INFO:tasks.workunit.client.0.smithi063.stdout:6/421: read d2/d1f/d29/d3f/f58 [1081090,92935] 0 2024-05-09T06:38:10.101 INFO:tasks.workunit.client.0.smithi063.stdout:1/372: mkdir d9/d7f 0 2024-05-09T06:38:10.103 INFO:tasks.workunit.client.1.smithi193.stdout:3/43: getdents d1/db 0 2024-05-09T06:38:10.104 INFO:tasks.workunit.client.1.smithi193.stdout:3/44: rename d1 to d1/de 22 2024-05-09T06:38:10.104 INFO:tasks.workunit.client.1.smithi193.stdout:3/45: fdatasync d1/f8 0 2024-05-09T06:38:10.105 INFO:tasks.workunit.client.1.smithi193.stdout:3/46: chown d1/db/cc 278 1 2024-05-09T06:38:10.105 INFO:tasks.workunit.client.1.smithi193.stdout:3/47: readlink d1/db/ld 0 2024-05-09T06:38:10.106 INFO:tasks.workunit.client.1.smithi193.stdout:5/61: mkdir d0/d13 0 2024-05-09T06:38:10.107 INFO:tasks.workunit.client.1.smithi193.stdout:3/48: dread d1/f9 [0,4194304] 0 2024-05-09T06:38:10.107 INFO:tasks.workunit.client.0.smithi063.stdout:3/438: truncate d7/d2a/d2d/f5d 3523934 0 2024-05-09T06:38:10.108 INFO:tasks.workunit.client.0.smithi063.stdout:3/439: fsync d7/d2a/d5b/f78 0 2024-05-09T06:38:10.109 INFO:tasks.workunit.client.0.smithi063.stdout:5/378: rmdir d8/d19/d30 39 2024-05-09T06:38:10.110 INFO:tasks.workunit.client.0.smithi063.stdout:5/379: fdatasync d8/d17/d1e/f42 0 2024-05-09T06:38:10.111 INFO:tasks.workunit.client.0.smithi063.stdout:4/417: rmdir d7/df/d1f/d28/d31/d7b 39 2024-05-09T06:38:10.111 INFO:tasks.workunit.client.0.smithi063.stdout:4/418: fdatasync d7/f2d 0 2024-05-09T06:38:10.112 INFO:tasks.workunit.client.0.smithi063.stdout:4/419: write d7/f42 [173044,16949] 0 2024-05-09T06:38:10.113 INFO:tasks.workunit.client.1.smithi193.stdout:8/58: creat d6/db/fe x:0 0 0 2024-05-09T06:38:10.116 INFO:tasks.workunit.client.1.smithi193.stdout:2/77: symlink db/l1c 0 2024-05-09T06:38:10.119 INFO:tasks.workunit.client.1.smithi193.stdout:6/126: creat d1/de/d11/f22 x:0 0 0 2024-05-09T06:38:10.119 INFO:tasks.workunit.client.0.smithi063.stdout:7/401: mknod d0/d15/d49/d54/c83 0 2024-05-09T06:38:10.121 INFO:tasks.workunit.client.1.smithi193.stdout:9/128: getdents dd/d1e/d1f 0 2024-05-09T06:38:10.122 INFO:tasks.workunit.client.1.smithi193.stdout:8/59: dwrite d6/db/fe [0,4194304] 0 2024-05-09T06:38:10.123 INFO:tasks.workunit.client.1.smithi193.stdout:9/129: write f0 [4481223,107522] 0 2024-05-09T06:38:10.124 INFO:tasks.workunit.client.1.smithi193.stdout:8/60: chown d6/ca 0 1 2024-05-09T06:38:10.124 INFO:tasks.workunit.client.1.smithi193.stdout:4/80: read f11 [354769,45068] 0 2024-05-09T06:38:10.131 INFO:tasks.workunit.client.0.smithi063.stdout:4/420: dread d7/df/d1a/f3b [0,4194304] 0 2024-05-09T06:38:10.132 INFO:tasks.workunit.client.0.smithi063.stdout:2/454: dwrite dc/dd/d2d/d60/f67 [4194304,4194304] 0 2024-05-09T06:38:10.132 INFO:tasks.workunit.client.1.smithi193.stdout:3/49: truncate f0 1144882 0 2024-05-09T06:38:10.136 INFO:tasks.workunit.client.1.smithi193.stdout:4/81: read f13 [777917,7307] 0 2024-05-09T06:38:10.137 INFO:tasks.workunit.client.0.smithi063.stdout:6/422: dread d2/d1f/f5b [0,4194304] 0 2024-05-09T06:38:10.139 INFO:tasks.workunit.client.1.smithi193.stdout:9/130: dwrite dd/d1e/f27 [0,4194304] 0 2024-05-09T06:38:10.139 INFO:tasks.workunit.client.1.smithi193.stdout:3/50: symlink d1/lf 0 2024-05-09T06:38:10.139 INFO:tasks.workunit.client.1.smithi193.stdout:5/62: dread d0/d1/da/dd/f11 [4194304,4194304] 0 2024-05-09T06:38:10.140 INFO:tasks.workunit.client.0.smithi063.stdout:0/473: fdatasync dd/d60/d63/d23/d33/f3e 0 2024-05-09T06:38:10.141 INFO:tasks.workunit.client.1.smithi193.stdout:5/63: write d0/fc [255218,33104] 0 2024-05-09T06:38:10.142 INFO:tasks.workunit.client.0.smithi063.stdout:0/474: write dd/d60/d63/d23/d33/f88 [392688,129122] 0 2024-05-09T06:38:10.142 INFO:tasks.workunit.client.0.smithi063.stdout:0/475: chown dd/d1a/d50 87 1 2024-05-09T06:38:10.147 INFO:tasks.workunit.client.0.smithi063.stdout:1/373: symlink d9/l80 0 2024-05-09T06:38:10.149 INFO:tasks.workunit.client.0.smithi063.stdout:9/433: sync 2024-05-09T06:38:10.150 INFO:tasks.workunit.client.0.smithi063.stdout:9/434: write d0/d4/d26/d39/f6c [3798982,26596] 0 2024-05-09T06:38:10.152 INFO:tasks.workunit.client.0.smithi063.stdout:2/455: mkdir dc/d6e/d7f 0 2024-05-09T06:38:10.154 INFO:tasks.workunit.client.1.smithi193.stdout:2/78: dwrite db/df/f10 [0,4194304] 0 2024-05-09T06:38:10.156 INFO:tasks.workunit.client.1.smithi193.stdout:9/131: mknod dd/df/d15/d16/c2c 0 2024-05-09T06:38:10.159 INFO:tasks.workunit.client.1.smithi193.stdout:3/51: mknod d1/db/c10 0 2024-05-09T06:38:10.160 INFO:tasks.workunit.client.1.smithi193.stdout:9/132: dread f0 [0,4194304] 0 2024-05-09T06:38:10.162 INFO:tasks.workunit.client.0.smithi063.stdout:5/380: mkdir d8/d19/d84 0 2024-05-09T06:38:10.164 INFO:tasks.workunit.client.1.smithi193.stdout:6/127: link d1/f3 d1/f23 0 2024-05-09T06:38:10.166 INFO:tasks.workunit.client.0.smithi063.stdout:6/423: rename d2/d1f/c36 to d2/d20/d4a/c8c 0 2024-05-09T06:38:10.167 INFO:tasks.workunit.client.1.smithi193.stdout:5/64: creat d0/d1/da/f14 x:0 0 0 2024-05-09T06:38:10.167 INFO:tasks.workunit.client.0.smithi063.stdout:6/424: chown d2/f85 67416296 1 2024-05-09T06:38:10.168 INFO:tasks.workunit.client.0.smithi063.stdout:6/425: chown d2/d20/f7b 524282 1 2024-05-09T06:38:10.169 INFO:tasks.workunit.client.1.smithi193.stdout:8/61: truncate f0 6708661 0 2024-05-09T06:38:10.170 INFO:tasks.workunit.client.0.smithi063.stdout:2/456: dread dc/dd/f23 [4194304,4194304] 0 2024-05-09T06:38:10.170 INFO:tasks.workunit.client.1.smithi193.stdout:8/62: dread - d6/fd zero size 2024-05-09T06:38:10.170 INFO:tasks.workunit.client.1.smithi193.stdout:8/63: read - d6/fd zero size 2024-05-09T06:38:10.171 INFO:tasks.workunit.client.1.smithi193.stdout:8/64: chown d6/db 1279066 1 2024-05-09T06:38:10.171 INFO:tasks.workunit.client.0.smithi063.stdout:3/440: truncate d7/f17 456505 0 2024-05-09T06:38:10.172 INFO:tasks.workunit.client.1.smithi193.stdout:2/79: mknod db/df/c1d 0 2024-05-09T06:38:10.173 INFO:tasks.workunit.client.1.smithi193.stdout:2/80: write f8 [904302,115490] 0 2024-05-09T06:38:10.179 INFO:tasks.workunit.client.0.smithi063.stdout:7/402: mknod d0/c84 0 2024-05-09T06:38:10.179 INFO:tasks.workunit.client.0.smithi063.stdout:2/457: dwrite dc/dd/f4f [0,4194304] 0 2024-05-09T06:38:10.179 INFO:tasks.workunit.client.0.smithi063.stdout:2/458: fsync dc/dd/f2f 0 2024-05-09T06:38:10.180 INFO:tasks.workunit.client.1.smithi193.stdout:3/52: creat d1/f11 x:0 0 0 2024-05-09T06:38:10.180 INFO:tasks.workunit.client.1.smithi193.stdout:3/53: truncate d1/f11 552305 0 2024-05-09T06:38:10.180 INFO:tasks.workunit.client.1.smithi193.stdout:9/133: rmdir dd/d1e 39 2024-05-09T06:38:10.180 INFO:tasks.workunit.client.1.smithi193.stdout:2/81: dwrite f8 [0,4194304] 0 2024-05-09T06:38:10.181 INFO:tasks.workunit.client.0.smithi063.stdout:3/441: dwrite d7/da/d2b/d66/d74/d7e/f8b [0,4194304] 0 2024-05-09T06:38:10.181 INFO:tasks.workunit.client.1.smithi193.stdout:2/82: write db/df/d12/f17 [622794,74779] 0 2024-05-09T06:38:10.183 INFO:tasks.workunit.client.1.smithi193.stdout:4/82: truncate f6 1403795 0 2024-05-09T06:38:10.190 INFO:tasks.workunit.client.1.smithi193.stdout:5/65: fdatasync d0/d1/da/dd/f11 0 2024-05-09T06:38:10.194 INFO:tasks.workunit.client.0.smithi063.stdout:4/421: creat d7/df/d35/d78/f86 x:0 0 0 2024-05-09T06:38:10.195 INFO:tasks.workunit.client.1.smithi193.stdout:8/65: creat d6/ff x:0 0 0 2024-05-09T06:38:10.196 INFO:tasks.workunit.client.1.smithi193.stdout:8/66: stat d6/db 0 2024-05-09T06:38:10.201 INFO:tasks.workunit.client.0.smithi063.stdout:6/426: creat d2/d1f/d29/d3f/d5c/d64/f8d x:0 0 0 2024-05-09T06:38:10.203 INFO:tasks.workunit.client.0.smithi063.stdout:6/427: truncate d2/d1f/d29/d3b/d4e/f6b 479553 0 2024-05-09T06:38:10.205 INFO:tasks.workunit.client.1.smithi193.stdout:9/134: stat dd/d1e/f28 0 2024-05-09T06:38:10.206 INFO:tasks.workunit.client.1.smithi193.stdout:2/83: creat db/f1e x:0 0 0 2024-05-09T06:38:10.206 INFO:tasks.workunit.client.0.smithi063.stdout:8/492: getdents d9 0 2024-05-09T06:38:10.207 INFO:tasks.workunit.client.0.smithi063.stdout:1/374: link d9/d2a/d35/d1e/f27 d9/d2a/d2e/f81 0 2024-05-09T06:38:10.208 INFO:tasks.workunit.client.1.smithi193.stdout:6/128: unlink d1/f3 0 2024-05-09T06:38:10.209 INFO:tasks.workunit.client.0.smithi063.stdout:1/375: write d9/d10/d2b/f33 [3839903,35351] 0 2024-05-09T06:38:10.212 INFO:tasks.workunit.client.0.smithi063.stdout:2/459: read dc/f21 [4212684,41764] 0 2024-05-09T06:38:10.213 INFO:tasks.workunit.client.1.smithi193.stdout:8/67: mknod d6/db/c10 0 2024-05-09T06:38:10.215 INFO:tasks.workunit.client.0.smithi063.stdout:7/403: rename d0/f80 to d0/d15/d49/d4f/d66/d67/d6b/f85 0 2024-05-09T06:38:10.217 INFO:tasks.workunit.client.1.smithi193.stdout:3/54: fdatasync d1/f4 0 2024-05-09T06:38:10.217 INFO:tasks.workunit.client.0.smithi063.stdout:2/460: dwrite dc/f1b [0,4194304] 0 2024-05-09T06:38:10.219 INFO:tasks.workunit.client.0.smithi063.stdout:4/422: fdatasync d7/df/f12 0 2024-05-09T06:38:10.221 INFO:tasks.workunit.client.1.smithi193.stdout:9/135: unlink dd/d1e/d1f/c23 0 2024-05-09T06:38:10.222 INFO:tasks.workunit.client.0.smithi063.stdout:5/381: symlink d8/d7e/l85 0 2024-05-09T06:38:10.223 INFO:tasks.workunit.client.1.smithi193.stdout:2/84: mknod db/df/c1f 0 2024-05-09T06:38:10.226 INFO:tasks.workunit.client.0.smithi063.stdout:4/423: dwrite d7/df/d1f/f3a [0,4194304] 0 2024-05-09T06:38:10.226 INFO:tasks.workunit.client.1.smithi193.stdout:4/83: creat d1a/f1b x:0 0 0 2024-05-09T06:38:10.227 INFO:tasks.workunit.client.0.smithi063.stdout:0/476: write dd/d1a/f65 [706331,109129] 0 2024-05-09T06:38:10.228 INFO:tasks.workunit.client.1.smithi193.stdout:6/129: creat d1/de/d11/d18/f24 x:0 0 0 2024-05-09T06:38:10.230 INFO:tasks.workunit.client.1.smithi193.stdout:5/66: link d0/d1/da/dd/f11 d0/d1/da/f15 0 2024-05-09T06:38:10.232 INFO:tasks.workunit.client.0.smithi063.stdout:7/404: dread d0/d15/f26 [0,4194304] 0 2024-05-09T06:38:10.233 INFO:tasks.workunit.client.0.smithi063.stdout:8/493: symlink d9/da/d45/la5 0 2024-05-09T06:38:10.234 INFO:tasks.workunit.client.0.smithi063.stdout:1/376: creat d9/d10/d2b/f82 x:0 0 0 2024-05-09T06:38:10.240 INFO:tasks.workunit.client.1.smithi193.stdout:2/85: chown db/df/c19 167430081 1 2024-05-09T06:38:10.242 INFO:tasks.workunit.client.0.smithi063.stdout:3/442: rename d7/d2a to d7/d4c/d90 0 2024-05-09T06:38:10.247 INFO:tasks.workunit.client.1.smithi193.stdout:5/67: unlink d0/c5 0 2024-05-09T06:38:10.248 INFO:tasks.workunit.client.1.smithi193.stdout:5/68: dread - d0/d1/da/f14 zero size 2024-05-09T06:38:10.250 INFO:tasks.workunit.client.1.smithi193.stdout:8/68: rename f0 to d6/db/f11 0 2024-05-09T06:38:10.250 INFO:tasks.workunit.client.1.smithi193.stdout:8/69: chown d6 10551 1 2024-05-09T06:38:10.251 INFO:tasks.workunit.client.1.smithi193.stdout:8/70: truncate d6/ff 248787 0 2024-05-09T06:38:10.254 INFO:tasks.workunit.client.0.smithi063.stdout:4/424: unlink d7/df/d35/c66 0 2024-05-09T06:38:10.254 INFO:tasks.workunit.client.1.smithi193.stdout:8/71: dread d6/db/fe [0,4194304] 0 2024-05-09T06:38:10.255 INFO:tasks.workunit.client.0.smithi063.stdout:4/425: write d7/df/d1f/f65 [1254737,99982] 0 2024-05-09T06:38:10.258 INFO:tasks.workunit.client.1.smithi193.stdout:6/130: mkdir d1/d25 0 2024-05-09T06:38:10.258 INFO:tasks.workunit.client.0.smithi063.stdout:6/428: getdents d2/d89 0 2024-05-09T06:38:10.260 INFO:tasks.workunit.client.1.smithi193.stdout:5/69: creat d0/d1/f16 x:0 0 0 2024-05-09T06:38:10.260 INFO:tasks.workunit.client.0.smithi063.stdout:4/426: dwrite d7/df/d35/f7f [4194304,4194304] 0 2024-05-09T06:38:10.262 INFO:tasks.workunit.client.0.smithi063.stdout:7/405: mkdir d0/d15/d49/d4f/d66/d67/d86 0 2024-05-09T06:38:10.264 INFO:tasks.workunit.client.0.smithi063.stdout:1/377: mkdir d9/d2a/d3c/d4c/d83 0 2024-05-09T06:38:10.268 INFO:tasks.workunit.client.0.smithi063.stdout:9/435: rename d0/d3b/c45 to d0/d8/d2d/c95 0 2024-05-09T06:38:10.271 INFO:tasks.workunit.client.1.smithi193.stdout:8/72: rename d6/ff to d6/db/f12 0 2024-05-09T06:38:10.271 INFO:tasks.workunit.client.1.smithi193.stdout:6/131: rename d1 to d1/de/d26 22 2024-05-09T06:38:10.273 INFO:tasks.workunit.client.1.smithi193.stdout:4/84: link c19 d1a/c1c 0 2024-05-09T06:38:10.274 INFO:tasks.workunit.client.0.smithi063.stdout:9/436: dwrite d0/d4/d26/d47/f2b [0,4194304] 0 2024-05-09T06:38:10.274 INFO:tasks.workunit.client.1.smithi193.stdout:8/73: dread d6/db/f12 [0,4194304] 0 2024-05-09T06:38:10.274 INFO:tasks.workunit.client.1.smithi193.stdout:4/85: dread - d1a/f1b zero size 2024-05-09T06:38:10.278 INFO:tasks.workunit.client.0.smithi063.stdout:3/443: dwrite d7/d22/f50 [0,4194304] 0 2024-05-09T06:38:10.281 INFO:tasks.workunit.client.1.smithi193.stdout:5/70: rename d0/d1/da/f12 to d0/d1/da/dd/f17 0 2024-05-09T06:38:10.282 INFO:tasks.workunit.client.0.smithi063.stdout:0/477: creat dd/d1a/d2c/d51/d76/d97/f9d x:0 0 0 2024-05-09T06:38:10.285 INFO:tasks.workunit.client.1.smithi193.stdout:4/86: readlink l2 0 2024-05-09T06:38:10.288 INFO:tasks.workunit.client.1.smithi193.stdout:8/74: creat d6/db/f13 x:0 0 0 2024-05-09T06:38:10.290 INFO:tasks.workunit.client.0.smithi063.stdout:4/427: symlink d7/df/d57/l87 0 2024-05-09T06:38:10.291 INFO:tasks.workunit.client.1.smithi193.stdout:2/86: getdents db/df 0 2024-05-09T06:38:10.293 INFO:tasks.workunit.client.1.smithi193.stdout:0/156: sync 2024-05-09T06:38:10.294 INFO:tasks.workunit.client.1.smithi193.stdout:0/157: chown d18/c2a 400678 1 2024-05-09T06:38:10.295 INFO:tasks.workunit.client.1.smithi193.stdout:0/158: chown d18/c19 447097 1 2024-05-09T06:38:10.296 INFO:tasks.workunit.client.1.smithi193.stdout:8/75: dwrite d6/db/f13 [0,4194304] 0 2024-05-09T06:38:10.297 INFO:tasks.workunit.client.1.smithi193.stdout:8/76: chown d6/db/fe 15871632 1 2024-05-09T06:38:10.298 INFO:tasks.workunit.client.0.smithi063.stdout:5/382: dwrite d8/d65/f80 [4194304,4194304] 0 2024-05-09T06:38:10.299 INFO:tasks.workunit.client.1.smithi193.stdout:6/132: creat d1/de/d11/d20/f27 x:0 0 0 2024-05-09T06:38:10.299 INFO:tasks.workunit.client.1.smithi193.stdout:8/77: read d6/db/f13 [1772568,38508] 0 2024-05-09T06:38:10.299 INFO:tasks.workunit.client.1.smithi193.stdout:6/133: chown d1 118390871 1 2024-05-09T06:38:10.299 INFO:tasks.workunit.client.0.smithi063.stdout:4/428: dwrite d7/df/d1f/d28/d31/d32/d34/f59 [0,4194304] 0 2024-05-09T06:38:10.301 INFO:tasks.workunit.client.0.smithi063.stdout:9/437: chown d0/d4/d26/f33 2407 1 2024-05-09T06:38:10.302 INFO:tasks.workunit.client.1.smithi193.stdout:2/87: creat db/df/f20 x:0 0 0 2024-05-09T06:38:10.302 INFO:tasks.workunit.client.1.smithi193.stdout:2/88: chown f8 907848 1 2024-05-09T06:38:10.305 INFO:tasks.workunit.client.0.smithi063.stdout:3/444: unlink d7/da/f79 0 2024-05-09T06:38:10.306 INFO:tasks.workunit.client.1.smithi193.stdout:5/71: symlink d0/l18 0 2024-05-09T06:38:10.312 INFO:tasks.workunit.client.0.smithi063.stdout:1/378: symlink d9/d2a/d35/d24/d68/l84 0 2024-05-09T06:38:10.313 INFO:tasks.workunit.client.1.smithi193.stdout:8/78: creat d6/db/f14 x:0 0 0 2024-05-09T06:38:10.314 INFO:tasks.workunit.client.1.smithi193.stdout:5/72: dread d0/d1/da/dd/f11 [0,4194304] 0 2024-05-09T06:38:10.315 INFO:tasks.workunit.client.1.smithi193.stdout:2/89: creat db/f21 x:0 0 0 2024-05-09T06:38:10.317 INFO:tasks.workunit.client.0.smithi063.stdout:1/379: dwrite d9/f4e [0,4194304] 0 2024-05-09T06:38:10.319 INFO:tasks.workunit.client.0.smithi063.stdout:9/438: rename d0/d4/c92 to d0/d4/d26/d39/d91/c96 0 2024-05-09T06:38:10.322 INFO:tasks.workunit.client.1.smithi193.stdout:2/90: dwrite f8 [0,4194304] 0 2024-05-09T06:38:10.322 INFO:tasks.workunit.client.1.smithi193.stdout:2/91: read f8 [3027748,20845] 0 2024-05-09T06:38:10.323 INFO:tasks.workunit.client.1.smithi193.stdout:8/79: stat c1 0 2024-05-09T06:38:10.324 INFO:tasks.workunit.client.1.smithi193.stdout:8/80: chown d6/db 7 1 2024-05-09T06:38:10.325 INFO:tasks.workunit.client.0.smithi063.stdout:5/383: dread d8/d17/d18/f4a [0,4194304] 0 2024-05-09T06:38:10.328 INFO:tasks.workunit.client.1.smithi193.stdout:8/81: dread d6/db/f13 [0,4194304] 0 2024-05-09T06:38:10.329 INFO:tasks.workunit.client.0.smithi063.stdout:2/461: link dc/dd/f13 dc/d36/f80 0 2024-05-09T06:38:10.330 INFO:tasks.workunit.client.1.smithi193.stdout:4/87: getdents d1a 0 2024-05-09T06:38:10.335 INFO:tasks.workunit.client.1.smithi193.stdout:2/92: mknod db/df/d12/c22 0 2024-05-09T06:38:10.335 INFO:tasks.workunit.client.0.smithi063.stdout:7/406: truncate d0/d15/d49/d4f/d66/d67/d6b/f6e 55026 0 2024-05-09T06:38:10.337 INFO:tasks.workunit.client.1.smithi193.stdout:2/93: truncate db/df/f20 384190 0 2024-05-09T06:38:10.340 INFO:tasks.workunit.client.1.smithi193.stdout:0/159: dwrite f15 [4194304,4194304] 0 2024-05-09T06:38:10.340 INFO:tasks.workunit.client.1.smithi193.stdout:8/82: unlink d6/db/fe 0 2024-05-09T06:38:10.340 INFO:tasks.workunit.client.1.smithi193.stdout:0/160: fdatasync f9 0 2024-05-09T06:38:10.341 INFO:tasks.workunit.client.1.smithi193.stdout:8/83: chown d6/db/f13 6 1 2024-05-09T06:38:10.344 INFO:tasks.workunit.client.0.smithi063.stdout:3/445: truncate f3 224302 0 2024-05-09T06:38:10.344 INFO:tasks.workunit.client.1.smithi193.stdout:4/88: dread f6 [0,4194304] 0 2024-05-09T06:38:10.345 INFO:tasks.workunit.client.0.smithi063.stdout:3/446: write d7/d4c/d90/d5b/f78 [1212799,20032] 0 2024-05-09T06:38:10.347 INFO:tasks.workunit.client.0.smithi063.stdout:3/447: write d7/d4c/d90/d5b/f78 [70397,117959] 0 2024-05-09T06:38:10.347 INFO:tasks.workunit.client.0.smithi063.stdout:6/429: getdents d2/d1f/d29/d3b/d4e 0 2024-05-09T06:38:10.348 INFO:tasks.workunit.client.0.smithi063.stdout:6/430: write d2/d1f/d29/f47 [465422,32251] 0 2024-05-09T06:38:10.350 INFO:tasks.workunit.client.0.smithi063.stdout:5/384: write d8/d19/d30/f61 [577453,44081] 0 2024-05-09T06:38:10.351 INFO:tasks.workunit.client.1.smithi193.stdout:2/94: creat db/df/d12/f23 x:0 0 0 2024-05-09T06:38:10.351 INFO:tasks.workunit.client.0.smithi063.stdout:5/385: dread - d8/d17/d1e/f76 zero size 2024-05-09T06:38:10.352 INFO:tasks.workunit.client.1.smithi193.stdout:2/95: chown db/df/d12/c22 84 1 2024-05-09T06:38:10.352 INFO:tasks.workunit.client.0.smithi063.stdout:2/462: symlink dc/dd/d45/d5a/l81 0 2024-05-09T06:38:10.352 INFO:tasks.workunit.client.1.smithi193.stdout:2/96: write f1 [6558381,7471] 0 2024-05-09T06:38:10.353 INFO:tasks.workunit.client.0.smithi063.stdout:2/463: fsync dc/d32/d33/f61 0 2024-05-09T06:38:10.355 INFO:tasks.workunit.client.0.smithi063.stdout:4/429: creat d7/df/d1f/f88 x:0 0 0 2024-05-09T06:38:10.356 INFO:tasks.workunit.client.0.smithi063.stdout:4/430: dread - d7/df/d35/d5a/f6f zero size 2024-05-09T06:38:10.357 INFO:tasks.workunit.client.0.smithi063.stdout:7/407: readlink d0/d15/d49/d4f/l7f 0 2024-05-09T06:38:10.361 INFO:tasks.workunit.client.1.smithi193.stdout:5/73: write d0/f9 [941150,88572] 0 2024-05-09T06:38:10.362 INFO:tasks.workunit.client.1.smithi193.stdout:8/84: mknod d6/db/c15 0 2024-05-09T06:38:10.363 INFO:tasks.workunit.client.0.smithi063.stdout:9/439: write d0/d8/f69 [4667180,122567] 0 2024-05-09T06:38:10.366 INFO:tasks.workunit.client.1.smithi193.stdout:8/85: dread d6/db/f13 [0,4194304] 0 2024-05-09T06:38:10.367 INFO:tasks.workunit.client.1.smithi193.stdout:2/97: creat db/df/d12/f24 x:0 0 0 2024-05-09T06:38:10.371 INFO:tasks.workunit.client.1.smithi193.stdout:5/74: unlink d0/d1/l4 0 2024-05-09T06:38:10.372 INFO:tasks.workunit.client.0.smithi063.stdout:5/386: truncate d8/d17/f1c 615268 0 2024-05-09T06:38:10.376 INFO:tasks.workunit.client.1.smithi193.stdout:8/86: rmdir d6 39 2024-05-09T06:38:10.378 INFO:tasks.workunit.client.0.smithi063.stdout:2/464: readlink dc/dd/l29 0 2024-05-09T06:38:10.378 INFO:tasks.workunit.client.1.smithi193.stdout:2/98: mkdir db/d25 0 2024-05-09T06:38:10.383 INFO:tasks.workunit.client.0.smithi063.stdout:7/408: symlink d0/d15/l87 0 2024-05-09T06:38:10.383 INFO:tasks.workunit.client.1.smithi193.stdout:2/99: fdatasync f9 0 2024-05-09T06:38:10.383 INFO:tasks.workunit.client.0.smithi063.stdout:1/380: link d9/d2a/d2e/l3b d9/d10/l85 0 2024-05-09T06:38:10.389 INFO:tasks.workunit.client.1.smithi193.stdout:4/89: dwrite f5 [0,4194304] 0 2024-05-09T06:38:10.390 INFO:tasks.workunit.client.1.smithi193.stdout:4/90: dread - d1a/f1b zero size 2024-05-09T06:38:10.391 INFO:tasks.workunit.client.1.smithi193.stdout:2/100: unlink db/cd 0 2024-05-09T06:38:10.393 INFO:tasks.workunit.client.1.smithi193.stdout:0/161: getdents d18/d1d 0 2024-05-09T06:38:10.393 INFO:tasks.workunit.client.0.smithi063.stdout:5/387: symlink d8/d17/d1e/l86 0 2024-05-09T06:38:10.394 INFO:tasks.workunit.client.1.smithi193.stdout:5/75: symlink d0/d13/l19 0 2024-05-09T06:38:10.395 INFO:tasks.workunit.client.1.smithi193.stdout:8/87: write d6/db/f12 [1060243,41604] 0 2024-05-09T06:38:10.396 INFO:tasks.workunit.client.1.smithi193.stdout:8/88: read d6/db/f13 [2760171,75177] 0 2024-05-09T06:38:10.398 INFO:tasks.workunit.client.0.smithi063.stdout:7/409: mknod d0/d15/d49/d4f/d66/d67/c88 0 2024-05-09T06:38:10.399 INFO:tasks.workunit.client.1.smithi193.stdout:4/91: creat d1a/f1d x:0 0 0 2024-05-09T06:38:10.400 INFO:tasks.workunit.client.0.smithi063.stdout:1/381: symlink d9/d10/d2b/d3f/d59/d62/l86 0 2024-05-09T06:38:10.400 INFO:tasks.workunit.client.0.smithi063.stdout:1/382: chown d9/d76 803283263 1 2024-05-09T06:38:10.401 INFO:tasks.workunit.client.1.smithi193.stdout:2/101: mknod db/df/d12/c26 0 2024-05-09T06:38:10.403 INFO:tasks.workunit.client.1.smithi193.stdout:0/162: symlink d18/d27/l31 0 2024-05-09T06:38:10.406 INFO:tasks.workunit.client.1.smithi193.stdout:5/76: creat d0/d1/da/dd/f1a x:0 0 0 2024-05-09T06:38:10.408 INFO:tasks.workunit.client.1.smithi193.stdout:8/89: symlink d6/l16 0 2024-05-09T06:38:10.410 INFO:tasks.workunit.client.1.smithi193.stdout:5/77: dread d0/d1/da/f15 [0,4194304] 0 2024-05-09T06:38:10.411 INFO:tasks.workunit.client.1.smithi193.stdout:4/92: mknod d1a/c1e 0 2024-05-09T06:38:10.411 INFO:tasks.workunit.client.1.smithi193.stdout:5/78: stat d0/f8 0 2024-05-09T06:38:10.411 INFO:tasks.workunit.client.0.smithi063.stdout:6/431: link d2/d12/l15 d2/d1f/d29/d3b/l8e 0 2024-05-09T06:38:10.415 INFO:tasks.workunit.client.1.smithi193.stdout:8/90: creat d6/db/f17 x:0 0 0 2024-05-09T06:38:10.416 INFO:tasks.workunit.client.1.smithi193.stdout:4/93: dwrite f17 [0,4194304] 0 2024-05-09T06:38:10.416 INFO:tasks.workunit.client.1.smithi193.stdout:8/91: chown d6/db/f14 125787 1 2024-05-09T06:38:10.418 INFO:tasks.workunit.client.0.smithi063.stdout:7/410: mkdir d0/d15/d89 0 2024-05-09T06:38:10.419 INFO:tasks.workunit.client.1.smithi193.stdout:8/92: dread d6/db/f12 [0,4194304] 0 2024-05-09T06:38:10.419 INFO:tasks.workunit.client.0.smithi063.stdout:7/411: fsync d0/d15/d49/f70 0 2024-05-09T06:38:10.419 INFO:tasks.workunit.client.0.smithi063.stdout:7/412: dread - d0/d15/d49/d4f/d66/f6c zero size 2024-05-09T06:38:10.419 INFO:tasks.workunit.client.1.smithi193.stdout:5/79: creat d0/d13/f1b x:0 0 0 2024-05-09T06:38:10.420 INFO:tasks.workunit.client.1.smithi193.stdout:5/80: dread - d0/d13/f1b zero size 2024-05-09T06:38:10.420 INFO:tasks.workunit.client.0.smithi063.stdout:1/383: chown d9/d2a/d35/d1e/l6d 376846265 1 2024-05-09T06:38:10.421 INFO:tasks.workunit.client.1.smithi193.stdout:2/102: creat db/d25/f27 x:0 0 0 2024-05-09T06:38:10.422 INFO:tasks.workunit.client.1.smithi193.stdout:2/103: chown db/d25/f27 1 1 2024-05-09T06:38:10.423 INFO:tasks.workunit.client.1.smithi193.stdout:0/163: symlink d18/l32 0 2024-05-09T06:38:10.424 INFO:tasks.workunit.client.0.smithi063.stdout:6/432: symlink d2/d1f/d29/d3b/l8f 0 2024-05-09T06:38:10.425 INFO:tasks.workunit.client.1.smithi193.stdout:4/94: write f7 [2896068,14808] 0 2024-05-09T06:38:10.428 INFO:tasks.workunit.client.0.smithi063.stdout:2/465: rename dc/dd/d2d/f38 to dc/dd/d2d/f82 0 2024-05-09T06:38:10.429 INFO:tasks.workunit.client.1.smithi193.stdout:8/93: fdatasync d6/db/f13 0 2024-05-09T06:38:10.429 INFO:tasks.workunit.client.1.smithi193.stdout:8/94: readlink d6/l16 0 2024-05-09T06:38:10.430 INFO:tasks.workunit.client.0.smithi063.stdout:3/448: truncate f3 169268 0 2024-05-09T06:38:10.431 INFO:tasks.workunit.client.1.smithi193.stdout:5/81: unlink d0/d1/l10 0 2024-05-09T06:38:10.431 INFO:tasks.workunit.client.0.smithi063.stdout:6/433: dread d2/d12/f53 [0,4194304] 0 2024-05-09T06:38:10.431 INFO:tasks.workunit.client.0.smithi063.stdout:4/431: link d7/df/c74 d7/df/d35/c89 0 2024-05-09T06:38:10.432 INFO:tasks.workunit.client.1.smithi193.stdout:2/104: creat db/f28 x:0 0 0 2024-05-09T06:38:10.434 INFO:tasks.workunit.client.0.smithi063.stdout:4/432: write d7/df/d1f/d28/f7d [945312,120260] 0 2024-05-09T06:38:10.434 INFO:tasks.workunit.client.1.smithi193.stdout:0/164: creat d18/d1d/f33 x:0 0 0 2024-05-09T06:38:10.436 INFO:tasks.workunit.client.0.smithi063.stdout:2/466: dwrite dc/dd/d2d/f34 [0,4194304] 0 2024-05-09T06:38:10.439 INFO:tasks.workunit.client.0.smithi063.stdout:9/440: getdents d0/d4/d26/d39/d48/d80/d84 0 2024-05-09T06:38:10.439 INFO:tasks.workunit.client.0.smithi063.stdout:7/413: dread d0/d15/d4a/d50/f23 [0,4194304] 0 2024-05-09T06:38:10.439 INFO:tasks.workunit.client.0.smithi063.stdout:3/449: dread d7/da/d2b/d66/d74/d7e/f8b [0,4194304] 0 2024-05-09T06:38:10.439 INFO:tasks.workunit.client.1.smithi193.stdout:5/82: dwrite d0/d1/f16 [0,4194304] 0 2024-05-09T06:38:10.443 INFO:tasks.workunit.client.0.smithi063.stdout:0/478: sync 2024-05-09T06:38:10.443 INFO:tasks.workunit.client.0.smithi063.stdout:8/494: sync 2024-05-09T06:38:10.443 INFO:tasks.workunit.client.0.smithi063.stdout:4/433: dwrite f5 [0,4194304] 0 2024-05-09T06:38:10.445 INFO:tasks.workunit.client.0.smithi063.stdout:8/495: dread - d9/da/d18/d46/f97 zero size 2024-05-09T06:38:10.445 INFO:tasks.workunit.client.0.smithi063.stdout:4/434: fsync d7/df/d1f/d28/d31/d61/f6d 0 2024-05-09T06:38:10.446 INFO:tasks.workunit.client.1.smithi193.stdout:5/83: dread d0/d1/f16 [0,4194304] 0 2024-05-09T06:38:10.448 INFO:tasks.workunit.client.1.smithi193.stdout:2/105: mknod db/c29 0 2024-05-09T06:38:10.450 INFO:tasks.workunit.client.0.smithi063.stdout:5/388: creat d8/f87 x:0 0 0 2024-05-09T06:38:10.451 INFO:tasks.workunit.client.0.smithi063.stdout:5/389: dread - d8/d17/d1e/f5f zero size 2024-05-09T06:38:10.452 INFO:tasks.workunit.client.0.smithi063.stdout:2/467: dread dc/d32/f42 [0,4194304] 0 2024-05-09T06:38:10.455 INFO:tasks.workunit.client.0.smithi063.stdout:4/435: dread d7/df/d1f/d28/d31/f5f [0,4194304] 0 2024-05-09T06:38:10.456 INFO:tasks.workunit.client.0.smithi063.stdout:4/436: write d7/df/d35/f36 [1739704,23655] 0 2024-05-09T06:38:10.457 INFO:tasks.workunit.client.1.smithi193.stdout:0/165: creat d18/d27/f34 x:0 0 0 2024-05-09T06:38:10.458 INFO:tasks.workunit.client.0.smithi063.stdout:4/437: write d7/df/d1f/f3a [1002505,99246] 0 2024-05-09T06:38:10.459 INFO:tasks.workunit.client.0.smithi063.stdout:6/434: rename d2/d4/d7c/f79 to d2/d1f/d29/d3f/d5c/f90 0 2024-05-09T06:38:10.461 INFO:tasks.workunit.client.0.smithi063.stdout:4/438: write d7/df/d1f/d28/d31/d32/d34/f59 [368939,125642] 0 2024-05-09T06:38:10.461 INFO:tasks.workunit.client.0.smithi063.stdout:6/435: chown d2/d20/f8a 478095482 1 2024-05-09T06:38:10.461 INFO:tasks.workunit.client.1.smithi193.stdout:8/95: truncate d6/db/f11 7168883 0 2024-05-09T06:38:10.463 INFO:tasks.workunit.client.0.smithi063.stdout:6/436: chown d2/d1f/d29/f47 1 1 2024-05-09T06:38:10.463 INFO:tasks.workunit.client.1.smithi193.stdout:2/106: creat db/d25/f2a x:0 0 0 2024-05-09T06:38:10.464 INFO:tasks.workunit.client.0.smithi063.stdout:6/437: stat d2/d12/f34 0 2024-05-09T06:38:10.464 INFO:tasks.workunit.client.1.smithi193.stdout:2/107: dread - db/d25/f2a zero size 2024-05-09T06:38:10.464 INFO:tasks.workunit.client.1.smithi193.stdout:0/166: mknod d18/d27/c35 0 2024-05-09T06:38:10.466 INFO:tasks.workunit.client.0.smithi063.stdout:4/439: dread d7/df/d1a/f22 [0,4194304] 0 2024-05-09T06:38:10.468 INFO:tasks.workunit.client.1.smithi193.stdout:2/108: dread f9 [0,4194304] 0 2024-05-09T06:38:10.469 INFO:tasks.workunit.client.1.smithi193.stdout:0/167: rename d18/d27/c2f to d18/d1d/c36 0 2024-05-09T06:38:10.469 INFO:tasks.workunit.client.1.smithi193.stdout:2/109: write db/df/d12/f23 [575857,94002] 0 2024-05-09T06:38:10.471 INFO:tasks.workunit.client.1.smithi193.stdout:0/168: dread - d18/d1d/f33 zero size 2024-05-09T06:38:10.472 INFO:tasks.workunit.client.1.smithi193.stdout:0/169: write f13 [1164669,103727] 0 2024-05-09T06:38:10.473 INFO:tasks.workunit.client.1.smithi193.stdout:0/170: chown d18/c1c 2239 1 2024-05-09T06:38:10.473 INFO:tasks.workunit.client.1.smithi193.stdout:4/95: write f11 [25855,56519] 0 2024-05-09T06:38:10.474 INFO:tasks.workunit.client.1.smithi193.stdout:0/171: write d18/d1d/f24 [721919,9362] 0 2024-05-09T06:38:10.476 INFO:tasks.workunit.client.1.smithi193.stdout:5/84: getdents d0/d1/da 0 2024-05-09T06:38:10.478 INFO:tasks.workunit.client.1.smithi193.stdout:2/110: symlink db/df/l2b 0 2024-05-09T06:38:10.479 INFO:tasks.workunit.client.0.smithi063.stdout:2/468: dread dc/dd/d2d/f82 [0,4194304] 0 2024-05-09T06:38:10.480 INFO:tasks.workunit.client.0.smithi063.stdout:2/469: read dc/dd/d2d/f82 [1831182,71631] 0 2024-05-09T06:38:10.480 INFO:tasks.workunit.client.0.smithi063.stdout:5/390: dread d8/d17/d1e/f23 [0,4194304] 0 2024-05-09T06:38:10.480 INFO:tasks.workunit.client.0.smithi063.stdout:2/470: chown dc/dd/f23 43790970 1 2024-05-09T06:38:10.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/391: write d8/f14 [1691025,29911] 0 2024-05-09T06:38:10.484 INFO:tasks.workunit.client.1.smithi193.stdout:4/96: mkdir d1a/d1f 0 2024-05-09T06:38:10.485 INFO:tasks.workunit.client.1.smithi193.stdout:2/111: dwrite db/d25/f2a [0,4194304] 0 2024-05-09T06:38:10.489 INFO:tasks.workunit.client.1.smithi193.stdout:5/85: rename d0/d13/f1b to d0/d1/da/dd/f1c 0 2024-05-09T06:38:10.490 INFO:tasks.workunit.client.1.smithi193.stdout:5/86: dread - d0/d1/da/dd/f1c zero size 2024-05-09T06:38:10.492 INFO:tasks.workunit.client.1.smithi193.stdout:8/96: unlink d6/db/f11 0 2024-05-09T06:38:10.495 INFO:tasks.workunit.client.1.smithi193.stdout:4/97: readlink l15 0 2024-05-09T06:38:10.497 INFO:tasks.workunit.client.0.smithi063.stdout:1/384: truncate d9/d2a/d35/d24/f26 6475332 0 2024-05-09T06:38:10.502 INFO:tasks.workunit.client.0.smithi063.stdout:6/438: write d2/d4/f3c [8839722,20177] 0 2024-05-09T06:38:10.502 INFO:tasks.workunit.client.1.smithi193.stdout:2/112: rmdir db 39 2024-05-09T06:38:10.507 INFO:tasks.workunit.client.1.smithi193.stdout:8/97: mkdir d6/db/d18 0 2024-05-09T06:38:10.507 INFO:tasks.workunit.client.0.smithi063.stdout:6/439: dwrite d2/d1f/d29/f47 [0,4194304] 0 2024-05-09T06:38:10.509 INFO:tasks.workunit.client.0.smithi063.stdout:7/414: symlink d0/d15/d49/d4f/d66/d67/d86/l8a 0 2024-05-09T06:38:10.509 INFO:tasks.workunit.client.0.smithi063.stdout:6/440: write d2/d1f/d29/d3f/d5c/d64/f8d [447961,111042] 0 2024-05-09T06:38:10.510 INFO:tasks.workunit.client.1.smithi193.stdout:5/87: dwrite d0/d1/f16 [0,4194304] 0 2024-05-09T06:38:10.510 INFO:tasks.workunit.client.0.smithi063.stdout:6/441: readlink d2/d20/d26/l7a 0 2024-05-09T06:38:10.511 INFO:tasks.workunit.client.1.smithi193.stdout:5/88: chown d0/d1/f16 1791 1 2024-05-09T06:38:10.513 INFO:tasks.workunit.client.1.smithi193.stdout:5/89: truncate d0/fc 904265 0 2024-05-09T06:38:10.514 INFO:tasks.workunit.client.1.smithi193.stdout:2/113: dread db/df/f20 [0,4194304] 0 2024-05-09T06:38:10.514 INFO:tasks.workunit.client.0.smithi063.stdout:8/496: creat d9/da/d18/d24/d71/fa6 x:0 0 0 2024-05-09T06:38:10.517 INFO:tasks.workunit.client.1.smithi193.stdout:8/98: mknod d6/c19 0 2024-05-09T06:38:10.519 INFO:tasks.workunit.client.1.smithi193.stdout:2/114: write db/df/d12/f17 [983663,6686] 0 2024-05-09T06:38:10.521 INFO:tasks.workunit.client.1.smithi193.stdout:4/98: creat d1a/d1f/f20 x:0 0 0 2024-05-09T06:38:10.526 INFO:tasks.workunit.client.0.smithi063.stdout:3/450: write d7/d4c/d90/d5b/f83 [1827830,9193] 0 2024-05-09T06:38:10.526 INFO:tasks.workunit.client.1.smithi193.stdout:5/90: dwrite d0/f8 [0,4194304] 0 2024-05-09T06:38:10.526 INFO:tasks.workunit.client.0.smithi063.stdout:3/451: dread - d7/d22/d28/f8c zero size 2024-05-09T06:38:10.527 INFO:tasks.workunit.client.0.smithi063.stdout:3/452: chown d7/d4c/d31/f6e 166 1 2024-05-09T06:38:10.528 INFO:tasks.workunit.client.0.smithi063.stdout:5/392: mkdir d8/d7e/d88 0 2024-05-09T06:38:10.529 INFO:tasks.workunit.client.1.smithi193.stdout:8/99: dwrite d6/db/f13 [0,4194304] 0 2024-05-09T06:38:10.530 INFO:tasks.workunit.client.1.smithi193.stdout:8/100: fsync d6/db/f14 0 2024-05-09T06:38:10.531 INFO:tasks.workunit.client.1.smithi193.stdout:2/115: unlink db/df/c1f 0 2024-05-09T06:38:10.533 INFO:tasks.workunit.client.1.smithi193.stdout:5/91: dread d0/d1/da/f15 [0,4194304] 0 2024-05-09T06:38:10.533 INFO:tasks.workunit.client.1.smithi193.stdout:4/99: mknod d1a/d1f/c21 0 2024-05-09T06:38:10.534 INFO:tasks.workunit.client.0.smithi063.stdout:8/497: dread d9/da/d18/d56/d58/f8e [0,4194304] 0 2024-05-09T06:38:10.535 INFO:tasks.workunit.client.0.smithi063.stdout:1/385: mknod d9/d10/d2b/c87 0 2024-05-09T06:38:10.546 INFO:tasks.workunit.client.1.smithi193.stdout:8/101: dread d6/db/f12 [0,4194304] 0 2024-05-09T06:38:10.546 INFO:tasks.workunit.client.1.smithi193.stdout:2/116: dwrite db/f1e [0,4194304] 0 2024-05-09T06:38:10.547 INFO:tasks.workunit.client.0.smithi063.stdout:7/415: symlink d0/d15/d49/d4f/d66/d67/d6b/l8b 0 2024-05-09T06:38:10.548 INFO:tasks.workunit.client.1.smithi193.stdout:4/100: dwrite d1a/f1b [0,4194304] 0 2024-05-09T06:38:10.548 INFO:tasks.workunit.client.0.smithi063.stdout:6/442: mkdir d2/df/d91 0 2024-05-09T06:38:10.549 INFO:tasks.workunit.client.1.smithi193.stdout:8/102: write d6/db/f17 [809952,85774] 0 2024-05-09T06:38:10.549 INFO:tasks.workunit.client.0.smithi063.stdout:7/416: dread d0/d15/d49/d54/f60 [0,4194304] 0 2024-05-09T06:38:10.549 INFO:tasks.workunit.client.1.smithi193.stdout:5/92: dwrite d0/d1/f6 [0,4194304] 0 2024-05-09T06:38:10.553 INFO:tasks.workunit.client.1.smithi193.stdout:8/103: truncate d6/db/f14 830181 0 2024-05-09T06:38:10.555 INFO:tasks.workunit.client.1.smithi193.stdout:2/117: symlink db/df/d12/l2c 0 2024-05-09T06:38:10.563 INFO:tasks.workunit.client.1.smithi193.stdout:5/93: rename d0/d1/cb to d0/d1/da/c1d 0 2024-05-09T06:38:10.564 INFO:tasks.workunit.client.1.smithi193.stdout:5/94: write d0/fc [395300,77620] 0 2024-05-09T06:38:10.565 INFO:tasks.workunit.client.0.smithi063.stdout:9/441: rename d0/d4/d26/f30 to d0/f97 0 2024-05-09T06:38:10.565 INFO:tasks.workunit.client.0.smithi063.stdout:2/471: rename dc to dc/dd/d45/d83 22 2024-05-09T06:38:10.566 INFO:tasks.workunit.client.0.smithi063.stdout:2/472: chown dc/dd/d2d/d60/d65/f7a 107338 1 2024-05-09T06:38:10.567 INFO:tasks.workunit.client.1.smithi193.stdout:8/104: symlink d6/l1a 0 2024-05-09T06:38:10.568 INFO:tasks.workunit.client.0.smithi063.stdout:5/393: creat d8/d19/d30/d74/f89 x:0 0 0 2024-05-09T06:38:10.568 INFO:tasks.workunit.client.0.smithi063.stdout:5/394: fdatasync d8/d17/d18/f58 0 2024-05-09T06:38:10.569 INFO:tasks.workunit.client.0.smithi063.stdout:5/395: dread - d8/f68 zero size 2024-05-09T06:38:10.571 INFO:tasks.workunit.client.0.smithi063.stdout:4/440: creat d7/df/d1f/d28/d31/d7b/d81/f8a x:0 0 0 2024-05-09T06:38:10.573 INFO:tasks.workunit.client.0.smithi063.stdout:0/479: getdents dd 0 2024-05-09T06:38:10.574 INFO:tasks.workunit.client.1.smithi193.stdout:5/95: creat d0/d1/da/dd/f1e x:0 0 0 2024-05-09T06:38:10.574 INFO:tasks.workunit.client.0.smithi063.stdout:4/441: write d7/df/d1f/d28/d31/d32/d34/f80 [879921,113850] 0 2024-05-09T06:38:10.575 INFO:tasks.workunit.client.1.smithi193.stdout:5/96: write d0/f7 [158833,106657] 0 2024-05-09T06:38:10.575 INFO:tasks.workunit.client.1.smithi193.stdout:8/105: mkdir d6/db/d1b 0 2024-05-09T06:38:10.577 INFO:tasks.workunit.client.0.smithi063.stdout:6/443: creat d2/d4/f92 x:0 0 0 2024-05-09T06:38:10.582 INFO:tasks.workunit.client.0.smithi063.stdout:6/444: dwrite d2/d1f/d29/f84 [0,4194304] 0 2024-05-09T06:38:10.586 INFO:tasks.workunit.client.0.smithi063.stdout:2/473: rename dc/dd/f17 to dc/d36/d43/d46/d63/d79/d7d/f84 0 2024-05-09T06:38:10.586 INFO:tasks.workunit.client.0.smithi063.stdout:8/498: creat d9/da/d18/d56/d58/d79/da1/fa7 x:0 0 0 2024-05-09T06:38:10.586 INFO:tasks.workunit.client.0.smithi063.stdout:5/396: creat d8/d17/d18/f8a x:0 0 0 2024-05-09T06:38:10.586 INFO:tasks.workunit.client.0.smithi063.stdout:1/386: mkdir d9/d2a/d3c/d4c/d83/d88 0 2024-05-09T06:38:10.591 INFO:tasks.workunit.client.0.smithi063.stdout:6/445: stat d2/d20/d4a/l77 0 2024-05-09T06:38:10.593 INFO:tasks.workunit.client.0.smithi063.stdout:3/453: dwrite f2 [0,4194304] 0 2024-05-09T06:38:10.594 INFO:tasks.workunit.client.0.smithi063.stdout:0/480: rename dd/d60/d63 to dd/d1a/d2c/d51/d72/d9e 0 2024-05-09T06:38:10.595 INFO:tasks.workunit.client.0.smithi063.stdout:8/499: unlink d9/d37/c7c 0 2024-05-09T06:38:10.595 INFO:tasks.workunit.client.1.smithi193.stdout:4/101: truncate f5 978993 0 2024-05-09T06:38:10.597 INFO:tasks.workunit.client.1.smithi193.stdout:4/102: write f11 [1237873,32689] 0 2024-05-09T06:38:10.597 INFO:tasks.workunit.client.0.smithi063.stdout:4/442: mkdir d7/df/d1f/d8b 0 2024-05-09T06:38:10.597 INFO:tasks.workunit.client.0.smithi063.stdout:8/500: fdatasync d9/da/d34/f59 0 2024-05-09T06:38:10.598 INFO:tasks.workunit.client.1.smithi193.stdout:4/103: dread - d1a/d1f/f20 zero size 2024-05-09T06:38:10.599 INFO:tasks.workunit.client.0.smithi063.stdout:2/474: rmdir dc/dd/d2d/d60/d65 39 2024-05-09T06:38:10.600 INFO:tasks.workunit.client.0.smithi063.stdout:4/443: write d7/f62 [4542232,106755] 0 2024-05-09T06:38:10.605 INFO:tasks.workunit.client.1.smithi193.stdout:4/104: mkdir d1a/d22 0 2024-05-09T06:38:10.606 INFO:tasks.workunit.client.0.smithi063.stdout:0/481: dwrite dd/d1a/d2c/d51/d72/d9e/d23/f2f [4194304,4194304] 0 2024-05-09T06:38:10.606 INFO:tasks.workunit.client.1.smithi193.stdout:8/106: rename d6/db to d6/d1c 0 2024-05-09T06:38:10.607 INFO:tasks.workunit.client.0.smithi063.stdout:0/482: write dd/fe [3293753,98435] 0 2024-05-09T06:38:10.609 INFO:tasks.workunit.client.0.smithi063.stdout:9/442: creat d0/f98 x:0 0 0 2024-05-09T06:38:10.610 INFO:tasks.workunit.client.1.smithi193.stdout:2/118: truncate db/df/f10 2865533 0 2024-05-09T06:38:10.611 INFO:tasks.workunit.client.1.smithi193.stdout:8/107: dread d6/d1c/f13 [0,4194304] 0 2024-05-09T06:38:10.611 INFO:tasks.workunit.client.1.smithi193.stdout:4/105: dread f11 [0,4194304] 0 2024-05-09T06:38:10.616 INFO:tasks.workunit.client.0.smithi063.stdout:1/387: dread d9/d2a/d35/d1e/f3a [4194304,4194304] 0 2024-05-09T06:38:10.616 INFO:tasks.workunit.client.1.smithi193.stdout:8/108: dread d6/d1c/f13 [0,4194304] 0 2024-05-09T06:38:10.616 INFO:tasks.workunit.client.0.smithi063.stdout:3/454: symlink d7/da/d2b/d66/l91 0 2024-05-09T06:38:10.617 INFO:tasks.workunit.client.1.smithi193.stdout:5/97: write d0/d1/da/dd/f11 [4182893,107810] 0 2024-05-09T06:38:10.617 INFO:tasks.workunit.client.0.smithi063.stdout:1/388: write d9/d10/f43 [1692684,43840] 0 2024-05-09T06:38:10.618 INFO:tasks.workunit.client.0.smithi063.stdout:3/455: truncate d7/d4c/f54 1053544 0 2024-05-09T06:38:10.619 INFO:tasks.workunit.client.0.smithi063.stdout:5/397: rename d8/d17/d18/f4a to d8/d53/f8b 0 2024-05-09T06:38:10.621 INFO:tasks.workunit.client.0.smithi063.stdout:5/398: write d8/d17/d18/f29 [748152,65889] 0 2024-05-09T06:38:10.623 INFO:tasks.workunit.client.0.smithi063.stdout:3/456: dread d7/da/d21/f4d [0,4194304] 0 2024-05-09T06:38:10.624 INFO:tasks.workunit.client.1.smithi193.stdout:8/109: dwrite d6/d1c/f17 [0,4194304] 0 2024-05-09T06:38:10.625 INFO:tasks.workunit.client.0.smithi063.stdout:3/457: fdatasync d7/d22/d28/f3f 0 2024-05-09T06:38:10.625 INFO:tasks.workunit.client.1.smithi193.stdout:5/98: dread d0/d1/f16 [0,4194304] 0 2024-05-09T06:38:10.629 INFO:tasks.workunit.client.0.smithi063.stdout:1/389: dread d9/d10/f1c [0,4194304] 0 2024-05-09T06:38:10.637 INFO:tasks.workunit.client.0.smithi063.stdout:7/417: link d0/d15/d4a/d50/l7c d0/d15/d49/d4f/d66/d67/d86/l8c 0 2024-05-09T06:38:10.641 INFO:tasks.workunit.client.1.smithi193.stdout:5/99: creat d0/d13/f1f x:0 0 0 2024-05-09T06:38:10.642 INFO:tasks.workunit.client.0.smithi063.stdout:9/443: creat d0/d4/d7d/f99 x:0 0 0 2024-05-09T06:38:10.645 INFO:tasks.workunit.client.1.smithi193.stdout:8/110: creat d6/d1c/d18/f1d x:0 0 0 2024-05-09T06:38:10.648 INFO:tasks.workunit.client.1.smithi193.stdout:8/111: dread d6/d1c/f14 [0,4194304] 0 2024-05-09T06:38:10.649 INFO:tasks.workunit.client.1.smithi193.stdout:5/100: mknod d0/d1/da/dd/c20 0 2024-05-09T06:38:10.649 INFO:tasks.workunit.client.1.smithi193.stdout:8/112: write d6/d1c/d18/f1d [300169,6993] 0 2024-05-09T06:38:10.650 INFO:tasks.workunit.client.1.smithi193.stdout:8/113: stat d6/c19 0 2024-05-09T06:38:10.650 INFO:tasks.workunit.client.1.smithi193.stdout:8/114: readlink d6/l1a 0 2024-05-09T06:38:10.653 INFO:tasks.workunit.client.0.smithi063.stdout:5/399: unlink c2 0 2024-05-09T06:38:10.654 INFO:tasks.workunit.client.1.smithi193.stdout:5/101: creat d0/d1/f21 x:0 0 0 2024-05-09T06:38:10.655 INFO:tasks.workunit.client.1.smithi193.stdout:5/102: fdatasync d0/d1/f21 0 2024-05-09T06:38:10.658 INFO:tasks.workunit.client.1.smithi193.stdout:4/106: dwrite f6 [0,4194304] 0 2024-05-09T06:38:10.659 INFO:tasks.workunit.client.1.smithi193.stdout:8/115: creat d6/d1c/f1e x:0 0 0 2024-05-09T06:38:10.661 INFO:tasks.workunit.client.1.smithi193.stdout:8/116: truncate d6/fd 328127 0 2024-05-09T06:38:10.661 INFO:tasks.workunit.client.0.smithi063.stdout:4/444: write d7/df/f12 [2963321,18355] 0 2024-05-09T06:38:10.661 INFO:tasks.workunit.client.0.smithi063.stdout:2/475: mknod dc/d6e/d7f/c85 0 2024-05-09T06:38:10.662 INFO:tasks.workunit.client.1.smithi193.stdout:2/119: truncate db/d25/f2a 1321059 0 2024-05-09T06:38:10.663 INFO:tasks.workunit.client.1.smithi193.stdout:5/103: creat d0/d1/da/dd/f22 x:0 0 0 2024-05-09T06:38:10.663 INFO:tasks.workunit.client.1.smithi193.stdout:2/120: write db/d25/f27 [360396,56707] 0 2024-05-09T06:38:10.665 INFO:tasks.workunit.client.1.smithi193.stdout:5/104: dread - d0/d1/da/dd/f1a zero size 2024-05-09T06:38:10.665 INFO:tasks.workunit.client.1.smithi193.stdout:1/98: sync 2024-05-09T06:38:10.666 INFO:tasks.workunit.client.1.smithi193.stdout:1/99: truncate d4/ff 105278 0 2024-05-09T06:38:10.667 INFO:tasks.workunit.client.1.smithi193.stdout:1/100: write d4/d16/f10 [603278,104162] 0 2024-05-09T06:38:10.668 INFO:tasks.workunit.client.0.smithi063.stdout:1/390: dwrite d9/d10/f3e [0,4194304] 0 2024-05-09T06:38:10.669 INFO:tasks.workunit.client.0.smithi063.stdout:6/446: write d2/d1f/d29/f63 [2421935,49323] 0 2024-05-09T06:38:10.671 INFO:tasks.workunit.client.0.smithi063.stdout:6/447: write d2/d1f/f69 [988457,36789] 0 2024-05-09T06:38:10.673 INFO:tasks.workunit.client.1.smithi193.stdout:2/121: mknod db/c2d 0 2024-05-09T06:38:10.676 INFO:tasks.workunit.client.1.smithi193.stdout:5/105: creat d0/d1/da/f23 x:0 0 0 2024-05-09T06:38:10.678 INFO:tasks.workunit.client.1.smithi193.stdout:5/106: dread - d0/d1/da/f14 zero size 2024-05-09T06:38:10.678 INFO:tasks.workunit.client.1.smithi193.stdout:2/122: dread f8 [0,4194304] 0 2024-05-09T06:38:10.679 INFO:tasks.workunit.client.1.smithi193.stdout:2/123: chown f6 194 1 2024-05-09T06:38:10.680 INFO:tasks.workunit.client.1.smithi193.stdout:8/117: creat d6/d1c/d1b/f1f x:0 0 0 2024-05-09T06:38:10.682 INFO:tasks.workunit.client.0.smithi063.stdout:5/400: unlink d8/d17/d18/f29 0 2024-05-09T06:38:10.682 INFO:tasks.workunit.client.1.smithi193.stdout:5/107: dread d0/d1/da/dd/f11 [0,4194304] 0 2024-05-09T06:38:10.683 INFO:tasks.workunit.client.0.smithi063.stdout:3/458: write d7/d4c/f80 [1018620,73360] 0 2024-05-09T06:38:10.685 INFO:tasks.workunit.client.0.smithi063.stdout:5/401: dread d8/d17/d18/f82 [0,4194304] 0 2024-05-09T06:38:10.687 INFO:tasks.workunit.client.0.smithi063.stdout:5/402: write d8/d19/d30/d46/f6c [851037,25301] 0 2024-05-09T06:38:10.687 INFO:tasks.workunit.client.0.smithi063.stdout:4/445: creat d7/df/d1f/d28/d31/d32/f8c x:0 0 0 2024-05-09T06:38:10.688 INFO:tasks.workunit.client.1.smithi193.stdout:2/124: mknod db/df/d12/c2e 0 2024-05-09T06:38:10.689 INFO:tasks.workunit.client.0.smithi063.stdout:8/501: link d9/da/d18/d24/l7f d9/da/d45/la8 0 2024-05-09T06:38:10.690 INFO:tasks.workunit.client.1.smithi193.stdout:8/118: creat d6/f20 x:0 0 0 2024-05-09T06:38:10.692 INFO:tasks.workunit.client.0.smithi063.stdout:7/418: mknod d0/d15/d82/c8d 0 2024-05-09T06:38:10.694 INFO:tasks.workunit.client.1.smithi193.stdout:5/108: symlink d0/d1/da/l24 0 2024-05-09T06:38:10.694 INFO:tasks.workunit.client.1.smithi193.stdout:5/109: stat d0/fc 0 2024-05-09T06:38:10.695 INFO:tasks.workunit.client.0.smithi063.stdout:7/419: fdatasync d0/d15/d4a/d50/f3f 0 2024-05-09T06:38:10.695 INFO:tasks.workunit.client.0.smithi063.stdout:0/483: creat dd/d1a/f9f x:0 0 0 2024-05-09T06:38:10.695 INFO:tasks.workunit.client.0.smithi063.stdout:8/502: dwrite d9/da/d34/f87 [0,4194304] 0 2024-05-09T06:38:10.696 INFO:tasks.workunit.client.1.smithi193.stdout:4/107: getdents d1a/d1f 0 2024-05-09T06:38:10.697 INFO:tasks.workunit.client.0.smithi063.stdout:6/448: creat d2/d20/d4a/f93 x:0 0 0 2024-05-09T06:38:10.699 INFO:tasks.workunit.client.0.smithi063.stdout:6/449: write d2/d1f/d29/f47 [1841230,105396] 0 2024-05-09T06:38:10.699 INFO:tasks.workunit.client.0.smithi063.stdout:9/444: mknod d0/d4/c9a 0 2024-05-09T06:38:10.701 INFO:tasks.workunit.client.1.smithi193.stdout:2/125: rename db/df/d12/f23 to db/df/f2f 0 2024-05-09T06:38:10.702 INFO:tasks.workunit.client.0.smithi063.stdout:8/503: dwrite d9/da/d18/f63 [0,4194304] 0 2024-05-09T06:38:10.704 INFO:tasks.workunit.client.0.smithi063.stdout:1/391: truncate d9/d10/d2b/d3f/d59/d62/f6c 1032835 0 2024-05-09T06:38:10.705 INFO:tasks.workunit.client.0.smithi063.stdout:1/392: chown d9/d2a/f74 199 1 2024-05-09T06:38:10.706 INFO:tasks.workunit.client.1.smithi193.stdout:5/110: unlink d0/d1/da/dd/f1e 0 2024-05-09T06:38:10.707 INFO:tasks.workunit.client.0.smithi063.stdout:1/393: fdatasync d9/d10/f43 0 2024-05-09T06:38:10.710 INFO:tasks.workunit.client.0.smithi063.stdout:9/445: dwrite d0/d4/d26/d39/d48/f5a [0,4194304] 0 2024-05-09T06:38:10.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/459: creat d7/da/d2b/d66/d74/f92 x:0 0 0 2024-05-09T06:38:10.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/460: chown d7/d4c/d25 14 1 2024-05-09T06:38:10.715 INFO:tasks.workunit.client.0.smithi063.stdout:2/476: creat dc/dd/d2d/d60/d65/f86 x:0 0 0 2024-05-09T06:38:10.718 INFO:tasks.workunit.client.1.smithi193.stdout:8/119: rename d6/d1c/f17 to d6/f21 0 2024-05-09T06:38:10.719 INFO:tasks.workunit.client.0.smithi063.stdout:5/403: write d8/f4b [4654155,103261] 0 2024-05-09T06:38:10.720 INFO:tasks.workunit.client.0.smithi063.stdout:4/446: mknod d7/df/d1f/d28/d31/d7b/d81/c8d 0 2024-05-09T06:38:10.721 INFO:tasks.workunit.client.1.smithi193.stdout:5/111: read d0/d1/f16 [2223985,40909] 0 2024-05-09T06:38:10.724 INFO:tasks.workunit.client.1.smithi193.stdout:4/108: unlink l9 0 2024-05-09T06:38:10.724 INFO:tasks.workunit.client.1.smithi193.stdout:4/109: readlink l15 0 2024-05-09T06:38:10.724 INFO:tasks.workunit.client.1.smithi193.stdout:4/110: readlink l2 0 2024-05-09T06:38:10.724 INFO:tasks.workunit.client.1.smithi193.stdout:4/111: stat fa 0 2024-05-09T06:38:10.727 INFO:tasks.workunit.client.0.smithi063.stdout:0/484: creat dd/d60/fa0 x:0 0 0 2024-05-09T06:38:10.727 INFO:tasks.workunit.client.0.smithi063.stdout:0/485: dread - dd/d1a/f9f zero size 2024-05-09T06:38:10.728 INFO:tasks.workunit.client.1.smithi193.stdout:8/120: write d6/d1c/f13 [4954195,82693] 0 2024-05-09T06:38:10.731 INFO:tasks.workunit.client.0.smithi063.stdout:6/450: creat d2/d20/d4a/f94 x:0 0 0 2024-05-09T06:38:10.732 INFO:tasks.workunit.client.1.smithi193.stdout:8/121: rmdir d6/d1c/d18 39 2024-05-09T06:38:10.735 INFO:tasks.workunit.client.0.smithi063.stdout:0/486: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d89/f8b [0,4194304] 0 2024-05-09T06:38:10.735 INFO:tasks.workunit.client.1.smithi193.stdout:5/112: dwrite d0/d1/da/dd/f17 [0,4194304] 0 2024-05-09T06:38:10.735 INFO:tasks.workunit.client.1.smithi193.stdout:4/112: link d1a/f1b d1a/f23 0 2024-05-09T06:38:10.745 INFO:tasks.workunit.client.1.smithi193.stdout:5/113: dwrite d0/fc [0,4194304] 0 2024-05-09T06:38:10.747 INFO:tasks.workunit.client.1.smithi193.stdout:2/126: rmdir db/df 39 2024-05-09T06:38:10.749 INFO:tasks.workunit.client.1.smithi193.stdout:2/127: truncate db/f1e 4380439 0 2024-05-09T06:38:10.751 INFO:tasks.workunit.client.0.smithi063.stdout:1/394: dwrite d9/d2a/d3c/d4a/f7b [0,4194304] 0 2024-05-09T06:38:10.753 INFO:tasks.workunit.client.0.smithi063.stdout:9/446: creat d0/d4/d57/f9b x:0 0 0 2024-05-09T06:38:10.754 INFO:tasks.workunit.client.1.smithi193.stdout:8/122: symlink d6/d1c/d18/l22 0 2024-05-09T06:38:10.757 INFO:tasks.workunit.client.0.smithi063.stdout:9/447: write d0/f98 [143130,80646] 0 2024-05-09T06:38:10.760 INFO:tasks.workunit.client.1.smithi193.stdout:5/114: unlink d0/d1/da/dd/f1a 0 2024-05-09T06:38:10.762 INFO:tasks.workunit.client.1.smithi193.stdout:8/123: dwrite d6/d1c/d1b/f1f [0,4194304] 0 2024-05-09T06:38:10.771 INFO:tasks.workunit.client.1.smithi193.stdout:5/115: dwrite d0/f9 [0,4194304] 0 2024-05-09T06:38:10.773 INFO:tasks.workunit.client.0.smithi063.stdout:5/404: creat d8/d17/d1e/f8c x:0 0 0 2024-05-09T06:38:10.778 INFO:tasks.workunit.client.0.smithi063.stdout:9/448: read d0/d3b/f6e [3637253,37168] 0 2024-05-09T06:38:10.782 INFO:tasks.workunit.client.1.smithi193.stdout:8/124: rename d6/d1c/f13 to d6/f23 0 2024-05-09T06:38:10.785 INFO:tasks.workunit.client.1.smithi193.stdout:2/128: symlink db/l30 0 2024-05-09T06:38:10.786 INFO:tasks.workunit.client.1.smithi193.stdout:2/129: fsync db/f1b 0 2024-05-09T06:38:10.786 INFO:tasks.workunit.client.1.smithi193.stdout:2/130: fsync f1 0 2024-05-09T06:38:10.792 INFO:tasks.workunit.client.1.smithi193.stdout:2/131: dread f9 [0,4194304] 0 2024-05-09T06:38:10.792 INFO:tasks.workunit.client.1.smithi193.stdout:2/132: truncate db/d25/f27 542977 0 2024-05-09T06:38:10.792 INFO:tasks.workunit.client.0.smithi063.stdout:9/449: chown d0/d4/d26/d39/f6c 5 1 2024-05-09T06:38:10.793 INFO:tasks.workunit.client.0.smithi063.stdout:7/420: dwrite d0/d15/d49/d54/f60 [0,4194304] 0 2024-05-09T06:38:10.793 INFO:tasks.workunit.client.0.smithi063.stdout:9/450: write d0/d4/f4d [1456899,37380] 0 2024-05-09T06:38:10.793 INFO:tasks.workunit.client.0.smithi063.stdout:4/447: dwrite d7/df/d1f/d28/d31/d32/f51 [0,4194304] 0 2024-05-09T06:38:10.794 INFO:tasks.workunit.client.0.smithi063.stdout:5/405: dwrite d8/d19/f1a [0,4194304] 0 2024-05-09T06:38:10.796 INFO:tasks.workunit.client.0.smithi063.stdout:4/448: write d7/df/f12 [2915680,121207] 0 2024-05-09T06:38:10.796 INFO:tasks.workunit.client.0.smithi063.stdout:3/461: rename d7/da/d21/d30/d44/d48/f81 to d7/d4c/d90/d2d/f93 0 2024-05-09T06:38:10.799 INFO:tasks.workunit.client.0.smithi063.stdout:4/449: readlink d7/df/d1f/d28/d31/d32/d34/l4d 0 2024-05-09T06:38:10.802 INFO:tasks.workunit.client.0.smithi063.stdout:6/451: getdents d2/df/d91 0 2024-05-09T06:38:10.803 INFO:tasks.workunit.client.0.smithi063.stdout:6/452: readlink d2/d20/d26/l43 0 2024-05-09T06:38:10.806 INFO:tasks.workunit.client.0.smithi063.stdout:8/504: link d9/d3b/f40 d9/d37/fa9 0 2024-05-09T06:38:10.806 INFO:tasks.workunit.client.0.smithi063.stdout:3/462: dwrite d7/d22/d28/f3f [0,4194304] 0 2024-05-09T06:38:10.807 INFO:tasks.workunit.client.1.smithi193.stdout:4/113: dwrite f17 [0,4194304] 0 2024-05-09T06:38:10.808 INFO:tasks.workunit.client.1.smithi193.stdout:5/116: rmdir d0/d1/da/dd 39 2024-05-09T06:38:10.809 INFO:tasks.workunit.client.1.smithi193.stdout:5/117: rename d0 to d0/d1/d25 22 2024-05-09T06:38:10.812 INFO:tasks.workunit.client.1.smithi193.stdout:4/114: dread f5 [0,4194304] 0 2024-05-09T06:38:10.818 INFO:tasks.workunit.client.0.smithi063.stdout:7/421: read d0/d15/d4a/f53 [582062,17274] 0 2024-05-09T06:38:10.819 INFO:tasks.workunit.client.0.smithi063.stdout:5/406: write d8/d17/d1e/f23 [733875,7078] 0 2024-05-09T06:38:10.824 INFO:tasks.workunit.client.1.smithi193.stdout:5/118: creat d0/d1/da/f26 x:0 0 0 2024-05-09T06:38:10.824 INFO:tasks.workunit.client.1.smithi193.stdout:5/119: stat d0/d1/da/f14 0 2024-05-09T06:38:10.827 INFO:tasks.workunit.client.0.smithi063.stdout:6/453: unlink d2/d4/d7c/l2e 0 2024-05-09T06:38:10.828 INFO:tasks.workunit.client.0.smithi063.stdout:2/477: dwrite dc/dd/f20 [0,4194304] 0 2024-05-09T06:38:10.830 INFO:tasks.workunit.client.0.smithi063.stdout:8/505: rename d9/da/d18/f2d to d9/da/d53/faa 0 2024-05-09T06:38:10.830 INFO:tasks.workunit.client.0.smithi063.stdout:3/463: dread f3 [0,4194304] 0 2024-05-09T06:38:10.831 INFO:tasks.workunit.client.0.smithi063.stdout:1/395: rmdir d9/d7f 0 2024-05-09T06:38:10.832 INFO:tasks.workunit.client.0.smithi063.stdout:3/464: write d7/d22/d28/f8c [154150,22689] 0 2024-05-09T06:38:10.833 INFO:tasks.workunit.client.0.smithi063.stdout:3/465: chown d7/l42 0 1 2024-05-09T06:38:10.837 INFO:tasks.workunit.client.0.smithi063.stdout:9/451: link d0/d4/f4d d0/d4/d7d/d8b/f9c 0 2024-05-09T06:38:10.839 INFO:tasks.workunit.client.0.smithi063.stdout:9/452: chown d0/d4/d26/d39/d48/f5a 4 1 2024-05-09T06:38:10.839 INFO:tasks.workunit.client.0.smithi063.stdout:9/453: stat d0/d4/d26/d47/f7a 0 2024-05-09T06:38:10.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/478: unlink dc/dd/d2d/f34 0 2024-05-09T06:38:10.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/479: chown dc/dd/d2d/d60 4223 1 2024-05-09T06:38:10.844 INFO:tasks.workunit.client.0.smithi063.stdout:7/422: mknod d0/d15/d4a/c8e 0 2024-05-09T06:38:10.846 INFO:tasks.workunit.client.0.smithi063.stdout:1/396: symlink d9/d2a/d3c/d4c/d83/d88/l89 0 2024-05-09T06:38:10.847 INFO:tasks.workunit.client.0.smithi063.stdout:5/407: rmdir d8/d7e/d88 0 2024-05-09T06:38:10.848 INFO:tasks.workunit.client.0.smithi063.stdout:2/480: dread dc/f2a [4194304,4194304] 0 2024-05-09T06:38:10.848 INFO:tasks.workunit.client.0.smithi063.stdout:7/423: mknod d0/d15/d49/d4f/d66/c8f 0 2024-05-09T06:38:10.855 INFO:tasks.workunit.client.0.smithi063.stdout:2/481: dwrite dc/dd/f4f [0,4194304] 0 2024-05-09T06:38:10.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:10 smithi063 ceph-mon[27171]: pgmap v8: 65 pgs: 65 active+clean; 750 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 93 MiB/s wr, 199 op/s 2024-05-09T06:38:10.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:10 smithi063 ceph-mon[27171]: mgrmap e25: smithi193.mmmqzh(active, since 10s), standbys: smithi063.egfguy 2024-05-09T06:38:10.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:10 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:38:10.856 INFO:tasks.workunit.client.0.smithi063.stdout:2/482: chown dc/d32/d33/f61 745300914 1 2024-05-09T06:38:10.856 INFO:tasks.workunit.client.0.smithi063.stdout:2/483: chown dc/dd/d2d/d60/d65/d72 5936080 1 2024-05-09T06:38:10.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:10 smithi193 ceph-mon[35141]: pgmap v8: 65 pgs: 65 active+clean; 750 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 93 MiB/s wr, 199 op/s 2024-05-09T06:38:10.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:10 smithi193 ceph-mon[35141]: mgrmap e25: smithi193.mmmqzh(active, since 10s), standbys: smithi063.egfguy 2024-05-09T06:38:10.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:10 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:38:10.872 INFO:tasks.workunit.client.1.smithi193.stdout:8/125: truncate d6/f23 2600954 0 2024-05-09T06:38:10.872 INFO:tasks.workunit.client.0.smithi063.stdout:1/397: mknod d9/d2a/d35/d24/d68/c8a 0 2024-05-09T06:38:10.873 INFO:tasks.workunit.client.0.smithi063.stdout:1/398: write d9/d10/f43 [178113,32541] 0 2024-05-09T06:38:10.874 INFO:tasks.workunit.client.0.smithi063.stdout:1/399: read - d9/d2a/d39/f5e zero size 2024-05-09T06:38:10.875 INFO:tasks.workunit.client.0.smithi063.stdout:7/424: unlink d0/d15/f2b 0 2024-05-09T06:38:10.877 INFO:tasks.workunit.client.1.smithi193.stdout:5/120: symlink d0/d1/l27 0 2024-05-09T06:38:10.878 INFO:tasks.workunit.client.1.smithi193.stdout:4/115: dwrite d1a/f1b [4194304,4194304] 0 2024-05-09T06:38:10.879 INFO:tasks.workunit.client.1.smithi193.stdout:5/121: write d0/f7 [2233797,7757] 0 2024-05-09T06:38:10.880 INFO:tasks.workunit.client.1.smithi193.stdout:8/126: rename d6/ca to d6/d1c/d1b/c24 0 2024-05-09T06:38:10.883 INFO:tasks.workunit.client.0.smithi063.stdout:4/450: truncate d7/df/d1f/d28/f7d 773873 0 2024-05-09T06:38:10.883 INFO:tasks.workunit.client.1.smithi193.stdout:8/127: dread d6/d1c/d1b/f1f [0,4194304] 0 2024-05-09T06:38:10.884 INFO:tasks.workunit.client.0.smithi063.stdout:4/451: stat d7/df/d57 0 2024-05-09T06:38:10.886 INFO:tasks.workunit.client.0.smithi063.stdout:2/484: mkdir dc/dd/d2d/d60/d65/d87 0 2024-05-09T06:38:10.886 INFO:tasks.workunit.client.0.smithi063.stdout:4/452: stat d7/df/d1f/d28/d31/d7b/d81/f8a 0 2024-05-09T06:38:10.886 INFO:tasks.workunit.client.0.smithi063.stdout:6/454: write d2/fd [3213553,102608] 0 2024-05-09T06:38:10.887 INFO:tasks.workunit.client.0.smithi063.stdout:1/400: creat d9/d10/d2b/d3f/d59/d62/f8b x:0 0 0 2024-05-09T06:38:10.888 INFO:tasks.workunit.client.1.smithi193.stdout:4/116: mkdir d1a/d24 0 2024-05-09T06:38:10.889 INFO:tasks.workunit.client.0.smithi063.stdout:2/485: write dc/d32/d33/f41 [4996842,123945] 0 2024-05-09T06:38:10.889 INFO:tasks.workunit.client.0.smithi063.stdout:6/455: chown d2/d1f/d29/d3b 171645 1 2024-05-09T06:38:10.890 INFO:tasks.workunit.client.0.smithi063.stdout:7/425: creat d0/d15/d49/d4f/d66/d67/d86/f90 x:0 0 0 2024-05-09T06:38:10.890 INFO:tasks.workunit.client.1.smithi193.stdout:5/122: rename d0/d1/da to d0/d1/da/dd/d28 22 2024-05-09T06:38:10.891 INFO:tasks.workunit.client.0.smithi063.stdout:1/401: dread - d9/d2a/d3c/d4c/f55 zero size 2024-05-09T06:38:10.891 INFO:tasks.workunit.client.0.smithi063.stdout:4/453: fsync d7/df/d1f/d28/d31/d7b/d81/f8a 0 2024-05-09T06:38:10.892 INFO:tasks.workunit.client.0.smithi063.stdout:6/456: write d2/d12/f34 [3589834,28322] 0 2024-05-09T06:38:10.892 INFO:tasks.workunit.client.0.smithi063.stdout:2/486: write dc/d32/d33/f61 [410993,9391] 0 2024-05-09T06:38:10.893 INFO:tasks.workunit.client.0.smithi063.stdout:2/487: fdatasync fa 0 2024-05-09T06:38:10.894 INFO:tasks.workunit.client.0.smithi063.stdout:2/488: readlink dc/d36/d43/d46/l4b 0 2024-05-09T06:38:10.896 INFO:tasks.workunit.client.1.smithi193.stdout:8/128: mkdir d6/d25 0 2024-05-09T06:38:10.896 INFO:tasks.workunit.client.1.smithi193.stdout:8/129: fsync d6/f20 0 2024-05-09T06:38:10.897 INFO:tasks.workunit.client.1.smithi193.stdout:4/117: fdatasync fa 0 2024-05-09T06:38:10.899 INFO:tasks.workunit.client.0.smithi063.stdout:7/426: creat d0/d15/d49/d4f/d66/f91 x:0 0 0 2024-05-09T06:38:10.900 INFO:tasks.workunit.client.1.smithi193.stdout:5/123: rmdir d0/d1/da 39 2024-05-09T06:38:10.903 INFO:tasks.workunit.client.0.smithi063.stdout:4/454: creat d7/df/d77/f8e x:0 0 0 2024-05-09T06:38:10.906 INFO:tasks.workunit.client.0.smithi063.stdout:6/457: unlink d2/d1f/c80 0 2024-05-09T06:38:10.907 INFO:tasks.workunit.client.0.smithi063.stdout:6/458: read - d2/df/f50 zero size 2024-05-09T06:38:10.909 INFO:tasks.workunit.client.0.smithi063.stdout:2/489: creat dc/d32/d33/d4e/f88 x:0 0 0 2024-05-09T06:38:10.911 INFO:tasks.workunit.client.1.smithi193.stdout:4/118: creat d1a/d1f/f25 x:0 0 0 2024-05-09T06:38:10.912 INFO:tasks.workunit.client.0.smithi063.stdout:7/427: symlink d0/d15/d49/d4f/d66/d67/d6b/l92 0 2024-05-09T06:38:10.913 INFO:tasks.workunit.client.0.smithi063.stdout:5/408: truncate d8/f21 5740000 0 2024-05-09T06:38:10.915 INFO:tasks.workunit.client.0.smithi063.stdout:4/455: unlink d7/df/d1f/d28/d31/l67 0 2024-05-09T06:38:10.916 INFO:tasks.workunit.client.1.smithi193.stdout:4/119: creat d1a/d1f/f26 x:0 0 0 2024-05-09T06:38:10.916 INFO:tasks.workunit.client.0.smithi063.stdout:4/456: read d7/d3d/f49 [149123,65425] 0 2024-05-09T06:38:10.919 INFO:tasks.workunit.client.1.smithi193.stdout:4/120: dread f5 [0,4194304] 0 2024-05-09T06:38:10.919 INFO:tasks.workunit.client.0.smithi063.stdout:2/490: creat dc/dd/d45/f89 x:0 0 0 2024-05-09T06:38:10.920 INFO:tasks.workunit.client.1.smithi193.stdout:4/121: chown fb 57635 1 2024-05-09T06:38:10.921 INFO:tasks.workunit.client.0.smithi063.stdout:1/402: creat d9/d2a/d39/f8c x:0 0 0 2024-05-09T06:38:10.921 INFO:tasks.workunit.client.0.smithi063.stdout:0/487: sync 2024-05-09T06:38:10.921 INFO:tasks.workunit.client.1.smithi193.stdout:4/122: dread f11 [0,4194304] 0 2024-05-09T06:38:10.922 INFO:tasks.workunit.client.1.smithi193.stdout:5/124: link d0/d1/f6 d0/f29 0 2024-05-09T06:38:10.923 INFO:tasks.workunit.client.0.smithi063.stdout:0/488: write dd/d1a/d2c/f43 [1478404,82766] 0 2024-05-09T06:38:10.923 INFO:tasks.workunit.client.1.smithi193.stdout:5/125: chown d0/f7 53833810 1 2024-05-09T06:38:10.923 INFO:tasks.workunit.client.0.smithi063.stdout:6/459: dread d2/df/f1d [0,4194304] 0 2024-05-09T06:38:10.924 INFO:tasks.workunit.client.1.smithi193.stdout:4/123: mknod d1a/d1f/c27 0 2024-05-09T06:38:10.924 INFO:tasks.workunit.client.1.smithi193.stdout:4/124: stat d1a/f23 0 2024-05-09T06:38:10.925 INFO:tasks.workunit.client.0.smithi063.stdout:6/460: dread - d2/d4/d7c/d7e/f87 zero size 2024-05-09T06:38:10.926 INFO:tasks.workunit.client.1.smithi193.stdout:5/126: stat d0/d1/da/l24 0 2024-05-09T06:38:10.927 INFO:tasks.workunit.client.1.smithi193.stdout:5/127: fdatasync d0/d1/da/dd/f22 0 2024-05-09T06:38:10.927 INFO:tasks.workunit.client.1.smithi193.stdout:4/125: creat d1a/f28 x:0 0 0 2024-05-09T06:38:10.928 INFO:tasks.workunit.client.1.smithi193.stdout:5/128: dread - d0/d1/da/dd/f22 zero size 2024-05-09T06:38:10.928 INFO:tasks.workunit.client.1.smithi193.stdout:4/126: dread - d1a/f28 zero size 2024-05-09T06:38:10.933 INFO:tasks.workunit.client.1.smithi193.stdout:4/127: fsync f12 0 2024-05-09T06:38:10.935 INFO:tasks.workunit.client.0.smithi063.stdout:5/409: link d8/d17/d18/f38 d8/d17/f8d 0 2024-05-09T06:38:10.936 INFO:tasks.workunit.client.0.smithi063.stdout:0/489: creat dd/d1a/d2c/fa1 x:0 0 0 2024-05-09T06:38:10.940 INFO:tasks.workunit.client.1.smithi193.stdout:5/129: creat d0/d1/f2a x:0 0 0 2024-05-09T06:38:10.941 INFO:tasks.workunit.client.0.smithi063.stdout:6/461: mkdir d2/d20/d95 0 2024-05-09T06:38:10.942 INFO:tasks.workunit.client.1.smithi193.stdout:4/128: mknod d1a/d22/c29 0 2024-05-09T06:38:10.944 INFO:tasks.workunit.client.1.smithi193.stdout:5/130: mkdir d0/d1/da/dd/d2b 0 2024-05-09T06:38:10.944 INFO:tasks.workunit.client.0.smithi063.stdout:2/491: getdents dc/dd/d2d/d60/d65/d87 0 2024-05-09T06:38:10.944 INFO:tasks.workunit.client.1.smithi193.stdout:5/131: stat d0/d1/da/dd/f11 0 2024-05-09T06:38:10.945 INFO:tasks.workunit.client.0.smithi063.stdout:2/492: read dc/dd/f20 [1761053,22570] 0 2024-05-09T06:38:10.946 INFO:tasks.workunit.client.1.smithi193.stdout:4/129: creat d1a/d1f/f2a x:0 0 0 2024-05-09T06:38:10.947 INFO:tasks.workunit.client.0.smithi063.stdout:1/403: mknod d9/d46/c8d 0 2024-05-09T06:38:10.950 INFO:tasks.workunit.client.1.smithi193.stdout:5/132: dread d0/d1/da/dd/f11 [0,4194304] 0 2024-05-09T06:38:10.950 INFO:tasks.workunit.client.1.smithi193.stdout:5/133: stat d0 0 2024-05-09T06:38:10.951 INFO:tasks.workunit.client.0.smithi063.stdout:6/462: mkdir d2/d4/d96 0 2024-05-09T06:38:10.953 INFO:tasks.workunit.client.0.smithi063.stdout:0/490: dwrite dd/f15 [0,4194304] 0 2024-05-09T06:38:10.954 INFO:tasks.workunit.client.1.smithi193.stdout:5/134: mkdir d0/d13/d2c 0 2024-05-09T06:38:10.956 INFO:tasks.workunit.client.0.smithi063.stdout:1/404: truncate d9/d10/f21 2327078 0 2024-05-09T06:38:10.957 INFO:tasks.workunit.client.0.smithi063.stdout:1/405: dread - d9/d2a/d2e/f7d zero size 2024-05-09T06:38:10.958 INFO:tasks.workunit.client.0.smithi063.stdout:4/457: getdents d7/df/d1f/d28/d31/d7b/d81 0 2024-05-09T06:38:10.958 INFO:tasks.workunit.client.0.smithi063.stdout:1/406: stat d9/d10/d2b/l34 0 2024-05-09T06:38:10.960 INFO:tasks.workunit.client.0.smithi063.stdout:6/463: creat d2/d20/d26/f97 x:0 0 0 2024-05-09T06:38:10.961 INFO:tasks.workunit.client.0.smithi063.stdout:0/491: chown dd/d1a/f2e 709118091 1 2024-05-09T06:38:10.962 INFO:tasks.workunit.client.1.smithi193.stdout:5/135: getdents d0/d1/da 0 2024-05-09T06:38:10.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/493: write dc/f27 [4277890,60852] 0 2024-05-09T06:38:10.964 INFO:tasks.workunit.client.0.smithi063.stdout:4/458: rmdir d7/d3d 39 2024-05-09T06:38:10.965 INFO:tasks.workunit.client.0.smithi063.stdout:1/407: unlink d9/d2a/d35/d24/d68/c8a 0 2024-05-09T06:38:10.966 INFO:tasks.workunit.client.1.smithi193.stdout:5/136: unlink d0/d1/f16 0 2024-05-09T06:38:10.967 INFO:tasks.workunit.client.1.smithi193.stdout:5/137: stat d0/d1/da/l24 0 2024-05-09T06:38:10.967 INFO:tasks.workunit.client.0.smithi063.stdout:4/459: truncate d7/df/d1f/d28/d31/d32/f8c 657800 0 2024-05-09T06:38:10.968 INFO:tasks.workunit.client.1.smithi193.stdout:5/138: write d0/f9 [1762272,31230] 0 2024-05-09T06:38:10.969 INFO:tasks.workunit.client.0.smithi063.stdout:0/492: rmdir dd/d1a/d2c/d51/d72/d9e/d23/d31 39 2024-05-09T06:38:10.969 INFO:tasks.workunit.client.0.smithi063.stdout:5/410: getdents d8/d53 0 2024-05-09T06:38:10.971 INFO:tasks.workunit.client.0.smithi063.stdout:5/411: fdatasync d8/f10 0 2024-05-09T06:38:10.971 INFO:tasks.workunit.client.1.smithi193.stdout:5/139: mknod d0/d1/da/dd/c2d 0 2024-05-09T06:38:10.971 INFO:tasks.workunit.client.0.smithi063.stdout:2/494: dwrite dc/dd/d2d/d60/d65/f86 [0,4194304] 0 2024-05-09T06:38:10.975 INFO:tasks.workunit.client.1.smithi193.stdout:5/140: write d0/d1/da/dd/f1c [25553,112731] 0 2024-05-09T06:38:10.977 INFO:tasks.workunit.client.1.smithi193.stdout:5/141: write d0/d1/da/f26 [950942,57509] 0 2024-05-09T06:38:10.980 INFO:tasks.workunit.client.1.smithi193.stdout:5/142: mkdir d0/d2e 0 2024-05-09T06:38:10.982 INFO:tasks.workunit.client.1.smithi193.stdout:5/143: mknod d0/d1/da/dd/c2f 0 2024-05-09T06:38:10.983 INFO:tasks.workunit.client.1.smithi193.stdout:5/144: chown d0/d1/da/dd/c20 106 1 2024-05-09T06:38:10.984 INFO:tasks.workunit.client.1.smithi193.stdout:5/145: write d0/d1/f21 [464699,41338] 0 2024-05-09T06:38:10.987 INFO:tasks.workunit.client.1.smithi193.stdout:5/146: dread d0/f8 [0,4194304] 0 2024-05-09T06:38:10.990 INFO:tasks.workunit.client.1.smithi193.stdout:5/147: creat d0/d13/d2c/f30 x:0 0 0 2024-05-09T06:38:10.995 INFO:tasks.workunit.client.1.smithi193.stdout:5/148: dwrite d0/d13/d2c/f30 [0,4194304] 0 2024-05-09T06:38:10.997 INFO:tasks.workunit.client.1.smithi193.stdout:5/149: write d0/f8 [3754272,106657] 0 2024-05-09T06:38:10.998 INFO:tasks.workunit.client.1.smithi193.stdout:5/150: mknod d0/d1/da/dd/c31 0 2024-05-09T06:38:10.999 INFO:tasks.workunit.client.1.smithi193.stdout:5/151: fdatasync d0/d1/da/dd/f1c 0 2024-05-09T06:38:11.016 INFO:tasks.workunit.client.0.smithi063.stdout:6/464: mkdir d2/d1f/d73/d88/d98 0 2024-05-09T06:38:11.016 INFO:tasks.workunit.client.0.smithi063.stdout:5/412: write d8/d17/f3d [3926520,111611] 0 2024-05-09T06:38:11.017 INFO:tasks.workunit.client.0.smithi063.stdout:6/465: readlink d2/d1f/l41 0 2024-05-09T06:38:11.019 INFO:tasks.workunit.client.0.smithi063.stdout:6/466: write d2/f85 [219391,104493] 0 2024-05-09T06:38:11.021 INFO:tasks.workunit.client.0.smithi063.stdout:5/413: creat d8/f8e x:0 0 0 2024-05-09T06:38:11.021 INFO:tasks.workunit.client.0.smithi063.stdout:6/467: dread - d2/d1f/d29/d3f/f82 zero size 2024-05-09T06:38:11.024 INFO:tasks.workunit.client.0.smithi063.stdout:0/493: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d31/f4a [0,4194304] 0 2024-05-09T06:38:11.025 INFO:tasks.workunit.client.0.smithi063.stdout:5/414: rename d8/f3a to d8/f8f 0 2024-05-09T06:38:11.028 INFO:tasks.workunit.client.0.smithi063.stdout:0/494: read dd/d1a/d2c/d51/d72/d9e/d23/f2f [4268951,124055] 0 2024-05-09T06:38:11.030 INFO:tasks.workunit.client.0.smithi063.stdout:2/495: dwrite dc/dd/f13 [0,4194304] 0 2024-05-09T06:38:11.034 INFO:tasks.workunit.client.0.smithi063.stdout:5/415: rename d8/d17/l49 to d8/d19/d30/d74/d77/l90 0 2024-05-09T06:38:11.035 INFO:tasks.workunit.client.0.smithi063.stdout:0/495: creat dd/d1a/d2c/d51/d72/d9e/d23/d31/fa2 x:0 0 0 2024-05-09T06:38:11.036 INFO:tasks.workunit.client.0.smithi063.stdout:5/416: mknod d8/d19/d30/d74/d77/c91 0 2024-05-09T06:38:11.037 INFO:tasks.workunit.client.0.smithi063.stdout:2/496: getdents dc/dd/d45/d5a/d6c 0 2024-05-09T06:38:11.037 INFO:tasks.workunit.client.0.smithi063.stdout:0/496: mknod dd/d1a/d2c/ca3 0 2024-05-09T06:38:11.038 INFO:tasks.workunit.client.0.smithi063.stdout:2/497: stat dc/dd/d45/d58/l78 0 2024-05-09T06:38:11.039 INFO:tasks.workunit.client.0.smithi063.stdout:5/417: symlink d8/l92 0 2024-05-09T06:38:11.042 INFO:tasks.workunit.client.0.smithi063.stdout:5/418: creat d8/d53/f93 x:0 0 0 2024-05-09T06:38:11.043 INFO:tasks.workunit.client.0.smithi063.stdout:0/497: mkdir dd/d60/d44/d90/da4 0 2024-05-09T06:38:11.044 INFO:tasks.workunit.client.0.smithi063.stdout:5/419: dread - d8/d65/f6f zero size 2024-05-09T06:38:11.046 INFO:tasks.workunit.client.0.smithi063.stdout:2/498: dwrite dc/dd/f23 [8388608,4194304] 0 2024-05-09T06:38:11.047 INFO:tasks.workunit.client.0.smithi063.stdout:5/420: creat d8/d19/d30/d46/f94 x:0 0 0 2024-05-09T06:38:11.047 INFO:tasks.workunit.client.0.smithi063.stdout:5/421: readlink d8/l2c 0 2024-05-09T06:38:11.048 INFO:tasks.workunit.client.0.smithi063.stdout:0/498: truncate dd/d1a/d2c/d51/d72/d9e/d23/d46/f82 118373 0 2024-05-09T06:38:11.050 INFO:tasks.workunit.client.0.smithi063.stdout:2/499: creat dc/dd/d45/f8a x:0 0 0 2024-05-09T06:38:11.051 INFO:tasks.workunit.client.0.smithi063.stdout:5/422: rename d8/d19/d30/d74/f89 to d8/d17/f95 0 2024-05-09T06:38:11.052 INFO:tasks.workunit.client.0.smithi063.stdout:2/500: chown dc/dd/le 423 1 2024-05-09T06:38:11.053 INFO:tasks.workunit.client.0.smithi063.stdout:8/506: sync 2024-05-09T06:38:11.055 INFO:tasks.workunit.client.0.smithi063.stdout:2/501: mknod dc/c8b 0 2024-05-09T06:38:11.056 INFO:tasks.workunit.client.0.smithi063.stdout:8/507: symlink d9/da/d18/d24/d71/lab 0 2024-05-09T06:38:11.057 INFO:tasks.workunit.client.0.smithi063.stdout:2/502: mkdir dc/dd/d45/d5a/d6c/d8c 0 2024-05-09T06:38:11.066 INFO:tasks.workunit.client.0.smithi063.stdout:0/499: dread dd/d1a/f25 [0,4194304] 0 2024-05-09T06:38:11.069 INFO:tasks.workunit.client.0.smithi063.stdout:0/500: creat dd/d1a/d2c/d51/d76/d97/fa5 x:0 0 0 2024-05-09T06:38:11.071 INFO:tasks.workunit.client.0.smithi063.stdout:0/501: write dd/d1a/d2c/f92 [154928,78652] 0 2024-05-09T06:38:11.075 INFO:tasks.workunit.client.0.smithi063.stdout:8/508: dread d9/da/fb [0,4194304] 0 2024-05-09T06:38:11.076 INFO:tasks.workunit.client.1.smithi193.stdout:7/134: sync 2024-05-09T06:38:11.077 INFO:tasks.workunit.client.0.smithi063.stdout:0/502: dread dd/d1a/d2c/d51/d72/d9e/d23/d89/f8b [0,4194304] 0 2024-05-09T06:38:11.078 INFO:tasks.workunit.client.1.smithi193.stdout:5/152: creat d0/d1/f32 x:0 0 0 2024-05-09T06:38:11.079 INFO:tasks.workunit.client.0.smithi063.stdout:2/503: dread dc/f1c [0,4194304] 0 2024-05-09T06:38:11.080 INFO:tasks.workunit.client.1.smithi193.stdout:7/135: rename d6/d8/d1a/d1f/f21 to d6/d8/d29/f2d 0 2024-05-09T06:38:11.082 INFO:tasks.workunit.client.0.smithi063.stdout:8/509: dwrite d9/da/d34/f59 [0,4194304] 0 2024-05-09T06:38:11.084 INFO:tasks.workunit.client.1.smithi193.stdout:5/153: dwrite d0/d1/da/dd/f17 [0,4194304] 0 2024-05-09T06:38:11.088 INFO:tasks.workunit.client.1.smithi193.stdout:5/154: dread d0/f9 [0,4194304] 0 2024-05-09T06:38:11.089 INFO:tasks.workunit.client.0.smithi063.stdout:8/510: rename d9/f47 to d9/da/d34/fac 0 2024-05-09T06:38:11.092 INFO:tasks.workunit.client.1.smithi193.stdout:5/155: dread d0/f8 [0,4194304] 0 2024-05-09T06:38:11.094 INFO:tasks.workunit.client.1.smithi193.stdout:5/156: write d0/d13/f1f [481470,85926] 0 2024-05-09T06:38:11.094 INFO:tasks.workunit.client.1.smithi193.stdout:7/136: creat d6/f2e x:0 0 0 2024-05-09T06:38:11.095 INFO:tasks.workunit.client.0.smithi063.stdout:2/504: getdents dc/dd/d45/d5a/d6c/d8c 0 2024-05-09T06:38:11.098 INFO:tasks.workunit.client.0.smithi063.stdout:0/503: getdents dd 0 2024-05-09T06:38:11.099 INFO:tasks.workunit.client.1.smithi193.stdout:7/137: mkdir d6/d8/d29/d2f 0 2024-05-09T06:38:11.102 INFO:tasks.workunit.client.0.smithi063.stdout:2/505: dwrite dc/f1b [4194304,4194304] 0 2024-05-09T06:38:11.103 INFO:tasks.workunit.client.0.smithi063.stdout:0/504: rmdir dd 39 2024-05-09T06:38:11.107 INFO:tasks.workunit.client.0.smithi063.stdout:8/511: symlink d9/d41/lad 0 2024-05-09T06:38:11.107 INFO:tasks.workunit.client.0.smithi063.stdout:0/505: write dd/d1a/f9f [777860,130674] 0 2024-05-09T06:38:11.108 INFO:tasks.workunit.client.1.smithi193.stdout:5/157: getdents d0 0 2024-05-09T06:38:11.109 INFO:tasks.workunit.client.1.smithi193.stdout:5/158: write d0/d1/da/dd/f17 [4498811,76682] 0 2024-05-09T06:38:11.110 INFO:tasks.workunit.client.1.smithi193.stdout:7/138: getdents d6 0 2024-05-09T06:38:11.113 INFO:tasks.workunit.client.0.smithi063.stdout:0/506: creat dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fa6 x:0 0 0 2024-05-09T06:38:11.118 INFO:tasks.workunit.client.1.smithi193.stdout:5/159: truncate d0/d13/d2c/f30 493112 0 2024-05-09T06:38:11.119 INFO:tasks.workunit.client.1.smithi193.stdout:7/139: truncate d6/d17/f19 1398283 0 2024-05-09T06:38:11.121 INFO:tasks.workunit.client.1.smithi193.stdout:5/160: rename d0/d1/da/dd/f17 to d0/d1/f33 0 2024-05-09T06:38:11.122 INFO:tasks.workunit.client.1.smithi193.stdout:5/161: stat d0/d1/da/dd/c2d 0 2024-05-09T06:38:11.123 INFO:tasks.workunit.client.1.smithi193.stdout:7/140: link d6/d8/fa d6/f30 0 2024-05-09T06:38:11.124 INFO:tasks.workunit.client.1.smithi193.stdout:5/162: read d0/d1/f21 [323,54704] 0 2024-05-09T06:38:11.124 INFO:tasks.workunit.client.0.smithi063.stdout:3/466: sync 2024-05-09T06:38:11.125 INFO:tasks.workunit.client.0.smithi063.stdout:0/507: stat dd/d1a/d2c/d51/d72/d9e/d23/d33/f3e 0 2024-05-09T06:38:11.125 INFO:tasks.workunit.client.0.smithi063.stdout:3/467: chown d7/da/fe 18456 1 2024-05-09T06:38:11.126 INFO:tasks.workunit.client.1.smithi193.stdout:5/163: truncate d0/d1/da/dd/f1c 969493 0 2024-05-09T06:38:11.126 INFO:tasks.workunit.client.0.smithi063.stdout:8/512: dwrite d9/da/d18/f23 [4194304,4194304] 0 2024-05-09T06:38:11.127 INFO:tasks.workunit.client.0.smithi063.stdout:8/513: rename d9 to d9/da/d18/d56/dae 22 2024-05-09T06:38:11.127 INFO:tasks.workunit.client.0.smithi063.stdout:3/468: fdatasync d7/da/d2b/d66/d74/f92 0 2024-05-09T06:38:11.136 INFO:tasks.workunit.client.0.smithi063.stdout:8/514: unlink d9/da/d18/d24/f92 0 2024-05-09T06:38:11.137 INFO:tasks.workunit.client.0.smithi063.stdout:3/469: mknod d7/da/d21/c94 0 2024-05-09T06:38:11.140 INFO:tasks.workunit.client.0.smithi063.stdout:8/515: fsync d9/da/d51/f88 0 2024-05-09T06:38:11.143 INFO:tasks.workunit.client.1.smithi193.stdout:7/141: dwrite d6/f16 [4194304,4194304] 0 2024-05-09T06:38:11.144 INFO:tasks.workunit.client.0.smithi063.stdout:3/470: unlink d7/da/d2b/d66/d74/d7e/l37 0 2024-05-09T06:38:11.145 INFO:tasks.workunit.client.0.smithi063.stdout:8/516: creat d9/da/d18/d56/d58/d79/faf x:0 0 0 2024-05-09T06:38:11.148 INFO:tasks.workunit.client.0.smithi063.stdout:3/471: creat d7/da/d2b/d66/f95 x:0 0 0 2024-05-09T06:38:11.150 INFO:tasks.workunit.client.0.smithi063.stdout:8/517: write d9/f77 [380850,57389] 0 2024-05-09T06:38:11.151 INFO:tasks.workunit.client.0.smithi063.stdout:8/518: chown d9/da/fb 8 1 2024-05-09T06:38:11.154 INFO:tasks.workunit.client.1.smithi193.stdout:7/142: symlink d6/dd/l31 0 2024-05-09T06:38:11.155 INFO:tasks.workunit.client.0.smithi063.stdout:8/519: symlink d9/da/d53/da3/lb0 0 2024-05-09T06:38:11.157 INFO:tasks.workunit.client.0.smithi063.stdout:3/472: dwrite d7/d22/f47 [4194304,4194304] 0 2024-05-09T06:38:11.158 INFO:tasks.workunit.client.1.smithi193.stdout:7/143: rename d6/dd/l31 to d6/d8/d29/l32 0 2024-05-09T06:38:11.160 INFO:tasks.workunit.client.0.smithi063.stdout:8/520: mknod d9/da/cb1 0 2024-05-09T06:38:11.161 INFO:tasks.workunit.client.0.smithi063.stdout:8/521: read d9/da/d34/f87 [1633870,71622] 0 2024-05-09T06:38:11.161 INFO:tasks.workunit.client.0.smithi063.stdout:8/522: chown d9/da/d53/faa 947891 1 2024-05-09T06:38:11.163 INFO:tasks.workunit.client.1.smithi193.stdout:7/144: write d6/d17/f23 [899815,71269] 0 2024-05-09T06:38:11.168 INFO:tasks.workunit.client.0.smithi063.stdout:8/523: stat d9/l6a 0 2024-05-09T06:38:11.176 INFO:tasks.workunit.client.1.smithi193.stdout:5/164: write d0/d13/d2c/f30 [1060601,90736] 0 2024-05-09T06:38:11.180 INFO:tasks.workunit.client.0.smithi063.stdout:8/524: dwrite d9/da/d18/d56/d8f/f89 [0,4194304] 0 2024-05-09T06:38:11.181 INFO:tasks.workunit.client.0.smithi063.stdout:8/525: chown d9/da/d18/d56 48522 1 2024-05-09T06:38:11.182 INFO:tasks.workunit.client.0.smithi063.stdout:3/473: dwrite d7/f17 [0,4194304] 0 2024-05-09T06:38:11.182 INFO:tasks.workunit.client.1.smithi193.stdout:7/145: dwrite f4 [0,4194304] 0 2024-05-09T06:38:11.185 INFO:tasks.workunit.client.0.smithi063.stdout:8/526: mkdir d9/da/d53/db2 0 2024-05-09T06:38:11.187 INFO:tasks.workunit.client.0.smithi063.stdout:3/474: write d7/d4c/d90/d2d/f93 [921121,118440] 0 2024-05-09T06:38:11.188 INFO:tasks.workunit.client.0.smithi063.stdout:3/475: fsync d7/d4c/f54 0 2024-05-09T06:38:11.189 INFO:tasks.workunit.client.1.smithi193.stdout:7/146: mkdir d6/d8/d1a/d33 0 2024-05-09T06:38:11.191 INFO:tasks.workunit.client.1.smithi193.stdout:7/147: chown d6/d8/d1a 119355547 1 2024-05-09T06:38:11.193 INFO:tasks.workunit.client.0.smithi063.stdout:8/527: dwrite d9/da/d18/d46/f9b [0,4194304] 0 2024-05-09T06:38:11.194 INFO:tasks.workunit.client.1.smithi193.stdout:5/165: dwrite d0/d1/f21 [0,4194304] 0 2024-05-09T06:38:11.194 INFO:tasks.workunit.client.1.smithi193.stdout:5/166: fsync d0/fc 0 2024-05-09T06:38:11.196 INFO:tasks.workunit.client.0.smithi063.stdout:3/476: rename d7/da/d2b/d66/l91 to d7/da/d2b/d66/l96 0 2024-05-09T06:38:11.202 INFO:tasks.workunit.client.1.smithi193.stdout:7/148: mknod d6/d8/d1a/d33/c34 0 2024-05-09T06:38:11.202 INFO:tasks.workunit.client.0.smithi063.stdout:3/477: dread d7/d22/d28/f3f [0,4194304] 0 2024-05-09T06:38:11.202 INFO:tasks.workunit.client.1.smithi193.stdout:7/149: stat d6/d17/c28 0 2024-05-09T06:38:11.204 INFO:tasks.workunit.client.0.smithi063.stdout:8/528: truncate d9/da/d34/fac 964213 0 2024-05-09T06:38:11.205 INFO:tasks.workunit.client.0.smithi063.stdout:8/529: dread - d9/da/d18/d24/d71/fa6 zero size 2024-05-09T06:38:11.207 INFO:tasks.workunit.client.0.smithi063.stdout:3/478: mkdir d7/da/d2b/d66/d74/d7e/d97 0 2024-05-09T06:38:11.208 INFO:tasks.workunit.client.1.smithi193.stdout:5/167: rmdir d0/d2e 0 2024-05-09T06:38:11.209 INFO:tasks.workunit.client.0.smithi063.stdout:8/530: read - d9/d37/f67 zero size 2024-05-09T06:38:11.210 INFO:tasks.workunit.client.0.smithi063.stdout:8/531: stat d9/da/d45/l6f 0 2024-05-09T06:38:11.211 INFO:tasks.workunit.client.0.smithi063.stdout:3/479: dread d7/da/f29 [0,4194304] 0 2024-05-09T06:38:11.211 INFO:tasks.workunit.client.0.smithi063.stdout:8/532: chown d9/da/d45/l4a 19301 1 2024-05-09T06:38:11.212 INFO:tasks.workunit.client.1.smithi193.stdout:7/150: rename d6/d17/f19 to d6/d8/f35 0 2024-05-09T06:38:11.212 INFO:tasks.workunit.client.0.smithi063.stdout:8/533: fsync d9/da/d18/d46/f52 0 2024-05-09T06:38:11.214 INFO:tasks.workunit.client.0.smithi063.stdout:3/480: rename d7/d4c/d90 to d7/d70/d98 0 2024-05-09T06:38:11.215 INFO:tasks.workunit.client.1.smithi193.stdout:7/151: mkdir d6/d8/d1a/d36 0 2024-05-09T06:38:11.220 INFO:tasks.workunit.client.0.smithi063.stdout:3/481: dwrite d7/da/d21/d30/f5a [0,4194304] 0 2024-05-09T06:38:11.225 INFO:tasks.workunit.client.0.smithi063.stdout:3/482: getdents d7/d70/d98/d5b 0 2024-05-09T06:38:11.231 INFO:tasks.workunit.client.0.smithi063.stdout:3/483: dwrite d7/d4c/d31/f6e [0,4194304] 0 2024-05-09T06:38:11.232 INFO:tasks.workunit.client.0.smithi063.stdout:3/484: fdatasync f5 0 2024-05-09T06:38:11.240 INFO:tasks.workunit.client.0.smithi063.stdout:3/485: unlink d7/d22/f50 0 2024-05-09T06:38:11.241 INFO:tasks.workunit.client.1.smithi193.stdout:5/168: dwrite d0/d1/da/f26 [0,4194304] 0 2024-05-09T06:38:11.242 INFO:tasks.workunit.client.0.smithi063.stdout:3/486: creat d7/da/d2b/d66/d74/d7e/f99 x:0 0 0 2024-05-09T06:38:11.247 INFO:tasks.workunit.client.0.smithi063.stdout:3/487: symlink d7/da/d21/d30/d44/l9a 0 2024-05-09T06:38:11.249 INFO:tasks.workunit.client.0.smithi063.stdout:3/488: dread - d7/da/d2b/d66/d74/d7e/f45 zero size 2024-05-09T06:38:11.250 INFO:tasks.workunit.client.1.smithi193.stdout:5/169: dwrite d0/f7 [0,4194304] 0 2024-05-09T06:38:11.255 INFO:tasks.workunit.client.0.smithi063.stdout:3/489: dwrite d7/d22/d28/f3c [0,4194304] 0 2024-05-09T06:38:11.257 INFO:tasks.workunit.client.1.smithi193.stdout:5/170: rename d0/le to d0/d13/l34 0 2024-05-09T06:38:11.259 INFO:tasks.workunit.client.0.smithi063.stdout:3/490: creat d7/d70/d98/d5b/f9b x:0 0 0 2024-05-09T06:38:11.268 INFO:tasks.workunit.client.1.smithi193.stdout:5/171: mknod d0/d1/da/dd/d2b/c35 0 2024-05-09T06:38:11.271 INFO:tasks.workunit.client.1.smithi193.stdout:5/172: rename d0/d1/da/dd/f22 to d0/d1/da/dd/f36 0 2024-05-09T06:38:11.272 INFO:tasks.workunit.client.1.smithi193.stdout:5/173: readlink d0/l18 0 2024-05-09T06:38:11.275 INFO:tasks.workunit.client.0.smithi063.stdout:3/491: dread d7/da/fb [0,4194304] 0 2024-05-09T06:38:11.278 INFO:tasks.workunit.client.0.smithi063.stdout:3/492: rmdir d7/d8e 0 2024-05-09T06:38:11.280 INFO:tasks.workunit.client.0.smithi063.stdout:3/493: rmdir d7/d70 39 2024-05-09T06:38:11.282 INFO:tasks.workunit.client.1.smithi193.stdout:5/174: rename d0/d1/da/dd to d0/d13/d2c/d37 0 2024-05-09T06:38:11.288 INFO:tasks.workunit.client.1.smithi193.stdout:5/175: dwrite d0/fc [0,4194304] 0 2024-05-09T06:38:11.290 INFO:tasks.workunit.client.1.smithi193.stdout:5/176: mkdir d0/d13/d2c/d38 0 2024-05-09T06:38:11.293 INFO:tasks.workunit.client.1.smithi193.stdout:5/177: unlink d0/d13/l19 0 2024-05-09T06:38:11.294 INFO:tasks.workunit.client.1.smithi193.stdout:5/178: write d0/d13/d2c/d37/f1c [1456502,36868] 0 2024-05-09T06:38:11.295 INFO:tasks.workunit.client.1.smithi193.stdout:5/179: stat d0/d13/d2c/d37/f11 0 2024-05-09T06:38:11.297 INFO:tasks.workunit.client.0.smithi063.stdout:4/460: sync 2024-05-09T06:38:11.297 INFO:tasks.workunit.client.0.smithi063.stdout:7/428: sync 2024-05-09T06:38:11.297 INFO:tasks.workunit.client.0.smithi063.stdout:9/454: sync 2024-05-09T06:38:11.297 INFO:tasks.workunit.client.0.smithi063.stdout:6/468: sync 2024-05-09T06:38:11.297 INFO:tasks.workunit.client.0.smithi063.stdout:1/408: sync 2024-05-09T06:38:11.297 INFO:tasks.workunit.client.0.smithi063.stdout:5/423: sync 2024-05-09T06:38:11.299 INFO:tasks.workunit.client.0.smithi063.stdout:5/424: write d8/f87 [318238,124768] 0 2024-05-09T06:38:11.299 INFO:tasks.workunit.client.1.smithi193.stdout:5/180: link d0/d13/d2c/d37/c20 d0/d13/d2c/d37/d2b/c39 0 2024-05-09T06:38:11.300 INFO:tasks.workunit.client.0.smithi063.stdout:7/429: mknod d0/d15/d82/c93 0 2024-05-09T06:38:11.301 INFO:tasks.workunit.client.0.smithi063.stdout:9/455: mkdir d0/d4/da/d9d 0 2024-05-09T06:38:11.302 INFO:tasks.workunit.client.0.smithi063.stdout:6/469: dread d2/df/f1d [0,4194304] 0 2024-05-09T06:38:11.303 INFO:tasks.workunit.client.0.smithi063.stdout:5/425: mkdir d8/d7e/d96 0 2024-05-09T06:38:11.305 INFO:tasks.workunit.client.0.smithi063.stdout:9/456: symlink d0/d3b/d63/l9e 0 2024-05-09T06:38:11.306 INFO:tasks.workunit.client.1.smithi193.stdout:5/181: dwrite d0/d1/f33 [0,4194304] 0 2024-05-09T06:38:11.307 INFO:tasks.workunit.client.0.smithi063.stdout:7/430: symlink d0/l94 0 2024-05-09T06:38:11.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/182: fsync d0/d1/f6 0 2024-05-09T06:38:11.310 INFO:tasks.workunit.client.0.smithi063.stdout:4/461: getdents d7/df/d1f/d28/d31/d32 0 2024-05-09T06:38:11.311 INFO:tasks.workunit.client.1.smithi193.stdout:5/183: stat d0/d13/d2c/f30 0 2024-05-09T06:38:11.316 INFO:tasks.workunit.client.0.smithi063.stdout:6/470: dwrite d2/d12/f24 [0,4194304] 0 2024-05-09T06:38:11.317 INFO:tasks.workunit.client.1.smithi193.stdout:5/184: mkdir d0/d1/da/d3a 0 2024-05-09T06:38:11.318 INFO:tasks.workunit.client.0.smithi063.stdout:5/426: dwrite d8/d17/d1e/f23 [4194304,4194304] 0 2024-05-09T06:38:11.319 INFO:tasks.workunit.client.0.smithi063.stdout:4/462: symlink d7/df/d1f/l8f 0 2024-05-09T06:38:11.320 INFO:tasks.workunit.client.0.smithi063.stdout:1/409: dread d9/d10/d2b/f33 [0,4194304] 0 2024-05-09T06:38:11.321 INFO:tasks.workunit.client.0.smithi063.stdout:1/410: fsync d9/d2a/d2e/f7d 0 2024-05-09T06:38:11.322 INFO:tasks.workunit.client.0.smithi063.stdout:6/471: mkdir d2/df/d59/d99 0 2024-05-09T06:38:11.322 INFO:tasks.workunit.client.0.smithi063.stdout:1/411: readlink d9/d2a/d35/d1e/l2f 0 2024-05-09T06:38:11.324 INFO:tasks.workunit.client.1.smithi193.stdout:5/185: symlink d0/d1/l3b 0 2024-05-09T06:38:11.328 INFO:tasks.workunit.client.0.smithi063.stdout:9/457: dread d0/d4/d26/d47/f2b [0,4194304] 0 2024-05-09T06:38:11.329 INFO:tasks.workunit.client.0.smithi063.stdout:5/427: mknod d8/c97 0 2024-05-09T06:38:11.329 INFO:tasks.workunit.client.0.smithi063.stdout:7/431: dread d0/d15/f25 [0,4194304] 0 2024-05-09T06:38:11.331 INFO:tasks.workunit.client.1.smithi193.stdout:5/186: truncate d0/d13/d2c/d37/f11 3920722 0 2024-05-09T06:38:11.333 INFO:tasks.workunit.client.0.smithi063.stdout:1/412: creat d9/d2a/d35/d1e/f8e x:0 0 0 2024-05-09T06:38:11.336 INFO:tasks.workunit.client.0.smithi063.stdout:6/472: mkdir d2/d9a 0 2024-05-09T06:38:11.336 INFO:tasks.workunit.client.0.smithi063.stdout:5/428: dwrite d8/f10 [0,4194304] 0 2024-05-09T06:38:11.338 INFO:tasks.workunit.client.0.smithi063.stdout:9/458: unlink d0/d4/d57/l7f 0 2024-05-09T06:38:11.340 INFO:tasks.workunit.client.0.smithi063.stdout:7/432: mknod d0/d15/d49/d4f/d66/c95 0 2024-05-09T06:38:11.341 INFO:tasks.workunit.client.0.smithi063.stdout:6/473: mknod d2/df/d59/c9b 0 2024-05-09T06:38:11.342 INFO:tasks.workunit.client.0.smithi063.stdout:5/429: dread d8/d17/d18/f58 [0,4194304] 0 2024-05-09T06:38:11.343 INFO:tasks.workunit.client.0.smithi063.stdout:6/474: write d2/d20/d26/f97 [601202,129515] 0 2024-05-09T06:38:11.344 INFO:tasks.workunit.client.0.smithi063.stdout:7/433: creat d0/d15/d4a/d50/f96 x:0 0 0 2024-05-09T06:38:11.345 INFO:tasks.workunit.client.0.smithi063.stdout:1/413: creat d9/d2a/d35/f8f x:0 0 0 2024-05-09T06:38:11.346 INFO:tasks.workunit.client.0.smithi063.stdout:5/430: write f6 [2102892,104438] 0 2024-05-09T06:38:11.347 INFO:tasks.workunit.client.0.smithi063.stdout:6/475: creat d2/d20/f9c x:0 0 0 2024-05-09T06:38:11.347 INFO:tasks.workunit.client.0.smithi063.stdout:1/414: write d9/d2a/d3c/d4a/f67 [468968,122544] 0 2024-05-09T06:38:11.350 INFO:tasks.workunit.client.0.smithi063.stdout:5/431: write d8/f66 [230561,34498] 0 2024-05-09T06:38:11.350 INFO:tasks.workunit.client.0.smithi063.stdout:1/415: chown d9/l5a 19408781 1 2024-05-09T06:38:11.350 INFO:tasks.workunit.client.0.smithi063.stdout:9/459: creat d0/f9f x:0 0 0 2024-05-09T06:38:11.353 INFO:tasks.workunit.client.0.smithi063.stdout:9/460: write d0/d4/d57/d60/f7c [680309,106843] 0 2024-05-09T06:38:11.354 INFO:tasks.workunit.client.0.smithi063.stdout:4/463: truncate d7/fb 3537237 0 2024-05-09T06:38:11.355 INFO:tasks.workunit.client.0.smithi063.stdout:4/464: chown d7/df/l5c 129693 1 2024-05-09T06:38:11.356 INFO:tasks.workunit.client.0.smithi063.stdout:7/434: dread d0/d15/f26 [0,4194304] 0 2024-05-09T06:38:11.356 INFO:tasks.workunit.client.0.smithi063.stdout:9/461: truncate d0/d4/d7d/f87 311915 0 2024-05-09T06:38:11.356 INFO:tasks.workunit.client.0.smithi063.stdout:4/465: stat d7/df/d1f/d28/c40 0 2024-05-09T06:38:11.357 INFO:tasks.workunit.client.0.smithi063.stdout:5/432: mknod d8/c98 0 2024-05-09T06:38:11.358 INFO:tasks.workunit.client.0.smithi063.stdout:6/476: creat d2/d1f/d29/d3f/d5c/f9d x:0 0 0 2024-05-09T06:38:11.359 INFO:tasks.workunit.client.0.smithi063.stdout:9/462: write d0/f5b [600754,15008] 0 2024-05-09T06:38:11.359 INFO:tasks.workunit.client.0.smithi063.stdout:5/433: stat d8/f16 0 2024-05-09T06:38:11.361 INFO:tasks.workunit.client.1.smithi193.stdout:5/187: write d0/d13/d2c/d37/f11 [2379364,2417] 0 2024-05-09T06:38:11.362 INFO:tasks.workunit.client.0.smithi063.stdout:1/416: getdents d9/d2a/d35/d24 0 2024-05-09T06:38:11.364 INFO:tasks.workunit.client.0.smithi063.stdout:4/466: rename d7/df/d1a/c38 to d7/df/d1f/d28/c90 0 2024-05-09T06:38:11.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/434: creat d8/d17/d75/f99 x:0 0 0 2024-05-09T06:38:11.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/435: dread - d8/d53/f93 zero size 2024-05-09T06:38:11.369 INFO:tasks.workunit.client.0.smithi063.stdout:1/417: write d9/d2a/d35/d1e/f3a [3310223,8770] 0 2024-05-09T06:38:11.370 INFO:tasks.workunit.client.1.smithi193.stdout:3/55: sync 2024-05-09T06:38:11.371 INFO:tasks.workunit.client.0.smithi063.stdout:9/463: rename d0/d4/d7d/d8b to d0/d3b/da0 0 2024-05-09T06:38:11.372 INFO:tasks.workunit.client.0.smithi063.stdout:4/467: creat d7/df/d57/f91 x:0 0 0 2024-05-09T06:38:11.373 INFO:tasks.workunit.client.0.smithi063.stdout:5/436: rmdir d8/d17 39 2024-05-09T06:38:11.375 INFO:tasks.workunit.client.1.smithi193.stdout:3/56: mknod d1/c12 0 2024-05-09T06:38:11.375 INFO:tasks.workunit.client.0.smithi063.stdout:7/435: dwrite d0/d15/d4a/d50/f2e [4194304,4194304] 0 2024-05-09T06:38:11.376 INFO:tasks.workunit.client.0.smithi063.stdout:6/477: link d2/d20/d4a/f93 d2/d1f/d29/d3f/d5c/f9e 0 2024-05-09T06:38:11.377 INFO:tasks.workunit.client.0.smithi063.stdout:7/436: write d0/d15/d4a/d50/f29 [1362239,86791] 0 2024-05-09T06:38:11.378 INFO:tasks.workunit.client.0.smithi063.stdout:1/418: rmdir d9/d2a/d2e 39 2024-05-09T06:38:11.379 INFO:tasks.workunit.client.0.smithi063.stdout:1/419: dread - d9/d10/d2b/d3f/d59/f79 zero size 2024-05-09T06:38:11.380 INFO:tasks.workunit.client.0.smithi063.stdout:6/478: dread d2/d12/f76 [0,4194304] 0 2024-05-09T06:38:11.380 INFO:tasks.workunit.client.1.smithi193.stdout:3/57: mknod d1/c13 0 2024-05-09T06:38:11.382 INFO:tasks.workunit.client.1.smithi193.stdout:5/188: link d0/d1/da/l24 d0/d1/l3c 0 2024-05-09T06:38:11.383 INFO:tasks.workunit.client.1.smithi193.stdout:5/189: write d0/d1/f21 [3777794,37612] 0 2024-05-09T06:38:11.384 INFO:tasks.workunit.client.0.smithi063.stdout:9/464: creat d0/d4/d57/fa1 x:0 0 0 2024-05-09T06:38:11.384 INFO:tasks.workunit.client.1.smithi193.stdout:5/190: dread - d0/d13/d2c/d37/f36 zero size 2024-05-09T06:38:11.385 INFO:tasks.workunit.client.0.smithi063.stdout:9/465: write d0/d4/d57/f64 [1481086,125992] 0 2024-05-09T06:38:11.385 INFO:tasks.workunit.client.0.smithi063.stdout:9/466: dread - d0/f9f zero size 2024-05-09T06:38:11.387 INFO:tasks.workunit.client.0.smithi063.stdout:4/468: mknod d7/d50/c92 0 2024-05-09T06:38:11.390 INFO:tasks.workunit.client.1.smithi193.stdout:5/191: chown d0/d1/da/c1d 344305 1 2024-05-09T06:38:11.392 INFO:tasks.workunit.client.0.smithi063.stdout:5/437: creat d8/d19/d30/d46/f9a x:0 0 0 2024-05-09T06:38:11.393 INFO:tasks.workunit.client.0.smithi063.stdout:4/469: dwrite d7/f42 [0,4194304] 0 2024-05-09T06:38:11.395 INFO:tasks.workunit.client.0.smithi063.stdout:7/437: mkdir d0/d15/d82/d97 0 2024-05-09T06:38:11.396 INFO:tasks.workunit.client.1.smithi193.stdout:3/58: write d1/f4 [1733012,93357] 0 2024-05-09T06:38:11.398 INFO:tasks.workunit.client.1.smithi193.stdout:5/192: link d0/d13/d2c/d37/f11 d0/d13/f3d 0 2024-05-09T06:38:11.406 INFO:tasks.workunit.client.1.smithi193.stdout:5/193: stat d0/f8 0 2024-05-09T06:38:11.406 INFO:tasks.workunit.client.1.smithi193.stdout:8/130: sync 2024-05-09T06:38:11.406 INFO:tasks.workunit.client.1.smithi193.stdout:9/136: sync 2024-05-09T06:38:11.406 INFO:tasks.workunit.client.1.smithi193.stdout:0/172: sync 2024-05-09T06:38:11.406 INFO:tasks.workunit.client.1.smithi193.stdout:6/134: sync 2024-05-09T06:38:11.407 INFO:tasks.workunit.client.1.smithi193.stdout:2/133: sync 2024-05-09T06:38:11.407 INFO:tasks.workunit.client.1.smithi193.stdout:1/101: sync 2024-05-09T06:38:11.407 INFO:tasks.workunit.client.1.smithi193.stdout:4/130: sync 2024-05-09T06:38:11.407 INFO:tasks.workunit.client.1.smithi193.stdout:7/152: sync 2024-05-09T06:38:11.407 INFO:tasks.workunit.client.1.smithi193.stdout:7/153: readlink d6/d8/d1a/l26 0 2024-05-09T06:38:11.407 INFO:tasks.workunit.client.1.smithi193.stdout:9/137: write dd/f29 [410505,18148] 0 2024-05-09T06:38:11.409 INFO:tasks.workunit.client.0.smithi063.stdout:9/467: dwrite d0/f97 [0,4194304] 0 2024-05-09T06:38:11.409 INFO:tasks.workunit.client.1.smithi193.stdout:1/102: truncate d4/ff 954188 0 2024-05-09T06:38:11.410 INFO:tasks.workunit.client.1.smithi193.stdout:4/131: dread - f10 zero size 2024-05-09T06:38:11.410 INFO:tasks.workunit.client.1.smithi193.stdout:7/154: write f4 [2945721,39217] 0 2024-05-09T06:38:11.411 INFO:tasks.workunit.client.1.smithi193.stdout:0/173: unlink c14 0 2024-05-09T06:38:11.412 INFO:tasks.workunit.client.0.smithi063.stdout:4/470: dread d7/df/d35/d5a/d60/f63 [0,4194304] 0 2024-05-09T06:38:11.412 INFO:tasks.workunit.client.1.smithi193.stdout:5/194: dread d0/f8 [0,4194304] 0 2024-05-09T06:38:11.415 INFO:tasks.workunit.client.0.smithi063.stdout:7/438: chown d0/d15/d49/d54/l78 0 1 2024-05-09T06:38:11.416 INFO:tasks.workunit.client.1.smithi193.stdout:2/134: stat db/df/d12/f24 0 2024-05-09T06:38:11.417 INFO:tasks.workunit.client.1.smithi193.stdout:8/131: read d6/f23 [2536071,114684] 0 2024-05-09T06:38:11.418 INFO:tasks.workunit.client.1.smithi193.stdout:9/138: dread dd/f13 [0,4194304] 0 2024-05-09T06:38:11.423 INFO:tasks.workunit.client.1.smithi193.stdout:0/174: rmdir d18 39 2024-05-09T06:38:11.425 INFO:tasks.workunit.client.1.smithi193.stdout:4/132: symlink d1a/d1f/l2b 0 2024-05-09T06:38:11.427 INFO:tasks.workunit.client.1.smithi193.stdout:7/155: dwrite f4 [0,4194304] 0 2024-05-09T06:38:11.428 INFO:tasks.workunit.client.0.smithi063.stdout:6/479: dwrite d2/d4/f5 [0,4194304] 0 2024-05-09T06:38:11.429 INFO:tasks.workunit.client.1.smithi193.stdout:4/133: dread - f10 zero size 2024-05-09T06:38:11.429 INFO:tasks.workunit.client.1.smithi193.stdout:3/59: dwrite d1/f9 [0,4194304] 0 2024-05-09T06:38:11.430 INFO:tasks.workunit.client.0.smithi063.stdout:9/468: truncate d0/d3b/f6e 159132 0 2024-05-09T06:38:11.431 INFO:tasks.workunit.client.1.smithi193.stdout:3/60: dread - d1/f8 zero size 2024-05-09T06:38:11.432 INFO:tasks.workunit.client.1.smithi193.stdout:1/103: dwrite d4/f13 [4194304,4194304] 0 2024-05-09T06:38:11.432 INFO:tasks.workunit.client.1.smithi193.stdout:5/195: fdatasync d0/d13/d2c/d37/f36 0 2024-05-09T06:38:11.436 INFO:tasks.workunit.client.1.smithi193.stdout:8/132: unlink c1 0 2024-05-09T06:38:11.437 INFO:tasks.workunit.client.1.smithi193.stdout:9/139: mknod dd/d1e/c2d 0 2024-05-09T06:38:11.437 INFO:tasks.workunit.client.0.smithi063.stdout:1/420: getdents d9/d2a/d35/d24 0 2024-05-09T06:38:11.438 INFO:tasks.workunit.client.1.smithi193.stdout:9/140: stat dd/f13 0 2024-05-09T06:38:11.438 INFO:tasks.workunit.client.1.smithi193.stdout:5/196: read d0/d13/d2c/d37/f1c [162294,119437] 0 2024-05-09T06:38:11.438 INFO:tasks.workunit.client.0.smithi063.stdout:1/421: truncate d9/d2a/d35/d1e/f27 4571794 0 2024-05-09T06:38:11.443 INFO:tasks.workunit.client.1.smithi193.stdout:2/135: truncate db/df/f20 911901 0 2024-05-09T06:38:11.443 INFO:tasks.workunit.client.1.smithi193.stdout:1/104: dwrite d4/fc [4194304,4194304] 0 2024-05-09T06:38:11.444 INFO:tasks.workunit.client.1.smithi193.stdout:1/105: readlink d4/l6 0 2024-05-09T06:38:11.445 INFO:tasks.workunit.client.0.smithi063.stdout:6/480: rmdir d2 39 2024-05-09T06:38:11.447 INFO:tasks.workunit.client.1.smithi193.stdout:2/136: write db/f1b [560343,34297] 0 2024-05-09T06:38:11.448 INFO:tasks.workunit.client.0.smithi063.stdout:3/494: sync 2024-05-09T06:38:11.448 INFO:tasks.workunit.client.0.smithi063.stdout:2/506: sync 2024-05-09T06:38:11.448 INFO:tasks.workunit.client.0.smithi063.stdout:0/508: sync 2024-05-09T06:38:11.448 INFO:tasks.workunit.client.0.smithi063.stdout:8/534: sync 2024-05-09T06:38:11.450 INFO:tasks.workunit.client.0.smithi063.stdout:4/471: link d7/df/d1f/f3a d7/df/d57/f93 0 2024-05-09T06:38:11.451 INFO:tasks.workunit.client.1.smithi193.stdout:7/156: unlink f4 0 2024-05-09T06:38:11.452 INFO:tasks.workunit.client.0.smithi063.stdout:9/469: mknod d0/d4/d26/d39/d3e/d66/ca2 0 2024-05-09T06:38:11.453 INFO:tasks.workunit.client.0.smithi063.stdout:3/495: truncate d7/da/d2b/d66/d74/d7e/f99 572134 0 2024-05-09T06:38:11.453 INFO:tasks.workunit.client.1.smithi193.stdout:4/134: creat d1a/d1f/f2c x:0 0 0 2024-05-09T06:38:11.454 INFO:tasks.workunit.client.0.smithi063.stdout:7/439: mkdir d0/d15/d89/d98 0 2024-05-09T06:38:11.454 INFO:tasks.workunit.client.0.smithi063.stdout:7/440: chown d0/d15/d82 505021985 1 2024-05-09T06:38:11.458 INFO:tasks.workunit.client.1.smithi193.stdout:3/61: creat d1/db/f14 x:0 0 0 2024-05-09T06:38:11.459 INFO:tasks.workunit.client.0.smithi063.stdout:8/535: creat d9/da/d18/d24/d71/fb3 x:0 0 0 2024-05-09T06:38:11.460 INFO:tasks.workunit.client.1.smithi193.stdout:8/133: rmdir d6 39 2024-05-09T06:38:11.462 INFO:tasks.workunit.client.0.smithi063.stdout:7/441: dwrite d0/d15/f25 [0,4194304] 0 2024-05-09T06:38:11.466 INFO:tasks.workunit.client.1.smithi193.stdout:1/106: fdatasync d4/f13 0 2024-05-09T06:38:11.467 INFO:tasks.workunit.client.0.smithi063.stdout:2/507: mkdir dc/d36/d43/d46/d63/d79/d7d/d8d 0 2024-05-09T06:38:11.469 INFO:tasks.workunit.client.1.smithi193.stdout:5/197: dread d0/f29 [0,4194304] 0 2024-05-09T06:38:11.471 INFO:tasks.workunit.client.0.smithi063.stdout:3/496: mkdir d7/d22/d9c 0 2024-05-09T06:38:11.474 INFO:tasks.workunit.client.1.smithi193.stdout:0/175: creat d18/d29/f37 x:0 0 0 2024-05-09T06:38:11.475 INFO:tasks.workunit.client.1.smithi193.stdout:0/176: truncate d18/d1d/f30 215916 0 2024-05-09T06:38:11.475 INFO:tasks.workunit.client.0.smithi063.stdout:9/470: symlink d0/d4/d26/d39/d91/la3 0 2024-05-09T06:38:11.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/137: mknod db/df/d12/c31 0 2024-05-09T06:38:11.480 INFO:tasks.workunit.client.1.smithi193.stdout:7/157: creat d6/d17/f37 x:0 0 0 2024-05-09T06:38:11.485 INFO:tasks.workunit.client.0.smithi063.stdout:8/536: creat d9/da/d18/d56/d58/fb4 x:0 0 0 2024-05-09T06:38:11.486 INFO:tasks.workunit.client.1.smithi193.stdout:3/62: symlink d1/l15 0 2024-05-09T06:38:11.486 INFO:tasks.workunit.client.1.smithi193.stdout:7/158: dwrite d6/d17/f23 [0,4194304] 0 2024-05-09T06:38:11.487 INFO:tasks.workunit.client.1.smithi193.stdout:7/159: fsync d6/dd/ff 0 2024-05-09T06:38:11.488 INFO:tasks.workunit.client.0.smithi063.stdout:6/481: dread d2/df/f5d [0,4194304] 0 2024-05-09T06:38:11.494 INFO:tasks.workunit.client.0.smithi063.stdout:7/442: dwrite d0/d15/f48 [0,4194304] 0 2024-05-09T06:38:11.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.513+0000 7f23f94e0700 1 -- 172.21.15.63:0/4122588605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4101e20 msgr2=0x7f23f4102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:11.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.513+0000 7f23f94e0700 1 --2- 172.21.15.63:0/4122588605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4101e20 0x7f23f4102280 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009910 tx=0x7f23e8009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:11.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.514+0000 7f23f94e0700 1 -- 172.21.15.63:0/4122588605 shutdown_connections 2024-05-09T06:38:11.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.514+0000 7f23f94e0700 1 --2- 172.21.15.63:0/4122588605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4101e20 0x7f23f4102280 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.514+0000 7f23f94e0700 1 --2- 172.21.15.63:0/4122588605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4100c20 0x7f23f4101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.514+0000 7f23f94e0700 1 -- 172.21.15.63:0/4122588605 >> 172.21.15.63:0/4122588605 conn(0x7f23f40fc200 msgr2=0x7f23f40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:11.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.514+0000 7f23f94e0700 1 -- 172.21.15.63:0/4122588605 shutdown_connections 2024-05-09T06:38:11.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.514+0000 7f23f94e0700 1 -- 172.21.15.63:0/4122588605 wait complete. 2024-05-09T06:38:11.496 INFO:tasks.workunit.client.1.smithi193.stdout:7/160: dwrite d6/f16 [8388608,4194304] 0 2024-05-09T06:38:11.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.516+0000 7f23f94e0700 1 Processor -- start 2024-05-09T06:38:11.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.516+0000 7f23f94e0700 1 -- start start 2024-05-09T06:38:11.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.516+0000 7f23f94e0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4100c20 0x7f23f4074b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:11.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.516+0000 7f23f94e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4101e20 0x7f23f40732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:11.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.516+0000 7f23f94e0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f40751a0 con 0x7f23f4100c20 2024-05-09T06:38:11.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.516+0000 7f23f94e0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f419a410 con 0x7f23f4101e20 2024-05-09T06:38:11.498 INFO:tasks.workunit.client.0.smithi063.stdout:0/509: dwrite dd/d1a/d2c/d51/d72/d9e/f64 [0,4194304] 0 2024-05-09T06:38:11.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.517+0000 7f23f2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4100c20 0x7f23f4074b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:11.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.517+0000 7f23f2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4100c20 0x7f23f4074b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55984/0 (socket says 172.21.15.63:55984) 2024-05-09T06:38:11.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.517+0000 7f23f2ffd700 1 -- 172.21.15.63:0/1818787715 learned_addr learned my addr 172.21.15.63:0/1818787715 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:11.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.517+0000 7f23f27fc700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4101e20 0x7f23f40732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:11.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.517+0000 7f23f27fc700 1 -- 172.21.15.63:0/1818787715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4100c20 msgr2=0x7f23f4074b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:11.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.517+0000 7f23f27fc700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4100c20 0x7f23f4074b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.517+0000 7f23f27fc700 1 -- 172.21.15.63:0/1818787715 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e80095f0 con 0x7f23f4101e20 2024-05-09T06:38:11.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.517+0000 7f23f2ffd700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4100c20 0x7f23f4074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:38:11.499 INFO:tasks.workunit.client.1.smithi193.stdout:0/177: rmdir d18 39 2024-05-09T06:38:11.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.518+0000 7f23f27fc700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4101e20 0x7f23f40732e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23e80059d0 tx=0x7f23e800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:11.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.518+0000 7f23dbfff700 1 -- 172.21.15.63:0/1818787715 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23e801d070 con 0x7f23f4101e20 2024-05-09T06:38:11.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.518+0000 7f23dbfff700 1 -- 172.21.15.63:0/1818787715 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23e8005dc0 con 0x7f23f4101e20 2024-05-09T06:38:11.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.518+0000 7f23dbfff700 1 -- 172.21.15.63:0/1818787715 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23e8021410 con 0x7f23f4101e20 2024-05-09T06:38:11.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.518+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f419a640 con 0x7f23f4101e20 2024-05-09T06:38:11.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.518+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f4073b80 con 0x7f23f4101e20 2024-05-09T06:38:11.500 INFO:tasks.workunit.client.0.smithi063.stdout:7/443: readlink d0/d15/d49/d4f/d66/d67/l7a 0 2024-05-09T06:38:11.500 INFO:tasks.workunit.client.0.smithi063.stdout:7/444: dread - d0/d15/d49/f5c zero size 2024-05-09T06:38:11.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.520+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f4190840 con 0x7f23f4101e20 2024-05-09T06:38:11.501 INFO:tasks.workunit.client.0.smithi063.stdout:9/471: rmdir d0/d4/d26/d39/d3e/d66 39 2024-05-09T06:38:11.502 INFO:tasks.workunit.client.0.smithi063.stdout:9/472: chown d0/d4/d26/d39/d48 10595748 1 2024-05-09T06:38:11.503 INFO:tasks.workunit.client.1.smithi193.stdout:2/138: creat db/df/d12/f32 x:0 0 0 2024-05-09T06:38:11.503 INFO:tasks.workunit.client.1.smithi193.stdout:2/139: readlink db/l30 0 2024-05-09T06:38:11.505 INFO:tasks.workunit.client.0.smithi063.stdout:1/422: creat d9/f90 x:0 0 0 2024-05-09T06:38:11.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.525+0000 7f23dbfff700 1 -- 172.21.15.63:0/1818787715 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f23e800f7b0 con 0x7f23f4101e20 2024-05-09T06:38:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.526+0000 7f23dbfff700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f23dc06d6f0 0x7f23dc06fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.526+0000 7f23dbfff700 1 -- 172.21.15.63:0/1818787715 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f23e808e5a0 con 0x7f23f4101e20 2024-05-09T06:38:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.526+0000 7f23f2ffd700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f23dc06d6f0 0x7f23dc06fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:11.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.527+0000 7f23f2ffd700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f23dc06d6f0 0x7f23dc06fbb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f23e400c530 tx=0x7f23e400b040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:11.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.532+0000 7f23dbfff700 1 -- 172.21.15.63:0/1818787715 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f23e805b7f0 con 0x7f23f4101e20 2024-05-09T06:38:11.513 INFO:tasks.workunit.client.0.smithi063.stdout:6/482: rmdir d2/d1f/d29/d3f/d5c/d64 39 2024-05-09T06:38:11.514 INFO:tasks.workunit.client.0.smithi063.stdout:6/483: fdatasync d2/d1f/d29/d3b/d4e/f67 0 2024-05-09T06:38:11.514 INFO:tasks.workunit.client.0.smithi063.stdout:4/472: creat d7/df/d1f/d28/d31/f94 x:0 0 0 2024-05-09T06:38:11.517 INFO:tasks.workunit.client.1.smithi193.stdout:3/63: unlink d1/db/c10 0 2024-05-09T06:38:11.518 INFO:tasks.workunit.client.0.smithi063.stdout:9/473: dread d0/d4/d26/f33 [0,4194304] 0 2024-05-09T06:38:11.519 INFO:tasks.workunit.client.0.smithi063.stdout:2/508: mkdir dc/d36/d43/d46/d63/d79/d7d/d8d/d8e 0 2024-05-09T06:38:11.520 INFO:tasks.workunit.client.1.smithi193.stdout:9/141: link l5 dd/d1e/d1f/l2e 0 2024-05-09T06:38:11.521 INFO:tasks.workunit.client.1.smithi193.stdout:1/107: creat d4/d14/f1b x:0 0 0 2024-05-09T06:38:11.522 INFO:tasks.workunit.client.0.smithi063.stdout:2/509: dread dc/dd/f23 [8388608,4194304] 0 2024-05-09T06:38:11.522 INFO:tasks.workunit.client.0.smithi063.stdout:0/510: truncate dd/d1a/d2c/d51/d72/d9e/d23/d33/f3e 880078 0 2024-05-09T06:38:11.522 INFO:tasks.workunit.client.1.smithi193.stdout:5/198: creat d0/d13/d2c/d38/f3e x:0 0 0 2024-05-09T06:38:11.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/511: chown dd/d1a/c49 6 1 2024-05-09T06:38:11.525 INFO:tasks.workunit.client.1.smithi193.stdout:0/178: dread - d18/f26 zero size 2024-05-09T06:38:11.527 INFO:tasks.workunit.client.1.smithi193.stdout:7/161: dread d6/dd/f22 [0,4194304] 0 2024-05-09T06:38:11.527 INFO:tasks.workunit.client.1.smithi193.stdout:1/108: dread d4/fc [4194304,4194304] 0 2024-05-09T06:38:11.528 INFO:tasks.workunit.client.0.smithi063.stdout:9/474: creat d0/d4/d57/fa4 x:0 0 0 2024-05-09T06:38:11.529 INFO:tasks.workunit.client.1.smithi193.stdout:3/64: mknod d1/c16 0 2024-05-09T06:38:11.530 INFO:tasks.workunit.client.0.smithi063.stdout:9/475: chown d0/d4/d26/f33 1077810 1 2024-05-09T06:38:11.531 INFO:tasks.workunit.client.1.smithi193.stdout:3/65: truncate d1/f8 308349 0 2024-05-09T06:38:11.532 INFO:tasks.workunit.client.1.smithi193.stdout:3/66: write d1/f8 [1263846,115385] 0 2024-05-09T06:38:11.533 INFO:tasks.workunit.client.0.smithi063.stdout:6/484: dwrite d2/d1f/d29/d3f/d5c/d64/f6f [0,4194304] 0 2024-05-09T06:38:11.533 INFO:tasks.workunit.client.1.smithi193.stdout:9/142: unlink dd/df/d15/f17 0 2024-05-09T06:38:11.535 INFO:tasks.workunit.client.1.smithi193.stdout:1/109: dwrite d4/fc [4194304,4194304] 0 2024-05-09T06:38:11.536 INFO:tasks.workunit.client.1.smithi193.stdout:1/110: chown d4/d16/fb 1347520791 1 2024-05-09T06:38:11.536 INFO:tasks.workunit.client.1.smithi193.stdout:1/111: readlink d4/l6 0 2024-05-09T06:38:11.537 INFO:tasks.workunit.client.0.smithi063.stdout:7/445: symlink d0/d15/d89/d98/l99 0 2024-05-09T06:38:11.540 INFO:tasks.workunit.client.0.smithi063.stdout:2/510: mkdir dc/dd/d45/d5a/d6c/d8f 0 2024-05-09T06:38:11.540 INFO:tasks.workunit.client.0.smithi063.stdout:8/537: dwrite d9/da/d18/d56/d58/f8e [0,4194304] 0 2024-05-09T06:38:11.542 INFO:tasks.workunit.client.0.smithi063.stdout:2/511: chown dc/dd/d45/f89 658064442 1 2024-05-09T06:38:11.544 INFO:tasks.workunit.client.0.smithi063.stdout:0/512: creat dd/d1a/d2c/d51/d72/d9e/d23/d46/fa7 x:0 0 0 2024-05-09T06:38:11.547 INFO:tasks.workunit.client.1.smithi193.stdout:8/134: write d6/d1c/f12 [1512099,21685] 0 2024-05-09T06:38:11.554 INFO:tasks.workunit.client.1.smithi193.stdout:3/67: mknod d1/db/c17 0 2024-05-09T06:38:11.556 INFO:tasks.workunit.client.0.smithi063.stdout:0/513: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/f4e [4194304,4194304] 0 2024-05-09T06:38:11.556 INFO:tasks.workunit.client.1.smithi193.stdout:9/143: creat dd/d1e/d1f/f2f x:0 0 0 2024-05-09T06:38:11.557 INFO:tasks.workunit.client.0.smithi063.stdout:0/514: stat dd/f15 0 2024-05-09T06:38:11.557 INFO:tasks.workunit.client.1.smithi193.stdout:3/68: dread f0 [0,4194304] 0 2024-05-09T06:38:11.558 INFO:tasks.workunit.client.1.smithi193.stdout:3/69: write d1/f4 [827337,52251] 0 2024-05-09T06:38:11.560 INFO:tasks.workunit.client.1.smithi193.stdout:1/112: mknod d4/d14/d15/c1c 0 2024-05-09T06:38:11.564 INFO:tasks.workunit.client.1.smithi193.stdout:5/199: rename d0/d1/da/f14 to d0/d13/d2c/f3f 0 2024-05-09T06:38:11.568 INFO:tasks.workunit.client.1.smithi193.stdout:8/135: creat d6/d1c/d18/f26 x:0 0 0 2024-05-09T06:38:11.569 INFO:tasks.workunit.client.1.smithi193.stdout:7/162: chown d6/d8/d1a/d1f 444 1 2024-05-09T06:38:11.570 INFO:tasks.workunit.client.1.smithi193.stdout:7/163: write d6/d17/f37 [951389,89884] 0 2024-05-09T06:38:11.571 INFO:tasks.workunit.client.1.smithi193.stdout:9/144: read f0 [1673249,84726] 0 2024-05-09T06:38:11.572 INFO:tasks.workunit.client.1.smithi193.stdout:9/145: fdatasync dd/d1e/f27 0 2024-05-09T06:38:11.573 INFO:tasks.workunit.client.1.smithi193.stdout:3/70: creat d1/db/f18 x:0 0 0 2024-05-09T06:38:11.574 INFO:tasks.workunit.client.1.smithi193.stdout:1/113: fsync d4/f9 0 2024-05-09T06:38:11.575 INFO:tasks.workunit.client.1.smithi193.stdout:1/114: chown d4/d14/c19 34978605 1 2024-05-09T06:38:11.578 INFO:tasks.workunit.client.1.smithi193.stdout:5/200: rename d0/d13/d2c/d37/f1c to d0/d13/f40 0 2024-05-09T06:38:11.579 INFO:tasks.workunit.client.1.smithi193.stdout:1/115: dread d4/d16/de/f12 [0,4194304] 0 2024-05-09T06:38:11.580 INFO:tasks.workunit.client.0.smithi063.stdout:1/423: link d9/d10/d2b/f33 d9/d2a/d35/f91 0 2024-05-09T06:38:11.582 INFO:tasks.workunit.client.1.smithi193.stdout:1/116: dread d4/f13 [4194304,4194304] 0 2024-05-09T06:38:11.582 INFO:tasks.workunit.client.0.smithi063.stdout:2/512: rename dc/dd/d45/d58 to dc/dd/d2d/d60/d65/d72/d90 0 2024-05-09T06:38:11.586 INFO:tasks.workunit.client.0.smithi063.stdout:0/515: unlink dd/d1a/d2c/d51/d72/d9e/d23/d46/c9a 0 2024-05-09T06:38:11.588 INFO:tasks.workunit.client.1.smithi193.stdout:0/179: truncate f13 131460 0 2024-05-09T06:38:11.590 INFO:tasks.workunit.client.1.smithi193.stdout:3/71: creat d1/db/f19 x:0 0 0 2024-05-09T06:38:11.591 INFO:tasks.workunit.client.0.smithi063.stdout:9/476: mkdir d0/da5 0 2024-05-09T06:38:11.591 INFO:tasks.workunit.client.1.smithi193.stdout:9/146: mknod dd/df/d15/d16/c30 0 2024-05-09T06:38:11.592 INFO:tasks.workunit.client.0.smithi063.stdout:6/485: mkdir d2/df/d59/d99/d9f 0 2024-05-09T06:38:11.594 INFO:tasks.workunit.client.0.smithi063.stdout:3/497: getdents d7/da/d21 0 2024-05-09T06:38:11.595 INFO:tasks.workunit.client.0.smithi063.stdout:5/438: sync 2024-05-09T06:38:11.595 INFO:tasks.workunit.client.0.smithi063.stdout:5/439: dread - d8/f8e zero size 2024-05-09T06:38:11.596 INFO:tasks.workunit.client.0.smithi063.stdout:7/446: link d0/d15/f72 d0/d15/f9a 0 2024-05-09T06:38:11.596 INFO:tasks.workunit.client.0.smithi063.stdout:7/447: fdatasync d0/d15/f76 0 2024-05-09T06:38:11.597 INFO:tasks.workunit.client.1.smithi193.stdout:5/201: mknod d0/d13/d2c/d37/c41 0 2024-05-09T06:38:11.598 INFO:tasks.workunit.client.0.smithi063.stdout:8/538: creat d9/da/d53/db2/fb5 x:0 0 0 2024-05-09T06:38:11.600 INFO:tasks.workunit.client.0.smithi063.stdout:1/424: creat d9/d2a/d3c/d4c/f92 x:0 0 0 2024-05-09T06:38:11.600 INFO:tasks.workunit.client.1.smithi193.stdout:8/136: mkdir d6/d1c/d27 0 2024-05-09T06:38:11.602 INFO:tasks.workunit.client.0.smithi063.stdout:2/513: symlink dc/d36/d43/d46/d63/d79/d7d/l91 0 2024-05-09T06:38:11.603 INFO:tasks.workunit.client.0.smithi063.stdout:2/514: chown dc/dd/d2d/d60/d65/d72/d90/l78 6 1 2024-05-09T06:38:11.603 INFO:tasks.workunit.client.1.smithi193.stdout:7/164: creat d6/d8/d29/d2f/f38 x:0 0 0 2024-05-09T06:38:11.604 INFO:tasks.workunit.client.1.smithi193.stdout:0/180: creat d18/d29/f38 x:0 0 0 2024-05-09T06:38:11.605 INFO:tasks.workunit.client.1.smithi193.stdout:3/72: mknod d1/db/c1a 0 2024-05-09T06:38:11.606 INFO:tasks.workunit.client.1.smithi193.stdout:9/147: mkdir dd/df/d31 0 2024-05-09T06:38:11.611 INFO:tasks.workunit.client.0.smithi063.stdout:8/539: mknod d9/da/d53/db2/cb6 0 2024-05-09T06:38:11.611 INFO:tasks.workunit.client.1.smithi193.stdout:8/137: mknod d6/d1c/d18/c28 0 2024-05-09T06:38:11.613 INFO:tasks.workunit.client.0.smithi063.stdout:9/477: mkdir d0/d8/da6 0 2024-05-09T06:38:11.613 INFO:tasks.workunit.client.0.smithi063.stdout:6/486: dwrite d2/d20/f21 [4194304,4194304] 0 2024-05-09T06:38:11.615 INFO:tasks.workunit.client.0.smithi063.stdout:2/515: rename dc/dd/d2d/d60/d65 to dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92 0 2024-05-09T06:38:11.616 INFO:tasks.workunit.client.1.smithi193.stdout:0/181: dread - f16 zero size 2024-05-09T06:38:11.618 INFO:tasks.workunit.client.0.smithi063.stdout:6/487: chown d2/d1f/d29 1 1 2024-05-09T06:38:11.619 INFO:tasks.workunit.client.1.smithi193.stdout:3/73: mkdir d1/db/d1b 0 2024-05-09T06:38:11.621 INFO:tasks.workunit.client.0.smithi063.stdout:3/498: dwrite d7/d70/d98/f4e [0,4194304] 0 2024-05-09T06:38:11.622 INFO:tasks.workunit.client.0.smithi063.stdout:5/440: dwrite d8/d17/f6d [0,4194304] 0 2024-05-09T06:38:11.626 INFO:tasks.workunit.client.0.smithi063.stdout:2/516: dread dc/dd/f23 [8388608,4194304] 0 2024-05-09T06:38:11.627 INFO:tasks.workunit.client.1.smithi193.stdout:8/138: mkdir d6/d1c/d1b/d29 0 2024-05-09T06:38:11.628 INFO:tasks.workunit.client.0.smithi063.stdout:2/517: write dc/d32/d33/d4e/f88 [1017153,19746] 0 2024-05-09T06:38:11.629 INFO:tasks.workunit.client.0.smithi063.stdout:4/473: sync 2024-05-09T06:38:11.633 INFO:tasks.workunit.client.0.smithi063.stdout:6/488: dwrite d2/d4/f92 [0,4194304] 0 2024-05-09T06:38:11.635 INFO:tasks.workunit.client.1.smithi193.stdout:0/182: creat d18/d27/f39 x:0 0 0 2024-05-09T06:38:11.639 INFO:tasks.workunit.client.1.smithi193.stdout:3/74: truncate f0 779189 0 2024-05-09T06:38:11.640 INFO:tasks.workunit.client.0.smithi063.stdout:8/540: rename d9/da/d45/l60 to d9/da/d18/d56/lb7 0 2024-05-09T06:38:11.642 INFO:tasks.workunit.client.0.smithi063.stdout:9/478: creat d0/d4/d26/d39/d91/fa7 x:0 0 0 2024-05-09T06:38:11.643 INFO:tasks.workunit.client.1.smithi193.stdout:9/148: creat dd/df/d15/d16/d26/f32 x:0 0 0 2024-05-09T06:38:11.645 INFO:tasks.workunit.client.0.smithi063.stdout:4/474: dwrite d7/df/d1f/d28/d31/d32/f8c [0,4194304] 0 2024-05-09T06:38:11.645 INFO:tasks.workunit.client.0.smithi063.stdout:0/516: getdents dd/d1a/d2c/d51/d72/d9e/d23/d33 0 2024-05-09T06:38:11.650 INFO:tasks.workunit.client.1.smithi193.stdout:1/117: getdents d4/d16 0 2024-05-09T06:38:11.650 INFO:tasks.workunit.client.1.smithi193.stdout:9/149: dwrite fa [0,4194304] 0 2024-05-09T06:38:11.656 INFO:tasks.workunit.client.1.smithi193.stdout:0/183: rmdir d18/d29 39 2024-05-09T06:38:11.658 INFO:tasks.workunit.client.0.smithi063.stdout:2/518: truncate dc/f21 7926191 0 2024-05-09T06:38:11.660 INFO:tasks.workunit.client.0.smithi063.stdout:7/448: dwrite d0/d15/d4a/d50/f23 [0,4194304] 0 2024-05-09T06:38:11.661 INFO:tasks.workunit.client.1.smithi193.stdout:7/165: write d6/d8/f35 [91378,52386] 0 2024-05-09T06:38:11.662 INFO:tasks.workunit.client.0.smithi063.stdout:6/489: dread d2/d4/d7c/d42/f61 [0,4194304] 0 2024-05-09T06:38:11.663 INFO:tasks.workunit.client.0.smithi063.stdout:5/441: dwrite d8/d19/d30/f43 [0,4194304] 0 2024-05-09T06:38:11.663 INFO:tasks.workunit.client.1.smithi193.stdout:8/139: unlink c4 0 2024-05-09T06:38:11.664 INFO:tasks.workunit.client.1.smithi193.stdout:5/202: dwrite d0/d13/f40 [0,4194304] 0 2024-05-09T06:38:11.668 INFO:tasks.workunit.client.0.smithi063.stdout:9/479: chown d0/d4/c46 874729 1 2024-05-09T06:38:11.672 INFO:tasks.workunit.client.1.smithi193.stdout:1/118: dread d4/fc [4194304,4194304] 0 2024-05-09T06:38:11.672 INFO:tasks.workunit.client.1.smithi193.stdout:9/150: unlink dd/d1e/c2d 0 2024-05-09T06:38:11.674 INFO:tasks.workunit.client.1.smithi193.stdout:1/119: write d4/d16/f10 [259880,14478] 0 2024-05-09T06:38:11.675 INFO:tasks.workunit.client.1.smithi193.stdout:9/151: write dd/f11 [735374,84534] 0 2024-05-09T06:38:11.676 INFO:tasks.workunit.client.0.smithi063.stdout:8/541: mkdir d9/da/d18/d56/d58/d79/db8 0 2024-05-09T06:38:11.680 INFO:tasks.workunit.client.1.smithi193.stdout:0/184: rmdir d18/d1d 39 2024-05-09T06:38:11.682 INFO:tasks.workunit.client.0.smithi063.stdout:1/425: link d9/d2a/d39/l41 d9/d46/l93 0 2024-05-09T06:38:11.682 INFO:tasks.workunit.client.0.smithi063.stdout:3/499: write d7/da/d2b/d66/d74/d7e/f8b [614435,129662] 0 2024-05-09T06:38:11.682 INFO:tasks.workunit.client.1.smithi193.stdout:3/75: symlink d1/db/d1b/l1c 0 2024-05-09T06:38:11.686 INFO:tasks.workunit.client.0.smithi063.stdout:0/517: rename dd/d1a/d2c/d51/d79 to dd/d1a/d2c/d51/d72/d9e/d23/d89/da8 0 2024-05-09T06:38:11.687 INFO:tasks.workunit.client.1.smithi193.stdout:7/166: readlink d6/d8/d1a/l25 0 2024-05-09T06:38:11.689 INFO:tasks.workunit.client.1.smithi193.stdout:8/140: symlink d6/d1c/d18/l2a 0 2024-05-09T06:38:11.690 INFO:tasks.workunit.client.1.smithi193.stdout:8/141: write d6/d1c/f12 [424076,6439] 0 2024-05-09T06:38:11.691 INFO:tasks.workunit.client.0.smithi063.stdout:2/519: mknod dc/dd/d45/d5a/c93 0 2024-05-09T06:38:11.692 INFO:tasks.workunit.client.0.smithi063.stdout:5/442: symlink d8/d19/d30/l9b 0 2024-05-09T06:38:11.693 INFO:tasks.workunit.client.1.smithi193.stdout:1/120: mknod d4/d14/c1d 0 2024-05-09T06:38:11.695 INFO:tasks.workunit.client.0.smithi063.stdout:9/480: creat d0/d4/d26/d47/fa8 x:0 0 0 2024-05-09T06:38:11.695 INFO:tasks.workunit.client.1.smithi193.stdout:1/121: dread d4/f9 [0,4194304] 0 2024-05-09T06:38:11.696 INFO:tasks.workunit.client.0.smithi063.stdout:9/481: truncate d0/f9f 399042 0 2024-05-09T06:38:11.696 INFO:tasks.workunit.client.1.smithi193.stdout:0/185: symlink d18/d27/l3a 0 2024-05-09T06:38:11.697 INFO:tasks.workunit.client.0.smithi063.stdout:8/542: symlink d9/da/d18/d56/lb9 0 2024-05-09T06:38:11.698 INFO:tasks.workunit.client.0.smithi063.stdout:5/443: dread d8/d19/d30/f61 [0,4194304] 0 2024-05-09T06:38:11.699 INFO:tasks.workunit.client.0.smithi063.stdout:8/543: dread - d9/da/d18/d56/d58/fb4 zero size 2024-05-09T06:38:11.699 INFO:tasks.workunit.client.1.smithi193.stdout:6/135: sync 2024-05-09T06:38:11.699 INFO:tasks.workunit.client.1.smithi193.stdout:2/140: sync 2024-05-09T06:38:11.699 INFO:tasks.workunit.client.1.smithi193.stdout:4/135: sync 2024-05-09T06:38:11.700 INFO:tasks.workunit.client.0.smithi063.stdout:5/444: chown d8/d7e 5757316 1 2024-05-09T06:38:11.702 INFO:tasks.workunit.client.0.smithi063.stdout:3/500: symlink d7/da/d21/d30/d44/d48/l9d 0 2024-05-09T06:38:11.704 INFO:tasks.workunit.client.1.smithi193.stdout:5/203: rename d0/d13/l34 to d0/d13/d2c/d37/l42 0 2024-05-09T06:38:11.708 INFO:tasks.workunit.client.0.smithi063.stdout:7/449: mkdir d0/d15/d82/d97/d9b 0 2024-05-09T06:38:11.709 INFO:tasks.workunit.client.0.smithi063.stdout:9/482: dread d0/d4/d26/d39/f55 [0,4194304] 0 2024-05-09T06:38:11.709 INFO:tasks.workunit.client.1.smithi193.stdout:9/152: mkdir dd/df/d31/d33 0 2024-05-09T06:38:11.710 INFO:tasks.workunit.client.0.smithi063.stdout:9/483: fsync d0/d4/d7d/f87 0 2024-05-09T06:38:11.710 INFO:tasks.workunit.client.1.smithi193.stdout:8/142: dwrite d6/f21 [0,4194304] 0 2024-05-09T06:38:11.711 INFO:tasks.workunit.client.0.smithi063.stdout:2/520: symlink dc/d36/d43/l94 0 2024-05-09T06:38:11.714 INFO:tasks.workunit.client.0.smithi063.stdout:8/544: dread d9/da/d18/f1d [0,4194304] 0 2024-05-09T06:38:11.715 INFO:tasks.workunit.client.0.smithi063.stdout:8/545: dread - d9/da/d45/fa4 zero size 2024-05-09T06:38:11.718 INFO:tasks.workunit.client.0.smithi063.stdout:9/484: dwrite d0/f5b [0,4194304] 0 2024-05-09T06:38:11.719 INFO:tasks.workunit.client.1.smithi193.stdout:7/167: symlink d6/d8/d1a/l39 0 2024-05-09T06:38:11.721 INFO:tasks.workunit.client.0.smithi063.stdout:1/426: mkdir d9/d76/d94 0 2024-05-09T06:38:11.721 INFO:tasks.workunit.client.1.smithi193.stdout:4/136: chown lc 1126 1 2024-05-09T06:38:11.722 INFO:tasks.workunit.client.0.smithi063.stdout:5/445: rmdir d8/d65 39 2024-05-09T06:38:11.723 INFO:tasks.workunit.client.0.smithi063.stdout:5/446: stat d8/d17/d18/f22 0 2024-05-09T06:38:11.725 INFO:tasks.workunit.client.0.smithi063.stdout:5/447: truncate d8/d17/f3d 4715669 0 2024-05-09T06:38:11.725 INFO:tasks.workunit.client.1.smithi193.stdout:4/137: dread fa [0,4194304] 0 2024-05-09T06:38:11.726 INFO:tasks.workunit.client.1.smithi193.stdout:4/138: readlink d1a/d1f/l2b 0 2024-05-09T06:38:11.727 INFO:tasks.workunit.client.1.smithi193.stdout:9/153: write dd/f13 [696063,96297] 0 2024-05-09T06:38:11.735 INFO:tasks.workunit.client.1.smithi193.stdout:5/204: dwrite d0/d1/da/f23 [0,4194304] 0 2024-05-09T06:38:11.737 INFO:tasks.workunit.client.1.smithi193.stdout:9/154: dwrite dd/f13 [0,4194304] 0 2024-05-09T06:38:11.738 INFO:tasks.workunit.client.0.smithi063.stdout:7/450: rename d0/d15/d49/d4f/d66/f6c to d0/d15/d49/d4f/d66/d67/d6b/f9c 0 2024-05-09T06:38:11.743 INFO:tasks.workunit.client.1.smithi193.stdout:7/168: truncate d6/f2e 707338 0 2024-05-09T06:38:11.744 INFO:tasks.workunit.client.0.smithi063.stdout:2/521: symlink dc/d36/d43/d46/d63/d79/d7d/d8d/l95 0 2024-05-09T06:38:11.748 INFO:tasks.workunit.client.0.smithi063.stdout:6/490: link d2/d4/cb d2/df/d91/ca0 0 2024-05-09T06:38:11.752 INFO:tasks.workunit.client.1.smithi193.stdout:4/139: mknod d1a/c2d 0 2024-05-09T06:38:11.754 INFO:tasks.workunit.client.1.smithi193.stdout:8/143: link d6/d1c/d18/f26 d6/d1c/d1b/f2b 0 2024-05-09T06:38:11.758 INFO:tasks.workunit.client.0.smithi063.stdout:9/485: write d0/d8/d2d/f7b [163250,47715] 0 2024-05-09T06:38:11.760 INFO:tasks.workunit.client.0.smithi063.stdout:1/427: mknod d9/d10/d2b/d3f/c95 0 2024-05-09T06:38:11.760 INFO:tasks.workunit.client.1.smithi193.stdout:0/186: dwrite d18/d1d/f1f [0,4194304] 0 2024-05-09T06:38:11.761 INFO:tasks.workunit.client.0.smithi063.stdout:1/428: read d9/d2a/d35/d1e/f3a [3750925,13514] 0 2024-05-09T06:38:11.762 INFO:tasks.workunit.client.0.smithi063.stdout:1/429: chown d9/d10/d2b/l34 7 1 2024-05-09T06:38:11.764 INFO:tasks.workunit.client.1.smithi193.stdout:1/122: link d4/d16/de/f12 d4/d16/d17/d1a/f1e 0 2024-05-09T06:38:11.765 INFO:tasks.workunit.client.1.smithi193.stdout:2/141: truncate db/f1e 3646911 0 2024-05-09T06:38:11.770 INFO:tasks.workunit.client.0.smithi063.stdout:0/518: link dd/d1a/c49 dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/ca9 0 2024-05-09T06:38:11.776 INFO:tasks.workunit.client.0.smithi063.stdout:2/522: mknod dc/d36/d43/d46/c96 0 2024-05-09T06:38:11.780 INFO:tasks.workunit.client.0.smithi063.stdout:2/523: read dc/d32/f62 [616929,110110] 0 2024-05-09T06:38:11.782 INFO:tasks.workunit.client.1.smithi193.stdout:0/187: dread - d18/d29/f38 zero size 2024-05-09T06:38:11.784 INFO:tasks.workunit.client.0.smithi063.stdout:9/486: creat d0/d4/d57/d60/d76/fa9 x:0 0 0 2024-05-09T06:38:11.786 INFO:tasks.workunit.client.0.smithi063.stdout:9/487: chown d0/d4/d26/d47 0 1 2024-05-09T06:38:11.786 INFO:tasks.workunit.client.1.smithi193.stdout:2/142: rename db/f1b to db/d25/f33 0 2024-05-09T06:38:11.786 INFO:tasks.workunit.client.0.smithi063.stdout:5/448: dread d8/d17/f24 [0,4194304] 0 2024-05-09T06:38:11.787 INFO:tasks.workunit.client.1.smithi193.stdout:5/205: symlink d0/d1/da/l43 0 2024-05-09T06:38:11.791 INFO:tasks.workunit.client.0.smithi063.stdout:2/524: dread dc/d32/d33/f61 [0,4194304] 0 2024-05-09T06:38:11.791 INFO:tasks.workunit.client.0.smithi063.stdout:0/519: creat dd/d60/d44/d90/faa x:0 0 0 2024-05-09T06:38:11.791 INFO:tasks.workunit.client.1.smithi193.stdout:3/76: dread d1/f4 [0,4194304] 0 2024-05-09T06:38:11.792 INFO:tasks.workunit.client.0.smithi063.stdout:3/501: write d7/da/d21/f4d [1577886,59832] 0 2024-05-09T06:38:11.792 INFO:tasks.workunit.client.1.smithi193.stdout:9/155: write dd/f18 [4939144,11510] 0 2024-05-09T06:38:11.794 INFO:tasks.workunit.client.1.smithi193.stdout:8/144: creat d6/d25/f2c x:0 0 0 2024-05-09T06:38:11.796 INFO:tasks.workunit.client.1.smithi193.stdout:9/156: truncate dd/d1e/d1f/f2f 594984 0 2024-05-09T06:38:11.797 INFO:tasks.workunit.client.1.smithi193.stdout:1/123: creat d4/d16/d17/d1a/f1f x:0 0 0 2024-05-09T06:38:11.797 INFO:tasks.workunit.client.1.smithi193.stdout:8/145: fsync d6/d1c/f12 0 2024-05-09T06:38:11.797 INFO:tasks.workunit.client.1.smithi193.stdout:7/169: dread d6/d8/f9 [0,4194304] 0 2024-05-09T06:38:11.798 INFO:tasks.workunit.client.0.smithi063.stdout:5/449: dread d8/f27 [0,4194304] 0 2024-05-09T06:38:11.799 INFO:tasks.workunit.client.1.smithi193.stdout:9/157: chown dd/d1e/d1f/f2f 1 1 2024-05-09T06:38:11.800 INFO:tasks.workunit.client.1.smithi193.stdout:8/146: stat d6/d1c/d1b/f2b 0 2024-05-09T06:38:11.801 INFO:tasks.workunit.client.0.smithi063.stdout:2/525: dread dc/d32/d33/f4c [0,4194304] 0 2024-05-09T06:38:11.801 INFO:tasks.workunit.client.0.smithi063.stdout:8/546: truncate d9/da/d53/faa 2734107 0 2024-05-09T06:38:11.802 INFO:tasks.workunit.client.1.smithi193.stdout:9/158: chown dd/df/d15/d16 7 1 2024-05-09T06:38:11.802 INFO:tasks.workunit.client.0.smithi063.stdout:0/520: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d89/f8a [0,4194304] 0 2024-05-09T06:38:11.803 INFO:tasks.workunit.client.1.smithi193.stdout:3/77: dwrite d1/db/f18 [0,4194304] 0 2024-05-09T06:38:11.803 INFO:tasks.workunit.client.1.smithi193.stdout:9/159: stat dd/df/d31/d33 0 2024-05-09T06:38:11.803 INFO:tasks.workunit.client.0.smithi063.stdout:0/521: dread - dd/d60/fa0 zero size 2024-05-09T06:38:11.805 INFO:tasks.workunit.client.0.smithi063.stdout:2/526: truncate dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f7a 7848 0 2024-05-09T06:38:11.807 INFO:tasks.workunit.client.1.smithi193.stdout:4/140: dwrite fa [0,4194304] 0 2024-05-09T06:38:11.807 INFO:tasks.workunit.client.0.smithi063.stdout:1/430: link d9/d46/f6e d9/d76/f96 0 2024-05-09T06:38:11.816 INFO:tasks.workunit.client.0.smithi063.stdout:9/488: creat d0/d4/d26/d39/d48/d80/d84/faa x:0 0 0 2024-05-09T06:38:11.817 INFO:tasks.workunit.client.0.smithi063.stdout:9/489: stat d0/d4/d7d/f82 0 2024-05-09T06:38:11.817 INFO:tasks.workunit.client.1.smithi193.stdout:7/170: fdatasync d6/d8/f9 0 2024-05-09T06:38:11.821 INFO:tasks.workunit.client.1.smithi193.stdout:8/147: rmdir d6/d1c 39 2024-05-09T06:38:11.822 INFO:tasks.workunit.client.1.smithi193.stdout:2/143: fdatasync db/df/f20 0 2024-05-09T06:38:11.823 INFO:tasks.workunit.client.0.smithi063.stdout:3/502: fsync f3 0 2024-05-09T06:38:11.824 INFO:tasks.workunit.client.1.smithi193.stdout:2/144: write db/df/d12/f17 [1764342,111381] 0 2024-05-09T06:38:11.824 INFO:tasks.workunit.client.0.smithi063.stdout:4/475: getdents d7/df 0 2024-05-09T06:38:11.825 INFO:tasks.workunit.client.0.smithi063.stdout:5/450: mkdir d8/d17/d18/d9c 0 2024-05-09T06:38:11.827 INFO:tasks.workunit.client.0.smithi063.stdout:8/547: mknod d9/da/d53/db2/cba 0 2024-05-09T06:38:11.829 INFO:tasks.workunit.client.0.smithi063.stdout:2/527: creat dc/d6e/d7f/f97 x:0 0 0 2024-05-09T06:38:11.830 INFO:tasks.workunit.client.1.smithi193.stdout:7/171: creat d6/dd/f3a x:0 0 0 2024-05-09T06:38:11.831 INFO:tasks.workunit.client.0.smithi063.stdout:6/491: truncate d2/d4/f92 3802939 0 2024-05-09T06:38:11.832 INFO:tasks.workunit.client.0.smithi063.stdout:3/503: dwrite d7/da/d2b/d66/d74/f92 [0,4194304] 0 2024-05-09T06:38:11.832 INFO:tasks.workunit.client.0.smithi063.stdout:1/431: chown d9/d2a/d39/d5d 32636608 1 2024-05-09T06:38:11.832 INFO:tasks.workunit.client.0.smithi063.stdout:2/528: readlink dc/d36/d43/d46/d63/d79/d7d/l91 0 2024-05-09T06:38:11.833 INFO:tasks.workunit.client.1.smithi193.stdout:0/188: rename l2 to d18/l3b 0 2024-05-09T06:38:11.836 INFO:tasks.workunit.client.0.smithi063.stdout:7/451: getdents d0/d15/d4a/d68 0 2024-05-09T06:38:11.838 INFO:tasks.workunit.client.0.smithi063.stdout:7/452: chown d0/d15/d49/d54/f60 1109 1 2024-05-09T06:38:11.838 INFO:tasks.workunit.client.1.smithi193.stdout:8/148: truncate d6/d1c/d1b/f2b 480371 0 2024-05-09T06:38:11.838 INFO:tasks.workunit.client.0.smithi063.stdout:9/490: symlink d0/d4/d26/d39/lab 0 2024-05-09T06:38:11.842 INFO:tasks.workunit.client.1.smithi193.stdout:5/206: rename d0/d1/c2 to d0/d1/c44 0 2024-05-09T06:38:11.844 INFO:tasks.workunit.client.0.smithi063.stdout:4/476: creat d7/df/f95 x:0 0 0 2024-05-09T06:38:11.850 INFO:tasks.workunit.client.1.smithi193.stdout:4/141: creat d1a/d24/f2e x:0 0 0 2024-05-09T06:38:11.852 INFO:tasks.workunit.client.1.smithi193.stdout:1/124: dwrite d4/d16/d17/d1a/f1e [4194304,4194304] 0 2024-05-09T06:38:11.854 INFO:tasks.workunit.client.0.smithi063.stdout:3/504: creat d7/da/d21/d30/f9e x:0 0 0 2024-05-09T06:38:11.855 INFO:tasks.workunit.client.1.smithi193.stdout:9/160: rename l4 to dd/df/l34 0 2024-05-09T06:38:11.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.876+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 --> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23f4073f50 con 0x7f23dc06d6f0 2024-05-09T06:38:11.857 INFO:tasks.workunit.client.1.smithi193.stdout:0/189: creat d18/d29/f3c x:0 0 0 2024-05-09T06:38:11.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.879+0000 7f23dbfff700 1 -- 172.21.15.63:0/1818787715 <== mgr.24415 v2:172.21.15.193:6828/1432929717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f23f4073f50 con 0x7f23dc06d6f0 2024-05-09T06:38:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f23dc06d6f0 msgr2=0x7f23dc06fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f23dc06d6f0 0x7f23dc06fbb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f23e400c530 tx=0x7f23e400b040 comp rx=0 tx=0).stop 2024-05-09T06:38:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4101e20 msgr2=0x7f23f40732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:11.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4101e20 0x7f23f40732e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23e80059d0 tx=0x7f23e800bd70 comp rx=0 tx=0).stop 2024-05-09T06:38:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 shutdown_connections 2024-05-09T06:38:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f23dc06d6f0 0x7f23dc06fbb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23f4100c20 0x7f23f4074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 --2- 172.21.15.63:0/1818787715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4101e20 0x7f23f40732e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 >> 172.21.15.63:0/1818787715 conn(0x7f23f40fc200 msgr2=0x7f23f4105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 shutdown_connections 2024-05-09T06:38:11.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:11.884+0000 7f23f94e0700 1 -- 172.21.15.63:0/1818787715 wait complete. 2024-05-09T06:38:11.866 INFO:tasks.workunit.client.0.smithi063.stdout:2/529: dwrite dc/d36/d43/d46/d63/d79/d7d/f84 [4194304,4194304] 0 2024-05-09T06:38:11.866 INFO:tasks.workunit.client.0.smithi063.stdout:9/491: symlink d0/d43/lac 0 2024-05-09T06:38:11.867 INFO:tasks.workunit.client.1.smithi193.stdout:4/142: symlink d1a/d1f/l2f 0 2024-05-09T06:38:11.867 INFO:tasks.workunit.client.0.smithi063.stdout:9/492: chown d0/f98 550867 1 2024-05-09T06:38:11.868 INFO:tasks.workunit.client.0.smithi063.stdout:2/530: read dc/dd/d2d/d60/f67 [3197196,128315] 0 2024-05-09T06:38:11.869 INFO:tasks.workunit.client.0.smithi063.stdout:4/477: unlink d7/df/d1f/l2a 0 2024-05-09T06:38:11.870 INFO:tasks.workunit.client.0.smithi063.stdout:1/432: dread d9/d2a/d35/d1e/f3a [0,4194304] 0 2024-05-09T06:38:11.872 INFO:tasks.workunit.client.0.smithi063.stdout:5/451: symlink d8/d7e/d96/l9d 0 2024-05-09T06:38:11.872 INFO:tasks.workunit.client.0.smithi063.stdout:1/433: fdatasync d9/f4e 0 2024-05-09T06:38:11.873 INFO:tasks.workunit.client.1.smithi193.stdout:1/125: rmdir d4/d14/d15 39 2024-05-09T06:38:11.874 INFO:tasks.workunit.client.1.smithi193.stdout:1/126: stat d4/d16/fb 0 2024-05-09T06:38:11.875 INFO:tasks.workunit.client.1.smithi193.stdout:9/161: dread dd/f11 [0,4194304] 0 2024-05-09T06:38:11.876 INFO:tasks.workunit.client.0.smithi063.stdout:1/434: dread d9/d10/f1c [0,4194304] 0 2024-05-09T06:38:11.877 INFO:tasks.workunit.client.1.smithi193.stdout:3/78: truncate d1/f11 435021 0 2024-05-09T06:38:11.879 INFO:tasks.workunit.client.0.smithi063.stdout:3/505: creat d7/d70/d98/f9f x:0 0 0 2024-05-09T06:38:11.880 INFO:tasks.workunit.client.0.smithi063.stdout:3/506: write d7/f11 [1287922,122265] 0 2024-05-09T06:38:11.880 INFO:tasks.workunit.client.1.smithi193.stdout:0/190: unlink d18/d27/f39 0 2024-05-09T06:38:11.882 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:38:11.882 INFO:tasks.workunit.client.0.smithi063.stdout:7/453: mkdir d0/d15/d82/d97/d9b/d9d 0 2024-05-09T06:38:11.882 INFO:tasks.workunit.client.0.smithi063.stdout:6/492: write d2/d12/f76 [1177840,24929] 0 2024-05-09T06:38:11.883 INFO:tasks.workunit.client.1.smithi193.stdout:8/149: symlink d6/d1c/d27/l2d 0 2024-05-09T06:38:11.883 INFO:tasks.workunit.client.0.smithi063.stdout:7/454: dread - d0/d15/d49/f5c zero size 2024-05-09T06:38:11.887 INFO:tasks.workunit.client.1.smithi193.stdout:8/150: dread d6/d1c/d18/f1d [0,4194304] 0 2024-05-09T06:38:11.887 INFO:tasks.workunit.client.1.smithi193.stdout:0/191: dwrite d18/d1d/f1f [0,4194304] 0 2024-05-09T06:38:11.888 INFO:tasks.workunit.client.0.smithi063.stdout:9/493: creat d0/d4/d26/d39/d48/d80/fad x:0 0 0 2024-05-09T06:38:11.888 INFO:tasks.workunit.client.1.smithi193.stdout:8/151: write d6/d1c/d18/f1d [568855,44212] 0 2024-05-09T06:38:11.889 INFO:tasks.workunit.client.1.smithi193.stdout:4/143: mkdir d1a/d24/d30 0 2024-05-09T06:38:11.889 INFO:tasks.workunit.client.1.smithi193.stdout:2/145: write db/df/f10 [2390750,76305] 0 2024-05-09T06:38:11.891 INFO:tasks.workunit.client.0.smithi063.stdout:4/478: creat d7/df/d1a/f96 x:0 0 0 2024-05-09T06:38:11.893 INFO:tasks.workunit.client.1.smithi193.stdout:1/127: symlink d4/d16/d17/d1a/l20 0 2024-05-09T06:38:11.894 INFO:tasks.workunit.client.0.smithi063.stdout:6/493: dread d2/d1f/d29/f63 [0,4194304] 0 2024-05-09T06:38:11.895 INFO:tasks.workunit.client.0.smithi063.stdout:6/494: fsync d2/d12/f34 0 2024-05-09T06:38:11.896 INFO:tasks.workunit.client.0.smithi063.stdout:6/495: read - d2/d1f/d6c/f7d zero size 2024-05-09T06:38:11.897 INFO:tasks.workunit.client.1.smithi193.stdout:7/172: rmdir d6/d8/d1a/d1f 0 2024-05-09T06:38:11.898 INFO:tasks.workunit.client.1.smithi193.stdout:4/144: dwrite d1a/d1f/f2a [0,4194304] 0 2024-05-09T06:38:11.898 INFO:tasks.workunit.client.1.smithi193.stdout:7/173: read d6/d17/f23 [2250602,6409] 0 2024-05-09T06:38:11.900 INFO:tasks.workunit.client.1.smithi193.stdout:9/162: creat dd/df/d15/d16/d26/f35 x:0 0 0 2024-05-09T06:38:11.902 INFO:tasks.workunit.client.0.smithi063.stdout:5/452: unlink d8/f14 0 2024-05-09T06:38:11.902 INFO:tasks.workunit.client.0.smithi063.stdout:5/453: read - d8/d17/d18/f8a zero size 2024-05-09T06:38:11.903 INFO:tasks.workunit.client.0.smithi063.stdout:5/454: chown c3 0 1 2024-05-09T06:38:11.905 INFO:tasks.workunit.client.0.smithi063.stdout:1/435: rename d9/l13 to d9/d2a/d39/d5d/l97 0 2024-05-09T06:38:11.906 INFO:tasks.workunit.client.1.smithi193.stdout:0/192: fdatasync d18/d1d/f1f 0 2024-05-09T06:38:11.912 INFO:tasks.workunit.client.0.smithi063.stdout:7/455: creat d0/d15/d4a/d68/f9e x:0 0 0 2024-05-09T06:38:11.913 INFO:tasks.workunit.client.0.smithi063.stdout:3/507: dwrite d7/d22/f47 [8388608,4194304] 0 2024-05-09T06:38:11.913 INFO:tasks.workunit.client.0.smithi063.stdout:7/456: fdatasync d0/d15/d49/d54/f60 0 2024-05-09T06:38:11.914 INFO:tasks.workunit.client.0.smithi063.stdout:7/457: stat d0/d15/f48 0 2024-05-09T06:38:11.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/458: chown d0/c19 0 1 2024-05-09T06:38:11.917 INFO:tasks.workunit.client.0.smithi063.stdout:7/459: truncate d0/d15/d49/f5e 587555 0 2024-05-09T06:38:11.917 INFO:tasks.workunit.client.0.smithi063.stdout:4/479: creat d7/df/d35/d5a/f97 x:0 0 0 2024-05-09T06:38:11.918 INFO:tasks.workunit.client.1.smithi193.stdout:7/174: readlink d6/l1d 0 2024-05-09T06:38:11.920 INFO:tasks.workunit.client.0.smithi063.stdout:2/531: mkdir dc/dd/d45/d5a/d6c/d8f/d98 0 2024-05-09T06:38:11.922 INFO:tasks.workunit.client.1.smithi193.stdout:4/145: unlink d1a/d1f/f25 0 2024-05-09T06:38:11.924 INFO:tasks.workunit.client.0.smithi063.stdout:2/532: dread dc/d32/d33/d4e/f88 [0,4194304] 0 2024-05-09T06:38:11.925 INFO:tasks.workunit.client.1.smithi193.stdout:9/163: mknod dd/d1e/d1f/c36 0 2024-05-09T06:38:11.926 INFO:tasks.workunit.client.1.smithi193.stdout:4/146: dread f12 [0,4194304] 0 2024-05-09T06:38:11.927 INFO:tasks.workunit.client.0.smithi063.stdout:1/436: dread d9/d2a/d35/d1e/f27 [0,4194304] 0 2024-05-09T06:38:11.929 INFO:tasks.workunit.client.1.smithi193.stdout:0/193: rmdir d18/d29 39 2024-05-09T06:38:11.933 INFO:tasks.workunit.client.0.smithi063.stdout:5/455: unlink d8/f4b 0 2024-05-09T06:38:11.933 INFO:tasks.workunit.client.0.smithi063.stdout:5/456: stat d8/f27 0 2024-05-09T06:38:11.934 INFO:tasks.workunit.client.1.smithi193.stdout:8/152: mknod d6/d1c/c2e 0 2024-05-09T06:38:11.935 INFO:tasks.workunit.client.1.smithi193.stdout:8/153: chown d6/l16 15 1 2024-05-09T06:38:11.937 INFO:tasks.workunit.client.1.smithi193.stdout:2/146: symlink db/l34 0 2024-05-09T06:38:11.937 INFO:tasks.workunit.client.1.smithi193.stdout:2/147: chown db/df 6869 1 2024-05-09T06:38:11.937 INFO:tasks.workunit.client.1.smithi193.stdout:2/148: stat db 0 2024-05-09T06:38:11.942 INFO:tasks.workunit.client.1.smithi193.stdout:2/149: dwrite f9 [0,4194304] 0 2024-05-09T06:38:11.945 INFO:tasks.workunit.client.0.smithi063.stdout:7/460: rename d0/d15/d82 to d0/d15/d49/d54/d9f 0 2024-05-09T06:38:11.949 INFO:tasks.workunit.client.1.smithi193.stdout:0/194: creat d18/d1d/f3d x:0 0 0 2024-05-09T06:38:11.951 INFO:tasks.workunit.client.1.smithi193.stdout:8/154: truncate d6/d1c/d1b/f1f 3381218 0 2024-05-09T06:38:11.951 INFO:tasks.workunit.client.1.smithi193.stdout:8/155: readlink d6/l16 0 2024-05-09T06:38:11.955 INFO:tasks.workunit.client.0.smithi063.stdout:1/437: creat d9/d2a/d35/d24/d68/f98 x:0 0 0 2024-05-09T06:38:11.960 INFO:tasks.workunit.client.0.smithi063.stdout:5/457: read - d8/d65/f83 zero size 2024-05-09T06:38:11.962 INFO:tasks.workunit.client.0.smithi063.stdout:9/494: creat d0/d4/d26/fae x:0 0 0 2024-05-09T06:38:11.962 INFO:tasks.workunit.client.1.smithi193.stdout:4/147: truncate f6 1080672 0 2024-05-09T06:38:11.963 INFO:tasks.workunit.client.0.smithi063.stdout:9/495: write d0/d4/d57/fa4 [1039321,79354] 0 2024-05-09T06:38:11.964 INFO:tasks.workunit.client.0.smithi063.stdout:9/496: fdatasync d0/d3b/f78 0 2024-05-09T06:38:11.965 INFO:tasks.workunit.client.1.smithi193.stdout:0/195: mknod d18/d27/c3e 0 2024-05-09T06:38:11.965 INFO:tasks.workunit.client.1.smithi193.stdout:0/196: read - d18/f26 zero size 2024-05-09T06:38:11.966 INFO:tasks.workunit.client.0.smithi063.stdout:9/497: truncate d0/d4/d26/d39/d91/fa7 680451 0 2024-05-09T06:38:11.966 INFO:tasks.workunit.client.1.smithi193.stdout:8/156: mkdir d6/d1c/d18/d2f 0 2024-05-09T06:38:11.968 INFO:tasks.workunit.client.1.smithi193.stdout:8/157: dread d6/fd [0,4194304] 0 2024-05-09T06:38:11.969 INFO:tasks.workunit.client.0.smithi063.stdout:7/461: rename d0/c42 to d0/d15/d49/d4f/d66/d67/d6b/ca0 0 2024-05-09T06:38:11.970 INFO:tasks.workunit.client.0.smithi063.stdout:6/496: write f1 [3594338,14828] 0 2024-05-09T06:38:11.971 INFO:tasks.workunit.client.1.smithi193.stdout:7/175: creat d6/d8/d1a/d36/f3b x:0 0 0 2024-05-09T06:38:11.971 INFO:tasks.workunit.client.0.smithi063.stdout:7/462: write d0/d15/d4a/d50/f61 [354293,97789] 0 2024-05-09T06:38:11.972 INFO:tasks.workunit.client.0.smithi063.stdout:7/463: fsync d0/d15/f58 0 2024-05-09T06:38:11.974 INFO:tasks.workunit.client.1.smithi193.stdout:9/164: truncate dd/f13 811742 0 2024-05-09T06:38:11.977 INFO:tasks.workunit.client.0.smithi063.stdout:8/548: dread d9/da/d53/faa [0,4194304] 0 2024-05-09T06:38:11.978 INFO:tasks.workunit.client.0.smithi063.stdout:8/549: dread - d9/da/f9a zero size 2024-05-09T06:38:11.978 INFO:tasks.workunit.client.1.smithi193.stdout:4/148: creat d1a/d22/f31 x:0 0 0 2024-05-09T06:38:11.979 INFO:tasks.workunit.client.0.smithi063.stdout:3/508: truncate d7/d22/f47 3855446 0 2024-05-09T06:38:11.979 INFO:tasks.workunit.client.0.smithi063.stdout:1/438: rmdir d9/d10/d2b/d3f 39 2024-05-09T06:38:11.980 INFO:tasks.workunit.client.1.smithi193.stdout:4/149: dread - d1a/d24/f2e zero size 2024-05-09T06:38:11.980 INFO:tasks.workunit.client.1.smithi193.stdout:9/165: dwrite dd/df/f10 [0,4194304] 0 2024-05-09T06:38:11.982 INFO:tasks.workunit.client.0.smithi063.stdout:8/550: dread d9/da/d18/d46/f64 [0,4194304] 0 2024-05-09T06:38:11.989 INFO:tasks.workunit.client.0.smithi063.stdout:9/498: rmdir d0/d3b/d63 39 2024-05-09T06:38:11.992 INFO:tasks.workunit.client.0.smithi063.stdout:7/464: symlink d0/d15/d49/d4f/d66/d67/d86/la1 0 2024-05-09T06:38:11.995 INFO:tasks.workunit.client.0.smithi063.stdout:2/533: creat dc/f99 x:0 0 0 2024-05-09T06:38:11.996 INFO:tasks.workunit.client.0.smithi063.stdout:2/534: write dc/dd/d45/f8a [70521,97441] 0 2024-05-09T06:38:11.998 INFO:tasks.workunit.client.0.smithi063.stdout:3/509: creat d7/d4c/d25/fa0 x:0 0 0 2024-05-09T06:38:11.999 INFO:tasks.workunit.client.0.smithi063.stdout:1/439: fsync d9/d76/f96 0 2024-05-09T06:38:12.001 INFO:tasks.workunit.client.0.smithi063.stdout:5/458: stat d8/d17/d18/f58 0 2024-05-09T06:38:12.002 INFO:tasks.workunit.client.0.smithi063.stdout:5/459: dread - d8/f8e zero size 2024-05-09T06:38:12.003 INFO:tasks.workunit.client.0.smithi063.stdout:4/480: getdents d7/df/d35/d78 0 2024-05-09T06:38:12.004 INFO:tasks.workunit.client.0.smithi063.stdout:1/440: chown d9/d2a/d35/c1f 899 1 2024-05-09T06:38:12.005 INFO:tasks.workunit.client.0.smithi063.stdout:9/499: creat d0/d8/d2d/faf x:0 0 0 2024-05-09T06:38:12.006 INFO:tasks.workunit.client.0.smithi063.stdout:7/465: readlink d0/d15/d49/d4f/d66/d67/d86/l8c 0 2024-05-09T06:38:12.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/497: dwrite d2/df/f5d [0,4194304] 0 2024-05-09T06:38:12.009 INFO:tasks.workunit.client.0.smithi063.stdout:3/510: rmdir d7/da/d2b/d66 39 2024-05-09T06:38:12.014 INFO:tasks.workunit.client.0.smithi063.stdout:6/498: dread d2/d20/d26/f97 [0,4194304] 0 2024-05-09T06:38:12.017 INFO:tasks.workunit.client.0.smithi063.stdout:0/522: sync 2024-05-09T06:38:12.018 INFO:tasks.workunit.client.0.smithi063.stdout:1/441: write d9/d2a/d35/d1e/f27 [2736508,61161] 0 2024-05-09T06:38:12.019 INFO:tasks.workunit.client.0.smithi063.stdout:7/466: rename d0/d15/d49/d4f/d66/c8f to d0/d15/d89/ca2 0 2024-05-09T06:38:12.019 INFO:tasks.workunit.client.0.smithi063.stdout:7/467: fdatasync d0/f81 0 2024-05-09T06:38:12.020 INFO:tasks.workunit.client.0.smithi063.stdout:8/551: link d9/da/d45/la5 d9/da/d18/d56/d58/d79/lbb 0 2024-05-09T06:38:12.020 INFO:tasks.workunit.client.0.smithi063.stdout:3/511: rmdir d7/d70/d98/d2d/d36 39 2024-05-09T06:38:12.022 INFO:tasks.workunit.client.0.smithi063.stdout:6/499: creat d2/d4/fa1 x:0 0 0 2024-05-09T06:38:12.022 INFO:tasks.workunit.client.0.smithi063.stdout:9/500: mkdir d0/d8/da6/db0 0 2024-05-09T06:38:12.024 INFO:tasks.workunit.client.0.smithi063.stdout:6/500: chown d2/d12/l4b 26 1 2024-05-09T06:38:12.025 INFO:tasks.workunit.client.0.smithi063.stdout:1/442: rmdir d9/d2a/d35/d1e 39 2024-05-09T06:38:12.029 INFO:tasks.workunit.client.0.smithi063.stdout:8/552: dwrite d9/da/d18/d24/f42 [0,4194304] 0 2024-05-09T06:38:12.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.049+0000 7f28c032c700 1 -- 172.21.15.63:0/208077760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b810a640 msgr2=0x7f28b810aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.049+0000 7f28c032c700 1 --2- 172.21.15.63:0/208077760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b810a640 0x7f28b810aac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009910 tx=0x7f28a8009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:12.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.050+0000 7f28c032c700 1 -- 172.21.15.63:0/208077760 shutdown_connections 2024-05-09T06:38:12.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.050+0000 7f28c032c700 1 --2- 172.21.15.63:0/208077760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b810a640 0x7f28b810aac0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.050+0000 7f28c032c700 1 --2- 172.21.15.63:0/208077760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b80695d0 0x7f28b80699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.050+0000 7f28c032c700 1 -- 172.21.15.63:0/208077760 >> 172.21.15.63:0/208077760 conn(0x7f28b806eaf0 msgr2=0x7f28b8070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:12.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.050+0000 7f28c032c700 1 -- 172.21.15.63:0/208077760 shutdown_connections 2024-05-09T06:38:12.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.050+0000 7f28c032c700 1 -- 172.21.15.63:0/208077760 wait complete. 2024-05-09T06:38:12.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.051+0000 7f28c032c700 1 Processor -- start 2024-05-09T06:38:12.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.051+0000 7f28c032c700 1 -- start start 2024-05-09T06:38:12.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28c032c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b80695d0 0x7f28b8109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28c032c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b810a640 0x7f28b81080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28c032c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b81a2840 con 0x7f28b810a640 2024-05-09T06:38:12.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28c032c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b81a2980 con 0x7f28b80695d0 2024-05-09T06:38:12.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28be0c8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b80695d0 0x7f28b8109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28be0c8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b80695d0 0x7f28b8109a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.63:55356/0 (socket says 172.21.15.63:55356) 2024-05-09T06:38:12.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28be0c8700 1 -- 172.21.15.63:0/2332109053 learned_addr learned my addr 172.21.15.63:0/2332109053 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28bd8c7700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b810a640 0x7f28b81080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28be0c8700 1 -- 172.21.15.63:0/2332109053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b810a640 msgr2=0x7f28b81080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28be0c8700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b810a640 0x7f28b81080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28be0c8700 1 -- 172.21.15.63:0/2332109053 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a80095f0 con 0x7f28b80695d0 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.052+0000 7f28bd8c7700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b810a640 0x7f28b81080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.053+0000 7f28be0c8700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b80695d0 0x7f28b8109a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e920 tx=0x7f28b400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.053+0000 7f28af7fe700 1 -- 172.21.15.63:0/2332109053 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b4009790 con 0x7f28b80695d0 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.053+0000 7f28af7fe700 1 -- 172.21.15.63:0/2332109053 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28b400cba0 con 0x7f28b80695d0 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.053+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b81085f0 con 0x7f28b80695d0 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.053+0000 7f28af7fe700 1 -- 172.21.15.63:0/2332109053 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b4018650 con 0x7f28b80695d0 2024-05-09T06:38:12.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.053+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b8108b90 con 0x7f28b80695d0 2024-05-09T06:38:12.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.055+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c005320 con 0x7f28b80695d0 2024-05-09T06:38:12.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.055+0000 7f28af7fe700 1 -- 172.21.15.63:0/2332109053 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f28b40188c0 con 0x7f28b80695d0 2024-05-09T06:38:12.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.056+0000 7f28af7fe700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f28a406d6e0 0x7f28a406fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.056+0000 7f28af7fe700 1 -- 172.21.15.63:0/2332109053 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f28b4014070 con 0x7f28b80695d0 2024-05-09T06:38:12.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.057+0000 7f28bd8c7700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f28a406d6e0 0x7f28a406fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.058+0000 7f28bd8c7700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f28a406d6e0 0x7f28a406fba0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f28a8006010 tx=0x7f28a8005a90 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:12.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.062+0000 7f28af7fe700 1 -- 172.21.15.63:0/2332109053 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f28b4053720 con 0x7f28b80695d0 2024-05-09T06:38:12.073 INFO:tasks.workunit.client.1.smithi193.stdout:1/128: getdents d4/d14/d15 0 2024-05-09T06:38:12.073 INFO:tasks.workunit.client.1.smithi193.stdout:4/150: truncate d1a/d24/f2e 843355 0 2024-05-09T06:38:12.073 INFO:tasks.workunit.client.1.smithi193.stdout:2/150: truncate db/df/d12/f17 406045 0 2024-05-09T06:38:12.077 INFO:tasks.workunit.client.1.smithi193.stdout:9/166: truncate dd/df/d15/d16/d26/f32 257024 0 2024-05-09T06:38:12.079 INFO:tasks.workunit.client.1.smithi193.stdout:7/176: creat d6/d8/d1a/f3c x:0 0 0 2024-05-09T06:38:12.088 INFO:tasks.workunit.client.1.smithi193.stdout:8/158: dwrite d6/d1c/f14 [0,4194304] 0 2024-05-09T06:38:12.088 INFO:tasks.workunit.client.1.smithi193.stdout:9/167: write f0 [1893375,112080] 0 2024-05-09T06:38:12.089 INFO:tasks.workunit.client.1.smithi193.stdout:4/151: dwrite fa [4194304,4194304] 0 2024-05-09T06:38:12.092 INFO:tasks.workunit.client.1.smithi193.stdout:2/151: dwrite db/f21 [0,4194304] 0 2024-05-09T06:38:12.096 INFO:tasks.workunit.client.1.smithi193.stdout:1/129: dwrite d4/f13 [4194304,4194304] 0 2024-05-09T06:38:12.096 INFO:tasks.workunit.client.1.smithi193.stdout:8/159: creat d6/d1c/d1b/f30 x:0 0 0 2024-05-09T06:38:12.098 INFO:tasks.workunit.client.1.smithi193.stdout:7/177: link d6/d8/d29/d2f/f38 d6/d8/d1a/f3d 0 2024-05-09T06:38:12.098 INFO:tasks.workunit.client.1.smithi193.stdout:2/152: readlink db/l30 0 2024-05-09T06:38:12.099 INFO:tasks.workunit.client.1.smithi193.stdout:1/130: read - d4/d16/d17/d1a/f1f zero size 2024-05-09T06:38:12.100 INFO:tasks.workunit.client.1.smithi193.stdout:9/168: mkdir dd/df/d15/d16/d26/d37 0 2024-05-09T06:38:12.100 INFO:tasks.workunit.client.1.smithi193.stdout:7/178: dread - d6/d8/d1a/d36/f3b zero size 2024-05-09T06:38:12.102 INFO:tasks.workunit.client.1.smithi193.stdout:8/160: truncate d6/d25/f2c 115228 0 2024-05-09T06:38:12.105 INFO:tasks.workunit.client.1.smithi193.stdout:7/179: truncate d6/dd/f3a 337394 0 2024-05-09T06:38:12.106 INFO:tasks.workunit.client.1.smithi193.stdout:4/152: creat d1a/d24/d30/f32 x:0 0 0 2024-05-09T06:38:12.107 INFO:tasks.workunit.client.1.smithi193.stdout:8/161: write d6/d25/f2c [864440,121413] 0 2024-05-09T06:38:12.110 INFO:tasks.workunit.client.1.smithi193.stdout:2/153: creat db/f35 x:0 0 0 2024-05-09T06:38:12.115 INFO:tasks.workunit.client.1.smithi193.stdout:2/154: symlink db/df/l36 0 2024-05-09T06:38:12.117 INFO:tasks.workunit.client.1.smithi193.stdout:1/131: dwrite d4/d16/d17/d1a/f1f [0,4194304] 0 2024-05-09T06:38:12.118 INFO:tasks.workunit.client.1.smithi193.stdout:9/169: dwrite dd/df/d15/d16/d26/f32 [0,4194304] 0 2024-05-09T06:38:12.119 INFO:tasks.workunit.client.1.smithi193.stdout:2/155: chown db/df/l36 10066 1 2024-05-09T06:38:12.119 INFO:tasks.workunit.client.1.smithi193.stdout:7/180: rmdir d6/d8/d29/d2f 39 2024-05-09T06:38:12.120 INFO:tasks.workunit.client.1.smithi193.stdout:4/153: dread d1a/d1f/f2a [0,4194304] 0 2024-05-09T06:38:12.121 INFO:tasks.workunit.client.1.smithi193.stdout:2/156: chown db/df/c1d 613 1 2024-05-09T06:38:12.124 INFO:tasks.workunit.client.1.smithi193.stdout:6/136: sync 2024-05-09T06:38:12.125 INFO:tasks.workunit.client.1.smithi193.stdout:9/170: mknod dd/c38 0 2024-05-09T06:38:12.126 INFO:tasks.workunit.client.1.smithi193.stdout:4/154: creat d1a/d1f/f33 x:0 0 0 2024-05-09T06:38:12.128 INFO:tasks.workunit.client.0.smithi063.stdout:4/481: write d7/df/d57/f93 [2965473,123656] 0 2024-05-09T06:38:12.128 INFO:tasks.workunit.client.0.smithi063.stdout:5/460: write d8/d19/f3f [881200,61947] 0 2024-05-09T06:38:12.128 INFO:tasks.workunit.client.0.smithi063.stdout:6/501: unlink d2/d20/c30 0 2024-05-09T06:38:12.129 INFO:tasks.workunit.client.1.smithi193.stdout:7/181: creat d6/d8/d1a/d33/f3e x:0 0 0 2024-05-09T06:38:12.130 INFO:tasks.workunit.client.0.smithi063.stdout:0/523: getdents dd/d1a/d2c/d51 0 2024-05-09T06:38:12.133 INFO:tasks.workunit.client.0.smithi063.stdout:7/468: rmdir d0/d15/d49/d4f/d66 39 2024-05-09T06:38:12.134 INFO:tasks.workunit.client.1.smithi193.stdout:9/171: mkdir dd/df/d15/d16/d39 0 2024-05-09T06:38:12.134 INFO:tasks.workunit.client.0.smithi063.stdout:5/461: dread d8/f66 [0,4194304] 0 2024-05-09T06:38:12.134 INFO:tasks.workunit.client.0.smithi063.stdout:4/482: fsync d7/df/d77/f8e 0 2024-05-09T06:38:12.135 INFO:tasks.workunit.client.1.smithi193.stdout:9/172: write dd/d1e/f28 [959880,40688] 0 2024-05-09T06:38:12.135 INFO:tasks.workunit.client.1.smithi193.stdout:4/155: creat d1a/d24/d30/f34 x:0 0 0 2024-05-09T06:38:12.136 INFO:tasks.workunit.client.0.smithi063.stdout:8/553: mkdir d9/da/d53/dbc 0 2024-05-09T06:38:12.137 INFO:tasks.workunit.client.1.smithi193.stdout:4/156: write fa [7256802,24400] 0 2024-05-09T06:38:12.138 INFO:tasks.workunit.client.0.smithi063.stdout:6/502: write d2/d1f/d29/d3f/d5c/f9d [887836,6845] 0 2024-05-09T06:38:12.138 INFO:tasks.workunit.client.1.smithi193.stdout:8/162: dwrite d6/d1c/d18/f26 [0,4194304] 0 2024-05-09T06:38:12.140 INFO:tasks.workunit.client.0.smithi063.stdout:1/443: dwrite d9/d46/d57/f65 [0,4194304] 0 2024-05-09T06:38:12.140 INFO:tasks.workunit.client.1.smithi193.stdout:9/173: rename dd/df/d15/d16/d39 to dd/df/d15/d16/d39/d3a 22 2024-05-09T06:38:12.140 INFO:tasks.workunit.client.0.smithi063.stdout:5/462: fdatasync d8/d19/d30/d46/f6c 0 2024-05-09T06:38:12.140 INFO:tasks.workunit.client.1.smithi193.stdout:4/157: chown l15 2969 1 2024-05-09T06:38:12.141 INFO:tasks.workunit.client.1.smithi193.stdout:7/182: creat d6/dd/f3f x:0 0 0 2024-05-09T06:38:12.141 INFO:tasks.workunit.client.1.smithi193.stdout:8/163: chown d6/d1c/d1b/f2b 14766 1 2024-05-09T06:38:12.142 INFO:tasks.workunit.client.1.smithi193.stdout:4/158: write fa [4340712,42532] 0 2024-05-09T06:38:12.144 INFO:tasks.workunit.client.0.smithi063.stdout:0/524: symlink dd/d60/d44/d90/lab 0 2024-05-09T06:38:12.145 INFO:tasks.workunit.client.0.smithi063.stdout:5/463: chown d8/d17/f6a 2003 1 2024-05-09T06:38:12.146 INFO:tasks.workunit.client.1.smithi193.stdout:1/132: getdents d4/d16/d17 0 2024-05-09T06:38:12.146 INFO:tasks.workunit.client.1.smithi193.stdout:1/133: chown d4/d16 57887849 1 2024-05-09T06:38:12.149 INFO:tasks.workunit.client.0.smithi063.stdout:3/512: dwrite d7/da/f18 [0,4194304] 0 2024-05-09T06:38:12.153 INFO:tasks.workunit.client.0.smithi063.stdout:7/469: dwrite d0/f18 [0,4194304] 0 2024-05-09T06:38:12.153 INFO:tasks.workunit.client.1.smithi193.stdout:1/134: dwrite d4/ff [0,4194304] 0 2024-05-09T06:38:12.155 INFO:tasks.workunit.client.1.smithi193.stdout:7/183: unlink d6/d8/f9 0 2024-05-09T06:38:12.157 INFO:tasks.workunit.client.1.smithi193.stdout:6/137: write d1/de/ff [516628,42870] 0 2024-05-09T06:38:12.160 INFO:tasks.workunit.client.0.smithi063.stdout:5/464: rmdir d8/d19/d30/d46 39 2024-05-09T06:38:12.161 INFO:tasks.workunit.client.1.smithi193.stdout:4/159: fsync d1a/d1f/f2a 0 2024-05-09T06:38:12.163 INFO:tasks.workunit.client.1.smithi193.stdout:1/135: dread d4/d16/fb [0,4194304] 0 2024-05-09T06:38:12.164 INFO:tasks.workunit.client.0.smithi063.stdout:4/483: mkdir d7/df/d1f/d28/d31/d71/d98 0 2024-05-09T06:38:12.166 INFO:tasks.workunit.client.1.smithi193.stdout:2/157: getdents db 0 2024-05-09T06:38:12.167 INFO:tasks.workunit.client.0.smithi063.stdout:0/525: mknod dd/d1a/d2c/d51/d72/cac 0 2024-05-09T06:38:12.167 INFO:tasks.workunit.client.0.smithi063.stdout:0/526: fsync dd/d60/d44/f69 0 2024-05-09T06:38:12.169 INFO:tasks.workunit.client.0.smithi063.stdout:0/527: chown dd/d1a/d2c/d51/d72/d9e/d23/d46 4552365 1 2024-05-09T06:38:12.170 INFO:tasks.workunit.client.1.smithi193.stdout:6/138: symlink d1/de/d11/l28 0 2024-05-09T06:38:12.171 INFO:tasks.workunit.client.0.smithi063.stdout:0/528: dread - dd/d1a/d2c/d51/d76/d97/fa5 zero size 2024-05-09T06:38:12.172 INFO:tasks.workunit.client.1.smithi193.stdout:8/164: unlink d6/d1c/d1b/f1f 0 2024-05-09T06:38:12.174 INFO:tasks.workunit.client.1.smithi193.stdout:8/165: stat d6/d1c/d18/d2f 0 2024-05-09T06:38:12.174 INFO:tasks.workunit.client.1.smithi193.stdout:1/136: dwrite d4/d16/de/f12 [4194304,4194304] 0 2024-05-09T06:38:12.174 INFO:tasks.workunit.client.0.smithi063.stdout:8/554: rename d9/f14 to d9/d3b/fbd 0 2024-05-09T06:38:12.175 INFO:tasks.workunit.client.1.smithi193.stdout:7/184: dread d6/d8/d1a/f27 [0,4194304] 0 2024-05-09T06:38:12.177 INFO:tasks.workunit.client.1.smithi193.stdout:4/160: dread f12 [0,4194304] 0 2024-05-09T06:38:12.178 INFO:tasks.workunit.client.1.smithi193.stdout:4/161: fsync d1a/d1f/f26 0 2024-05-09T06:38:12.179 INFO:tasks.workunit.client.0.smithi063.stdout:4/484: creat d7/d50/f99 x:0 0 0 2024-05-09T06:38:12.180 INFO:tasks.workunit.client.0.smithi063.stdout:0/529: dwrite dd/fe [0,4194304] 0 2024-05-09T06:38:12.181 INFO:tasks.workunit.client.0.smithi063.stdout:1/444: creat d9/d10/d2b/f99 x:0 0 0 2024-05-09T06:38:12.181 INFO:tasks.workunit.client.0.smithi063.stdout:4/485: chown d7/df/d35/c89 32369 1 2024-05-09T06:38:12.182 INFO:tasks.workunit.client.1.smithi193.stdout:2/158: mkdir db/d37 0 2024-05-09T06:38:12.185 INFO:tasks.workunit.client.0.smithi063.stdout:1/445: chown d9/d2a/d35/l1b 2338 1 2024-05-09T06:38:12.187 INFO:tasks.workunit.client.1.smithi193.stdout:1/137: unlink d4/c8 0 2024-05-09T06:38:12.189 INFO:tasks.workunit.client.0.smithi063.stdout:4/486: dread d7/f42 [0,4194304] 0 2024-05-09T06:38:12.189 INFO:tasks.workunit.client.1.smithi193.stdout:1/138: dread d4/f9 [0,4194304] 0 2024-05-09T06:38:12.195 INFO:tasks.workunit.client.0.smithi063.stdout:7/470: rename d0/d15/d49/d4f/d66/d67/d6b/f9c to d0/d15/d49/d54/d9f/d97/d9b/fa3 0 2024-05-09T06:38:12.195 INFO:tasks.workunit.client.1.smithi193.stdout:7/185: rename d6/dd/f14 to d6/dd/f40 0 2024-05-09T06:38:12.197 INFO:tasks.workunit.client.1.smithi193.stdout:4/162: mknod d1a/d22/c35 0 2024-05-09T06:38:12.198 INFO:tasks.workunit.client.1.smithi193.stdout:6/139: dread d1/fa [0,4194304] 0 2024-05-09T06:38:12.199 INFO:tasks.workunit.client.1.smithi193.stdout:9/174: getdents dd/df/d15 0 2024-05-09T06:38:12.200 INFO:tasks.workunit.client.1.smithi193.stdout:5/207: sync 2024-05-09T06:38:12.200 INFO:tasks.workunit.client.1.smithi193.stdout:3/79: sync 2024-05-09T06:38:12.201 INFO:tasks.workunit.client.1.smithi193.stdout:5/208: chown d0/d13/f1f 11239178 1 2024-05-09T06:38:12.203 INFO:tasks.workunit.client.0.smithi063.stdout:6/503: dwrite d2/d1f/f5b [0,4194304] 0 2024-05-09T06:38:12.203 INFO:tasks.workunit.client.0.smithi063.stdout:1/446: fdatasync d9/d2a/d35/d24/f53 0 2024-05-09T06:38:12.205 INFO:tasks.workunit.client.0.smithi063.stdout:4/487: rmdir d7/df/d35/d5a/d60 39 2024-05-09T06:38:12.207 INFO:tasks.workunit.client.0.smithi063.stdout:7/471: mknod d0/d15/d49/d4f/d66/d67/d86/ca4 0 2024-05-09T06:38:12.210 INFO:tasks.workunit.client.1.smithi193.stdout:2/159: dwrite f8 [4194304,4194304] 0 2024-05-09T06:38:12.210 INFO:tasks.workunit.client.0.smithi063.stdout:3/513: write d7/da/fb [928905,121244] 0 2024-05-09T06:38:12.211 INFO:tasks.workunit.client.1.smithi193.stdout:3/80: dwrite d1/db/f19 [0,4194304] 0 2024-05-09T06:38:12.220 INFO:tasks.workunit.client.1.smithi193.stdout:3/81: dwrite d1/f8 [0,4194304] 0 2024-05-09T06:38:12.221 INFO:tasks.workunit.client.1.smithi193.stdout:7/186: chown d6/d8/d1a/c1e 181377386 1 2024-05-09T06:38:12.230 INFO:tasks.workunit.client.1.smithi193.stdout:6/140: chown d1/f6 3104 1 2024-05-09T06:38:12.231 INFO:tasks.workunit.client.1.smithi193.stdout:9/175: fsync dd/f11 0 2024-05-09T06:38:12.231 INFO:tasks.workunit.client.0.smithi063.stdout:6/504: dwrite d2/d4/f1c [0,4194304] 0 2024-05-09T06:38:12.232 INFO:tasks.workunit.client.1.smithi193.stdout:6/141: readlink d1/l15 0 2024-05-09T06:38:12.233 INFO:tasks.workunit.client.1.smithi193.stdout:4/163: dwrite f10 [0,4194304] 0 2024-05-09T06:38:12.234 INFO:tasks.workunit.client.0.smithi063.stdout:0/530: truncate dd/d1a/d2c/f92 90948 0 2024-05-09T06:38:12.234 INFO:tasks.workunit.client.1.smithi193.stdout:4/164: write d1a/d24/d30/f34 [540771,8776] 0 2024-05-09T06:38:12.235 INFO:tasks.workunit.client.0.smithi063.stdout:7/472: rmdir d0/d15/d49/d4f/d66/d67 39 2024-05-09T06:38:12.238 INFO:tasks.workunit.client.0.smithi063.stdout:4/488: rename d7/df/d1f/d28/d31/d32/d34/c43 to d7/df/d1f/d28/d31/d61/c9a 0 2024-05-09T06:38:12.241 INFO:tasks.workunit.client.1.smithi193.stdout:1/139: mknod d4/d14/c21 0 2024-05-09T06:38:12.242 INFO:tasks.workunit.client.0.smithi063.stdout:3/514: symlink d7/da/d2b/la1 0 2024-05-09T06:38:12.245 INFO:tasks.workunit.client.0.smithi063.stdout:0/531: fsync dd/d1a/d2c/d51/d72/d9e/d23/f2f 0 2024-05-09T06:38:12.246 INFO:tasks.workunit.client.1.smithi193.stdout:7/187: write d6/d8/d29/f2d [23181,60257] 0 2024-05-09T06:38:12.247 INFO:tasks.workunit.client.0.smithi063.stdout:7/473: creat d0/d15/d49/d54/fa5 x:0 0 0 2024-05-09T06:38:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:1/140: dread d4/f13 [4194304,4194304] 0 2024-05-09T06:38:12.249 INFO:tasks.workunit.client.0.smithi063.stdout:1/447: getdents d9/d2a/d35/d24 0 2024-05-09T06:38:12.251 INFO:tasks.workunit.client.1.smithi193.stdout:9/176: symlink dd/df/d31/l3b 0 2024-05-09T06:38:12.251 INFO:tasks.workunit.client.0.smithi063.stdout:7/474: read d0/d15/f35 [384641,42935] 0 2024-05-09T06:38:12.253 INFO:tasks.workunit.client.1.smithi193.stdout:5/209: write d0/d13/d2c/d37/f11 [1595195,105961] 0 2024-05-09T06:38:12.253 INFO:tasks.workunit.client.1.smithi193.stdout:4/165: fsync f17 0 2024-05-09T06:38:12.254 INFO:tasks.workunit.client.1.smithi193.stdout:4/166: stat f11 0 2024-05-09T06:38:12.254 INFO:tasks.workunit.client.1.smithi193.stdout:4/167: fsync fa 0 2024-05-09T06:38:12.257 INFO:tasks.workunit.client.0.smithi063.stdout:0/532: creat dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/fad x:0 0 0 2024-05-09T06:38:12.259 INFO:tasks.workunit.client.1.smithi193.stdout:8/166: link d6/d1c/c10 d6/c31 0 2024-05-09T06:38:12.261 INFO:tasks.workunit.client.0.smithi063.stdout:2/535: sync 2024-05-09T06:38:12.264 INFO:tasks.workunit.client.0.smithi063.stdout:8/555: dread d9/da/d18/f63 [0,4194304] 0 2024-05-09T06:38:12.265 INFO:tasks.workunit.client.1.smithi193.stdout:0/197: sync 2024-05-09T06:38:12.265 INFO:tasks.workunit.client.1.smithi193.stdout:0/198: stat f15 0 2024-05-09T06:38:12.265 INFO:tasks.workunit.client.0.smithi063.stdout:1/448: rename d9/d46/f6f to d9/d2a/d39/d5d/f9a 0 2024-05-09T06:38:12.266 INFO:tasks.workunit.client.0.smithi063.stdout:1/449: chown d9/d2a/d3c/d4c/d83/d88/l89 256260849 1 2024-05-09T06:38:12.268 INFO:tasks.workunit.client.1.smithi193.stdout:1/141: mknod d4/d16/d17/d1a/c22 0 2024-05-09T06:38:12.271 INFO:tasks.workunit.client.1.smithi193.stdout:3/82: dwrite f0 [0,4194304] 0 2024-05-09T06:38:12.273 INFO:tasks.workunit.client.1.smithi193.stdout:3/83: fdatasync d1/f9 0 2024-05-09T06:38:12.274 INFO:tasks.workunit.client.0.smithi063.stdout:0/533: dwrite dd/d1a/d2c/d51/d72/d9e/f55 [4194304,4194304] 0 2024-05-09T06:38:12.275 INFO:tasks.workunit.client.1.smithi193.stdout:9/177: rename dd/df/d31/d33 to dd/d1e/d3c 0 2024-05-09T06:38:12.275 INFO:tasks.workunit.client.0.smithi063.stdout:4/489: link d7/df/d35/c89 d7/df/d1f/d28/d31/d61/c9b 0 2024-05-09T06:38:12.277 INFO:tasks.workunit.client.1.smithi193.stdout:1/142: dwrite d4/ff [0,4194304] 0 2024-05-09T06:38:12.281 INFO:tasks.workunit.client.0.smithi063.stdout:2/536: dread dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f86 [0,4194304] 0 2024-05-09T06:38:12.285 INFO:tasks.workunit.client.0.smithi063.stdout:0/534: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/f7b [0,4194304] 0 2024-05-09T06:38:12.285 INFO:tasks.workunit.client.0.smithi063.stdout:0/535: chown dd/d60/d44 91651978 1 2024-05-09T06:38:12.288 INFO:tasks.workunit.client.1.smithi193.stdout:1/143: dwrite d4/d14/d15/f18 [8388608,4194304] 0 2024-05-09T06:38:12.290 INFO:tasks.workunit.client.0.smithi063.stdout:6/505: getdents d2/d1f/d29 0 2024-05-09T06:38:12.290 INFO:tasks.workunit.client.1.smithi193.stdout:2/160: mkdir db/d25/d38 0 2024-05-09T06:38:12.294 INFO:tasks.workunit.client.1.smithi193.stdout:0/199: fdatasync d18/f28 0 2024-05-09T06:38:12.296 INFO:tasks.workunit.client.1.smithi193.stdout:7/188: mknod d6/d8/d29/d2f/c41 0 2024-05-09T06:38:12.297 INFO:tasks.workunit.client.1.smithi193.stdout:6/142: dwrite d1/de/f13 [0,4194304] 0 2024-05-09T06:38:12.298 INFO:tasks.workunit.client.1.smithi193.stdout:7/189: truncate d6/dd/ff 10387001 0 2024-05-09T06:38:12.304 INFO:tasks.workunit.client.0.smithi063.stdout:0/536: mknod dd/d1a/d2c/d51/d76/d97/cae 0 2024-05-09T06:38:12.306 INFO:tasks.workunit.client.1.smithi193.stdout:3/84: creat d1/f1d x:0 0 0 2024-05-09T06:38:12.306 INFO:tasks.workunit.client.0.smithi063.stdout:0/537: dread - dd/d1a/d2c/d51/d72/d9e/d23/d46/fa7 zero size 2024-05-09T06:38:12.306 INFO:tasks.workunit.client.0.smithi063.stdout:6/506: symlink d2/d20/d26/la2 0 2024-05-09T06:38:12.307 INFO:tasks.workunit.client.1.smithi193.stdout:9/178: symlink dd/df/d31/l3d 0 2024-05-09T06:38:12.307 INFO:tasks.workunit.client.1.smithi193.stdout:3/85: write d1/db/f19 [5147285,6827] 0 2024-05-09T06:38:12.308 INFO:tasks.workunit.client.0.smithi063.stdout:1/450: chown d9/d10/d2b/d3f/d59/d62/f6c 3 1 2024-05-09T06:38:12.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/210: mknod d0/d1/da/d3a/c45 0 2024-05-09T06:38:12.309 INFO:tasks.workunit.client.0.smithi063.stdout:1/451: stat d9/l5a 0 2024-05-09T06:38:12.309 INFO:tasks.workunit.client.0.smithi063.stdout:7/475: link d0/d15/d49/l51 d0/d15/d89/la6 0 2024-05-09T06:38:12.310 INFO:tasks.workunit.client.0.smithi063.stdout:1/452: truncate d9/d46/d57/f65 4243274 0 2024-05-09T06:38:12.310 INFO:tasks.workunit.client.0.smithi063.stdout:4/490: mknod d7/d3d/c9c 0 2024-05-09T06:38:12.313 INFO:tasks.workunit.client.1.smithi193.stdout:1/144: mknod d4/d14/d15/c23 0 2024-05-09T06:38:12.313 INFO:tasks.workunit.client.0.smithi063.stdout:6/507: mkdir d2/df/d59/da3 0 2024-05-09T06:38:12.315 INFO:tasks.workunit.client.0.smithi063.stdout:0/538: unlink dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/fad 0 2024-05-09T06:38:12.317 INFO:tasks.workunit.client.0.smithi063.stdout:6/508: read d2/d12/f24 [2392262,5883] 0 2024-05-09T06:38:12.317 INFO:tasks.workunit.client.0.smithi063.stdout:8/556: link d9/da/d45/la8 d9/da/d18/d56/d8f/lbe 0 2024-05-09T06:38:12.318 INFO:tasks.workunit.client.0.smithi063.stdout:4/491: dwrite d7/df/d1f/f53 [0,4194304] 0 2024-05-09T06:38:12.322 INFO:tasks.workunit.client.1.smithi193.stdout:8/167: write d6/f23 [2736340,5787] 0 2024-05-09T06:38:12.324 INFO:tasks.workunit.client.1.smithi193.stdout:3/86: chown d1/f11 1035510 1 2024-05-09T06:38:12.327 INFO:tasks.workunit.client.1.smithi193.stdout:9/179: mknod dd/d1e/c3e 0 2024-05-09T06:38:12.329 INFO:tasks.workunit.client.1.smithi193.stdout:4/168: dwrite f13 [0,4194304] 0 2024-05-09T06:38:12.329 INFO:tasks.workunit.client.1.smithi193.stdout:9/180: chown dd/df/d31/l3b 4 1 2024-05-09T06:38:12.330 INFO:tasks.workunit.client.1.smithi193.stdout:4/169: fdatasync f10 0 2024-05-09T06:38:12.332 INFO:tasks.workunit.client.1.smithi193.stdout:5/211: write d0/d13/d2c/f3f [81147,51544] 0 2024-05-09T06:38:12.332 INFO:tasks.workunit.client.0.smithi063.stdout:7/476: truncate d0/ff 1533824 0 2024-05-09T06:38:12.333 INFO:tasks.workunit.client.1.smithi193.stdout:8/168: dwrite d6/f21 [0,4194304] 0 2024-05-09T06:38:12.333 INFO:tasks.workunit.client.1.smithi193.stdout:9/181: read dd/d1e/d1f/f21 [628152,127804] 0 2024-05-09T06:38:12.334 INFO:tasks.workunit.client.1.smithi193.stdout:5/212: dread - d0/d13/d2c/d37/f36 zero size 2024-05-09T06:38:12.335 INFO:tasks.workunit.client.0.smithi063.stdout:6/509: mknod d2/d12/ca4 0 2024-05-09T06:38:12.337 INFO:tasks.workunit.client.0.smithi063.stdout:1/453: truncate d9/d2a/d35/f91 82026 0 2024-05-09T06:38:12.338 INFO:tasks.workunit.client.0.smithi063.stdout:1/454: stat d9/d10/d2b/d3f/d59/l60 0 2024-05-09T06:38:12.341 INFO:tasks.workunit.client.0.smithi063.stdout:8/557: rename d9/da/d18/d24/d71/c95 to d9/da/d18/d56/d58/d79/da1/cbf 0 2024-05-09T06:38:12.342 INFO:tasks.workunit.client.0.smithi063.stdout:8/558: readlink d9/da/d34/l44 0 2024-05-09T06:38:12.343 INFO:tasks.workunit.client.1.smithi193.stdout:7/190: link d6/d17/f37 d6/d17/f42 0 2024-05-09T06:38:12.347 INFO:tasks.workunit.client.1.smithi193.stdout:3/87: symlink d1/db/l1e 0 2024-05-09T06:38:12.347 INFO:tasks.workunit.client.1.smithi193.stdout:3/88: chown f0 1131885 1 2024-05-09T06:38:12.348 INFO:tasks.workunit.client.1.smithi193.stdout:3/89: write d1/f9 [869256,91931] 0 2024-05-09T06:38:12.352 INFO:tasks.workunit.client.0.smithi063.stdout:4/492: getdents d7/df/d1f/d28/d31/d7b/d81 0 2024-05-09T06:38:12.354 INFO:tasks.workunit.client.1.smithi193.stdout:4/170: creat d1a/f36 x:0 0 0 2024-05-09T06:38:12.358 INFO:tasks.workunit.client.1.smithi193.stdout:0/200: dwrite f8 [0,4194304] 0 2024-05-09T06:38:12.358 INFO:tasks.workunit.client.0.smithi063.stdout:8/559: symlink d9/da/d18/d46/lc0 0 2024-05-09T06:38:12.359 INFO:tasks.workunit.client.0.smithi063.stdout:4/493: dwrite d7/df/f95 [0,4194304] 0 2024-05-09T06:38:12.359 INFO:tasks.workunit.client.0.smithi063.stdout:8/560: dread - d9/da/d18/d56/d58/d79/faf zero size 2024-05-09T06:38:12.360 INFO:tasks.workunit.client.1.smithi193.stdout:6/143: dwrite d1/f6 [0,4194304] 0 2024-05-09T06:38:12.361 INFO:tasks.workunit.client.0.smithi063.stdout:7/477: mknod d0/d15/d49/d4f/d66/ca7 0 2024-05-09T06:38:12.364 INFO:tasks.workunit.client.0.smithi063.stdout:7/478: write d0/d15/f76 [998709,124090] 0 2024-05-09T06:38:12.364 INFO:tasks.workunit.client.0.smithi063.stdout:4/494: truncate d7/df/d1f/d28/d31/d61/f6d 857796 0 2024-05-09T06:38:12.366 INFO:tasks.workunit.client.0.smithi063.stdout:4/495: chown d7/df/d1f/d28/d31 84251 1 2024-05-09T06:38:12.366 INFO:tasks.workunit.client.1.smithi193.stdout:8/169: symlink d6/d25/l32 0 2024-05-09T06:38:12.368 INFO:tasks.workunit.client.0.smithi063.stdout:4/496: truncate d7/df/d1f/f65 4900839 0 2024-05-09T06:38:12.371 INFO:tasks.workunit.client.1.smithi193.stdout:0/201: dwrite d18/f28 [0,4194304] 0 2024-05-09T06:38:12.374 INFO:tasks.workunit.client.1.smithi193.stdout:5/213: mknod d0/d13/d2c/d37/d2b/c46 0 2024-05-09T06:38:12.380 INFO:tasks.workunit.client.0.smithi063.stdout:6/510: write d2/d1f/d29/d3f/f58 [1793045,85406] 0 2024-05-09T06:38:12.386 INFO:tasks.workunit.client.0.smithi063.stdout:4/497: rename d7/df/d1f/f53 to d7/d50/f9d 0 2024-05-09T06:38:12.387 INFO:tasks.workunit.client.0.smithi063.stdout:4/498: fdatasync d7/df/f12 0 2024-05-09T06:38:12.387 INFO:tasks.workunit.client.1.smithi193.stdout:9/182: mkdir dd/d3f 0 2024-05-09T06:38:12.388 INFO:tasks.workunit.client.1.smithi193.stdout:9/183: stat c3 0 2024-05-09T06:38:12.388 INFO:tasks.workunit.client.1.smithi193.stdout:6/144: creat d1/de/d11/d18/f29 x:0 0 0 2024-05-09T06:38:12.390 INFO:tasks.workunit.client.0.smithi063.stdout:1/455: dread d9/d2a/d35/d1e/f25 [4194304,4194304] 0 2024-05-09T06:38:12.390 INFO:tasks.workunit.client.0.smithi063.stdout:6/511: symlink d2/d1f/d6c/la5 0 2024-05-09T06:38:12.391 INFO:tasks.workunit.client.0.smithi063.stdout:6/512: chown d2/d1f/d6c/f7d 952528 1 2024-05-09T06:38:12.391 INFO:tasks.workunit.client.0.smithi063.stdout:7/479: creat d0/d15/d49/d54/d9f/d97/d9b/d9d/fa8 x:0 0 0 2024-05-09T06:38:12.392 INFO:tasks.workunit.client.1.smithi193.stdout:4/171: dread f7 [0,4194304] 0 2024-05-09T06:38:12.392 INFO:tasks.workunit.client.0.smithi063.stdout:6/513: fsync d2/d20/f9c 0 2024-05-09T06:38:12.393 INFO:tasks.workunit.client.1.smithi193.stdout:1/145: link d4/d16/f10 d4/d14/f24 0 2024-05-09T06:38:12.393 INFO:tasks.workunit.client.1.smithi193.stdout:1/146: chown d4/c7 1988276744 1 2024-05-09T06:38:12.394 INFO:tasks.workunit.client.0.smithi063.stdout:6/514: write d2/df/f50 [232157,42026] 0 2024-05-09T06:38:12.394 INFO:tasks.workunit.client.0.smithi063.stdout:6/515: fdatasync d2/d1f/d29/d3f/f1e 0 2024-05-09T06:38:12.395 INFO:tasks.workunit.client.1.smithi193.stdout:5/214: mkdir d0/d13/d2c/d37/d2b/d47 0 2024-05-09T06:38:12.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.418+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 --> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f289c000bf0 con 0x7f28a406d6e0 2024-05-09T06:38:12.399 INFO:tasks.workunit.client.0.smithi063.stdout:1/456: creat d9/d46/d57/f9b x:0 0 0 2024-05-09T06:38:12.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.420+0000 7f28af7fe700 1 -- 172.21.15.63:0/2332109053 <== mgr.24415 v2:172.21.15.193:6828/1432929717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f289c000bf0 con 0x7f28a406d6e0 2024-05-09T06:38:12.401 INFO:tasks.workunit.client.0.smithi063.stdout:1/457: write d9/d2a/d35/f8f [122583,84808] 0 2024-05-09T06:38:12.401 INFO:tasks.workunit.client.0.smithi063.stdout:7/480: rename d0/d15/d4a/d50/f61 to d0/d15/d89/fa9 0 2024-05-09T06:38:12.403 INFO:tasks.workunit.client.0.smithi063.stdout:1/458: write d9/d2a/d3c/d4a/f7b [3036971,61957] 0 2024-05-09T06:38:12.403 INFO:tasks.workunit.client.0.smithi063.stdout:7/481: chown d0/d15/f4e 1771 1 2024-05-09T06:38:12.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.425+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f28a406d6e0 msgr2=0x7f28a406fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.425+0000 7f28c032c700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f28a406d6e0 0x7f28a406fba0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f28a8006010 tx=0x7f28a8005a90 comp rx=0 tx=0).stop 2024-05-09T06:38:12.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.425+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b80695d0 msgr2=0x7f28b8109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.425+0000 7f28c032c700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b80695d0 0x7f28b8109a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e920 tx=0x7f28b400ece0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.425+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 shutdown_connections 2024-05-09T06:38:12.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.425+0000 7f28c032c700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f28a406d6e0 0x7f28a406fba0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.425+0000 7f28c032c700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28b80695d0 0x7f28b8109a10 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e920 tx=0x7f28b400ece0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.425+0000 7f28c032c700 1 --2- 172.21.15.63:0/2332109053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b810a640 0x7f28b81080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.425+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 >> 172.21.15.63:0/2332109053 conn(0x7f28b806eaf0 msgr2=0x7f28b810da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:12.407 INFO:tasks.workunit.client.1.smithi193.stdout:6/145: creat d1/de/d11/d18/f2a x:0 0 0 2024-05-09T06:38:12.407 INFO:tasks.workunit.client.1.smithi193.stdout:6/146: write d1/f2 [7828167,118070] 0 2024-05-09T06:38:12.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.426+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 shutdown_connections 2024-05-09T06:38:12.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.426+0000 7f28c032c700 1 -- 172.21.15.63:0/2332109053 wait complete. 2024-05-09T06:38:12.407 INFO:tasks.workunit.client.0.smithi063.stdout:6/516: dread d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:12.408 INFO:tasks.workunit.client.0.smithi063.stdout:5/465: sync 2024-05-09T06:38:12.408 INFO:tasks.workunit.client.0.smithi063.stdout:9/501: sync 2024-05-09T06:38:12.408 INFO:tasks.workunit.client.0.smithi063.stdout:8/561: dread d9/da/d18/f80 [0,4194304] 0 2024-05-09T06:38:12.409 INFO:tasks.workunit.client.1.smithi193.stdout:4/172: unlink lc 0 2024-05-09T06:38:12.409 INFO:tasks.workunit.client.0.smithi063.stdout:9/502: chown d0/d4/da/l2f 2613603 1 2024-05-09T06:38:12.410 INFO:tasks.workunit.client.0.smithi063.stdout:9/503: dread - d0/d4/d57/d60/d76/fa9 zero size 2024-05-09T06:38:12.410 INFO:tasks.workunit.client.1.smithi193.stdout:4/173: write fa [1504576,67682] 0 2024-05-09T06:38:12.412 INFO:tasks.workunit.client.1.smithi193.stdout:4/174: write d1a/f36 [292815,114527] 0 2024-05-09T06:38:12.412 INFO:tasks.workunit.client.1.smithi193.stdout:1/147: creat d4/d16/f25 x:0 0 0 2024-05-09T06:38:12.413 INFO:tasks.workunit.client.1.smithi193.stdout:6/147: dwrite d1/f6 [8388608,4194304] 0 2024-05-09T06:38:12.413 INFO:tasks.workunit.client.1.smithi193.stdout:4/175: fdatasync f5 0 2024-05-09T06:38:12.415 INFO:tasks.workunit.client.0.smithi063.stdout:9/504: dread d0/d4/f4d [0,4194304] 0 2024-05-09T06:38:12.420 INFO:tasks.workunit.client.1.smithi193.stdout:3/90: link d1/c13 d1/db/c1f 0 2024-05-09T06:38:12.422 INFO:tasks.workunit.client.1.smithi193.stdout:3/91: write d1/f4 [1254935,45991] 0 2024-05-09T06:38:12.422 INFO:tasks.workunit.client.1.smithi193.stdout:9/184: mknod dd/c40 0 2024-05-09T06:38:12.422 INFO:tasks.workunit.client.1.smithi193.stdout:4/176: dwrite f13 [4194304,4194304] 0 2024-05-09T06:38:12.423 INFO:tasks.workunit.client.1.smithi193.stdout:9/185: chown f0 23041 1 2024-05-09T06:38:12.423 INFO:tasks.workunit.client.1.smithi193.stdout:4/177: truncate d1a/f28 972579 0 2024-05-09T06:38:12.424 INFO:tasks.workunit.client.1.smithi193.stdout:9/186: write dd/d1e/f27 [5145743,62308] 0 2024-05-09T06:38:12.426 INFO:tasks.workunit.client.0.smithi063.stdout:5/466: write d8/d17/f95 [982998,97622] 0 2024-05-09T06:38:12.429 INFO:tasks.workunit.client.0.smithi063.stdout:5/467: fsync d8/d19/f3f 0 2024-05-09T06:38:12.432 INFO:tasks.workunit.client.0.smithi063.stdout:9/505: dread d0/d4/d26/d47/f5d [0,4194304] 0 2024-05-09T06:38:12.465 INFO:tasks.workunit.client.0.smithi063.stdout:3/515: sync 2024-05-09T06:38:12.466 INFO:tasks.workunit.client.0.smithi063.stdout:2/537: sync 2024-05-09T06:38:12.466 INFO:tasks.workunit.client.0.smithi063.stdout:2/538: stat dc/dd/d2d/d60/l69 0 2024-05-09T06:38:12.467 INFO:tasks.workunit.client.0.smithi063.stdout:2/539: fdatasync dc/d6e/d7f/f97 0 2024-05-09T06:38:12.509 INFO:tasks.workunit.client.1.smithi193.stdout:5/215: rmdir d0/d13/d2c 39 2024-05-09T06:38:12.511 INFO:tasks.workunit.client.1.smithi193.stdout:7/191: write d6/f30 [3242663,94981] 0 2024-05-09T06:38:12.512 INFO:tasks.workunit.client.0.smithi063.stdout:4/499: write d7/df/d1f/f23 [288406,72603] 0 2024-05-09T06:38:12.514 INFO:tasks.workunit.client.1.smithi193.stdout:6/148: mknod d1/de/d11/d20/c2b 0 2024-05-09T06:38:12.515 INFO:tasks.workunit.client.1.smithi193.stdout:6/149: stat d1/de/d11/d18/f1b 0 2024-05-09T06:38:12.516 INFO:tasks.workunit.client.1.smithi193.stdout:6/150: write d1/de/f13 [765785,20723] 0 2024-05-09T06:38:12.518 INFO:tasks.workunit.client.0.smithi063.stdout:7/482: mkdir d0/daa 0 2024-05-09T06:38:12.519 INFO:tasks.workunit.client.1.smithi193.stdout:3/92: mkdir d1/d20 0 2024-05-09T06:38:12.522 INFO:tasks.workunit.client.1.smithi193.stdout:8/170: getdents d6 0 2024-05-09T06:38:12.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/539: sync 2024-05-09T06:38:12.524 INFO:tasks.workunit.client.1.smithi193.stdout:3/93: dread d1/f9 [0,4194304] 0 2024-05-09T06:38:12.524 INFO:tasks.workunit.client.0.smithi063.stdout:5/468: mkdir d8/d65/d9e 0 2024-05-09T06:38:12.524 INFO:tasks.workunit.client.0.smithi063.stdout:1/459: write d9/d2a/d35/d1e/f3a [3788396,62492] 0 2024-05-09T06:38:12.525 INFO:tasks.workunit.client.1.smithi193.stdout:5/216: write d0/d13/d2c/f30 [747011,130268] 0 2024-05-09T06:38:12.529 INFO:tasks.workunit.client.1.smithi193.stdout:7/192: rename d6/d8/d29/d2f/f38 to d6/d17/f43 0 2024-05-09T06:38:12.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.551+0000 7fc1cebce700 1 -- 172.21.15.63:0/2827759940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 msgr2=0x7fc1c8106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.551+0000 7fc1cebce700 1 --2- 172.21.15.63:0/2827759940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 0x7fc1c8106410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009910 tx=0x7fc1b4009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:12.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.551+0000 7fc1cebce700 1 -- 172.21.15.63:0/2827759940 shutdown_connections 2024-05-09T06:38:12.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.551+0000 7fc1cebce700 1 --2- 172.21.15.63:0/2827759940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 0x7fc1c8106410 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.551+0000 7fc1cebce700 1 --2- 172.21.15.63:0/2827759940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c81016f0 0x7fc1c8103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.551+0000 7fc1cebce700 1 -- 172.21.15.63:0/2827759940 >> 172.21.15.63:0/2827759940 conn(0x7fc1c80fb0c0 msgr2=0x7fc1c80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:12.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.551+0000 7fc1cebce700 1 -- 172.21.15.63:0/2827759940 shutdown_connections 2024-05-09T06:38:12.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.551+0000 7fc1cebce700 1 -- 172.21.15.63:0/2827759940 wait complete. 2024-05-09T06:38:12.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.552+0000 7fc1cebce700 1 Processor -- start 2024-05-09T06:38:12.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.552+0000 7fc1cebce700 1 -- start start 2024-05-09T06:38:12.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1cebce700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c81016f0 0x7fc1c8101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1cebce700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 0x7fc1c80ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1cebce700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80ffbe0 con 0x7fc1c8104020 2024-05-09T06:38:12.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1cebce700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80ffd20 con 0x7fc1c81016f0 2024-05-09T06:38:12.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1c7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 0x7fc1c80ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1c7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 0x7fc1c80ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56014/0 (socket says 172.21.15.63:56014) 2024-05-09T06:38:12.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1c7fff700 1 -- 172.21.15.63:0/2715859383 learned_addr learned my addr 172.21.15.63:0/2715859383 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:12.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1cc96a700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c81016f0 0x7fc1c8101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1c7fff700 1 -- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c81016f0 msgr2=0x7fc1c8101000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1c7fff700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c81016f0 0x7fc1c8101000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.553+0000 7fc1c7fff700 1 -- 172.21.15.63:0/2715859383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40095f0 con 0x7fc1c8104020 2024-05-09T06:38:12.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.554+0000 7fc1c7fff700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 0x7fc1c80ff6a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400b800 tx=0x7fc1b4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:12.535 INFO:tasks.workunit.client.0.smithi063.stdout:5/469: dread d8/d17/f6d [0,4194304] 0 2024-05-09T06:38:12.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.554+0000 7fc1c5ffb700 1 -- 172.21.15.63:0/2715859383 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b401d070 con 0x7fc1c8104020 2024-05-09T06:38:12.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.554+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c80fffa0 con 0x7fc1c8104020 2024-05-09T06:38:12.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.554+0000 7fc1c5ffb700 1 -- 172.21.15.63:0/2715859383 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1b40052a0 con 0x7fc1c8104020 2024-05-09T06:38:12.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.554+0000 7fc1c5ffb700 1 -- 172.21.15.63:0/2715859383 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b4017400 con 0x7fc1c8104020 2024-05-09T06:38:12.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.554+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c8100510 con 0x7fc1c8104020 2024-05-09T06:38:12.536 INFO:tasks.workunit.client.0.smithi063.stdout:8/562: creat d9/da/d53/dbc/fc1 x:0 0 0 2024-05-09T06:38:12.537 INFO:tasks.workunit.client.0.smithi063.stdout:4/500: readlink d7/df/d35/l84 0 2024-05-09T06:38:12.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.555+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c804ebb0 con 0x7fc1c8104020 2024-05-09T06:38:12.537 INFO:tasks.workunit.client.0.smithi063.stdout:0/540: dwrite dd/d1a/d2c/d51/d72/d9e/f55 [0,4194304] 0 2024-05-09T06:38:12.537 INFO:tasks.workunit.client.0.smithi063.stdout:5/470: read - d8/d17/d75/f7f zero size 2024-05-09T06:38:12.538 INFO:tasks.workunit.client.1.smithi193.stdout:9/187: mkdir dd/d3f/d41 0 2024-05-09T06:38:12.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.557+0000 7fc1c5ffb700 1 -- 172.21.15.63:0/2715859383 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7fc1b4003780 con 0x7fc1c8104020 2024-05-09T06:38:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.557+0000 7fc1c5ffb700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fc1b806d6f0 0x7fc1b806fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.558+0000 7fc1c5ffb700 1 -- 172.21.15.63:0/2715859383 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc1b408e510 con 0x7fc1c8104020 2024-05-09T06:38:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.558+0000 7fc1cc96a700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fc1b806d6f0 0x7fc1b806fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:12.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.558+0000 7fc1cc96a700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fc1b806d6f0 0x7fc1b806fbb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009c60 tx=0x7fc1bc009260 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:12.539 INFO:tasks.workunit.client.0.smithi063.stdout:7/483: dread - d0/d15/d49/d4f/d66/d67/d86/f90 zero size 2024-05-09T06:38:12.540 INFO:tasks.workunit.client.0.smithi063.stdout:7/484: fsync d0/d15/d49/d54/f60 0 2024-05-09T06:38:12.541 INFO:tasks.workunit.client.1.smithi193.stdout:3/94: symlink d1/db/l21 0 2024-05-09T06:38:12.543 INFO:tasks.workunit.client.1.smithi193.stdout:7/193: dread d6/dd/ff [0,4194304] 0 2024-05-09T06:38:12.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.564+0000 7fc1c5ffb700 1 -- 172.21.15.63:0/2715859383 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7fc1b405b8a0 con 0x7fc1c8104020 2024-05-09T06:38:12.544 INFO:tasks.workunit.client.1.smithi193.stdout:1/148: creat d4/f26 x:0 0 0 2024-05-09T06:38:12.544 INFO:tasks.workunit.client.1.smithi193.stdout:7/194: read - d6/d8/d1a/f3d zero size 2024-05-09T06:38:12.545 INFO:tasks.workunit.client.0.smithi063.stdout:9/506: creat d0/d8/da6/db0/fb1 x:0 0 0 2024-05-09T06:38:12.549 INFO:tasks.workunit.client.0.smithi063.stdout:3/516: link d7/f1b d7/d4c/d31/fa2 0 2024-05-09T06:38:12.550 INFO:tasks.workunit.client.0.smithi063.stdout:3/517: fsync f5 0 2024-05-09T06:38:12.551 INFO:tasks.workunit.client.1.smithi193.stdout:6/151: mkdir d1/d25/d2c 0 2024-05-09T06:38:12.553 INFO:tasks.workunit.client.0.smithi063.stdout:3/518: dread d7/da/f18 [0,4194304] 0 2024-05-09T06:38:12.554 INFO:tasks.workunit.client.1.smithi193.stdout:8/171: mknod d6/d1c/c33 0 2024-05-09T06:38:12.555 INFO:tasks.workunit.client.1.smithi193.stdout:8/172: chown d6/d1c/d18/c28 91 1 2024-05-09T06:38:12.559 INFO:tasks.workunit.client.0.smithi063.stdout:3/519: read d7/d4c/d31/f6e [277582,82369] 0 2024-05-09T06:38:12.563 INFO:tasks.workunit.client.1.smithi193.stdout:7/195: readlink d6/d8/d1a/l20 0 2024-05-09T06:38:12.566 INFO:tasks.workunit.client.1.smithi193.stdout:6/152: mkdir d1/de/d11/d2d 0 2024-05-09T06:38:12.567 INFO:tasks.workunit.client.1.smithi193.stdout:9/188: write dd/f11 [930367,26407] 0 2024-05-09T06:38:12.569 INFO:tasks.workunit.client.1.smithi193.stdout:3/95: creat d1/d20/f22 x:0 0 0 2024-05-09T06:38:12.570 INFO:tasks.workunit.client.1.smithi193.stdout:9/189: dread dd/f29 [0,4194304] 0 2024-05-09T06:38:12.570 INFO:tasks.workunit.client.1.smithi193.stdout:3/96: fsync d1/db/f19 0 2024-05-09T06:38:12.571 INFO:tasks.workunit.client.1.smithi193.stdout:8/173: fdatasync d6/d1c/d18/f1d 0 2024-05-09T06:38:12.571 INFO:tasks.workunit.client.1.smithi193.stdout:3/97: dread - d1/db/f14 zero size 2024-05-09T06:38:12.571 INFO:tasks.workunit.client.1.smithi193.stdout:9/190: write dd/f18 [621176,73988] 0 2024-05-09T06:38:12.573 INFO:tasks.workunit.client.1.smithi193.stdout:3/98: read d1/db/f19 [1969467,126969] 0 2024-05-09T06:38:12.578 INFO:tasks.workunit.client.1.smithi193.stdout:8/174: dwrite d6/d1c/f14 [0,4194304] 0 2024-05-09T06:38:12.579 INFO:tasks.workunit.client.1.smithi193.stdout:1/149: dwrite d4/f13 [0,4194304] 0 2024-05-09T06:38:12.580 INFO:tasks.workunit.client.0.smithi063.stdout:2/540: truncate f6 18953 0 2024-05-09T06:38:12.585 INFO:tasks.workunit.client.0.smithi063.stdout:8/563: write d9/da/d34/f4b [3492951,2228] 0 2024-05-09T06:38:12.586 INFO:tasks.workunit.client.1.smithi193.stdout:8/175: dwrite d6/f23 [0,4194304] 0 2024-05-09T06:38:12.587 INFO:tasks.workunit.client.0.smithi063.stdout:4/501: mknod d7/df/c9e 0 2024-05-09T06:38:12.589 INFO:tasks.workunit.client.0.smithi063.stdout:5/471: chown d8/d17/d18/c72 2 1 2024-05-09T06:38:12.595 INFO:tasks.workunit.client.0.smithi063.stdout:7/485: mknod d0/d15/d49/d54/d9f/cab 0 2024-05-09T06:38:12.597 INFO:tasks.workunit.client.1.smithi193.stdout:9/191: mknod dd/df/c42 0 2024-05-09T06:38:12.599 INFO:tasks.workunit.client.0.smithi063.stdout:9/507: rename d0/f32 to d0/d4/d57/d60/fb2 0 2024-05-09T06:38:12.600 INFO:tasks.workunit.client.1.smithi193.stdout:3/99: mknod d1/db/c23 0 2024-05-09T06:38:12.601 INFO:tasks.workunit.client.1.smithi193.stdout:5/217: link d0/l18 d0/d1/l48 0 2024-05-09T06:38:12.604 INFO:tasks.workunit.client.1.smithi193.stdout:1/150: creat d4/d16/f27 x:0 0 0 2024-05-09T06:38:12.605 INFO:tasks.workunit.client.0.smithi063.stdout:3/520: write d7/da/d2b/d66/d74/d7e/f45 [132815,60328] 0 2024-05-09T06:38:12.605 INFO:tasks.workunit.client.0.smithi063.stdout:6/517: getdents d2/d1f/d6c 0 2024-05-09T06:38:12.606 INFO:tasks.workunit.client.1.smithi193.stdout:6/153: creat d1/de/d11/d2d/f2e x:0 0 0 2024-05-09T06:38:12.606 INFO:tasks.workunit.client.0.smithi063.stdout:3/521: chown d7/da/d21/d30/d44/d48/l7a 197344 1 2024-05-09T06:38:12.607 INFO:tasks.workunit.client.0.smithi063.stdout:8/564: truncate d9/da/d18/d24/f3c 4858126 0 2024-05-09T06:38:12.609 INFO:tasks.workunit.client.1.smithi193.stdout:8/176: chown d6/d1c/d1b/c24 63 1 2024-05-09T06:38:12.609 INFO:tasks.workunit.client.1.smithi193.stdout:8/177: chown d6/c9 62127723 1 2024-05-09T06:38:12.609 INFO:tasks.workunit.client.0.smithi063.stdout:4/502: rmdir d7/df/d57 39 2024-05-09T06:38:12.611 INFO:tasks.workunit.client.0.smithi063.stdout:4/503: write d7/df/d1f/f3a [3513127,125094] 0 2024-05-09T06:38:12.613 INFO:tasks.workunit.client.1.smithi193.stdout:9/192: read dd/f29 [309975,11992] 0 2024-05-09T06:38:12.613 INFO:tasks.workunit.client.1.smithi193.stdout:9/193: readlink dd/df/d15/l19 0 2024-05-09T06:38:12.614 INFO:tasks.workunit.client.0.smithi063.stdout:7/486: creat d0/d15/d49/d54/d9f/d97/d9b/fac x:0 0 0 2024-05-09T06:38:12.614 INFO:tasks.workunit.client.0.smithi063.stdout:7/487: read - d0/d15/d49/d4f/d66/d67/d86/f90 zero size 2024-05-09T06:38:12.614 INFO:tasks.workunit.client.1.smithi193.stdout:3/100: unlink d1/db/c17 0 2024-05-09T06:38:12.616 INFO:tasks.workunit.client.0.smithi063.stdout:5/472: rename d8/d19/f4f to d8/d17/d1e/f9f 0 2024-05-09T06:38:12.616 INFO:tasks.workunit.client.0.smithi063.stdout:5/473: fsync d8/d65/f6f 0 2024-05-09T06:38:12.617 INFO:tasks.workunit.client.0.smithi063.stdout:9/508: creat d0/d4/d7d/fb3 x:0 0 0 2024-05-09T06:38:12.618 INFO:tasks.workunit.client.0.smithi063.stdout:9/509: fsync d0/d4/d26/d39/d91/fa7 0 2024-05-09T06:38:12.618 INFO:tasks.workunit.client.1.smithi193.stdout:7/196: creat d6/d8/f44 x:0 0 0 2024-05-09T06:38:12.619 INFO:tasks.workunit.client.0.smithi063.stdout:1/460: link d9/d2a/d39/c5f d9/d76/d94/c9c 0 2024-05-09T06:38:12.621 INFO:tasks.workunit.client.1.smithi193.stdout:1/151: creat d4/d16/de/f28 x:0 0 0 2024-05-09T06:38:12.621 INFO:tasks.workunit.client.1.smithi193.stdout:1/152: stat d4/d16 0 2024-05-09T06:38:12.623 INFO:tasks.workunit.client.0.smithi063.stdout:1/461: dwrite d9/d10/f15 [0,4194304] 0 2024-05-09T06:38:12.625 INFO:tasks.workunit.client.1.smithi193.stdout:6/154: dread d1/de/d11/d18/f1e [0,4194304] 0 2024-05-09T06:38:12.626 INFO:tasks.workunit.client.0.smithi063.stdout:6/518: symlink d2/d20/la6 0 2024-05-09T06:38:12.626 INFO:tasks.workunit.client.1.smithi193.stdout:9/194: mkdir dd/d1e/d1f/d43 0 2024-05-09T06:38:12.628 INFO:tasks.workunit.client.1.smithi193.stdout:9/195: dread dd/f29 [0,4194304] 0 2024-05-09T06:38:12.628 INFO:tasks.workunit.client.0.smithi063.stdout:2/541: mkdir dc/dd/d45/d5a/d6c/d8c/d9a 0 2024-05-09T06:38:12.629 INFO:tasks.workunit.client.1.smithi193.stdout:3/101: creat d1/f24 x:0 0 0 2024-05-09T06:38:12.630 INFO:tasks.workunit.client.0.smithi063.stdout:1/462: dwrite d9/d10/d2b/d3f/d59/d62/f8b [0,4194304] 0 2024-05-09T06:38:12.630 INFO:tasks.workunit.client.0.smithi063.stdout:3/522: creat d7/da/d21/d30/fa3 x:0 0 0 2024-05-09T06:38:12.631 INFO:tasks.workunit.client.0.smithi063.stdout:1/463: read d9/d46/d57/f65 [3712679,106238] 0 2024-05-09T06:38:12.631 INFO:tasks.workunit.client.0.smithi063.stdout:3/523: read - d7/da/d21/d30/d44/d48/f6f zero size 2024-05-09T06:38:12.632 INFO:tasks.workunit.client.1.smithi193.stdout:5/218: symlink d0/d13/d2c/d37/d2b/d47/l49 0 2024-05-09T06:38:12.633 INFO:tasks.workunit.client.0.smithi063.stdout:3/524: write d7/da/d2b/d66/f95 [685076,43481] 0 2024-05-09T06:38:12.637 INFO:tasks.workunit.client.0.smithi063.stdout:8/565: mkdir d9/da/d51/dc2 0 2024-05-09T06:38:12.637 INFO:tasks.workunit.client.1.smithi193.stdout:9/196: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:12.638 INFO:tasks.workunit.client.0.smithi063.stdout:1/464: dwrite d9/d10/d2b/d3f/d59/f79 [0,4194304] 0 2024-05-09T06:38:12.638 INFO:tasks.workunit.client.0.smithi063.stdout:8/566: dread - d9/da/d18/d56/f78 zero size 2024-05-09T06:38:12.639 INFO:tasks.workunit.client.1.smithi193.stdout:1/153: creat d4/d16/d17/f29 x:0 0 0 2024-05-09T06:38:12.642 INFO:tasks.workunit.client.0.smithi063.stdout:0/541: creat dd/d1a/d2c/d51/d72/d9e/faf x:0 0 0 2024-05-09T06:38:12.642 INFO:tasks.workunit.client.0.smithi063.stdout:4/504: dread - d7/df/d35/d5a/d60/f6e zero size 2024-05-09T06:38:12.643 INFO:tasks.workunit.client.1.smithi193.stdout:1/154: read d4/d14/d15/f18 [3215482,728] 0 2024-05-09T06:38:12.644 INFO:tasks.workunit.client.0.smithi063.stdout:0/542: stat dd/d1a/d2c/d51/d72/d9e/d23/d89/f8a 0 2024-05-09T06:38:12.645 INFO:tasks.workunit.client.1.smithi193.stdout:1/155: stat d4/d16/d17/d1a 0 2024-05-09T06:38:12.647 INFO:tasks.workunit.client.0.smithi063.stdout:7/488: unlink d0/l73 0 2024-05-09T06:38:12.647 INFO:tasks.workunit.client.0.smithi063.stdout:5/474: stat d8/d19/d30/d46 0 2024-05-09T06:38:12.649 INFO:tasks.workunit.client.1.smithi193.stdout:3/102: mknod d1/c25 0 2024-05-09T06:38:12.651 INFO:tasks.workunit.client.0.smithi063.stdout:9/510: unlink d0/d4/da/f15 0 2024-05-09T06:38:12.652 INFO:tasks.workunit.client.1.smithi193.stdout:6/155: dwrite d1/fa [4194304,4194304] 0 2024-05-09T06:38:12.653 INFO:tasks.workunit.client.1.smithi193.stdout:5/219: rename d0/d13/f1f to d0/d13/d2c/f4a 0 2024-05-09T06:38:12.655 INFO:tasks.workunit.client.1.smithi193.stdout:6/156: write d1/de/d11/f1d [352650,102826] 0 2024-05-09T06:38:12.656 INFO:tasks.workunit.client.1.smithi193.stdout:1/156: dwrite d4/d16/f25 [0,4194304] 0 2024-05-09T06:38:12.657 INFO:tasks.workunit.client.0.smithi063.stdout:6/519: symlink d2/df/la7 0 2024-05-09T06:38:12.658 INFO:tasks.workunit.client.1.smithi193.stdout:3/103: mkdir d1/db/d26 0 2024-05-09T06:38:12.662 INFO:tasks.workunit.client.0.smithi063.stdout:2/542: dread dc/d32/d33/f61 [0,4194304] 0 2024-05-09T06:38:12.662 INFO:tasks.workunit.client.1.smithi193.stdout:9/197: symlink dd/df/d15/d16/d39/l44 0 2024-05-09T06:38:12.663 INFO:tasks.workunit.client.0.smithi063.stdout:3/525: mkdir d7/d4c/d31/da4 0 2024-05-09T06:38:12.667 INFO:tasks.workunit.client.1.smithi193.stdout:6/157: creat d1/de/f2f x:0 0 0 2024-05-09T06:38:12.668 INFO:tasks.workunit.client.0.smithi063.stdout:8/567: rmdir d9/da/d51 39 2024-05-09T06:38:12.669 INFO:tasks.workunit.client.1.smithi193.stdout:1/157: unlink d4/d14/f24 0 2024-05-09T06:38:12.669 INFO:tasks.workunit.client.1.smithi193.stdout:1/158: chown d4/d16/de 1424832220 1 2024-05-09T06:38:12.670 INFO:tasks.workunit.client.0.smithi063.stdout:4/505: rmdir d7/d3d/d73 39 2024-05-09T06:38:12.671 INFO:tasks.workunit.client.0.smithi063.stdout:3/526: dwrite d7/da/d21/d30/f57 [0,4194304] 0 2024-05-09T06:38:12.671 INFO:tasks.workunit.client.1.smithi193.stdout:1/159: truncate d4/d16/f27 369423 0 2024-05-09T06:38:12.671 INFO:tasks.workunit.client.1.smithi193.stdout:3/104: symlink d1/l27 0 2024-05-09T06:38:12.672 INFO:tasks.workunit.client.0.smithi063.stdout:0/543: mknod dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/cb0 0 2024-05-09T06:38:12.674 INFO:tasks.workunit.client.1.smithi193.stdout:9/198: creat dd/df/d15/d16/d39/f45 x:0 0 0 2024-05-09T06:38:12.677 INFO:tasks.workunit.client.1.smithi193.stdout:6/158: creat d1/d25/f30 x:0 0 0 2024-05-09T06:38:12.678 INFO:tasks.workunit.client.0.smithi063.stdout:9/511: creat d0/d8/da6/db0/fb4 x:0 0 0 2024-05-09T06:38:12.678 INFO:tasks.workunit.client.1.smithi193.stdout:6/159: stat d1/de/d11/d20/f27 0 2024-05-09T06:38:12.679 INFO:tasks.workunit.client.0.smithi063.stdout:9/512: dread - d0/d8/d2d/faf zero size 2024-05-09T06:38:12.679 INFO:tasks.workunit.client.1.smithi193.stdout:1/160: dwrite d4/d16/d17/d1a/f1e [0,4194304] 0 2024-05-09T06:38:12.680 INFO:tasks.workunit.client.1.smithi193.stdout:6/160: write d1/de/d11/f22 [868440,3905] 0 2024-05-09T06:38:12.681 INFO:tasks.workunit.client.1.smithi193.stdout:1/161: rename d4/d14 to d4/d14/d15/d2a 22 2024-05-09T06:38:12.683 INFO:tasks.workunit.client.1.smithi193.stdout:6/161: rename d1 to d1/de/d11/d18/d31 22 2024-05-09T06:38:12.683 INFO:tasks.workunit.client.1.smithi193.stdout:5/220: getdents d0 0 2024-05-09T06:38:12.685 INFO:tasks.workunit.client.0.smithi063.stdout:9/513: dwrite d0/f65 [0,4194304] 0 2024-05-09T06:38:12.685 INFO:tasks.workunit.client.1.smithi193.stdout:5/221: read d0/d13/d2c/f4a [326652,51012] 0 2024-05-09T06:38:12.687 INFO:tasks.workunit.client.0.smithi063.stdout:2/543: symlink dc/dd/d45/d5a/d6c/l9b 0 2024-05-09T06:38:12.688 INFO:tasks.workunit.client.0.smithi063.stdout:1/465: rmdir d9/d2a/d2e 39 2024-05-09T06:38:12.690 INFO:tasks.workunit.client.1.smithi193.stdout:1/162: mkdir d4/d14/d15/d2b 0 2024-05-09T06:38:12.691 INFO:tasks.workunit.client.1.smithi193.stdout:1/163: truncate d4/d16/f27 919932 0 2024-05-09T06:38:12.692 INFO:tasks.workunit.client.1.smithi193.stdout:1/164: fdatasync d4/f13 0 2024-05-09T06:38:12.693 INFO:tasks.workunit.client.0.smithi063.stdout:8/568: symlink d9/da/d53/da3/lc3 0 2024-05-09T06:38:12.693 INFO:tasks.workunit.client.1.smithi193.stdout:1/165: chown d4/f26 5 1 2024-05-09T06:38:12.693 INFO:tasks.workunit.client.1.smithi193.stdout:6/162: symlink d1/de/d11/l32 0 2024-05-09T06:38:12.694 INFO:tasks.workunit.client.0.smithi063.stdout:2/544: dwrite dc/dd/d2d/f77 [0,4194304] 0 2024-05-09T06:38:12.695 INFO:tasks.workunit.client.0.smithi063.stdout:2/545: fsync dc/f1b 0 2024-05-09T06:38:12.696 INFO:tasks.workunit.client.1.smithi193.stdout:9/199: symlink dd/df/d15/d16/d26/d37/l46 0 2024-05-09T06:38:12.697 INFO:tasks.workunit.client.0.smithi063.stdout:2/546: truncate dc/dd/d45/f89 1044159 0 2024-05-09T06:38:12.698 INFO:tasks.workunit.client.1.smithi193.stdout:9/200: stat dd/d1e/d1f/d43 0 2024-05-09T06:38:12.700 INFO:tasks.workunit.client.1.smithi193.stdout:5/222: creat d0/d13/d2c/f4b x:0 0 0 2024-05-09T06:38:12.701 INFO:tasks.workunit.client.0.smithi063.stdout:0/544: creat dd/d60/d44/d90/fb1 x:0 0 0 2024-05-09T06:38:12.702 INFO:tasks.workunit.client.1.smithi193.stdout:1/166: dwrite d4/d16/d17/f29 [0,4194304] 0 2024-05-09T06:38:12.704 INFO:tasks.workunit.client.1.smithi193.stdout:1/167: write d4/d16/d17/d1a/f1e [6222673,106234] 0 2024-05-09T06:38:12.705 INFO:tasks.workunit.client.1.smithi193.stdout:1/168: stat d4/d16/d17/d1a 0 2024-05-09T06:38:12.706 INFO:tasks.workunit.client.0.smithi063.stdout:5/475: unlink d8/d17/d18/f69 0 2024-05-09T06:38:12.706 INFO:tasks.workunit.client.1.smithi193.stdout:8/178: truncate d6/d25/f2c 827948 0 2024-05-09T06:38:12.710 INFO:tasks.workunit.client.0.smithi063.stdout:0/545: dwrite dd/fe [0,4194304] 0 2024-05-09T06:38:12.711 INFO:tasks.workunit.client.1.smithi193.stdout:9/201: creat dd/df/f47 x:0 0 0 2024-05-09T06:38:12.715 INFO:tasks.workunit.client.0.smithi063.stdout:9/514: rmdir d0/d4/d57/d60/d76 39 2024-05-09T06:38:12.718 INFO:tasks.workunit.client.0.smithi063.stdout:0/546: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d46/fa7 [0,4194304] 0 2024-05-09T06:38:12.723 INFO:tasks.workunit.client.0.smithi063.stdout:1/466: rename d9/d10/d2b to d9/d2a/d35/d1e/d9d 0 2024-05-09T06:38:12.723 INFO:tasks.workunit.client.0.smithi063.stdout:1/467: chown d9/d10/f15 867 1 2024-05-09T06:38:12.727 INFO:tasks.workunit.client.1.smithi193.stdout:1/169: rmdir d4/d16/d17 39 2024-05-09T06:38:12.729 INFO:tasks.workunit.client.1.smithi193.stdout:3/105: getdents d1/db 0 2024-05-09T06:38:12.732 INFO:tasks.workunit.client.1.smithi193.stdout:9/202: stat dd/d1e/d1f/l2e 0 2024-05-09T06:38:12.733 INFO:tasks.workunit.client.0.smithi063.stdout:4/506: mknod d7/df/c9f 0 2024-05-09T06:38:12.735 INFO:tasks.workunit.client.0.smithi063.stdout:3/527: symlink d7/da/d2b/la5 0 2024-05-09T06:38:12.735 INFO:tasks.workunit.client.0.smithi063.stdout:3/528: fdatasync d7/da/fb 0 2024-05-09T06:38:12.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/489: link d0/d15/d49/d4f/d66/d67/l7a d0/d15/d49/d54/d9f/d97/lad 0 2024-05-09T06:38:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:12 smithi193 ceph-mon[35141]: pgmap v9: 65 pgs: 65 active+clean; 1.3 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 181 MiB/s wr, 365 op/s 2024-05-09T06:38:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:12 smithi193 ceph-mon[35141]: from='client.24465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:12.741 INFO:tasks.workunit.client.1.smithi193.stdout:9/203: rmdir dd/d1e 39 2024-05-09T06:38:12.741 INFO:tasks.workunit.client.0.smithi063.stdout:2/547: dwrite dc/dd/f23 [0,4194304] 0 2024-05-09T06:38:12.745 INFO:tasks.workunit.client.1.smithi193.stdout:9/204: dread f0 [4194304,4194304] 0 2024-05-09T06:38:12.749 INFO:tasks.workunit.client.0.smithi063.stdout:0/547: creat dd/fb2 x:0 0 0 2024-05-09T06:38:12.752 INFO:tasks.workunit.client.0.smithi063.stdout:2/548: dread dc/dd/d45/f89 [0,4194304] 0 2024-05-09T06:38:12.753 INFO:tasks.workunit.client.0.smithi063.stdout:0/548: dread dd/d1a/f35 [0,4194304] 0 2024-05-09T06:38:12.755 INFO:tasks.workunit.client.1.smithi193.stdout:5/223: getdents d0/d1/da/d3a 0 2024-05-09T06:38:12.755 INFO:tasks.workunit.client.1.smithi193.stdout:5/224: chown d0/d1/f21 1 1 2024-05-09T06:38:12.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:12 smithi063 ceph-mon[27171]: pgmap v9: 65 pgs: 65 active+clean; 1.3 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 181 MiB/s wr, 365 op/s 2024-05-09T06:38:12.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:12 smithi063 ceph-mon[27171]: from='client.24465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:12.760 INFO:tasks.workunit.client.0.smithi063.stdout:2/549: dwrite dc/f1b [4194304,4194304] 0 2024-05-09T06:38:12.761 INFO:tasks.workunit.client.0.smithi063.stdout:2/550: write dc/f1c [2096637,57382] 0 2024-05-09T06:38:12.763 INFO:tasks.workunit.client.0.smithi063.stdout:3/529: rename d7/d70/d98/d5b/f83 to d7/da/d2b/d66/d74/d7e/fa6 0 2024-05-09T06:38:12.763 INFO:tasks.workunit.client.1.smithi193.stdout:8/179: getdents d6/d1c/d27 0 2024-05-09T06:38:12.764 INFO:tasks.workunit.client.1.smithi193.stdout:8/180: chown d6/d1c/d18/l22 1 1 2024-05-09T06:38:12.765 INFO:tasks.workunit.client.0.smithi063.stdout:7/490: creat d0/d15/d49/d54/d9f/d97/fae x:0 0 0 2024-05-09T06:38:12.767 INFO:tasks.workunit.client.0.smithi063.stdout:7/491: truncate d0/d15/f58 222160 0 2024-05-09T06:38:12.768 INFO:tasks.workunit.client.0.smithi063.stdout:6/520: getdents d2/df/d59/d99 0 2024-05-09T06:38:12.770 INFO:tasks.workunit.client.1.smithi193.stdout:1/170: link d4/d14/c19 d4/c2c 0 2024-05-09T06:38:12.770 INFO:tasks.workunit.client.0.smithi063.stdout:6/521: write d2/d1f/d29/d3f/d5c/d64/f6f [884074,74554] 0 2024-05-09T06:38:12.773 INFO:tasks.workunit.client.0.smithi063.stdout:9/515: unlink d0/d8/c51 0 2024-05-09T06:38:12.774 INFO:tasks.workunit.client.1.smithi193.stdout:5/225: dwrite d0/f8 [0,4194304] 0 2024-05-09T06:38:12.774 INFO:tasks.workunit.client.1.smithi193.stdout:9/205: mkdir dd/d1e/d1f/d48 0 2024-05-09T06:38:12.775 INFO:tasks.workunit.client.1.smithi193.stdout:3/106: write d1/f9 [1654519,65342] 0 2024-05-09T06:38:12.776 INFO:tasks.workunit.client.1.smithi193.stdout:3/107: readlink d1/db/l21 0 2024-05-09T06:38:12.781 INFO:tasks.workunit.client.1.smithi193.stdout:1/171: rename d4/f11 to d4/d16/de/f2d 0 2024-05-09T06:38:12.782 INFO:tasks.workunit.client.1.smithi193.stdout:5/226: readlink d0/d13/d2c/d37/l42 0 2024-05-09T06:38:12.784 INFO:tasks.workunit.client.0.smithi063.stdout:8/569: creat d9/da/d18/fc4 x:0 0 0 2024-05-09T06:38:12.785 INFO:tasks.workunit.client.1.smithi193.stdout:9/206: mkdir dd/df/d15/d16/d39/d49 0 2024-05-09T06:38:12.786 INFO:tasks.workunit.client.0.smithi063.stdout:0/549: symlink dd/d1a/d2c/lb3 0 2024-05-09T06:38:12.790 INFO:tasks.workunit.client.0.smithi063.stdout:4/507: creat d7/df/d1f/d28/d31/d71/d98/fa0 x:0 0 0 2024-05-09T06:38:12.792 INFO:tasks.workunit.client.1.smithi193.stdout:3/108: symlink d1/d20/l28 0 2024-05-09T06:38:12.793 INFO:tasks.workunit.client.1.smithi193.stdout:3/109: write d1/db/f18 [2415597,33651] 0 2024-05-09T06:38:12.801 INFO:tasks.workunit.client.1.smithi193.stdout:2/161: sync 2024-05-09T06:38:12.802 INFO:tasks.workunit.client.1.smithi193.stdout:5/227: rmdir d0/d1 39 2024-05-09T06:38:12.808 INFO:tasks.workunit.client.0.smithi063.stdout:9/516: chown d0/d4/d26/f34 1784720508 1 2024-05-09T06:38:12.809 INFO:tasks.workunit.client.1.smithi193.stdout:3/110: creat d1/d20/f29 x:0 0 0 2024-05-09T06:38:12.809 INFO:tasks.workunit.client.0.smithi063.stdout:1/468: link d9/d2a/d35/d1e/c37 d9/d2a/d39/d5d/c9e 0 2024-05-09T06:38:12.809 INFO:tasks.workunit.client.1.smithi193.stdout:3/111: truncate d1/f8 4460815 0 2024-05-09T06:38:12.811 INFO:tasks.workunit.client.0.smithi063.stdout:8/570: mkdir d9/da/d18/d56/d58/d79/dc5 0 2024-05-09T06:38:12.813 INFO:tasks.workunit.client.0.smithi063.stdout:0/550: symlink dd/d1a/d50/lb4 0 2024-05-09T06:38:12.814 INFO:tasks.workunit.client.0.smithi063.stdout:4/508: creat d7/d50/fa1 x:0 0 0 2024-05-09T06:38:12.816 INFO:tasks.workunit.client.0.smithi063.stdout:2/551: creat dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72/d7b/f9c x:0 0 0 2024-05-09T06:38:12.817 INFO:tasks.workunit.client.0.smithi063.stdout:2/552: write fa [3150263,102890] 0 2024-05-09T06:38:12.817 INFO:tasks.workunit.client.1.smithi193.stdout:1/172: creat d4/d14/f2e x:0 0 0 2024-05-09T06:38:12.818 INFO:tasks.workunit.client.0.smithi063.stdout:2/553: chown dc/d36/d43/d46/d63/d79 6265707 1 2024-05-09T06:38:12.818 INFO:tasks.workunit.client.0.smithi063.stdout:2/554: chown dc/dd 41229847 1 2024-05-09T06:38:12.820 INFO:tasks.workunit.client.0.smithi063.stdout:9/517: mkdir d0/d8/da6/db0/db5 0 2024-05-09T06:38:12.821 INFO:tasks.workunit.client.0.smithi063.stdout:1/469: mknod d9/d46/d57/c9f 0 2024-05-09T06:38:12.824 INFO:tasks.workunit.client.0.smithi063.stdout:0/551: mknod dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/cb5 0 2024-05-09T06:38:12.826 INFO:tasks.workunit.client.1.smithi193.stdout:1/173: dread d4/d16/f10 [0,4194304] 0 2024-05-09T06:38:12.828 INFO:tasks.workunit.client.1.smithi193.stdout:2/162: creat db/f39 x:0 0 0 2024-05-09T06:38:12.828 INFO:tasks.workunit.client.0.smithi063.stdout:7/492: write d0/d15/d49/f5f [914989,73242] 0 2024-05-09T06:38:12.833 INFO:tasks.workunit.client.1.smithi193.stdout:8/181: dread d6/d25/f2c [0,4194304] 0 2024-05-09T06:38:12.833 INFO:tasks.workunit.client.0.smithi063.stdout:9/518: rmdir d0/d4/d26/d39/d42 39 2024-05-09T06:38:12.834 INFO:tasks.workunit.client.1.smithi193.stdout:8/182: fdatasync d6/d1c/d1b/f2b 0 2024-05-09T06:38:12.834 INFO:tasks.workunit.client.1.smithi193.stdout:3/112: link d1/db/c1a d1/db/d26/c2a 0 2024-05-09T06:38:12.838 INFO:tasks.workunit.client.0.smithi063.stdout:0/552: creat dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fb6 x:0 0 0 2024-05-09T06:38:12.842 INFO:tasks.workunit.client.0.smithi063.stdout:3/530: getdents d7/da/d21/d30 0 2024-05-09T06:38:12.844 INFO:tasks.workunit.client.1.smithi193.stdout:2/163: dwrite db/df/d12/f32 [0,4194304] 0 2024-05-09T06:38:12.844 INFO:tasks.workunit.client.0.smithi063.stdout:7/493: mknod d0/d15/d49/d54/d9f/d97/d9b/d9d/caf 0 2024-05-09T06:38:12.846 INFO:tasks.workunit.client.0.smithi063.stdout:4/509: dwrite d7/df/f21 [4194304,4194304] 0 2024-05-09T06:38:12.847 INFO:tasks.workunit.client.0.smithi063.stdout:4/510: stat d7/f39 0 2024-05-09T06:38:12.848 INFO:tasks.workunit.client.1.smithi193.stdout:3/113: rmdir d1 39 2024-05-09T06:38:12.850 INFO:tasks.workunit.client.0.smithi063.stdout:9/519: mknod d0/d4/d26/d47/cb6 0 2024-05-09T06:38:12.851 INFO:tasks.workunit.client.1.smithi193.stdout:8/183: dread d6/d1c/d18/f1d [0,4194304] 0 2024-05-09T06:38:12.852 INFO:tasks.workunit.client.1.smithi193.stdout:8/184: dread d6/fd [0,4194304] 0 2024-05-09T06:38:12.853 INFO:tasks.workunit.client.1.smithi193.stdout:8/185: readlink d6/d1c/d18/l2a 0 2024-05-09T06:38:12.854 INFO:tasks.workunit.client.1.smithi193.stdout:8/186: fdatasync d6/d1c/f12 0 2024-05-09T06:38:12.854 INFO:tasks.workunit.client.0.smithi063.stdout:2/555: truncate dc/f2a 197317 0 2024-05-09T06:38:12.856 INFO:tasks.workunit.client.0.smithi063.stdout:8/571: rename d9/d3b/l35 to d9/da/d18/d56/d58/d79/lc6 0 2024-05-09T06:38:12.856 INFO:tasks.workunit.client.0.smithi063.stdout:2/556: truncate dc/f25 4762284 0 2024-05-09T06:38:12.857 INFO:tasks.workunit.client.0.smithi063.stdout:2/557: stat dc/f99 0 2024-05-09T06:38:12.857 INFO:tasks.workunit.client.0.smithi063.stdout:0/553: mknod dd/d60/d44/d90/cb7 0 2024-05-09T06:38:12.858 INFO:tasks.workunit.client.0.smithi063.stdout:8/572: write d9/da/d18/d56/f91 [168562,123349] 0 2024-05-09T06:38:12.859 INFO:tasks.workunit.client.0.smithi063.stdout:3/531: symlink d7/d70/d98/la7 0 2024-05-09T06:38:12.860 INFO:tasks.workunit.client.1.smithi193.stdout:3/114: write d1/f4 [5139838,113336] 0 2024-05-09T06:38:12.861 INFO:tasks.workunit.client.1.smithi193.stdout:1/174: link d4/fc d4/d14/f2f 0 2024-05-09T06:38:12.862 INFO:tasks.workunit.client.1.smithi193.stdout:1/175: read - d4/d14/f1b zero size 2024-05-09T06:38:12.862 INFO:tasks.workunit.client.1.smithi193.stdout:5/228: link d0/d13/d2c/d37/d2b/c46 d0/c4c 0 2024-05-09T06:38:12.864 INFO:tasks.workunit.client.1.smithi193.stdout:1/176: read d4/d16/f27 [360078,69389] 0 2024-05-09T06:38:12.866 INFO:tasks.workunit.client.0.smithi063.stdout:3/532: dwrite d7/da/d2b/d66/d74/d7e/f99 [0,4194304] 0 2024-05-09T06:38:12.867 INFO:tasks.workunit.client.1.smithi193.stdout:3/115: dwrite d1/db/f18 [0,4194304] 0 2024-05-09T06:38:12.867 INFO:tasks.workunit.client.0.smithi063.stdout:3/533: fdatasync d7/da/d21/d30/f9e 0 2024-05-09T06:38:12.868 INFO:tasks.workunit.client.0.smithi063.stdout:7/494: symlink d0/d15/d4a/lb0 0 2024-05-09T06:38:12.869 INFO:tasks.workunit.client.0.smithi063.stdout:7/495: write d0/d15/d49/f65 [3609044,14580] 0 2024-05-09T06:38:12.885 INFO:tasks.workunit.client.0.smithi063.stdout:9/520: creat d0/d4/da/d9d/fb7 x:0 0 0 2024-05-09T06:38:12.886 INFO:tasks.workunit.client.0.smithi063.stdout:9/521: dread - d0/d4/d57/f9b zero size 2024-05-09T06:38:12.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.906+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 --> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1c8100510 con 0x7fc1b806d6f0 2024-05-09T06:38:12.887 INFO:tasks.workunit.client.1.smithi193.stdout:3/116: symlink d1/l2b 0 2024-05-09T06:38:12.888 INFO:tasks.workunit.client.0.smithi063.stdout:1/470: link d9/d2a/d39/l69 d9/d2a/d35/d1e/d9d/la0 0 2024-05-09T06:38:12.889 INFO:tasks.workunit.client.0.smithi063.stdout:1/471: chown d9/d46/d57/f65 0 1 2024-05-09T06:38:12.889 INFO:tasks.workunit.client.0.smithi063.stdout:3/534: rmdir d7/da/d2b 39 2024-05-09T06:38:12.890 INFO:tasks.workunit.client.1.smithi193.stdout:5/229: dread d0/d13/f3d [0,4194304] 0 2024-05-09T06:38:12.890 INFO:tasks.workunit.client.0.smithi063.stdout:3/535: readlink d7/l1f 0 2024-05-09T06:38:12.891 INFO:tasks.workunit.client.1.smithi193.stdout:8/187: creat d6/d1c/f34 x:0 0 0 2024-05-09T06:38:12.892 INFO:tasks.workunit.client.1.smithi193.stdout:2/164: getdents db/d25 0 2024-05-09T06:38:12.892 INFO:tasks.workunit.client.0.smithi063.stdout:1/472: dread d9/d2a/d35/f8f [0,4194304] 0 2024-05-09T06:38:12.893 INFO:tasks.workunit.client.0.smithi063.stdout:9/522: dwrite d0/d8/d2d/faf [0,4194304] 0 2024-05-09T06:38:12.893 INFO:tasks.workunit.client.1.smithi193.stdout:1/177: rmdir d4/d16/d17/d1a 39 2024-05-09T06:38:12.896 INFO:tasks.workunit.client.0.smithi063.stdout:0/554: truncate dd/d1a/d2c/d51/d72/d9e/d23/d33/f3e 353677 0 2024-05-09T06:38:12.897 INFO:tasks.workunit.client.0.smithi063.stdout:0/555: chown dd/d1a/f65 0 1 2024-05-09T06:38:12.897 INFO:tasks.workunit.client.1.smithi193.stdout:5/230: mkdir d0/d13/d2c/d4d 0 2024-05-09T06:38:12.897 INFO:tasks.workunit.client.0.smithi063.stdout:0/556: fdatasync dd/fb2 0 2024-05-09T06:38:12.898 INFO:tasks.workunit.client.0.smithi063.stdout:9/523: dread d0/d8/f69 [4194304,4194304] 0 2024-05-09T06:38:12.899 INFO:tasks.workunit.client.0.smithi063.stdout:9/524: chown d0/d8/da6/db0 521 1 2024-05-09T06:38:12.899 INFO:tasks.workunit.client.0.smithi063.stdout:0/557: read f2 [661639,53891] 0 2024-05-09T06:38:12.900 INFO:tasks.workunit.client.0.smithi063.stdout:9/525: chown d0/d4/c4f 155972852 1 2024-05-09T06:38:12.900 INFO:tasks.workunit.client.0.smithi063.stdout:9/526: chown d0/d4/d26/d39/d91 54950269 1 2024-05-09T06:38:12.902 INFO:tasks.workunit.client.1.smithi193.stdout:2/165: chown db/f35 4461331 1 2024-05-09T06:38:12.904 INFO:tasks.workunit.client.0.smithi063.stdout:2/558: rename dc/d32/f62 to dc/f9d 0 2024-05-09T06:38:12.906 INFO:tasks.workunit.client.1.smithi193.stdout:1/178: rmdir d4/d16 39 2024-05-09T06:38:12.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.925+0000 7fc1c5ffb700 1 -- 172.21.15.63:0/2715859383 <== mgr.24415 v2:172.21.15.193:6828/1432929717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc1c8100510 con 0x7fc1b806d6f0 2024-05-09T06:38:12.906 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:38:12.906 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (7m) 2m ago 8m 25.3M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:38:12.906 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (8m) 2m ago 8m 7465k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:38:12.906 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (7m) 20s ago 7m 7642k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (8m) 2m ago 8m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (7m) 20s ago 7m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6m) 2m ago 8m 87.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (3m) 2m ago 3m 11.7M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (2m) 2m ago 2m 14.7M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (3m) 20s ago 2m 142M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (2m) 20s ago 2m 14.3M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:9283,8765,8443 running (10m) 2m ago 10m 500M - 18.2.2-1297-g1758e8ea 196422f1b641 a66e4acc828f 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (38s) 20s ago 7m 96.9M - 19.0.0-2515-g96805dcf d7acac7d5657 b3fcf25fc9f0 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (10m) 2m ago 10m 45.4M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 7d88727e4bbb 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (7m) 20s ago 7m 40.2M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (8m) 2m ago 8m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (7m) 20s ago 7m 16.0M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (6m) 2m ago 6m 47.2M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:38:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (6m) 2m ago 6m 48.3M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:38:12.908 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (5m) 2m ago 5m 46.4M 3946M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:38:12.908 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (5m) 20s ago 5m 113M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:38:12.908 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (5m) 20s ago 5m 113M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:38:12.908 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (4m) 20s ago 4m 108M 4970M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:38:12.908 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 2m ago 8m 58.2M - 2.43.0 a07b618ecd1d 9f4deb8cc8b4 2024-05-09T06:38:12.909 INFO:tasks.workunit.client.1.smithi193.stdout:3/117: mknod d1/db/d26/c2c 0 2024-05-09T06:38:12.909 INFO:tasks.workunit.client.1.smithi193.stdout:2/166: dread db/d25/f33 [0,4194304] 0 2024-05-09T06:38:12.909 INFO:tasks.workunit.client.1.smithi193.stdout:2/167: dread - db/df/d12/f14 zero size 2024-05-09T06:38:12.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fc1b806d6f0 msgr2=0x7fc1b806fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fc1b806d6f0 0x7fc1b806fbb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009c60 tx=0x7fc1bc009260 comp rx=0 tx=0).stop 2024-05-09T06:38:12.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 msgr2=0x7fc1c80ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:12.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 0x7fc1c80ff6a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400b800 tx=0x7fc1b4005f00 comp rx=0 tx=0).stop 2024-05-09T06:38:12.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 shutdown_connections 2024-05-09T06:38:12.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fc1b806d6f0 0x7fc1b806fbb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c81016f0 0x7fc1c8101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 --2- 172.21.15.63:0/2715859383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c8104020 0x7fc1c80ff6a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:12.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 >> 172.21.15.63:0/2715859383 conn(0x7fc1c80fb0c0 msgr2=0x7fc1c80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:12.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 shutdown_connections 2024-05-09T06:38:12.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:12.928+0000 7fc1cebce700 1 -- 172.21.15.63:0/2715859383 wait complete. 2024-05-09T06:38:12.910 INFO:tasks.workunit.client.1.smithi193.stdout:3/118: truncate d1/db/f14 482621 0 2024-05-09T06:38:12.911 INFO:tasks.workunit.client.1.smithi193.stdout:2/168: truncate db/f35 502679 0 2024-05-09T06:38:12.912 INFO:tasks.workunit.client.0.smithi063.stdout:8/573: write d9/da/f2f [2881210,80355] 0 2024-05-09T06:38:12.913 INFO:tasks.workunit.client.1.smithi193.stdout:1/179: fdatasync d4/f9 0 2024-05-09T06:38:12.918 INFO:tasks.workunit.client.1.smithi193.stdout:8/188: write d6/d1c/d18/f1d [1328996,89272] 0 2024-05-09T06:38:12.918 INFO:tasks.workunit.client.1.smithi193.stdout:3/119: dwrite d1/db/f14 [0,4194304] 0 2024-05-09T06:38:12.919 INFO:tasks.workunit.client.1.smithi193.stdout:8/189: readlink d6/d25/l32 0 2024-05-09T06:38:12.919 INFO:tasks.workunit.client.0.smithi063.stdout:1/473: unlink f3 0 2024-05-09T06:38:12.920 INFO:tasks.workunit.client.0.smithi063.stdout:1/474: stat d9/d2a/d35/d24 0 2024-05-09T06:38:12.921 INFO:tasks.workunit.client.1.smithi193.stdout:2/169: symlink db/d25/l3a 0 2024-05-09T06:38:12.922 INFO:tasks.workunit.client.1.smithi193.stdout:2/170: write f8 [8804140,39875] 0 2024-05-09T06:38:12.927 INFO:tasks.workunit.client.0.smithi063.stdout:4/511: getdents d7/df/d1f/d28 0 2024-05-09T06:38:12.928 INFO:tasks.workunit.client.1.smithi193.stdout:2/171: dread db/df/f20 [0,4194304] 0 2024-05-09T06:38:12.929 INFO:tasks.workunit.client.1.smithi193.stdout:8/190: dwrite d6/d1c/d1b/f2b [0,4194304] 0 2024-05-09T06:38:12.931 INFO:tasks.workunit.client.1.smithi193.stdout:3/120: rmdir d1 39 2024-05-09T06:38:12.931 INFO:tasks.workunit.client.1.smithi193.stdout:5/231: dread d0/f9 [0,4194304] 0 2024-05-09T06:38:12.932 INFO:tasks.workunit.client.1.smithi193.stdout:5/232: chown d0/d13/d2c 3090111 1 2024-05-09T06:38:12.933 INFO:tasks.workunit.client.0.smithi063.stdout:0/558: unlink dd/f2d 0 2024-05-09T06:38:12.933 INFO:tasks.workunit.client.1.smithi193.stdout:5/233: chown d0/d13/d2c/f30 3769 1 2024-05-09T06:38:12.936 INFO:tasks.workunit.client.0.smithi063.stdout:2/559: rename dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d87 to dc/d36/d43/d46/d63/d9e 0 2024-05-09T06:38:12.937 INFO:tasks.workunit.client.1.smithi193.stdout:8/191: dwrite d6/f23 [0,4194304] 0 2024-05-09T06:38:12.999 INFO:tasks.workunit.client.1.smithi193.stdout:4/178: sync 2024-05-09T06:38:12.999 INFO:tasks.workunit.client.1.smithi193.stdout:0/202: sync 2024-05-09T06:38:13.000 INFO:tasks.workunit.client.1.smithi193.stdout:0/203: write d18/d1d/f30 [1067404,92574] 0 2024-05-09T06:38:13.017 INFO:tasks.workunit.client.1.smithi193.stdout:4/179: dread d1a/f23 [0,4194304] 0 2024-05-09T06:38:13.038 INFO:tasks.workunit.client.0.smithi063.stdout:5/476: sync 2024-05-09T06:38:13.041 INFO:tasks.workunit.client.1.smithi193.stdout:4/180: dread f5 [0,4194304] 0 2024-05-09T06:38:13.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.063+0000 7f18cc604700 1 -- 172.21.15.63:0/3345802750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18c4100d30 msgr2=0x7f18c4101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.063+0000 7f18cc604700 1 --2- 172.21.15.63:0/3345802750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18c4100d30 0x7f18c4101190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009910 tx=0x7f18b4009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:13.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.063+0000 7f18cc604700 1 -- 172.21.15.63:0/3345802750 shutdown_connections 2024-05-09T06:38:13.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.063+0000 7f18cc604700 1 --2- 172.21.15.63:0/3345802750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18c4100d30 0x7f18c4101190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.063+0000 7f18cc604700 1 --2- 172.21.15.63:0/3345802750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18c40ffb30 0x7f18c40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.063+0000 7f18cc604700 1 -- 172.21.15.63:0/3345802750 >> 172.21.15.63:0/3345802750 conn(0x7f18c40fb0d0 msgr2=0x7f18c40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:13.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.063+0000 7f18cc604700 1 -- 172.21.15.63:0/3345802750 shutdown_connections 2024-05-09T06:38:13.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.063+0000 7f18cc604700 1 -- 172.21.15.63:0/3345802750 wait complete. 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.064+0000 7f18cc604700 1 Processor -- start 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.064+0000 7f18cc604700 1 -- start start 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.064+0000 7f18cc604700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18c40ffb30 0x7f18c4069be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.064+0000 7f18cc604700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18c4100d30 0x7f18c419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.064+0000 7f18cc604700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419b910 con 0x7f18c4100d30 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.064+0000 7f18cc604700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419ba50 con 0x7f18c40ffb30 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.065+0000 7f18ca3a0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18c40ffb30 0x7f18c4069be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.065+0000 7f18c9b9f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18c4100d30 0x7f18c419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.065+0000 7f18c9b9f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18c4100d30 0x7f18c419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56036/0 (socket says 172.21.15.63:56036) 2024-05-09T06:38:13.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.065+0000 7f18c9b9f700 1 -- 172.21.15.63:0/1247040273 learned_addr learned my addr 172.21.15.63:0/1247040273 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.065+0000 7f18ca3a0700 1 -- 172.21.15.63:0/1247040273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18c4100d30 msgr2=0x7f18c419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.065+0000 7f18ca3a0700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18c4100d30 0x7f18c419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.065+0000 7f18ca3a0700 1 -- 172.21.15.63:0/1247040273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40095f0 con 0x7f18c40ffb30 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.065+0000 7f18ca3a0700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18c40ffb30 0x7f18c4069be0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18c000e940 tx=0x7f18c000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.065+0000 7f18bb7fe700 1 -- 172.21.15.63:0/1247040273 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c00041f0 con 0x7f18c40ffb30 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.066+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c419bce0 con 0x7f18c40ffb30 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.066+0000 7f18bb7fe700 1 -- 172.21.15.63:0/1247040273 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18c0004350 con 0x7f18c40ffb30 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.066+0000 7f18bb7fe700 1 -- 172.21.15.63:0/1247040273 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c0003c90 con 0x7f18c40ffb30 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.066+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c419c250 con 0x7f18c40ffb30 2024-05-09T06:38:13.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.067+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c418f6b0 con 0x7f18c40ffb30 2024-05-09T06:38:13.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.068+0000 7f18bb7fe700 1 -- 172.21.15.63:0/1247040273 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f18c000c7c0 con 0x7f18c40ffb30 2024-05-09T06:38:13.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.069+0000 7f18bb7fe700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f18b006d730 0x7f18b006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.069+0000 7f18bb7fe700 1 -- 172.21.15.63:0/1247040273 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f18c0014070 con 0x7f18c40ffb30 2024-05-09T06:38:13.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.069+0000 7f18c9b9f700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f18b006d730 0x7f18b006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.069+0000 7f18c9b9f700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f18b006d730 0x7f18b006fbf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18c406a1a0 tx=0x7f18b400b770 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:13.054 INFO:tasks.workunit.client.1.smithi193.stdout:3/121: readlink d1/l15 0 2024-05-09T06:38:13.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.075+0000 7f18bb7fe700 1 -- 172.21.15.63:0/1247040273 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f18c005a730 con 0x7f18c40ffb30 2024-05-09T06:38:13.056 INFO:tasks.workunit.client.0.smithi063.stdout:7/496: rmdir d0/daa 0 2024-05-09T06:38:13.056 INFO:tasks.workunit.client.0.smithi063.stdout:8/574: read d9/d3b/f40 [2075841,36439] 0 2024-05-09T06:38:13.062 INFO:tasks.workunit.client.0.smithi063.stdout:1/475: creat d9/d2a/d35/d1e/fa1 x:0 0 0 2024-05-09T06:38:13.063 INFO:tasks.workunit.client.1.smithi193.stdout:8/192: symlink d6/d1c/d18/l35 0 2024-05-09T06:38:13.063 INFO:tasks.workunit.client.1.smithi193.stdout:8/193: stat d6/fd 0 2024-05-09T06:38:13.071 INFO:tasks.workunit.client.1.smithi193.stdout:4/181: mkdir d1a/d24/d37 0 2024-05-09T06:38:13.075 INFO:tasks.workunit.client.1.smithi193.stdout:3/122: mkdir d1/d2d 0 2024-05-09T06:38:13.076 INFO:tasks.workunit.client.0.smithi063.stdout:7/497: dread d0/f10 [0,4194304] 0 2024-05-09T06:38:13.080 INFO:tasks.workunit.client.1.smithi193.stdout:3/123: dread f0 [4194304,4194304] 0 2024-05-09T06:38:13.082 INFO:tasks.workunit.client.0.smithi063.stdout:3/536: creat d7/fa8 x:0 0 0 2024-05-09T06:38:13.085 INFO:tasks.workunit.client.1.smithi193.stdout:3/124: dwrite d1/db/f18 [0,4194304] 0 2024-05-09T06:38:13.086 INFO:tasks.workunit.client.1.smithi193.stdout:7/197: sync 2024-05-09T06:38:13.086 INFO:tasks.workunit.client.1.smithi193.stdout:0/204: creat d18/d29/f3f x:0 0 0 2024-05-09T06:38:13.086 INFO:tasks.workunit.client.0.smithi063.stdout:5/477: dwrite d8/d17/d18/f2d [0,4194304] 0 2024-05-09T06:38:13.089 INFO:tasks.workunit.client.0.smithi063.stdout:6/522: sync 2024-05-09T06:38:13.089 INFO:tasks.workunit.client.0.smithi063.stdout:9/527: sync 2024-05-09T06:38:13.089 INFO:tasks.workunit.client.1.smithi193.stdout:4/182: write d1a/f1b [7676545,120498] 0 2024-05-09T06:38:13.091 INFO:tasks.workunit.client.1.smithi193.stdout:4/183: write d1a/d1f/f33 [485025,47142] 0 2024-05-09T06:38:13.091 INFO:tasks.workunit.client.0.smithi063.stdout:6/523: write f1 [2618595,82199] 0 2024-05-09T06:38:13.093 INFO:tasks.workunit.client.0.smithi063.stdout:6/524: read - d2/d20/d4a/f94 zero size 2024-05-09T06:38:13.093 INFO:tasks.workunit.client.0.smithi063.stdout:9/528: read d0/d4/d57/f64 [1165237,21275] 0 2024-05-09T06:38:13.096 INFO:tasks.workunit.client.1.smithi193.stdout:5/234: mkdir d0/d1/d4e 0 2024-05-09T06:38:13.098 INFO:tasks.workunit.client.1.smithi193.stdout:4/184: dwrite fa [4194304,4194304] 0 2024-05-09T06:38:13.099 INFO:tasks.workunit.client.0.smithi063.stdout:5/478: dread d8/d17/f95 [0,4194304] 0 2024-05-09T06:38:13.100 INFO:tasks.workunit.client.0.smithi063.stdout:5/479: truncate d8/f8e 444072 0 2024-05-09T06:38:13.107 INFO:tasks.workunit.client.1.smithi193.stdout:4/185: dwrite d1a/d24/d30/f34 [0,4194304] 0 2024-05-09T06:38:13.108 INFO:tasks.workunit.client.1.smithi193.stdout:4/186: chown d1a/f1b 32110002 1 2024-05-09T06:38:13.116 INFO:tasks.workunit.client.1.smithi193.stdout:2/172: getdents db/df 0 2024-05-09T06:38:13.117 INFO:tasks.workunit.client.1.smithi193.stdout:2/173: write db/df/d12/f14 [517355,2472] 0 2024-05-09T06:38:13.125 INFO:tasks.workunit.client.1.smithi193.stdout:1/180: link d4/c2c d4/d16/d17/c30 0 2024-05-09T06:38:13.126 INFO:tasks.workunit.client.0.smithi063.stdout:3/537: rename f5 to d7/d4c/d31/fa9 0 2024-05-09T06:38:13.127 INFO:tasks.workunit.client.0.smithi063.stdout:3/538: fdatasync d7/d70/d98/d2d/f65 0 2024-05-09T06:38:13.129 INFO:tasks.workunit.client.0.smithi063.stdout:5/480: rmdir d8/d19 39 2024-05-09T06:38:13.129 INFO:tasks.workunit.client.0.smithi063.stdout:3/539: write d7/d22/d28/f3c [2473920,63432] 0 2024-05-09T06:38:13.129 INFO:tasks.workunit.client.1.smithi193.stdout:8/194: link d6/d1c/c33 d6/d25/c36 0 2024-05-09T06:38:13.135 INFO:tasks.workunit.client.0.smithi063.stdout:7/498: symlink d0/d15/d4a/lb1 0 2024-05-09T06:38:13.136 INFO:tasks.workunit.client.1.smithi193.stdout:4/187: symlink d1a/l38 0 2024-05-09T06:38:13.140 INFO:tasks.workunit.client.0.smithi063.stdout:8/575: rename d9/da/d18/d56 to d9/da/d18/dc7 0 2024-05-09T06:38:13.142 INFO:tasks.workunit.client.0.smithi063.stdout:9/529: mknod d0/cb8 0 2024-05-09T06:38:13.143 INFO:tasks.workunit.client.0.smithi063.stdout:6/525: getdents d2/d4/d7c/d42 0 2024-05-09T06:38:13.145 INFO:tasks.workunit.client.1.smithi193.stdout:2/174: symlink db/df/d12/l3b 0 2024-05-09T06:38:13.149 INFO:tasks.workunit.client.0.smithi063.stdout:3/540: creat d7/d4c/d31/faa x:0 0 0 2024-05-09T06:38:13.150 INFO:tasks.workunit.client.1.smithi193.stdout:8/195: creat d6/d1c/d18/f37 x:0 0 0 2024-05-09T06:38:13.150 INFO:tasks.workunit.client.1.smithi193.stdout:8/196: readlink d6/d1c/d18/l2a 0 2024-05-09T06:38:13.152 INFO:tasks.workunit.client.1.smithi193.stdout:9/207: sync 2024-05-09T06:38:13.152 INFO:tasks.workunit.client.1.smithi193.stdout:6/163: sync 2024-05-09T06:38:13.153 INFO:tasks.workunit.client.1.smithi193.stdout:9/208: fdatasync dd/d1e/f28 0 2024-05-09T06:38:13.153 INFO:tasks.workunit.client.1.smithi193.stdout:6/164: truncate d1/de/f2f 883703 0 2024-05-09T06:38:13.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/188: rename l16 to d1a/d24/d37/l39 0 2024-05-09T06:38:13.156 INFO:tasks.workunit.client.0.smithi063.stdout:9/530: rename d0/d4/c46 to d0/da5/cb9 0 2024-05-09T06:38:13.158 INFO:tasks.workunit.client.0.smithi063.stdout:8/576: dwrite d9/d3b/fbd [0,4194304] 0 2024-05-09T06:38:13.161 INFO:tasks.workunit.client.0.smithi063.stdout:3/541: mkdir d7/d22/d28/dab 0 2024-05-09T06:38:13.163 INFO:tasks.workunit.client.0.smithi063.stdout:7/499: mknod d0/d15/d4a/cb2 0 2024-05-09T06:38:13.164 INFO:tasks.workunit.client.0.smithi063.stdout:7/500: fsync d0/f10 0 2024-05-09T06:38:13.166 INFO:tasks.workunit.client.1.smithi193.stdout:6/165: symlink d1/de/d11/d18/l33 0 2024-05-09T06:38:13.167 INFO:tasks.workunit.client.1.smithi193.stdout:6/166: stat d1/de/d11/d20 0 2024-05-09T06:38:13.168 INFO:tasks.workunit.client.1.smithi193.stdout:4/189: mknod d1a/d24/c3a 0 2024-05-09T06:38:13.171 INFO:tasks.workunit.client.1.smithi193.stdout:2/175: creat db/d25/d38/f3c x:0 0 0 2024-05-09T06:38:13.172 INFO:tasks.workunit.client.0.smithi063.stdout:6/526: fdatasync d2/d4/f92 0 2024-05-09T06:38:13.175 INFO:tasks.workunit.client.1.smithi193.stdout:0/205: symlink d18/l40 0 2024-05-09T06:38:13.176 INFO:tasks.workunit.client.0.smithi063.stdout:8/577: mknod d9/da/d34/cc8 0 2024-05-09T06:38:13.177 INFO:tasks.workunit.client.1.smithi193.stdout:6/167: write d1/de/d11/d18/f1c [3630787,101117] 0 2024-05-09T06:38:13.177 INFO:tasks.workunit.client.0.smithi063.stdout:0/559: write dd/d1a/f25 [1519115,32955] 0 2024-05-09T06:38:13.178 INFO:tasks.workunit.client.0.smithi063.stdout:5/481: symlink d8/d19/d30/la0 0 2024-05-09T06:38:13.181 INFO:tasks.workunit.client.0.smithi063.stdout:5/482: chown d8/d35/f41 121132526 1 2024-05-09T06:38:13.182 INFO:tasks.workunit.client.0.smithi063.stdout:6/527: dwrite d2/d1f/d29/d3f/d5c/d64/f8d [0,4194304] 0 2024-05-09T06:38:13.182 INFO:tasks.workunit.client.0.smithi063.stdout:0/560: truncate dd/d60/d44/f69 966266 0 2024-05-09T06:38:13.182 INFO:tasks.workunit.client.1.smithi193.stdout:2/176: unlink db/d25/d38/f3c 0 2024-05-09T06:38:13.183 INFO:tasks.workunit.client.1.smithi193.stdout:8/197: link d6/d25/l32 d6/l38 0 2024-05-09T06:38:13.183 INFO:tasks.workunit.client.1.smithi193.stdout:6/168: dwrite d1/de/ff [0,4194304] 0 2024-05-09T06:38:13.184 INFO:tasks.workunit.client.0.smithi063.stdout:2/560: link dc/dd/d2d/c5d dc/d36/d43/c9f 0 2024-05-09T06:38:13.184 INFO:tasks.workunit.client.0.smithi063.stdout:5/483: stat d8/d17/d1e/f23 0 2024-05-09T06:38:13.185 INFO:tasks.workunit.client.0.smithi063.stdout:3/542: dread d7/da/d21/f4d [0,4194304] 0 2024-05-09T06:38:13.186 INFO:tasks.workunit.client.1.smithi193.stdout:0/206: creat d18/d27/f41 x:0 0 0 2024-05-09T06:38:13.186 INFO:tasks.workunit.client.0.smithi063.stdout:6/528: dread - d2/d1f/d6c/f7d zero size 2024-05-09T06:38:13.187 INFO:tasks.workunit.client.1.smithi193.stdout:0/207: write d18/d29/f38 [66296,103477] 0 2024-05-09T06:38:13.190 INFO:tasks.workunit.client.1.smithi193.stdout:4/190: link d1a/d24/f2e d1a/d24/f3b 0 2024-05-09T06:38:13.191 INFO:tasks.workunit.client.1.smithi193.stdout:4/191: dread f12 [0,4194304] 0 2024-05-09T06:38:13.194 INFO:tasks.workunit.client.0.smithi063.stdout:0/561: truncate dd/d1a/d2c/d51/d72/d9e/d23/d31/fa2 866359 0 2024-05-09T06:38:13.195 INFO:tasks.workunit.client.0.smithi063.stdout:8/578: creat d9/da/d18/dc7/d8f/fc9 x:0 0 0 2024-05-09T06:38:13.196 INFO:tasks.workunit.client.0.smithi063.stdout:0/562: dread - dd/d1a/d50/f86 zero size 2024-05-09T06:38:13.197 INFO:tasks.workunit.client.1.smithi193.stdout:1/181: rmdir d4/d14 39 2024-05-09T06:38:13.202 INFO:tasks.workunit.client.1.smithi193.stdout:6/169: rename d1/f5 to d1/de/d11/d2d/f34 0 2024-05-09T06:38:13.205 INFO:tasks.workunit.client.0.smithi063.stdout:3/543: creat d7/d4c/d25/fac x:0 0 0 2024-05-09T06:38:13.209 INFO:tasks.workunit.client.0.smithi063.stdout:4/512: mkdir d7/df/d1f/da2 0 2024-05-09T06:38:13.210 INFO:tasks.workunit.client.0.smithi063.stdout:4/513: dread - d7/df/d1f/d28/d31/f94 zero size 2024-05-09T06:38:13.211 INFO:tasks.workunit.client.0.smithi063.stdout:4/514: readlink d7/df/d1f/l4f 0 2024-05-09T06:38:13.211 INFO:tasks.workunit.client.1.smithi193.stdout:4/192: link d1a/d24/f2e d1a/d24/d37/f3c 0 2024-05-09T06:38:13.211 INFO:tasks.workunit.client.1.smithi193.stdout:4/193: stat fb 0 2024-05-09T06:38:13.212 INFO:tasks.workunit.client.1.smithi193.stdout:4/194: write d1a/d1f/f2c [262353,52728] 0 2024-05-09T06:38:13.213 INFO:tasks.workunit.client.1.smithi193.stdout:1/182: mknod d4/d16/d17/c31 0 2024-05-09T06:38:13.219 INFO:tasks.workunit.client.0.smithi063.stdout:5/484: mknod d8/d65/d9e/ca1 0 2024-05-09T06:38:13.220 INFO:tasks.workunit.client.1.smithi193.stdout:4/195: dwrite d1a/f36 [0,4194304] 0 2024-05-09T06:38:13.220 INFO:tasks.workunit.client.0.smithi063.stdout:5/485: read - d8/d17/d1e/f5f zero size 2024-05-09T06:38:13.221 INFO:tasks.workunit.client.1.smithi193.stdout:9/209: link dd/df/d15/d16/c25 dd/c4a 0 2024-05-09T06:38:13.223 INFO:tasks.workunit.client.0.smithi063.stdout:8/579: dread d9/f70 [0,4194304] 0 2024-05-09T06:38:13.225 INFO:tasks.workunit.client.0.smithi063.stdout:7/501: getdents d0/d15 0 2024-05-09T06:38:13.225 INFO:tasks.workunit.client.1.smithi193.stdout:6/170: dread d1/de/d11/d2d/f34 [0,4194304] 0 2024-05-09T06:38:13.226 INFO:tasks.workunit.client.1.smithi193.stdout:6/171: fdatasync d1/de/f13 0 2024-05-09T06:38:13.227 INFO:tasks.workunit.client.1.smithi193.stdout:4/196: creat d1a/d24/f3d x:0 0 0 2024-05-09T06:38:13.229 INFO:tasks.workunit.client.1.smithi193.stdout:9/210: creat dd/df/d15/d16/d26/f4b x:0 0 0 2024-05-09T06:38:13.231 INFO:tasks.workunit.client.0.smithi063.stdout:7/502: dwrite d0/d15/f58 [0,4194304] 0 2024-05-09T06:38:13.231 INFO:tasks.workunit.client.1.smithi193.stdout:9/211: write dd/df/d15/d16/d26/f35 [85510,59993] 0 2024-05-09T06:38:13.232 INFO:tasks.workunit.client.1.smithi193.stdout:1/183: rename d4/d16/de/f2d to d4/f32 0 2024-05-09T06:38:13.233 INFO:tasks.workunit.client.1.smithi193.stdout:7/198: dwrite d6/f13 [0,4194304] 0 2024-05-09T06:38:13.234 INFO:tasks.workunit.client.1.smithi193.stdout:7/199: chown d6/d8/d1a/f27 7674 1 2024-05-09T06:38:13.235 INFO:tasks.workunit.client.1.smithi193.stdout:7/200: fdatasync d6/dd/f3a 0 2024-05-09T06:38:13.237 INFO:tasks.workunit.client.1.smithi193.stdout:6/172: mknod d1/de/d11/d2d/c35 0 2024-05-09T06:38:13.239 INFO:tasks.workunit.client.0.smithi063.stdout:6/529: link d2/d20/d26/l43 d2/d20/d95/la8 0 2024-05-09T06:38:13.241 INFO:tasks.workunit.client.0.smithi063.stdout:6/530: write d2/d1f/d6c/f71 [353066,40797] 0 2024-05-09T06:38:13.241 INFO:tasks.workunit.client.1.smithi193.stdout:3/125: truncate d1/db/f14 1658179 0 2024-05-09T06:38:13.243 INFO:tasks.workunit.client.0.smithi063.stdout:6/531: chown d2/df/la7 146391702 1 2024-05-09T06:38:13.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/184: truncate d4/d14/f2e 196663 0 2024-05-09T06:38:13.245 INFO:tasks.workunit.client.0.smithi063.stdout:1/476: link d9/d2a/d35/d1e/d9d/f82 d9/d2a/fa2 0 2024-05-09T06:38:13.246 INFO:tasks.workunit.client.1.smithi193.stdout:9/212: creat dd/df/d31/f4c x:0 0 0 2024-05-09T06:38:13.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/177: rmdir db/df/d12 39 2024-05-09T06:38:13.250 INFO:tasks.workunit.client.1.smithi193.stdout:5/235: truncate d0/d13/f3d 632516 0 2024-05-09T06:38:13.251 INFO:tasks.workunit.client.1.smithi193.stdout:7/201: chown d6/d8/c2b 245766 1 2024-05-09T06:38:13.252 INFO:tasks.workunit.client.0.smithi063.stdout:9/531: mkdir d0/d4/d57/dba 0 2024-05-09T06:38:13.255 INFO:tasks.workunit.client.0.smithi063.stdout:6/532: dread d2/f38 [0,4194304] 0 2024-05-09T06:38:13.257 INFO:tasks.workunit.client.0.smithi063.stdout:6/533: chown d2/d20/d4a/f93 125 1 2024-05-09T06:38:13.257 INFO:tasks.workunit.client.1.smithi193.stdout:3/126: mknod d1/db/c2e 0 2024-05-09T06:38:13.258 INFO:tasks.workunit.client.1.smithi193.stdout:3/127: read - d1/f24 zero size 2024-05-09T06:38:13.259 INFO:tasks.workunit.client.0.smithi063.stdout:7/503: write d0/d15/d49/d4f/d66/f71 [152350,96548] 0 2024-05-09T06:38:13.259 INFO:tasks.workunit.client.1.smithi193.stdout:1/185: fdatasync d4/d14/d15/f18 0 2024-05-09T06:38:13.266 INFO:tasks.workunit.client.1.smithi193.stdout:2/178: dread f1 [4194304,4194304] 0 2024-05-09T06:38:13.266 INFO:tasks.workunit.client.1.smithi193.stdout:7/202: symlink d6/d8/d1a/d36/l45 0 2024-05-09T06:38:13.267 INFO:tasks.workunit.client.0.smithi063.stdout:8/580: symlink d9/da/d18/d24/lca 0 2024-05-09T06:38:13.273 INFO:tasks.workunit.client.0.smithi063.stdout:4/515: rmdir d7/d50/d85 0 2024-05-09T06:38:13.274 INFO:tasks.workunit.client.1.smithi193.stdout:3/128: chown d1/db/c1f 35437 1 2024-05-09T06:38:13.275 INFO:tasks.workunit.client.1.smithi193.stdout:0/208: rmdir d18/d27 39 2024-05-09T06:38:13.276 INFO:tasks.workunit.client.0.smithi063.stdout:9/532: truncate d0/d8/f69 2662483 0 2024-05-09T06:38:13.276 INFO:tasks.workunit.client.1.smithi193.stdout:0/209: write d18/f28 [3936260,102869] 0 2024-05-09T06:38:13.277 INFO:tasks.workunit.client.0.smithi063.stdout:0/563: getdents dd/d1a/d2c/d51/d72/d9e/d23/d33/d48 0 2024-05-09T06:38:13.279 INFO:tasks.workunit.client.0.smithi063.stdout:4/516: dwrite d7/f39 [0,4194304] 0 2024-05-09T06:38:13.280 INFO:tasks.workunit.client.0.smithi063.stdout:0/564: chown dd/d60/d44/d90/faa 44277106 1 2024-05-09T06:38:13.281 INFO:tasks.workunit.client.0.smithi063.stdout:2/561: link dc/f2a dc/dd/d2d/fa0 0 2024-05-09T06:38:13.282 INFO:tasks.workunit.client.1.smithi193.stdout:5/236: getdents d0/d1/da/d3a 0 2024-05-09T06:38:13.285 INFO:tasks.workunit.client.0.smithi063.stdout:2/562: readlink dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72/d90/l71 0 2024-05-09T06:38:13.285 INFO:tasks.workunit.client.1.smithi193.stdout:2/179: rename db/df/c1a to db/d25/c3d 0 2024-05-09T06:38:13.286 INFO:tasks.workunit.client.0.smithi063.stdout:7/504: unlink d0/d15/d49/d54/d9f/d97/d9b/d9d/caf 0 2024-05-09T06:38:13.288 INFO:tasks.workunit.client.1.smithi193.stdout:4/197: getdents d1a/d24/d30 0 2024-05-09T06:38:13.289 INFO:tasks.workunit.client.0.smithi063.stdout:8/581: write d9/da/d18/f1d [2362914,19214] 0 2024-05-09T06:38:13.289 INFO:tasks.workunit.client.1.smithi193.stdout:4/198: write d1a/f28 [769598,43594] 0 2024-05-09T06:38:13.290 INFO:tasks.workunit.client.1.smithi193.stdout:4/199: readlink d1a/d1f/l2f 0 2024-05-09T06:38:13.290 INFO:tasks.workunit.client.1.smithi193.stdout:3/129: read f0 [1022824,39956] 0 2024-05-09T06:38:13.294 INFO:tasks.workunit.client.1.smithi193.stdout:9/213: link dd/d1e/d1f/f21 dd/df/d15/d16/d26/d37/f4d 0 2024-05-09T06:38:13.297 INFO:tasks.workunit.client.1.smithi193.stdout:1/186: rmdir d4 39 2024-05-09T06:38:13.299 INFO:tasks.workunit.client.0.smithi063.stdout:0/565: rename dd/l1f to dd/d60/d44/d90/da4/lb8 0 2024-05-09T06:38:13.299 INFO:tasks.workunit.client.1.smithi193.stdout:8/198: readlink d6/d25/l32 0 2024-05-09T06:38:13.300 INFO:tasks.workunit.client.0.smithi063.stdout:2/563: creat dc/d6e/d7f/fa1 x:0 0 0 2024-05-09T06:38:13.302 INFO:tasks.workunit.client.0.smithi063.stdout:2/564: fdatasync dc/d6e/d7f/fa1 0 2024-05-09T06:38:13.302 INFO:tasks.workunit.client.0.smithi063.stdout:4/517: dwrite d7/df/f2f [0,4194304] 0 2024-05-09T06:38:13.303 INFO:tasks.workunit.client.0.smithi063.stdout:7/505: mknod d0/d15/d49/d4f/d66/d67/cb3 0 2024-05-09T06:38:13.304 INFO:tasks.workunit.client.0.smithi063.stdout:7/506: chown d0/d15/d89/d98/l99 185561 1 2024-05-09T06:38:13.305 INFO:tasks.workunit.client.1.smithi193.stdout:3/130: mkdir d1/db/d2f 0 2024-05-09T06:38:13.306 INFO:tasks.workunit.client.0.smithi063.stdout:8/582: creat d9/da/d53/fcb x:0 0 0 2024-05-09T06:38:13.310 INFO:tasks.workunit.client.1.smithi193.stdout:3/131: dread d1/db/f18 [0,4194304] 0 2024-05-09T06:38:13.312 INFO:tasks.workunit.client.0.smithi063.stdout:9/533: mknod d0/d3b/cbb 0 2024-05-09T06:38:13.316 INFO:tasks.workunit.client.0.smithi063.stdout:6/534: creat d2/fa9 x:0 0 0 2024-05-09T06:38:13.316 INFO:tasks.workunit.client.1.smithi193.stdout:7/203: rename d6/f24 to d6/d8/d1a/d33/f46 0 2024-05-09T06:38:13.317 INFO:tasks.workunit.client.1.smithi193.stdout:3/132: dread f0 [0,4194304] 0 2024-05-09T06:38:13.317 INFO:tasks.workunit.client.0.smithi063.stdout:6/535: dread - d2/d1f/d29/d3f/d5c/f9e zero size 2024-05-09T06:38:13.318 INFO:tasks.workunit.client.0.smithi063.stdout:5/486: write d8/d17/f8d [1498742,65927] 0 2024-05-09T06:38:13.320 INFO:tasks.workunit.client.0.smithi063.stdout:5/487: fdatasync d8/d19/d30/d46/f6c 0 2024-05-09T06:38:13.322 INFO:tasks.workunit.client.1.smithi193.stdout:1/187: dwrite d4/d16/d17/f29 [0,4194304] 0 2024-05-09T06:38:13.324 INFO:tasks.workunit.client.1.smithi193.stdout:6/173: truncate d1/de/ff 1644869 0 2024-05-09T06:38:13.325 INFO:tasks.workunit.client.0.smithi063.stdout:7/507: write d0/d15/d89/fa9 [805830,46183] 0 2024-05-09T06:38:13.325 INFO:tasks.workunit.client.0.smithi063.stdout:7/508: stat d0/d15/d4a/d50/c44 0 2024-05-09T06:38:13.327 INFO:tasks.workunit.client.1.smithi193.stdout:9/214: creat dd/d3f/d41/f4e x:0 0 0 2024-05-09T06:38:13.332 INFO:tasks.workunit.client.1.smithi193.stdout:6/174: dwrite d1/de/d11/d18/f24 [0,4194304] 0 2024-05-09T06:38:13.334 INFO:tasks.workunit.client.0.smithi063.stdout:1/477: getdents d9/d2a/d35/d1e/d9d/d3f 0 2024-05-09T06:38:13.340 INFO:tasks.workunit.client.1.smithi193.stdout:3/133: symlink d1/d20/l30 0 2024-05-09T06:38:13.345 INFO:tasks.workunit.client.1.smithi193.stdout:4/200: write d1a/d24/d37/f3c [1791477,119472] 0 2024-05-09T06:38:13.346 INFO:tasks.workunit.client.1.smithi193.stdout:4/201: chown d1a/d22/c35 108 1 2024-05-09T06:38:13.347 INFO:tasks.workunit.client.1.smithi193.stdout:4/202: truncate d1a/d24/f3d 902406 0 2024-05-09T06:38:13.348 INFO:tasks.workunit.client.0.smithi063.stdout:2/565: fsync f6 0 2024-05-09T06:38:13.350 INFO:tasks.workunit.client.1.smithi193.stdout:9/215: symlink dd/df/d15/l4f 0 2024-05-09T06:38:13.354 INFO:tasks.workunit.client.1.smithi193.stdout:0/210: rename d18/d1d/c36 to d18/d29/c42 0 2024-05-09T06:38:13.356 INFO:tasks.workunit.client.1.smithi193.stdout:0/211: write d18/d29/f2b [489797,79015] 0 2024-05-09T06:38:13.356 INFO:tasks.workunit.client.1.smithi193.stdout:0/212: fsync f9 0 2024-05-09T06:38:13.359 INFO:tasks.workunit.client.0.smithi063.stdout:5/488: dread d8/d65/f80 [4194304,4194304] 0 2024-05-09T06:38:13.359 INFO:tasks.workunit.client.1.smithi193.stdout:6/175: fsync d1/de/d11/d18/f1e 0 2024-05-09T06:38:13.361 INFO:tasks.workunit.client.1.smithi193.stdout:3/134: symlink d1/l31 0 2024-05-09T06:38:13.362 INFO:tasks.workunit.client.1.smithi193.stdout:3/135: chown d1/l15 24348 1 2024-05-09T06:38:13.363 INFO:tasks.workunit.client.0.smithi063.stdout:8/583: rename d9/d37/c61 to d9/d37/ccc 0 2024-05-09T06:38:13.366 INFO:tasks.workunit.client.0.smithi063.stdout:1/478: symlink d9/d2a/d3c/la3 0 2024-05-09T06:38:13.367 INFO:tasks.workunit.client.1.smithi193.stdout:8/199: creat d6/d1c/d18/d2f/f39 x:0 0 0 2024-05-09T06:38:13.370 INFO:tasks.workunit.client.0.smithi063.stdout:3/544: sync 2024-05-09T06:38:13.370 INFO:tasks.workunit.client.0.smithi063.stdout:6/536: mknod d2/df/d59/da3/caa 0 2024-05-09T06:38:13.372 INFO:tasks.workunit.client.0.smithi063.stdout:0/566: write dd/d1a/d2c/f92 [723900,60620] 0 2024-05-09T06:38:13.373 INFO:tasks.workunit.client.1.smithi193.stdout:4/203: creat d1a/d1f/f3e x:0 0 0 2024-05-09T06:38:13.373 INFO:tasks.workunit.client.0.smithi063.stdout:0/567: write dd/d60/fa0 [182123,68137] 0 2024-05-09T06:38:13.376 INFO:tasks.workunit.client.1.smithi193.stdout:4/204: dread d1a/d1f/f33 [0,4194304] 0 2024-05-09T06:38:13.379 INFO:tasks.workunit.client.1.smithi193.stdout:9/216: creat dd/df/d15/d16/f50 x:0 0 0 2024-05-09T06:38:13.380 INFO:tasks.workunit.client.0.smithi063.stdout:4/518: truncate d7/df/d1f/d28/d31/d61/f6d 304671 0 2024-05-09T06:38:13.383 INFO:tasks.workunit.client.1.smithi193.stdout:5/237: rename d0/d13/d2c/d37/f36 to d0/d13/d2c/f4f 0 2024-05-09T06:38:13.384 INFO:tasks.workunit.client.1.smithi193.stdout:7/204: dwrite d6/d8/d1a/f3c [0,4194304] 0 2024-05-09T06:38:13.386 INFO:tasks.workunit.client.1.smithi193.stdout:4/205: dread d1a/f1b [4194304,4194304] 0 2024-05-09T06:38:13.389 INFO:tasks.workunit.client.1.smithi193.stdout:6/176: mkdir d1/de/d11/d18/d36 0 2024-05-09T06:38:13.389 INFO:tasks.workunit.client.1.smithi193.stdout:5/238: truncate d0/d13/d2c/d38/f3e 393989 0 2024-05-09T06:38:13.389 INFO:tasks.workunit.client.0.smithi063.stdout:0/568: dread dd/f1d [0,4194304] 0 2024-05-09T06:38:13.390 INFO:tasks.workunit.client.1.smithi193.stdout:6/177: chown d1/de/d11/d20/f27 28 1 2024-05-09T06:38:13.392 INFO:tasks.workunit.client.1.smithi193.stdout:8/200: mkdir d6/d1c/d1b/d3a 0 2024-05-09T06:38:13.393 INFO:tasks.workunit.client.0.smithi063.stdout:9/534: creat d0/d3b/fbc x:0 0 0 2024-05-09T06:38:13.395 INFO:tasks.workunit.client.0.smithi063.stdout:3/545: chown d7/da/d2b/f43 126801277 1 2024-05-09T06:38:13.396 INFO:tasks.workunit.client.0.smithi063.stdout:3/546: readlink d7/da/d2b/d66/d74/d7e/l5c 0 2024-05-09T06:38:13.403 INFO:tasks.workunit.client.0.smithi063.stdout:5/489: write d8/d19/d30/f61 [353668,9844] 0 2024-05-09T06:38:13.405 INFO:tasks.workunit.client.0.smithi063.stdout:7/509: creat d0/d15/d4a/fb4 x:0 0 0 2024-05-09T06:38:13.408 INFO:tasks.workunit.client.1.smithi193.stdout:7/205: read - d6/d17/f43 zero size 2024-05-09T06:38:13.409 INFO:tasks.workunit.client.0.smithi063.stdout:1/479: dread d9/f12 [4194304,4194304] 0 2024-05-09T06:38:13.410 INFO:tasks.workunit.client.0.smithi063.stdout:0/569: creat dd/d1a/d2c/d51/d72/d9e/d23/d31/fb9 x:0 0 0 2024-05-09T06:38:13.413 INFO:tasks.workunit.client.0.smithi063.stdout:6/537: symlink d2/d1f/d73/d88/d98/lab 0 2024-05-09T06:38:13.413 INFO:tasks.workunit.client.0.smithi063.stdout:6/538: stat d2/f7 0 2024-05-09T06:38:13.415 INFO:tasks.workunit.client.0.smithi063.stdout:3/547: symlink d7/d70/lad 0 2024-05-09T06:38:13.415 INFO:tasks.workunit.client.0.smithi063.stdout:9/535: dread d0/d4/da/f1c [0,4194304] 0 2024-05-09T06:38:13.416 INFO:tasks.workunit.client.0.smithi063.stdout:3/548: chown d7/d4c/d25/fac 0 1 2024-05-09T06:38:13.418 INFO:tasks.workunit.client.0.smithi063.stdout:2/566: truncate dc/dd/d2d/f77 700588 0 2024-05-09T06:38:13.419 INFO:tasks.workunit.client.1.smithi193.stdout:4/206: dwrite f7 [0,4194304] 0 2024-05-09T06:38:13.420 INFO:tasks.workunit.client.0.smithi063.stdout:7/510: dread d0/d15/f9a [0,4194304] 0 2024-05-09T06:38:13.422 INFO:tasks.workunit.client.1.smithi193.stdout:1/188: getdents d4/d16/de 0 2024-05-09T06:38:13.423 INFO:tasks.workunit.client.0.smithi063.stdout:4/519: readlink d7/d3d/d73/l82 0 2024-05-09T06:38:13.425 INFO:tasks.workunit.client.1.smithi193.stdout:4/207: dread f13 [0,4194304] 0 2024-05-09T06:38:13.425 INFO:tasks.workunit.client.0.smithi063.stdout:8/584: creat d9/da/d51/dc2/fcd x:0 0 0 2024-05-09T06:38:13.429 INFO:tasks.workunit.client.1.smithi193.stdout:4/208: dread f10 [0,4194304] 0 2024-05-09T06:38:13.429 INFO:tasks.workunit.client.1.smithi193.stdout:9/217: creat dd/df/d15/d16/d39/d49/f51 x:0 0 0 2024-05-09T06:38:13.430 INFO:tasks.workunit.client.0.smithi063.stdout:9/536: dread d0/d4/da/f21 [0,4194304] 0 2024-05-09T06:38:13.431 INFO:tasks.workunit.client.0.smithi063.stdout:0/570: mkdir dd/d1a/d2c/dba 0 2024-05-09T06:38:13.432 INFO:tasks.workunit.client.0.smithi063.stdout:0/571: chown dd/d1a/d2c/d51/d72/d9e/d23 249917 1 2024-05-09T06:38:13.433 INFO:tasks.workunit.client.0.smithi063.stdout:6/539: creat d2/d1f/d29/d3b/fac x:0 0 0 2024-05-09T06:38:13.435 INFO:tasks.workunit.client.0.smithi063.stdout:9/537: dwrite d0/f71 [0,4194304] 0 2024-05-09T06:38:13.436 INFO:tasks.workunit.client.1.smithi193.stdout:7/206: dread - d6/dd/f40 zero size 2024-05-09T06:38:13.438 INFO:tasks.workunit.client.1.smithi193.stdout:7/207: write d6/d8/f44 [748847,120960] 0 2024-05-09T06:38:13.439 INFO:tasks.workunit.client.0.smithi063.stdout:2/567: readlink dc/dd/l4a 0 2024-05-09T06:38:13.440 INFO:tasks.workunit.client.0.smithi063.stdout:5/490: write d8/d17/f56 [189475,124086] 0 2024-05-09T06:38:13.441 INFO:tasks.workunit.client.0.smithi063.stdout:2/568: write dc/d36/d43/d46/d63/d79/d7d/f84 [5922986,27969] 0 2024-05-09T06:38:13.445 INFO:tasks.workunit.client.1.smithi193.stdout:6/178: mkdir d1/de/d11/d18/d36/d37 0 2024-05-09T06:38:13.445 INFO:tasks.workunit.client.0.smithi063.stdout:9/538: dread d0/d4/d26/f33 [0,4194304] 0 2024-05-09T06:38:13.446 INFO:tasks.workunit.client.0.smithi063.stdout:9/539: write d0/d8/d2d/f7b [508090,129948] 0 2024-05-09T06:38:13.450 INFO:tasks.workunit.client.0.smithi063.stdout:7/511: fsync d0/d15/d49/f4c 0 2024-05-09T06:38:13.450 INFO:tasks.workunit.client.1.smithi193.stdout:1/189: fsync d4/f32 0 2024-05-09T06:38:13.451 INFO:tasks.workunit.client.0.smithi063.stdout:7/512: write d0/d15/d4a/d68/f9e [296895,108609] 0 2024-05-09T06:38:13.451 INFO:tasks.workunit.client.0.smithi063.stdout:8/585: chown d9/da/d18/d24/d71/l74 31348354 1 2024-05-09T06:38:13.452 INFO:tasks.workunit.client.0.smithi063.stdout:7/513: fsync d0/d15/f48 0 2024-05-09T06:38:13.454 INFO:tasks.workunit.client.0.smithi063.stdout:1/480: creat d9/d2a/d35/d1e/d9d/d3f/d59/d62/d7e/fa4 x:0 0 0 2024-05-09T06:38:13.457 INFO:tasks.workunit.client.1.smithi193.stdout:2/180: rename db/df/l2b to db/l3e 0 2024-05-09T06:38:13.459 INFO:tasks.workunit.client.0.smithi063.stdout:9/540: dread d0/d4/d26/d47/f2b [0,4194304] 0 2024-05-09T06:38:13.459 INFO:tasks.workunit.client.0.smithi063.stdout:9/541: stat d0/d4/d57/f64 0 2024-05-09T06:38:13.460 INFO:tasks.workunit.client.1.smithi193.stdout:0/213: getdents d18 0 2024-05-09T06:38:13.460 INFO:tasks.workunit.client.1.smithi193.stdout:8/201: truncate d6/d1c/d1b/f2b 2665826 0 2024-05-09T06:38:13.460 INFO:tasks.workunit.client.1.smithi193.stdout:0/214: truncate d18/d29/f3f 979825 0 2024-05-09T06:38:13.461 INFO:tasks.workunit.client.1.smithi193.stdout:7/208: creat d6/dd/f47 x:0 0 0 2024-05-09T06:38:13.463 INFO:tasks.workunit.client.1.smithi193.stdout:5/239: link d0/d13/d2c/d37/l42 d0/d13/d2c/d4d/l50 0 2024-05-09T06:38:13.464 INFO:tasks.workunit.client.1.smithi193.stdout:0/215: dread f17 [0,4194304] 0 2024-05-09T06:38:13.465 INFO:tasks.workunit.client.0.smithi063.stdout:5/491: mkdir d8/d17/da2 0 2024-05-09T06:38:13.466 INFO:tasks.workunit.client.1.smithi193.stdout:1/190: mknod d4/d14/d15/c33 0 2024-05-09T06:38:13.468 INFO:tasks.workunit.client.1.smithi193.stdout:9/218: creat dd/d1e/d1f/d48/f52 x:0 0 0 2024-05-09T06:38:13.470 INFO:tasks.workunit.client.1.smithi193.stdout:3/136: rename d1/c25 to d1/d2d/c32 0 2024-05-09T06:38:13.470 INFO:tasks.workunit.client.1.smithi193.stdout:2/181: rmdir db/df 39 2024-05-09T06:38:13.472 INFO:tasks.workunit.client.1.smithi193.stdout:8/202: chown d6/l16 190082 1 2024-05-09T06:38:13.472 INFO:tasks.workunit.client.1.smithi193.stdout:8/203: dread - d6/d1c/d18/d2f/f39 zero size 2024-05-09T06:38:13.473 INFO:tasks.workunit.client.0.smithi063.stdout:8/586: creat d9/da/d18/dc7/d8f/fce x:0 0 0 2024-05-09T06:38:13.474 INFO:tasks.workunit.client.1.smithi193.stdout:7/209: mknod d6/d8/d1a/d33/c48 0 2024-05-09T06:38:13.475 INFO:tasks.workunit.client.1.smithi193.stdout:7/210: chown d6/dd/f40 426003 1 2024-05-09T06:38:13.476 INFO:tasks.workunit.client.0.smithi063.stdout:1/481: creat d9/d2a/d35/d1e/d9d/d3f/d59/d62/fa5 x:0 0 0 2024-05-09T06:38:13.477 INFO:tasks.workunit.client.0.smithi063.stdout:0/572: mkdir dd/d1a/d2c/d51/d72/d9e/d23/dbb 0 2024-05-09T06:38:13.477 INFO:tasks.workunit.client.0.smithi063.stdout:0/573: stat dd/c84 0 2024-05-09T06:38:13.477 INFO:tasks.workunit.client.0.smithi063.stdout:0/574: stat dd/d1a/l20 0 2024-05-09T06:38:13.479 INFO:tasks.workunit.client.1.smithi193.stdout:0/216: mknod d18/d29/c43 0 2024-05-09T06:38:13.482 INFO:tasks.workunit.client.0.smithi063.stdout:6/540: link d2/d1f/d29/d3f/f25 d2/d1f/d73/d88/fad 0 2024-05-09T06:38:13.483 INFO:tasks.workunit.client.0.smithi063.stdout:6/541: chown d2/d1f/d29/d3b/fac 16284 1 2024-05-09T06:38:13.488 INFO:tasks.workunit.client.1.smithi193.stdout:9/219: dread f0 [4194304,4194304] 0 2024-05-09T06:38:13.491 INFO:tasks.workunit.client.0.smithi063.stdout:4/520: rename d7/df/d1f to d7/da3 0 2024-05-09T06:38:13.492 INFO:tasks.workunit.client.0.smithi063.stdout:4/521: fdatasync d7/df/d77/f8e 0 2024-05-09T06:38:13.495 INFO:tasks.workunit.client.0.smithi063.stdout:8/587: mknod d9/da/d18/dc7/d58/ccf 0 2024-05-09T06:38:13.496 INFO:tasks.workunit.client.0.smithi063.stdout:8/588: dread - d9/da/d18/d46/f97 zero size 2024-05-09T06:38:13.496 INFO:tasks.workunit.client.0.smithi063.stdout:8/589: dread - d9/da/f9a zero size 2024-05-09T06:38:13.497 INFO:tasks.workunit.client.0.smithi063.stdout:1/482: creat d9/d2a/d35/d24/fa6 x:0 0 0 2024-05-09T06:38:13.498 INFO:tasks.workunit.client.1.smithi193.stdout:7/211: symlink d6/d8/d1a/d36/l49 0 2024-05-09T06:38:13.498 INFO:tasks.workunit.client.1.smithi193.stdout:7/212: dread - d6/d8/d1a/d33/f3e zero size 2024-05-09T06:38:13.501 INFO:tasks.workunit.client.1.smithi193.stdout:5/240: fsync d0/d1/da/f15 0 2024-05-09T06:38:13.502 INFO:tasks.workunit.client.0.smithi063.stdout:9/542: write d0/d3b/da0/f9c [902949,4595] 0 2024-05-09T06:38:13.504 INFO:tasks.workunit.client.1.smithi193.stdout:5/241: read d0/d13/d2c/d38/f3e [193688,120106] 0 2024-05-09T06:38:13.504 INFO:tasks.workunit.client.0.smithi063.stdout:3/549: getdents d7 0 2024-05-09T06:38:13.505 INFO:tasks.workunit.client.1.smithi193.stdout:4/209: getdents d1a/d1f 0 2024-05-09T06:38:13.506 INFO:tasks.workunit.client.1.smithi193.stdout:5/242: fsync d0/d13/d2c/d38/f3e 0 2024-05-09T06:38:13.506 INFO:tasks.workunit.client.1.smithi193.stdout:4/210: stat f12 0 2024-05-09T06:38:13.507 INFO:tasks.workunit.client.1.smithi193.stdout:6/179: dwrite d1/de/ff [0,4194304] 0 2024-05-09T06:38:13.507 INFO:tasks.workunit.client.0.smithi063.stdout:6/542: mknod d2/d4/d7c/d7e/cae 0 2024-05-09T06:38:13.509 INFO:tasks.workunit.client.0.smithi063.stdout:5/492: write d8/f4c [430702,129157] 0 2024-05-09T06:38:13.510 INFO:tasks.workunit.client.0.smithi063.stdout:2/569: creat dc/dd/d2d/d60/fa2 x:0 0 0 2024-05-09T06:38:13.511 INFO:tasks.workunit.client.1.smithi193.stdout:3/137: creat d1/db/d2f/f33 x:0 0 0 2024-05-09T06:38:13.515 INFO:tasks.workunit.client.1.smithi193.stdout:2/182: rename db/df/l36 to db/d37/l3f 0 2024-05-09T06:38:13.517 INFO:tasks.workunit.client.0.smithi063.stdout:7/514: creat d0/d15/d4a/fb5 x:0 0 0 2024-05-09T06:38:13.518 INFO:tasks.workunit.client.1.smithi193.stdout:8/204: fsync d6/fd 0 2024-05-09T06:38:13.519 INFO:tasks.workunit.client.0.smithi063.stdout:8/590: creat d9/da/d53/db2/fd0 x:0 0 0 2024-05-09T06:38:13.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/575: dwrite dd/d1a/f35 [0,4194304] 0 2024-05-09T06:38:13.523 INFO:tasks.workunit.client.1.smithi193.stdout:5/243: write d0/fc [493565,54688] 0 2024-05-09T06:38:13.524 INFO:tasks.workunit.client.0.smithi063.stdout:3/550: unlink d7/da/d21/d30/f8a 0 2024-05-09T06:38:13.525 INFO:tasks.workunit.client.0.smithi063.stdout:3/551: dread - d7/d4c/d31/faa zero size 2024-05-09T06:38:13.525 INFO:tasks.workunit.client.1.smithi193.stdout:4/211: mkdir d1a/d24/d3f 0 2024-05-09T06:38:13.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.549+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f18c404ebb0 con 0x7f18c40ffb30 2024-05-09T06:38:13.529 INFO:tasks.workunit.client.0.smithi063.stdout:5/493: mkdir d8/d17/da3 0 2024-05-09T06:38:13.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.550+0000 7f18bb7fe700 1 -- 172.21.15.63:0/1247040273 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+841 (secure 0 0 0) 0x7f18c005a2c0 con 0x7f18c40ffb30 2024-05-09T06:38:13.530 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 1, 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 1 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:38:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:38:13.532 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 13, 2024-05-09T06:38:13.532 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 1 2024-05-09T06:38:13.532 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:38:13.532 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:38:13.532 INFO:tasks.workunit.client.1.smithi193.stdout:6/180: creat d1/de/f38 x:0 0 0 2024-05-09T06:38:13.532 INFO:tasks.workunit.client.1.smithi193.stdout:6/181: read - d1/fd zero size 2024-05-09T06:38:13.532 INFO:tasks.workunit.client.1.smithi193.stdout:6/182: fsync d1/de/f10 0 2024-05-09T06:38:13.533 INFO:tasks.workunit.client.0.smithi063.stdout:2/570: mkdir dc/d36/d43/d46/d63/d79/da3 0 2024-05-09T06:38:13.533 INFO:tasks.workunit.client.1.smithi193.stdout:1/191: link d4/c7 d4/d16/de/c34 0 2024-05-09T06:38:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.553+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f18b006d730 msgr2=0x7f18b006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.553+0000 7f18cc604700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f18b006d730 0x7f18b006fbf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18c406a1a0 tx=0x7f18b400b770 comp rx=0 tx=0).stop 2024-05-09T06:38:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.553+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18c40ffb30 msgr2=0x7f18c4069be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.553+0000 7f18cc604700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18c40ffb30 0x7f18c4069be0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18c000e940 tx=0x7f18c000ec50 comp rx=0 tx=0).stop 2024-05-09T06:38:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.553+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 shutdown_connections 2024-05-09T06:38:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.553+0000 7f18cc604700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f18b006d730 0x7f18b006fbf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.553+0000 7f18cc604700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18c40ffb30 0x7f18c4069be0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.553+0000 7f18cc604700 1 --2- 172.21.15.63:0/1247040273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18c4100d30 0x7f18c419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.553+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 >> 172.21.15.63:0/1247040273 conn(0x7f18c40fb0d0 msgr2=0x7f18c4104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:13.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.554+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 shutdown_connections 2024-05-09T06:38:13.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.554+0000 7f18cc604700 1 -- 172.21.15.63:0/1247040273 wait complete. 2024-05-09T06:38:13.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/183: chown db/df/d12/f32 162978 1 2024-05-09T06:38:13.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/184: write db/df/d12/f24 [838259,13799] 0 2024-05-09T06:38:13.542 INFO:tasks.workunit.client.0.smithi063.stdout:7/515: fsync d0/d15/d4a/d50/f30 0 2024-05-09T06:38:13.542 INFO:tasks.workunit.client.0.smithi063.stdout:1/483: creat d9/d76/d94/fa7 x:0 0 0 2024-05-09T06:38:13.544 INFO:tasks.workunit.client.0.smithi063.stdout:9/543: truncate d0/d43/f5f 150296 0 2024-05-09T06:38:13.544 INFO:tasks.workunit.client.0.smithi063.stdout:5/494: dwrite d8/f4c [0,4194304] 0 2024-05-09T06:38:13.545 INFO:tasks.workunit.client.1.smithi193.stdout:8/205: read d6/d1c/d1b/f2b [1309477,104494] 0 2024-05-09T06:38:13.546 INFO:tasks.workunit.client.0.smithi063.stdout:9/544: chown d0/l4e 18 1 2024-05-09T06:38:13.547 INFO:tasks.workunit.client.0.smithi063.stdout:7/516: write d0/d15/d49/d54/d9f/d97/fae [676658,32486] 0 2024-05-09T06:38:13.547 INFO:tasks.workunit.client.1.smithi193.stdout:8/206: truncate d6/d1c/d18/d2f/f39 461902 0 2024-05-09T06:38:13.547 INFO:tasks.workunit.client.0.smithi063.stdout:0/576: truncate dd/d1a/d50/f86 803881 0 2024-05-09T06:38:13.548 INFO:tasks.workunit.client.0.smithi063.stdout:6/543: dread d2/d1f/f4c [0,4194304] 0 2024-05-09T06:38:13.549 INFO:tasks.workunit.client.0.smithi063.stdout:3/552: rename d7/l1f to d7/d70/lae 0 2024-05-09T06:38:13.550 INFO:tasks.workunit.client.1.smithi193.stdout:3/138: write d1/f4 [3161029,18862] 0 2024-05-09T06:38:13.550 INFO:tasks.workunit.client.1.smithi193.stdout:3/139: dread - d1/f24 zero size 2024-05-09T06:38:13.551 INFO:tasks.workunit.client.1.smithi193.stdout:3/140: fsync d1/f24 0 2024-05-09T06:38:13.558 INFO:tasks.workunit.client.0.smithi063.stdout:5/495: dread d8/d19/f44 [0,4194304] 0 2024-05-09T06:38:13.558 INFO:tasks.workunit.client.1.smithi193.stdout:5/244: truncate d0/d13/d2c/d37/f11 1458769 0 2024-05-09T06:38:13.558 INFO:tasks.workunit.client.1.smithi193.stdout:5/245: chown d0/d1 13 1 2024-05-09T06:38:13.559 INFO:tasks.workunit.client.1.smithi193.stdout:6/183: symlink d1/de/d11/d18/l39 0 2024-05-09T06:38:13.564 INFO:tasks.workunit.client.0.smithi063.stdout:5/496: dwrite d8/d19/d30/f61 [0,4194304] 0 2024-05-09T06:38:13.568 INFO:tasks.workunit.client.1.smithi193.stdout:2/185: mkdir db/d25/d38/d40 0 2024-05-09T06:38:13.569 INFO:tasks.workunit.client.1.smithi193.stdout:2/186: readlink db/df/l11 0 2024-05-09T06:38:13.569 INFO:tasks.workunit.client.1.smithi193.stdout:2/187: chown db/df/d12/f32 29221040 1 2024-05-09T06:38:13.573 INFO:tasks.workunit.client.1.smithi193.stdout:3/141: creat d1/db/f34 x:0 0 0 2024-05-09T06:38:13.577 INFO:tasks.workunit.client.1.smithi193.stdout:6/184: mkdir d1/de/d11/d2d/d3a 0 2024-05-09T06:38:13.578 INFO:tasks.workunit.client.1.smithi193.stdout:1/192: mkdir d4/d16/d35 0 2024-05-09T06:38:13.579 INFO:tasks.workunit.client.1.smithi193.stdout:9/220: getdents dd/df/d15/d16/d39 0 2024-05-09T06:38:13.581 INFO:tasks.workunit.client.1.smithi193.stdout:8/207: link d6/d1c/d18/f37 d6/d1c/d27/f3b 0 2024-05-09T06:38:13.582 INFO:tasks.workunit.client.1.smithi193.stdout:2/188: mknod db/d25/c41 0 2024-05-09T06:38:13.583 INFO:tasks.workunit.client.1.smithi193.stdout:2/189: chown db/df/d12/c2e 1096566 1 2024-05-09T06:38:13.584 INFO:tasks.workunit.client.1.smithi193.stdout:2/190: read db/df/d12/f32 [2858362,74346] 0 2024-05-09T06:38:13.584 INFO:tasks.workunit.client.1.smithi193.stdout:4/212: rmdir d1a/d24/d3f 0 2024-05-09T06:38:13.586 INFO:tasks.workunit.client.1.smithi193.stdout:5/246: symlink d0/d13/l51 0 2024-05-09T06:38:13.588 INFO:tasks.workunit.client.1.smithi193.stdout:9/221: unlink dd/df/d15/l19 0 2024-05-09T06:38:13.588 INFO:tasks.workunit.client.1.smithi193.stdout:9/222: readlink dd/df/d31/l3d 0 2024-05-09T06:38:13.589 INFO:tasks.workunit.client.1.smithi193.stdout:8/208: chown d6/d25/c36 103 1 2024-05-09T06:38:13.592 INFO:tasks.workunit.client.1.smithi193.stdout:2/191: creat db/d37/f42 x:0 0 0 2024-05-09T06:38:13.593 INFO:tasks.workunit.client.1.smithi193.stdout:2/192: write db/df/d12/f13 [711488,83531] 0 2024-05-09T06:38:13.594 INFO:tasks.workunit.client.1.smithi193.stdout:4/213: rename f13 to d1a/d24/f40 0 2024-05-09T06:38:13.596 INFO:tasks.workunit.client.1.smithi193.stdout:6/185: creat d1/de/d11/d2d/d3a/f3b x:0 0 0 2024-05-09T06:38:13.599 INFO:tasks.workunit.client.0.smithi063.stdout:2/571: read dc/f2a [178908,99989] 0 2024-05-09T06:38:13.599 INFO:tasks.workunit.client.1.smithi193.stdout:5/247: dread d0/d13/d2c/d37/f11 [0,4194304] 0 2024-05-09T06:38:13.600 INFO:tasks.workunit.client.1.smithi193.stdout:8/209: write d6/f21 [4410470,66761] 0 2024-05-09T06:38:13.600 INFO:tasks.workunit.client.0.smithi063.stdout:2/572: fdatasync dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f7a 0 2024-05-09T06:38:13.601 INFO:tasks.workunit.client.0.smithi063.stdout:4/522: creat d7/df/d35/d5a/fa4 x:0 0 0 2024-05-09T06:38:13.602 INFO:tasks.workunit.client.0.smithi063.stdout:2/573: write dc/d6e/d7f/fa1 [784472,106900] 0 2024-05-09T06:38:13.602 INFO:tasks.workunit.client.1.smithi193.stdout:9/223: creat dd/d1e/d1f/d48/f53 x:0 0 0 2024-05-09T06:38:13.603 INFO:tasks.workunit.client.0.smithi063.stdout:8/591: creat d9/da/d18/dc7/d58/d79/dc5/fd1 x:0 0 0 2024-05-09T06:38:13.604 INFO:tasks.workunit.client.0.smithi063.stdout:2/574: readlink dc/dd/d45/d5a/d6c/l9b 0 2024-05-09T06:38:13.604 INFO:tasks.workunit.client.1.smithi193.stdout:3/142: link d1/db/c1f d1/c35 0 2024-05-09T06:38:13.607 INFO:tasks.workunit.client.0.smithi063.stdout:1/484: unlink d9/d2a/d35/d1e/d9d/d3f/d59/d62/f6c 0 2024-05-09T06:38:13.608 INFO:tasks.workunit.client.1.smithi193.stdout:2/193: mknod db/d25/c43 0 2024-05-09T06:38:13.609 INFO:tasks.workunit.client.0.smithi063.stdout:6/544: creat d2/df/d59/faf x:0 0 0 2024-05-09T06:38:13.610 INFO:tasks.workunit.client.1.smithi193.stdout:4/214: truncate d1a/f1b 3668860 0 2024-05-09T06:38:13.611 INFO:tasks.workunit.client.0.smithi063.stdout:6/545: readlink d2/d1f/d29/d3f/l46 0 2024-05-09T06:38:13.612 INFO:tasks.workunit.client.0.smithi063.stdout:1/485: dwrite d9/d10/f3e [4194304,4194304] 0 2024-05-09T06:38:13.613 INFO:tasks.workunit.client.0.smithi063.stdout:0/577: rmdir dd/d1a/d2c/d51/d72/d9e/d23/d33 39 2024-05-09T06:38:13.615 INFO:tasks.workunit.client.1.smithi193.stdout:5/248: unlink d0/fc 0 2024-05-09T06:38:13.616 INFO:tasks.workunit.client.1.smithi193.stdout:5/249: truncate d0/d1/f2a 794828 0 2024-05-09T06:38:13.620 INFO:tasks.workunit.client.1.smithi193.stdout:9/224: rename dd/df/d15/d16/d39/l44 to dd/d3f/l54 0 2024-05-09T06:38:13.621 INFO:tasks.workunit.client.0.smithi063.stdout:0/578: dwrite f6 [12582912,4194304] 0 2024-05-09T06:38:13.622 INFO:tasks.workunit.client.1.smithi193.stdout:3/143: unlink d1/d20/f22 0 2024-05-09T06:38:13.623 INFO:tasks.workunit.client.1.smithi193.stdout:3/144: chown d1/d20/f29 0 1 2024-05-09T06:38:13.624 INFO:tasks.workunit.client.0.smithi063.stdout:5/497: symlink d8/d7e/la4 0 2024-05-09T06:38:13.628 INFO:tasks.workunit.client.1.smithi193.stdout:4/215: creat d1a/d24/f41 x:0 0 0 2024-05-09T06:38:13.629 INFO:tasks.workunit.client.1.smithi193.stdout:4/216: chown d1a/d24/d30/f34 2635 1 2024-05-09T06:38:13.630 INFO:tasks.workunit.client.1.smithi193.stdout:6/186: mkdir d1/d25/d2c/d3c 0 2024-05-09T06:38:13.631 INFO:tasks.workunit.client.1.smithi193.stdout:4/217: dread f12 [0,4194304] 0 2024-05-09T06:38:13.632 INFO:tasks.workunit.client.1.smithi193.stdout:6/187: read d1/de/f10 [962220,79331] 0 2024-05-09T06:38:13.633 INFO:tasks.workunit.client.0.smithi063.stdout:2/575: read dc/d32/d33/f4c [1042681,122744] 0 2024-05-09T06:38:13.633 INFO:tasks.workunit.client.1.smithi193.stdout:5/250: creat d0/d1/da/d3a/f52 x:0 0 0 2024-05-09T06:38:13.634 INFO:tasks.workunit.client.1.smithi193.stdout:4/218: dread fa [4194304,4194304] 0 2024-05-09T06:38:13.634 INFO:tasks.workunit.client.1.smithi193.stdout:8/210: mknod d6/c3c 0 2024-05-09T06:38:13.635 INFO:tasks.workunit.client.1.smithi193.stdout:4/219: write d1a/d24/f3b [1704861,120284] 0 2024-05-09T06:38:13.639 INFO:tasks.workunit.client.1.smithi193.stdout:3/145: symlink d1/db/l36 0 2024-05-09T06:38:13.641 INFO:tasks.workunit.client.0.smithi063.stdout:1/486: symlink d9/d10/la8 0 2024-05-09T06:38:13.642 INFO:tasks.workunit.client.0.smithi063.stdout:1/487: dread - d9/d2a/d39/f8c zero size 2024-05-09T06:38:13.644 INFO:tasks.workunit.client.1.smithi193.stdout:6/188: rmdir d1/d25/d2c 39 2024-05-09T06:38:13.646 INFO:tasks.workunit.client.1.smithi193.stdout:5/251: mknod d0/d13/d2c/c53 0 2024-05-09T06:38:13.646 INFO:tasks.workunit.client.0.smithi063.stdout:1/488: dread d9/d2a/d35/d1e/d9d/d3f/d59/f79 [0,4194304] 0 2024-05-09T06:38:13.648 INFO:tasks.workunit.client.1.smithi193.stdout:5/252: write d0/d13/d2c/f30 [2007547,122571] 0 2024-05-09T06:38:13.649 INFO:tasks.workunit.client.0.smithi063.stdout:0/579: read dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/f7b [1759045,43058] 0 2024-05-09T06:38:13.650 INFO:tasks.workunit.client.1.smithi193.stdout:5/253: write d0/d1/f33 [133286,2113] 0 2024-05-09T06:38:13.651 INFO:tasks.workunit.client.1.smithi193.stdout:6/189: dwrite d1/d25/f30 [0,4194304] 0 2024-05-09T06:38:13.652 INFO:tasks.workunit.client.0.smithi063.stdout:5/498: unlink d8/d17/f95 0 2024-05-09T06:38:13.653 INFO:tasks.workunit.client.1.smithi193.stdout:5/254: dread d0/d13/f3d [0,4194304] 0 2024-05-09T06:38:13.657 INFO:tasks.workunit.client.1.smithi193.stdout:2/194: creat db/f44 x:0 0 0 2024-05-09T06:38:13.661 INFO:tasks.workunit.client.1.smithi193.stdout:3/146: fsync d1/db/f18 0 2024-05-09T06:38:13.662 INFO:tasks.workunit.client.1.smithi193.stdout:3/147: stat d1/db/cc 0 2024-05-09T06:38:13.662 INFO:tasks.workunit.client.1.smithi193.stdout:3/148: chown d1/c6 28823150 1 2024-05-09T06:38:13.662 INFO:tasks.workunit.client.0.smithi063.stdout:2/576: readlink dc/d32/d33/l54 0 2024-05-09T06:38:13.663 INFO:tasks.workunit.client.1.smithi193.stdout:9/225: rename dd/d3f to dd/d1e/d1f/d48/d55 0 2024-05-09T06:38:13.664 INFO:tasks.workunit.client.1.smithi193.stdout:9/226: readlink dd/df/d31/l3b 0 2024-05-09T06:38:13.665 INFO:tasks.workunit.client.1.smithi193.stdout:9/227: chown dd/d1e/d1f/f2f 13 1 2024-05-09T06:38:13.666 INFO:tasks.workunit.client.1.smithi193.stdout:9/228: dread - dd/df/d15/d16/d39/d49/f51 zero size 2024-05-09T06:38:13.666 INFO:tasks.workunit.client.0.smithi063.stdout:9/545: link d0/d3b/cbb d0/d8/da6/db0/db5/cbd 0 2024-05-09T06:38:13.668 INFO:tasks.workunit.client.0.smithi063.stdout:3/553: readlink d7/d4c/d31/l76 0 2024-05-09T06:38:13.680 INFO:tasks.workunit.client.1.smithi193.stdout:3/149: creat d1/db/d26/f37 x:0 0 0 2024-05-09T06:38:13.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.700+0000 7fee17eca700 1 -- 172.21.15.63:0/3279739234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee10100c20 msgr2=0x7fee10101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:13.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.700+0000 7fee17eca700 1 --2- 172.21.15.63:0/3279739234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee10100c20 0x7fee10101040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009910 tx=0x7fee0c009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.700+0000 7fee17eca700 1 -- 172.21.15.63:0/3279739234 shutdown_connections 2024-05-09T06:38:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.700+0000 7fee17eca700 1 --2- 172.21.15.63:0/3279739234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10101e20 0x7fee10102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.681 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:13 smithi063 ceph-mon[27171]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:13.681 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:13 smithi063 ceph-mon[27171]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.700+0000 7fee17eca700 1 --2- 172.21.15.63:0/3279739234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee10100c20 0x7fee10101040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.700+0000 7fee17eca700 1 -- 172.21.15.63:0/3279739234 >> 172.21.15.63:0/3279739234 conn(0x7fee100fc200 msgr2=0x7fee100fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:13.682 INFO:tasks.workunit.client.0.smithi063.stdout:6/546: dwrite d2/d20/d4a/f93 [0,4194304] 0 2024-05-09T06:38:13.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.701+0000 7fee17eca700 1 -- 172.21.15.63:0/3279739234 shutdown_connections 2024-05-09T06:38:13.682 INFO:tasks.workunit.client.0.smithi063.stdout:7/517: truncate d0/d15/d49/f5e 217000 0 2024-05-09T06:38:13.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.702+0000 7fee17eca700 1 -- 172.21.15.63:0/3279739234 wait complete. 2024-05-09T06:38:13.683 INFO:tasks.workunit.client.0.smithi063.stdout:6/547: stat d2/df 0 2024-05-09T06:38:13.684 INFO:tasks.workunit.client.1.smithi193.stdout:5/255: rename d0/d13/d2c/f30 to d0/d13/d2c/d38/f54 0 2024-05-09T06:38:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.703+0000 7fee17eca700 1 Processor -- start 2024-05-09T06:38:13.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.703+0000 7fee17eca700 1 -- start start 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee17eca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee10100c20 0x7fee101966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee17eca700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10101e20 0x7fee10196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee17eca700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10197220 con 0x7fee10101e20 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee17eca700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10197360 con 0x7fee10100c20 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee15465700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10101e20 0x7fee10196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee15465700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10101e20 0x7fee10196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56058/0 (socket says 172.21.15.63:56058) 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee15465700 1 -- 172.21.15.63:0/3291182600 learned_addr learned my addr 172.21.15.63:0/3291182600 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee15465700 1 -- 172.21.15.63:0/3291182600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee10100c20 msgr2=0x7fee101966c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee15465700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee10100c20 0x7fee101966c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.704+0000 7fee15465700 1 -- 172.21.15.63:0/3291182600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c0095f0 con 0x7fee10101e20 2024-05-09T06:38:13.686 INFO:tasks.workunit.client.1.smithi193.stdout:5/256: fsync d0/d13/d2c/f4b 0 2024-05-09T06:38:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.705+0000 7fee15465700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10101e20 0x7fee10196c00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fee0000e920 tx=0x7fee0000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.705+0000 7fee06ffd700 1 -- 172.21.15.63:0/3291182600 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee00009790 con 0x7fee10101e20 2024-05-09T06:38:13.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.706+0000 7fee06ffd700 1 -- 172.21.15.63:0/3291182600 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee0000cba0 con 0x7fee10101e20 2024-05-09T06:38:13.687 INFO:tasks.workunit.client.0.smithi063.stdout:5/499: dwrite d8/d17/d1e/f9f [0,4194304] 0 2024-05-09T06:38:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.706+0000 7fee06ffd700 1 -- 172.21.15.63:0/3291182600 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee00018650 con 0x7fee10101e20 2024-05-09T06:38:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.706+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee100732c0 con 0x7fee10101e20 2024-05-09T06:38:13.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.706+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee10073890 con 0x7fee10101e20 2024-05-09T06:38:13.687 INFO:tasks.workunit.client.0.smithi063.stdout:7/518: truncate d0/d15/d4a/f7e 276186 0 2024-05-09T06:38:13.688 INFO:tasks.workunit.client.0.smithi063.stdout:7/519: chown d0/d15/d4a/d50/f3f 250 1 2024-05-09T06:38:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.708+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf4005320 con 0x7fee10101e20 2024-05-09T06:38:13.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.709+0000 7fee06ffd700 1 -- 172.21.15.63:0/3291182600 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7fee0000cd10 con 0x7fee10101e20 2024-05-09T06:38:13.689 INFO:tasks.workunit.client.0.smithi063.stdout:5/500: readlink d8/d53/l5a 0 2024-05-09T06:38:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.710+0000 7fee06ffd700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fedfc06d730 0x7fedfc06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:13.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.710+0000 7fee06ffd700 1 -- 172.21.15.63:0/3291182600 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fee00014070 con 0x7fee10101e20 2024-05-09T06:38:13.690 INFO:tasks.workunit.client.0.smithi063.stdout:5/501: write d8/d17/d1e/f81 [184080,117771] 0 2024-05-09T06:38:13.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.711+0000 7fee15c66700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fedfc06d730 0x7fedfc06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:13.693 INFO:tasks.workunit.client.1.smithi193.stdout:6/190: truncate d1/de/f13 3738524 0 2024-05-09T06:38:13.694 INFO:tasks.workunit.client.1.smithi193.stdout:6/191: write d1/de/d11/d18/f2a [620393,81028] 0 2024-05-09T06:38:13.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.714+0000 7fee15c66700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fedfc06d730 0x7fedfc06fbf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009fd0 tx=0x7fee0c00b7c0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:13.695 INFO:tasks.workunit.client.0.smithi063.stdout:1/489: creat d9/d2a/d39/d5d/d75/fa9 x:0 0 0 2024-05-09T06:38:13.696 INFO:tasks.workunit.client.0.smithi063.stdout:1/490: readlink d9/d2a/d39/l40 0 2024-05-09T06:38:13.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:13.717+0000 7fee06ffd700 1 -- 172.21.15.63:0/3291182600 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7fee00053720 con 0x7fee10101e20 2024-05-09T06:38:13.704 INFO:tasks.workunit.client.0.smithi063.stdout:4/523: getdents d7/da3/d28/d31 0 2024-05-09T06:38:13.705 INFO:tasks.workunit.client.0.smithi063.stdout:4/524: write d7/df/f95 [827625,12340] 0 2024-05-09T06:38:13.706 INFO:tasks.workunit.client.1.smithi193.stdout:9/229: unlink dd/df/d15/d16/c25 0 2024-05-09T06:38:13.709 INFO:tasks.workunit.client.1.smithi193.stdout:5/257: mknod d0/d13/d2c/d37/d2b/d47/c55 0 2024-05-09T06:38:13.712 INFO:tasks.workunit.client.1.smithi193.stdout:8/211: creat d6/d1c/f3d x:0 0 0 2024-05-09T06:38:13.715 INFO:tasks.workunit.client.1.smithi193.stdout:8/212: dread d6/d1c/d18/f26 [0,4194304] 0 2024-05-09T06:38:13.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/554: creat d7/da/d2b/d66/d74/faf x:0 0 0 2024-05-09T06:38:13.716 INFO:tasks.workunit.client.0.smithi063.stdout:9/546: dread d0/d8/d2d/f52 [0,4194304] 0 2024-05-09T06:38:13.717 INFO:tasks.workunit.client.1.smithi193.stdout:4/220: getdents d1a/d22 0 2024-05-09T06:38:13.717 INFO:tasks.workunit.client.0.smithi063.stdout:9/547: stat d0/d4/d57/f64 0 2024-05-09T06:38:13.718 INFO:tasks.workunit.client.0.smithi063.stdout:5/502: fdatasync d8/d17/d18/f38 0 2024-05-09T06:38:13.718 INFO:tasks.workunit.client.0.smithi063.stdout:5/503: chown d8/f27 15664 1 2024-05-09T06:38:13.719 INFO:tasks.workunit.client.1.smithi193.stdout:6/192: unlink d1/de/d11/l28 0 2024-05-09T06:38:13.719 INFO:tasks.workunit.client.0.smithi063.stdout:9/548: dread d0/d4/d7d/f87 [0,4194304] 0 2024-05-09T06:38:13.719 INFO:tasks.workunit.client.0.smithi063.stdout:5/504: dread - d8/d19/d30/d46/f94 zero size 2024-05-09T06:38:13.721 INFO:tasks.workunit.client.1.smithi193.stdout:2/195: creat db/f45 x:0 0 0 2024-05-09T06:38:13.721 INFO:tasks.workunit.client.0.smithi063.stdout:9/549: write d0/d8/da6/db0/fb1 [897021,51725] 0 2024-05-09T06:38:13.722 INFO:tasks.workunit.client.0.smithi063.stdout:6/548: write d2/d20/d26/f97 [1049376,26302] 0 2024-05-09T06:38:13.723 INFO:tasks.workunit.client.0.smithi063.stdout:6/549: truncate d2/d1f/d29/d3b/fac 371172 0 2024-05-09T06:38:13.726 INFO:tasks.workunit.client.1.smithi193.stdout:9/230: rmdir dd/df/d15 39 2024-05-09T06:38:13.727 INFO:tasks.workunit.client.0.smithi063.stdout:4/525: mkdir d7/df/d35/da5 0 2024-05-09T06:38:13.729 INFO:tasks.workunit.client.0.smithi063.stdout:2/577: link dc/d32/d33/f61 dc/dd/d45/d5a/d6a/fa4 0 2024-05-09T06:38:13.730 INFO:tasks.workunit.client.1.smithi193.stdout:8/213: rmdir d6/d1c/d1b 39 2024-05-09T06:38:13.730 INFO:tasks.workunit.client.0.smithi063.stdout:2/578: dread f3 [0,4194304] 0 2024-05-09T06:38:13.736 INFO:tasks.workunit.client.1.smithi193.stdout:2/196: creat db/df/d12/f46 x:0 0 0 2024-05-09T06:38:13.736 INFO:tasks.workunit.client.1.smithi193.stdout:3/150: dwrite d1/db/f14 [0,4194304] 0 2024-05-09T06:38:13.737 INFO:tasks.workunit.client.0.smithi063.stdout:5/505: read d8/d17/f1c [356495,82058] 0 2024-05-09T06:38:13.741 INFO:tasks.workunit.client.0.smithi063.stdout:0/580: getdents dd/d1a/d2c 0 2024-05-09T06:38:13.744 INFO:tasks.workunit.client.0.smithi063.stdout:1/491: dwrite d9/d2a/d35/d1e/d9d/d3f/d59/d62/f8b [0,4194304] 0 2024-05-09T06:38:13.748 INFO:tasks.workunit.client.0.smithi063.stdout:3/555: mkdir d7/d4c/d31/da4/db0 0 2024-05-09T06:38:13.750 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:13 smithi193 ceph-mon[35141]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:13.750 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:13 smithi193 ceph-mon[35141]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:13.752 INFO:tasks.workunit.client.1.smithi193.stdout:2/197: unlink db/df/f2f 0 2024-05-09T06:38:13.753 INFO:tasks.workunit.client.0.smithi063.stdout:1/492: dwrite d9/d2a/d3c/d4c/f78 [0,4194304] 0 2024-05-09T06:38:13.754 INFO:tasks.workunit.client.1.smithi193.stdout:3/151: rmdir d1/db/d1b 39 2024-05-09T06:38:13.754 INFO:tasks.workunit.client.0.smithi063.stdout:5/506: creat d8/d53/fa5 x:0 0 0 2024-05-09T06:38:13.756 INFO:tasks.workunit.client.0.smithi063.stdout:2/579: dread dc/d32/d33/d4e/f88 [0,4194304] 0 2024-05-09T06:38:13.756 INFO:tasks.workunit.client.1.smithi193.stdout:5/258: rename d0/d1/f33 to d0/f56 0 2024-05-09T06:38:13.760 INFO:tasks.workunit.client.0.smithi063.stdout:0/581: mknod dd/d60/cbc 0 2024-05-09T06:38:13.763 INFO:tasks.workunit.client.1.smithi193.stdout:6/193: creat d1/d25/d2c/d3c/f3d x:0 0 0 2024-05-09T06:38:13.765 INFO:tasks.workunit.client.1.smithi193.stdout:2/198: readlink db/l34 0 2024-05-09T06:38:13.766 INFO:tasks.workunit.client.1.smithi193.stdout:9/231: dwrite dd/f18 [0,4194304] 0 2024-05-09T06:38:13.767 INFO:tasks.workunit.client.1.smithi193.stdout:3/152: mknod d1/d20/c38 0 2024-05-09T06:38:13.767 INFO:tasks.workunit.client.0.smithi063.stdout:0/582: chown dd/d1a/d2c/d51/d76/d97/fa5 597534 1 2024-05-09T06:38:13.767 INFO:tasks.workunit.client.0.smithi063.stdout:4/526: dwrite d7/da3/d28/f79 [0,4194304] 0 2024-05-09T06:38:13.768 INFO:tasks.workunit.client.1.smithi193.stdout:3/153: stat d1/db/d2f 0 2024-05-09T06:38:13.768 INFO:tasks.workunit.client.0.smithi063.stdout:9/550: creat d0/d4/da/fbe x:0 0 0 2024-05-09T06:38:13.768 INFO:tasks.workunit.client.1.smithi193.stdout:3/154: read f0 [2282594,63629] 0 2024-05-09T06:38:13.771 INFO:tasks.workunit.client.0.smithi063.stdout:2/580: dwrite dc/d6e/d7f/fa1 [0,4194304] 0 2024-05-09T06:38:13.772 INFO:tasks.workunit.client.1.smithi193.stdout:8/214: mknod d6/d1c/d1b/d29/c3e 0 2024-05-09T06:38:13.775 INFO:tasks.workunit.client.1.smithi193.stdout:4/221: getdents d1a/d24 0 2024-05-09T06:38:13.778 INFO:tasks.workunit.client.1.smithi193.stdout:6/194: mkdir d1/d25/d2c/d3c/d3e 0 2024-05-09T06:38:13.781 INFO:tasks.workunit.client.1.smithi193.stdout:6/195: truncate d1/de/d11/d18/f29 547940 0 2024-05-09T06:38:13.781 INFO:tasks.workunit.client.0.smithi063.stdout:6/550: rename d2/f85 to d2/d1f/d29/d3f/d5c/fb0 0 2024-05-09T06:38:13.782 INFO:tasks.workunit.client.0.smithi063.stdout:3/556: rename d7/d70/d98 to d7/d70/d98/d2d/d36/db1 22 2024-05-09T06:38:13.785 INFO:tasks.workunit.client.1.smithi193.stdout:4/222: dwrite f7 [4194304,4194304] 0 2024-05-09T06:38:13.786 INFO:tasks.workunit.client.1.smithi193.stdout:4/223: write d1a/d1f/f2c [329148,114709] 0 2024-05-09T06:38:13.787 INFO:tasks.workunit.client.1.smithi193.stdout:4/224: readlink d1a/d1f/l2b 0 2024-05-09T06:38:13.788 INFO:tasks.workunit.client.0.smithi063.stdout:6/551: read d2/d4/f5 [1369134,68112] 0 2024-05-09T06:38:13.789 INFO:tasks.workunit.client.1.smithi193.stdout:4/225: write d1a/d1f/f26 [653588,24132] 0 2024-05-09T06:38:13.789 INFO:tasks.workunit.client.1.smithi193.stdout:4/226: readlink l2 0 2024-05-09T06:38:13.790 INFO:tasks.workunit.client.1.smithi193.stdout:4/227: write f5 [3103032,8192] 0 2024-05-09T06:38:13.790 INFO:tasks.workunit.client.1.smithi193.stdout:4/228: readlink d1a/l38 0 2024-05-09T06:38:13.792 INFO:tasks.workunit.client.0.smithi063.stdout:5/507: mknod d8/ca6 0 2024-05-09T06:38:13.794 INFO:tasks.workunit.client.0.smithi063.stdout:5/508: chown d8/d17/f6d 18148 1 2024-05-09T06:38:13.794 INFO:tasks.workunit.client.0.smithi063.stdout:9/551: rmdir d0/d4/d26/d39/d91 39 2024-05-09T06:38:13.798 INFO:tasks.workunit.client.0.smithi063.stdout:2/581: mkdir dc/d36/d43/d46/d63/d79/da5 0 2024-05-09T06:38:13.799 INFO:tasks.workunit.client.0.smithi063.stdout:2/582: readlink dc/dd/l4a 0 2024-05-09T06:38:13.806 INFO:tasks.workunit.client.0.smithi063.stdout:3/557: rmdir d7 39 2024-05-09T06:38:13.808 INFO:tasks.workunit.client.0.smithi063.stdout:4/527: mkdir d7/da6 0 2024-05-09T06:38:13.811 INFO:tasks.workunit.client.1.smithi193.stdout:4/229: creat d1a/d24/d37/f42 x:0 0 0 2024-05-09T06:38:13.812 INFO:tasks.workunit.client.1.smithi193.stdout:4/230: read d1a/d1f/f26 [665541,126002] 0 2024-05-09T06:38:13.813 INFO:tasks.workunit.client.0.smithi063.stdout:5/509: creat d8/d7e/fa7 x:0 0 0 2024-05-09T06:38:13.818 INFO:tasks.workunit.client.1.smithi193.stdout:8/215: fdatasync d6/fd 0 2024-05-09T06:38:13.818 INFO:tasks.workunit.client.0.smithi063.stdout:5/510: dread d8/d17/f6d [0,4194304] 0 2024-05-09T06:38:13.818 INFO:tasks.workunit.client.0.smithi063.stdout:2/583: dread dc/f9d [0,4194304] 0 2024-05-09T06:38:13.821 INFO:tasks.workunit.client.1.smithi193.stdout:2/199: creat db/df/f47 x:0 0 0 2024-05-09T06:38:13.822 INFO:tasks.workunit.client.0.smithi063.stdout:4/528: write d7/df/d35/d5a/d60/f63 [1403484,80752] 0 2024-05-09T06:38:13.827 INFO:tasks.workunit.client.1.smithi193.stdout:4/231: mkdir d1a/d22/d43 0 2024-05-09T06:38:13.830 INFO:tasks.workunit.client.0.smithi063.stdout:2/584: mknod dc/d36/d43/d46/d63/d79/d7d/d8d/ca6 0 2024-05-09T06:38:13.831 INFO:tasks.workunit.client.1.smithi193.stdout:8/216: truncate d6/d1c/d18/f26 1623602 0 2024-05-09T06:38:13.832 INFO:tasks.workunit.client.0.smithi063.stdout:6/552: truncate d2/d1f/d29/d3f/d5c/d64/f6f 1851364 0 2024-05-09T06:38:13.833 INFO:tasks.workunit.client.0.smithi063.stdout:6/553: fdatasync d2/d20/f7b 0 2024-05-09T06:38:13.836 INFO:tasks.workunit.client.1.smithi193.stdout:2/200: dread db/df/d12/f32 [0,4194304] 0 2024-05-09T06:38:13.837 INFO:tasks.workunit.client.0.smithi063.stdout:9/552: rename d0/d4/d26/d39/d48/c81 to d0/d4/d57/d60/cbf 0 2024-05-09T06:38:13.838 INFO:tasks.workunit.client.0.smithi063.stdout:6/554: dread d2/d20/d4a/f93 [0,4194304] 0 2024-05-09T06:38:13.839 INFO:tasks.workunit.client.0.smithi063.stdout:3/558: dread d7/d4c/d31/fa2 [4194304,4194304] 0 2024-05-09T06:38:13.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/585: mknod dc/dd/d45/d5a/ca7 0 2024-05-09T06:38:13.844 INFO:tasks.workunit.client.1.smithi193.stdout:3/155: getdents d1/db/d26 0 2024-05-09T06:38:13.847 INFO:tasks.workunit.client.0.smithi063.stdout:5/511: write d8/d17/d18/f1f [1655548,8206] 0 2024-05-09T06:38:13.850 INFO:tasks.workunit.client.1.smithi193.stdout:2/201: creat db/d25/f48 x:0 0 0 2024-05-09T06:38:13.852 INFO:tasks.workunit.client.1.smithi193.stdout:2/202: read db/d25/f27 [290074,101840] 0 2024-05-09T06:38:13.858 INFO:tasks.workunit.client.0.smithi063.stdout:6/555: write d2/d4/f92 [3425222,90836] 0 2024-05-09T06:38:13.858 INFO:tasks.workunit.client.0.smithi063.stdout:2/586: symlink dc/d36/d43/d46/d63/d9e/la8 0 2024-05-09T06:38:13.858 INFO:tasks.workunit.client.1.smithi193.stdout:4/232: truncate f17 1967299 0 2024-05-09T06:38:13.858 INFO:tasks.workunit.client.1.smithi193.stdout:3/156: write d1/f11 [150491,73365] 0 2024-05-09T06:38:13.859 INFO:tasks.workunit.client.1.smithi193.stdout:8/217: getdents d6/d1c/d1b/d3a 0 2024-05-09T06:38:13.861 INFO:tasks.workunit.client.0.smithi063.stdout:6/556: dwrite d2/d20/f7b [0,4194304] 0 2024-05-09T06:38:13.878 INFO:tasks.workunit.client.0.smithi063.stdout:6/557: dread d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:13.991 INFO:tasks.workunit.client.0.smithi063.stdout:8/592: sync 2024-05-09T06:38:13.991 INFO:tasks.workunit.client.0.smithi063.stdout:0/583: sync 2024-05-09T06:38:13.992 INFO:tasks.workunit.client.0.smithi063.stdout:7/520: sync 2024-05-09T06:38:13.992 INFO:tasks.workunit.client.0.smithi063.stdout:1/493: sync 2024-05-09T06:38:14.001 INFO:tasks.workunit.client.0.smithi063.stdout:8/593: dread d9/da/fb [0,4194304] 0 2024-05-09T06:38:14.016 INFO:tasks.workunit.client.0.smithi063.stdout:4/529: creat d7/d3d/fa7 x:0 0 0 2024-05-09T06:38:14.016 INFO:tasks.workunit.client.1.smithi193.stdout:9/232: getdents dd/df/d15/d16/d26/d37 0 2024-05-09T06:38:14.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.036+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fedf40058a0 con 0x7fee10101e20 2024-05-09T06:38:14.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.037+0000 7fee06ffd700 1 -- 172.21.15.63:0/3291182600 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fee0005fe60 con 0x7fee10101e20 2024-05-09T06:38:14.017 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:38:14.017 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:38:14.017 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:38:14.017 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:38:14.017 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:38:14.017 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:38:14.018 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:38:14.019 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:38:14.020 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:38:14.020 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:38:14.020 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:38:14.020 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:38:14.020 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:38:14.020 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:38:14.020 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:38:14.020 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:38:14.021 INFO:tasks.workunit.client.1.smithi193.stdout:2/203: write db/f21 [4077237,86705] 0 2024-05-09T06:38:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fedfc06d730 msgr2=0x7fedfc06fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fedfc06d730 0x7fedfc06fbf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009fd0 tx=0x7fee0c00b7c0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10101e20 msgr2=0x7fee10196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10101e20 0x7fee10196c00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fee0000e920 tx=0x7fee0000ece0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 shutdown_connections 2024-05-09T06:38:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fedfc06d730 0x7fedfc06fbf0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee10100c20 0x7fee101966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 --2- 172.21.15.63:0/3291182600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10101e20 0x7fee10196c00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 >> 172.21.15.63:0/3291182600 conn(0x7fee100fc200 msgr2=0x7fee101053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 shutdown_connections 2024-05-09T06:38:14.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.039+0000 7fee17eca700 1 -- 172.21.15.63:0/3291182600 wait complete. 2024-05-09T06:38:14.022 INFO:tasks.workunit.client.0.smithi063.stdout:5/512: mknod d8/d19/d30/d74/ca8 0 2024-05-09T06:38:14.023 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:38:14.023 INFO:tasks.workunit.client.0.smithi063.stdout:3/559: unlink d7/c1c 0 2024-05-09T06:38:14.023 INFO:tasks.workunit.client.1.smithi193.stdout:4/233: creat d1a/d24/d37/f44 x:0 0 0 2024-05-09T06:38:14.100 INFO:tasks.workunit.client.1.smithi193.stdout:8/218: write d6/f20 [350510,23240] 0 2024-05-09T06:38:14.101 INFO:tasks.workunit.client.0.smithi063.stdout:2/587: rmdir dc/d36/d43/d46/d63/d79/d7d/d8d 39 2024-05-09T06:38:14.101 INFO:tasks.workunit.client.1.smithi193.stdout:3/157: chown d1/db/d26/c2a 14906766 1 2024-05-09T06:38:14.103 INFO:tasks.workunit.client.1.smithi193.stdout:9/233: chown dd/d1e 74536 1 2024-05-09T06:38:14.104 INFO:tasks.workunit.client.0.smithi063.stdout:6/558: fsync d2/df/f14 0 2024-05-09T06:38:14.105 INFO:tasks.workunit.client.0.smithi063.stdout:1/494: readlink d9/d2a/d39/l41 0 2024-05-09T06:38:14.110 INFO:tasks.workunit.client.1.smithi193.stdout:2/204: rename db/f35 to db/d25/f49 0 2024-05-09T06:38:14.112 INFO:tasks.workunit.client.0.smithi063.stdout:0/584: creat dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/fbd x:0 0 0 2024-05-09T06:38:14.112 INFO:tasks.workunit.client.1.smithi193.stdout:4/234: symlink d1a/d1f/l45 0 2024-05-09T06:38:14.113 INFO:tasks.workunit.client.1.smithi193.stdout:4/235: read - d1a/d22/f31 zero size 2024-05-09T06:38:14.114 INFO:tasks.workunit.client.1.smithi193.stdout:4/236: write d1a/d24/d37/f44 [694975,125285] 0 2024-05-09T06:38:14.115 INFO:tasks.workunit.client.0.smithi063.stdout:8/594: truncate d9/f70 2277526 0 2024-05-09T06:38:14.117 INFO:tasks.workunit.client.1.smithi193.stdout:3/158: mknod d1/c39 0 2024-05-09T06:38:14.120 INFO:tasks.workunit.client.0.smithi063.stdout:0/585: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/f6e [0,4194304] 0 2024-05-09T06:38:14.122 INFO:tasks.workunit.client.0.smithi063.stdout:3/560: write f2 [5774026,31758] 0 2024-05-09T06:38:14.125 INFO:tasks.workunit.client.0.smithi063.stdout:9/553: rename d0/d8/da6/db0/db5/cbd to d0/d4/d26/d39/d42/cc0 0 2024-05-09T06:38:14.126 INFO:tasks.workunit.client.1.smithi193.stdout:4/237: mknod d1a/d24/d37/c46 0 2024-05-09T06:38:14.131 INFO:tasks.workunit.client.1.smithi193.stdout:8/219: write d6/d1c/d27/f3b [124656,106124] 0 2024-05-09T06:38:14.133 INFO:tasks.workunit.client.1.smithi193.stdout:2/205: getdents db/d25/d38/d40 0 2024-05-09T06:38:14.138 INFO:tasks.workunit.client.0.smithi063.stdout:5/513: dwrite d8/d35/f41 [0,4194304] 0 2024-05-09T06:38:14.139 INFO:tasks.workunit.client.1.smithi193.stdout:8/220: mkdir d6/d1c/d1b/d3f 0 2024-05-09T06:38:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.161+0000 7f003dd11700 1 -- 172.21.15.63:0/154080376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0038103fe0 msgr2=0x7f00381063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.161+0000 7f003dd11700 1 --2- 172.21.15.63:0/154080376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0038103fe0 0x7f00381063d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f002c009910 tx=0x7f002c009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.161+0000 7f003dd11700 1 -- 172.21.15.63:0/154080376 shutdown_connections 2024-05-09T06:38:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.161+0000 7f003dd11700 1 --2- 172.21.15.63:0/154080376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0038103fe0 0x7f00381063d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.161+0000 7f003dd11700 1 --2- 172.21.15.63:0/154080376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00381016b0 0x7f0038103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.161+0000 7f003dd11700 1 -- 172.21.15.63:0/154080376 >> 172.21.15.63:0/154080376 conn(0x7f00380fb0c0 msgr2=0x7f00380fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.161+0000 7f003dd11700 1 -- 172.21.15.63:0/154080376 shutdown_connections 2024-05-09T06:38:14.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.161+0000 7f003dd11700 1 -- 172.21.15.63:0/154080376 wait complete. 2024-05-09T06:38:14.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.162+0000 7f003dd11700 1 Processor -- start 2024-05-09T06:38:14.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.163+0000 7f003dd11700 1 -- start start 2024-05-09T06:38:14.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.164+0000 7f003dd11700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00381016b0 0x7f00381965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.164+0000 7f003dd11700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0038103fe0 0x7f0038196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.164+0000 7f003dd11700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0038197140 con 0x7f00381016b0 2024-05-09T06:38:14.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.164+0000 7f003dd11700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0038197280 con 0x7f0038103fe0 2024-05-09T06:38:14.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.164+0000 7f0036ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0038103fe0 0x7f0038196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.164+0000 7f0036ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0038103fe0 0x7f0038196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.63:55440/0 (socket says 172.21.15.63:55440) 2024-05-09T06:38:14.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.164+0000 7f0036ffd700 1 -- 172.21.15.63:0/1326672302 learned_addr learned my addr 172.21.15.63:0/1326672302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:14.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.165+0000 7f0036ffd700 1 -- 172.21.15.63:0/1326672302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00381016b0 msgr2=0x7f00381965e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:38:14.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.165+0000 7f00377fe700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00381016b0 0x7f00381965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.165+0000 7f0036ffd700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00381016b0 0x7f00381965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.165+0000 7f0036ffd700 1 -- 172.21.15.63:0/1326672302 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c0095f0 con 0x7f0038103fe0 2024-05-09T06:38:14.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.165+0000 7f00377fe700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00381016b0 0x7f00381965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:38:14.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.166+0000 7f0036ffd700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0038103fe0 0x7f0038196b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f002c00b800 tx=0x7f002c00f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:14.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.166+0000 7f0034ff9700 1 -- 172.21.15.63:0/1326672302 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f002c01d070 con 0x7f0038103fe0 2024-05-09T06:38:14.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.166+0000 7f0034ff9700 1 -- 172.21.15.63:0/1326672302 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f002c0052a0 con 0x7f0038103fe0 2024-05-09T06:38:14.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.166+0000 7f0034ff9700 1 -- 172.21.15.63:0/1326672302 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f002c017400 con 0x7f0038103fe0 2024-05-09T06:38:14.147 INFO:tasks.workunit.client.0.smithi063.stdout:9/554: creat d0/d4/d26/d39/d48/d80/fc1 x:0 0 0 2024-05-09T06:38:14.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.167+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003819bc80 con 0x7f0038103fe0 2024-05-09T06:38:14.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.167+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003819c1f0 con 0x7f0038103fe0 2024-05-09T06:38:14.148 INFO:tasks.workunit.client.1.smithi193.stdout:3/159: getdents d1 0 2024-05-09T06:38:14.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.169+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003804ebb0 con 0x7f0038103fe0 2024-05-09T06:38:14.149 INFO:tasks.workunit.client.1.smithi193.stdout:8/221: rename d6/d25 to d6/d1c/d18/d2f/d40 0 2024-05-09T06:38:14.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.170+0000 7f0034ff9700 1 -- 172.21.15.63:0/1326672302 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f002c0037f0 con 0x7f0038103fe0 2024-05-09T06:38:14.150 INFO:tasks.workunit.client.0.smithi063.stdout:1/495: dwrite d9/d2a/d35/d1e/d9d/d3f/d59/f79 [0,4194304] 0 2024-05-09T06:38:14.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.170+0000 7f0034ff9700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f002006d720 0x7f002006fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.171+0000 7f0034ff9700 1 -- 172.21.15.63:0/1326672302 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f002c08e510 con 0x7f0038103fe0 2024-05-09T06:38:14.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.171+0000 7f00377fe700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f002006d720 0x7f002006fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.171+0000 7f00377fe700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f002006d720 0x7f002006fbe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0028009be0 tx=0x7f0028009260 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:14.152 INFO:tasks.workunit.client.1.smithi193.stdout:2/206: link f8 db/df/f4a 0 2024-05-09T06:38:14.152 INFO:tasks.workunit.client.1.smithi193.stdout:8/222: dread d6/d1c/d27/f3b [0,4194304] 0 2024-05-09T06:38:14.153 INFO:tasks.workunit.client.1.smithi193.stdout:4/238: getdents d1a 0 2024-05-09T06:38:14.154 INFO:tasks.workunit.client.0.smithi063.stdout:7/521: link d0/d15/d4a/d50/f37 d0/d15/d89/d98/fb6 0 2024-05-09T06:38:14.155 INFO:tasks.workunit.client.1.smithi193.stdout:3/160: mknod d1/d20/c3a 0 2024-05-09T06:38:14.156 INFO:tasks.workunit.client.1.smithi193.stdout:4/239: dread - d1a/f1d zero size 2024-05-09T06:38:14.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.178+0000 7f0034ff9700 1 -- 172.21.15.63:0/1326672302 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f002c05b8a0 con 0x7f0038103fe0 2024-05-09T06:38:14.159 INFO:tasks.workunit.client.1.smithi193.stdout:8/223: creat d6/d1c/d1b/d29/f41 x:0 0 0 2024-05-09T06:38:14.160 INFO:tasks.workunit.client.1.smithi193.stdout:3/161: truncate d1/db/d26/f37 740225 0 2024-05-09T06:38:14.160 INFO:tasks.workunit.client.0.smithi063.stdout:5/514: mkdir d8/d35/da9 0 2024-05-09T06:38:14.160 INFO:tasks.workunit.client.0.smithi063.stdout:5/515: fdatasync d8/d65/f83 0 2024-05-09T06:38:14.161 INFO:tasks.workunit.client.0.smithi063.stdout:5/516: chown d8/d19/d30/d46/f94 27522 1 2024-05-09T06:38:14.163 INFO:tasks.workunit.client.1.smithi193.stdout:2/207: dwrite db/d25/f48 [0,4194304] 0 2024-05-09T06:38:14.167 INFO:tasks.workunit.client.0.smithi063.stdout:8/595: rename d9/f70 to d9/da/d18/dc7/d58/d79/dc5/fd2 0 2024-05-09T06:38:14.167 INFO:tasks.workunit.client.0.smithi063.stdout:5/517: dwrite d8/d35/f64 [0,4194304] 0 2024-05-09T06:38:14.167 INFO:tasks.workunit.client.1.smithi193.stdout:4/240: dwrite d1a/d1f/f2c [0,4194304] 0 2024-05-09T06:38:14.174 INFO:tasks.workunit.client.0.smithi063.stdout:9/555: mkdir d0/d4/d26/d39/d48/d80/d84/dc2 0 2024-05-09T06:38:14.175 INFO:tasks.workunit.client.0.smithi063.stdout:3/561: dwrite d7/d4c/d31/f6e [0,4194304] 0 2024-05-09T06:38:14.178 INFO:tasks.workunit.client.0.smithi063.stdout:2/588: truncate dc/dd/f4f 1893679 0 2024-05-09T06:38:14.181 INFO:tasks.workunit.client.0.smithi063.stdout:1/496: creat d9/d2a/d35/d24/faa x:0 0 0 2024-05-09T06:38:14.187 INFO:tasks.workunit.client.0.smithi063.stdout:4/530: link d7/d3d/c9c d7/ca8 0 2024-05-09T06:38:14.189 INFO:tasks.workunit.client.1.smithi193.stdout:3/162: creat d1/db/d1b/f3b x:0 0 0 2024-05-09T06:38:14.191 INFO:tasks.workunit.client.1.smithi193.stdout:8/224: symlink d6/d1c/d1b/d3a/l42 0 2024-05-09T06:38:14.200 INFO:tasks.workunit.client.0.smithi063.stdout:9/556: creat d0/d4/d26/d39/d48/d80/fc3 x:0 0 0 2024-05-09T06:38:14.200 INFO:tasks.workunit.client.0.smithi063.stdout:9/557: dread - d0/d4/d26/d39/d48/d80/fc1 zero size 2024-05-09T06:38:14.202 INFO:tasks.workunit.client.0.smithi063.stdout:3/562: rmdir d7/d70/d98/d5b 39 2024-05-09T06:38:14.203 INFO:tasks.workunit.client.0.smithi063.stdout:3/563: read f2 [5780741,83961] 0 2024-05-09T06:38:14.204 INFO:tasks.workunit.client.0.smithi063.stdout:3/564: fdatasync d7/da/d2b/d66/d74/d7e/f45 0 2024-05-09T06:38:14.205 INFO:tasks.workunit.client.0.smithi063.stdout:3/565: stat d7/d4c/l5f 0 2024-05-09T06:38:14.206 INFO:tasks.workunit.client.1.smithi193.stdout:2/208: write db/df/d12/f17 [323142,68784] 0 2024-05-09T06:38:14.206 INFO:tasks.workunit.client.1.smithi193.stdout:8/225: dwrite d6/f21 [4194304,4194304] 0 2024-05-09T06:38:14.207 INFO:tasks.workunit.client.0.smithi063.stdout:0/586: sync 2024-05-09T06:38:14.208 INFO:tasks.workunit.client.1.smithi193.stdout:8/226: chown d6/d1c/c15 320971 1 2024-05-09T06:38:14.209 INFO:tasks.workunit.client.0.smithi063.stdout:5/518: write d8/d17/f1c [825546,103193] 0 2024-05-09T06:38:14.212 INFO:tasks.workunit.client.1.smithi193.stdout:3/163: write f0 [1169615,129233] 0 2024-05-09T06:38:14.213 INFO:tasks.workunit.client.1.smithi193.stdout:4/241: dwrite d1a/f1b [0,4194304] 0 2024-05-09T06:38:14.216 INFO:tasks.workunit.client.0.smithi063.stdout:6/559: rename d2/d1f/d29/d3b/d4e/f67 to d2/d1f/d29/fb1 0 2024-05-09T06:38:14.218 INFO:tasks.workunit.client.1.smithi193.stdout:3/164: symlink d1/db/d26/l3c 0 2024-05-09T06:38:14.219 INFO:tasks.workunit.client.1.smithi193.stdout:3/165: write d1/db/d1b/f3b [621097,14077] 0 2024-05-09T06:38:14.220 INFO:tasks.workunit.client.1.smithi193.stdout:3/166: stat d1/db/d1b/f3b 0 2024-05-09T06:38:14.220 INFO:tasks.workunit.client.1.smithi193.stdout:2/209: write f8 [9500315,44421] 0 2024-05-09T06:38:14.221 INFO:tasks.workunit.client.1.smithi193.stdout:4/242: mknod d1a/d22/d43/c47 0 2024-05-09T06:38:14.222 INFO:tasks.workunit.client.1.smithi193.stdout:3/167: write d1/db/f34 [927660,98668] 0 2024-05-09T06:38:14.222 INFO:tasks.workunit.client.1.smithi193.stdout:3/168: readlink d1/lf 0 2024-05-09T06:38:14.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/497: dwrite d9/d2a/d35/f29 [0,4194304] 0 2024-05-09T06:38:14.224 INFO:tasks.workunit.client.1.smithi193.stdout:4/243: readlink l8 0 2024-05-09T06:38:14.226 INFO:tasks.workunit.client.1.smithi193.stdout:2/210: mkdir db/d4b 0 2024-05-09T06:38:14.228 INFO:tasks.workunit.client.0.smithi063.stdout:8/596: dwrite d9/da/d34/f4b [0,4194304] 0 2024-05-09T06:38:14.229 INFO:tasks.workunit.client.0.smithi063.stdout:0/587: unlink dd/d60/cbc 0 2024-05-09T06:38:14.230 INFO:tasks.workunit.client.1.smithi193.stdout:3/169: creat d1/d20/f3d x:0 0 0 2024-05-09T06:38:14.233 INFO:tasks.workunit.client.0.smithi063.stdout:8/597: dread d9/da/f2f [4194304,4194304] 0 2024-05-09T06:38:14.233 INFO:tasks.workunit.client.0.smithi063.stdout:7/522: link d0/d15/d4a/d50/f3c d0/d15/d49/d4f/d66/fb7 0 2024-05-09T06:38:14.234 INFO:tasks.workunit.client.1.smithi193.stdout:4/244: unlink d1a/d24/f3b 0 2024-05-09T06:38:14.235 INFO:tasks.workunit.client.0.smithi063.stdout:9/558: dread d0/d4/d26/d39/f55 [0,4194304] 0 2024-05-09T06:38:14.236 INFO:tasks.workunit.client.0.smithi063.stdout:5/519: creat d8/d17/faa x:0 0 0 2024-05-09T06:38:14.240 INFO:tasks.workunit.client.1.smithi193.stdout:4/245: mknod d1a/d24/d30/c48 0 2024-05-09T06:38:14.242 INFO:tasks.workunit.client.1.smithi193.stdout:2/211: symlink db/d4b/l4c 0 2024-05-09T06:38:14.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/212: unlink db/c2d 0 2024-05-09T06:38:14.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/213: dread - db/f44 zero size 2024-05-09T06:38:14.249 INFO:tasks.workunit.client.0.smithi063.stdout:1/498: creat d9/d2a/d3c/d4c/d83/d88/fab x:0 0 0 2024-05-09T06:38:14.251 INFO:tasks.workunit.client.1.smithi193.stdout:3/170: rename d1/db/d26 to d1/d3e 0 2024-05-09T06:38:14.255 INFO:tasks.workunit.client.0.smithi063.stdout:0/588: unlink dd/d1a/d2c/d51/d72/d9e/c68 0 2024-05-09T06:38:14.257 INFO:tasks.workunit.client.0.smithi063.stdout:7/523: unlink d0/l13 0 2024-05-09T06:38:14.260 INFO:tasks.workunit.client.0.smithi063.stdout:0/589: dwrite dd/d60/fa0 [0,4194304] 0 2024-05-09T06:38:14.260 INFO:tasks.workunit.client.1.smithi193.stdout:4/246: rename d1a/f28 to d1a/d24/d30/f49 0 2024-05-09T06:38:14.261 INFO:tasks.workunit.client.0.smithi063.stdout:0/590: fdatasync dd/d1a/d2c/d51/d72/d9e/f55 0 2024-05-09T06:38:14.261 INFO:tasks.workunit.client.0.smithi063.stdout:8/598: mkdir d9/da/d18/dc7/dd3 0 2024-05-09T06:38:14.265 INFO:tasks.workunit.client.1.smithi193.stdout:2/214: dread db/f21 [0,4194304] 0 2024-05-09T06:38:14.266 INFO:tasks.workunit.client.0.smithi063.stdout:9/559: creat d0/d4/d26/d39/d48/d80/d84/fc4 x:0 0 0 2024-05-09T06:38:14.270 INFO:tasks.workunit.client.0.smithi063.stdout:3/566: rename d7/da/d2b/d66/l96 to d7/da/d21/lb2 0 2024-05-09T06:38:14.271 INFO:tasks.workunit.client.1.smithi193.stdout:3/171: creat d1/db/f3f x:0 0 0 2024-05-09T06:38:14.271 INFO:tasks.workunit.client.0.smithi063.stdout:2/589: link dc/c1a dc/dd/ca9 0 2024-05-09T06:38:14.272 INFO:tasks.workunit.client.0.smithi063.stdout:2/590: truncate dc/dd/d45/f8a 704588 0 2024-05-09T06:38:14.274 INFO:tasks.workunit.client.1.smithi193.stdout:3/172: mknod d1/c40 0 2024-05-09T06:38:14.276 INFO:tasks.workunit.client.0.smithi063.stdout:0/591: creat dd/fbe x:0 0 0 2024-05-09T06:38:14.277 INFO:tasks.workunit.client.0.smithi063.stdout:0/592: dread - dd/d60/d44/d90/fb1 zero size 2024-05-09T06:38:14.278 INFO:tasks.workunit.client.1.smithi193.stdout:3/173: creat d1/db/f41 x:0 0 0 2024-05-09T06:38:14.279 INFO:tasks.workunit.client.1.smithi193.stdout:3/174: write d1/f24 [434951,121940] 0 2024-05-09T06:38:14.280 INFO:tasks.workunit.client.0.smithi063.stdout:5/520: rename d8/d19/d30/d46/f9a to d8/d53/fab 0 2024-05-09T06:38:14.282 INFO:tasks.workunit.client.0.smithi063.stdout:3/567: truncate d7/d4c/f69 1015922 0 2024-05-09T06:38:14.282 INFO:tasks.workunit.client.0.smithi063.stdout:6/560: link d2/d12/f24 d2/d1f/d6c/fb2 0 2024-05-09T06:38:14.283 INFO:tasks.workunit.client.1.smithi193.stdout:4/247: fsync d1a/d24/f2e 0 2024-05-09T06:38:14.285 INFO:tasks.workunit.client.0.smithi063.stdout:7/524: unlink d0/d15/d49/d4f/d66/d67/l7a 0 2024-05-09T06:38:14.287 INFO:tasks.workunit.client.0.smithi063.stdout:4/531: getdents d7/df/d57 0 2024-05-09T06:38:14.289 INFO:tasks.workunit.client.0.smithi063.stdout:8/599: creat d9/da/d18/dc7/d58/d79/db8/fd4 x:0 0 0 2024-05-09T06:38:14.290 INFO:tasks.workunit.client.1.smithi193.stdout:4/248: creat d1a/d24/d37/f4a x:0 0 0 2024-05-09T06:38:14.291 INFO:tasks.workunit.client.1.smithi193.stdout:4/249: stat l4 0 2024-05-09T06:38:14.291 INFO:tasks.workunit.client.1.smithi193.stdout:4/250: dread - d1a/d24/d30/f32 zero size 2024-05-09T06:38:14.291 INFO:tasks.workunit.client.0.smithi063.stdout:9/560: symlink d0/d4/d26/d39/d48/d80/d84/dc2/lc5 0 2024-05-09T06:38:14.293 INFO:tasks.workunit.client.0.smithi063.stdout:0/593: dread dd/d1a/d2c/d51/d72/d9e/f55 [0,4194304] 0 2024-05-09T06:38:14.293 INFO:tasks.workunit.client.0.smithi063.stdout:2/591: dread dc/dd/d45/d5a/f5e [0,4194304] 0 2024-05-09T06:38:14.294 INFO:tasks.workunit.client.1.smithi193.stdout:4/251: rmdir d1a/d22/d43 39 2024-05-09T06:38:14.294 INFO:tasks.workunit.client.0.smithi063.stdout:9/561: truncate d0/d4/d7d/f82 59894 0 2024-05-09T06:38:14.295 INFO:tasks.workunit.client.1.smithi193.stdout:4/252: write d1a/d24/d30/f32 [531688,42866] 0 2024-05-09T06:38:14.296 INFO:tasks.workunit.client.1.smithi193.stdout:3/175: link d1/d20/l30 d1/d2d/l42 0 2024-05-09T06:38:14.298 INFO:tasks.workunit.client.0.smithi063.stdout:3/568: creat d7/d4c/d31/fb3 x:0 0 0 2024-05-09T06:38:14.300 INFO:tasks.workunit.client.0.smithi063.stdout:6/561: creat d2/df/d59/d99/fb3 x:0 0 0 2024-05-09T06:38:14.301 INFO:tasks.workunit.client.0.smithi063.stdout:3/569: truncate d7/d4c/d31/faa 3543 0 2024-05-09T06:38:14.301 INFO:tasks.workunit.client.0.smithi063.stdout:2/592: read dc/d32/f42 [2391242,2833] 0 2024-05-09T06:38:14.302 INFO:tasks.workunit.client.1.smithi193.stdout:3/176: creat d1/d3e/f43 x:0 0 0 2024-05-09T06:38:14.303 INFO:tasks.workunit.client.0.smithi063.stdout:0/594: dwrite dd/d1a/d2c/d51/d72/d9e/faf [0,4194304] 0 2024-05-09T06:38:14.303 INFO:tasks.workunit.client.0.smithi063.stdout:2/593: write dc/d32/f7c [97591,59826] 0 2024-05-09T06:38:14.304 INFO:tasks.workunit.client.0.smithi063.stdout:7/525: write d0/d15/d49/d54/d9f/d97/d9b/fa3 [804301,45396] 0 2024-05-09T06:38:14.304 INFO:tasks.workunit.client.1.smithi193.stdout:2/215: write db/df/d12/f32 [97692,111700] 0 2024-05-09T06:38:14.305 INFO:tasks.workunit.client.0.smithi063.stdout:0/595: stat dd/d60/d44/l83 0 2024-05-09T06:38:14.305 INFO:tasks.workunit.client.0.smithi063.stdout:6/562: truncate d2/d1f/d29/d3f/d5c/fb0 645915 0 2024-05-09T06:38:14.306 INFO:tasks.workunit.client.0.smithi063.stdout:7/526: chown d0/d15/d49/d4f/d66/fb7 58617276 1 2024-05-09T06:38:14.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/594: chown dc/dd/d2d/d60/f67 14 1 2024-05-09T06:38:14.307 INFO:tasks.workunit.client.0.smithi063.stdout:7/527: fdatasync d0/d15/f48 0 2024-05-09T06:38:14.307 INFO:tasks.workunit.client.1.smithi193.stdout:7/213: sync 2024-05-09T06:38:14.307 INFO:tasks.workunit.client.1.smithi193.stdout:1/193: sync 2024-05-09T06:38:14.307 INFO:tasks.workunit.client.1.smithi193.stdout:0/217: sync 2024-05-09T06:38:14.308 INFO:tasks.workunit.client.0.smithi063.stdout:7/528: chown d0/d15/d49/d4f/d66/d67 4690131 1 2024-05-09T06:38:14.308 INFO:tasks.workunit.client.1.smithi193.stdout:0/218: chown f17 225237769 1 2024-05-09T06:38:14.308 INFO:tasks.workunit.client.1.smithi193.stdout:1/194: chown d4/f13 280 1 2024-05-09T06:38:14.309 INFO:tasks.workunit.client.0.smithi063.stdout:8/600: rename d9/da/d18/dc7/d58/d79/faf to d9/da/d34/fd5 0 2024-05-09T06:38:14.310 INFO:tasks.workunit.client.1.smithi193.stdout:3/177: dwrite d1/d3e/f43 [0,4194304] 0 2024-05-09T06:38:14.312 INFO:tasks.workunit.client.1.smithi193.stdout:2/216: mknod db/df/d12/c4d 0 2024-05-09T06:38:14.331 INFO:tasks.workunit.client.1.smithi193.stdout:7/214: readlink d6/l1d 0 2024-05-09T06:38:14.333 INFO:tasks.workunit.client.0.smithi063.stdout:3/570: creat d7/da/d2b/d66/fb4 x:0 0 0 2024-05-09T06:38:14.333 INFO:tasks.workunit.client.1.smithi193.stdout:7/215: dread d6/dd/f3a [0,4194304] 0 2024-05-09T06:38:14.334 INFO:tasks.workunit.client.1.smithi193.stdout:4/253: getdents d1a/d24/d30 0 2024-05-09T06:38:14.334 INFO:tasks.workunit.client.1.smithi193.stdout:7/216: chown d6/d8/d1a 3949001 1 2024-05-09T06:38:14.335 INFO:tasks.workunit.client.0.smithi063.stdout:4/532: mknod d7/df/d35/d5a/d60/d75/ca9 0 2024-05-09T06:38:14.335 INFO:tasks.workunit.client.0.smithi063.stdout:4/533: write d7/d50/f99 [850307,37491] 0 2024-05-09T06:38:14.336 INFO:tasks.workunit.client.1.smithi193.stdout:0/219: creat d18/d1d/f44 x:0 0 0 2024-05-09T06:38:14.337 INFO:tasks.workunit.client.1.smithi193.stdout:0/220: chown d18/d29/f37 631753148 1 2024-05-09T06:38:14.338 INFO:tasks.workunit.client.1.smithi193.stdout:3/178: fsync d1/db/f18 0 2024-05-09T06:38:14.338 INFO:tasks.workunit.client.1.smithi193.stdout:0/221: write f9 [7443535,66325] 0 2024-05-09T06:38:14.339 INFO:tasks.workunit.client.1.smithi193.stdout:3/179: dread - d1/d20/f3d zero size 2024-05-09T06:38:14.339 INFO:tasks.workunit.client.1.smithi193.stdout:0/222: stat d18/f28 0 2024-05-09T06:38:14.342 INFO:tasks.workunit.client.0.smithi063.stdout:1/499: sync 2024-05-09T06:38:14.342 INFO:tasks.workunit.client.1.smithi193.stdout:2/217: mknod db/d4b/c4e 0 2024-05-09T06:38:14.344 INFO:tasks.workunit.client.0.smithi063.stdout:0/596: symlink dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/lbf 0 2024-05-09T06:38:14.344 INFO:tasks.workunit.client.0.smithi063.stdout:0/597: chown dd/c5b 2144311218 1 2024-05-09T06:38:14.345 INFO:tasks.workunit.client.0.smithi063.stdout:0/598: readlink dd/d60/d44/d90/lab 0 2024-05-09T06:38:14.347 INFO:tasks.workunit.client.0.smithi063.stdout:5/521: creat d8/d17/d18/d9c/fac x:0 0 0 2024-05-09T06:38:14.348 INFO:tasks.workunit.client.1.smithi193.stdout:1/195: dread d4/f13 [4194304,4194304] 0 2024-05-09T06:38:14.352 INFO:tasks.workunit.client.0.smithi063.stdout:5/522: dwrite d8/d17/d18/f38 [4194304,4194304] 0 2024-05-09T06:38:14.353 INFO:tasks.workunit.client.0.smithi063.stdout:5/523: stat d8/d19/d84 0 2024-05-09T06:38:14.354 INFO:tasks.workunit.client.0.smithi063.stdout:5/524: chown d8/d19/d30/d46/f94 0 1 2024-05-09T06:38:14.360 INFO:tasks.workunit.client.1.smithi193.stdout:3/180: unlink d1/c12 0 2024-05-09T06:38:14.365 INFO:tasks.workunit.client.1.smithi193.stdout:0/223: rename d18/d29/f37 to d18/d29/f45 0 2024-05-09T06:38:14.367 INFO:tasks.workunit.client.1.smithi193.stdout:3/181: dwrite d1/f4 [0,4194304] 0 2024-05-09T06:38:14.367 INFO:tasks.workunit.client.1.smithi193.stdout:2/218: mknod db/d4b/c4f 0 2024-05-09T06:38:14.369 INFO:tasks.workunit.client.1.smithi193.stdout:2/219: write db/df/d12/f17 [918109,24236] 0 2024-05-09T06:38:14.369 INFO:tasks.workunit.client.1.smithi193.stdout:2/220: fsync db/f45 0 2024-05-09T06:38:14.377 INFO:tasks.workunit.client.1.smithi193.stdout:4/254: rmdir d1a/d22/d43 39 2024-05-09T06:38:14.379 INFO:tasks.workunit.client.1.smithi193.stdout:7/217: dwrite d6/d17/f42 [0,4194304] 0 2024-05-09T06:38:14.385 INFO:tasks.workunit.client.1.smithi193.stdout:4/255: dwrite d1a/d1f/f20 [0,4194304] 0 2024-05-09T06:38:14.390 INFO:tasks.workunit.client.1.smithi193.stdout:3/182: mkdir d1/d44 0 2024-05-09T06:38:14.391 INFO:tasks.workunit.client.1.smithi193.stdout:1/196: creat d4/d14/d15/d2b/f36 x:0 0 0 2024-05-09T06:38:14.393 INFO:tasks.workunit.client.1.smithi193.stdout:7/218: symlink d6/dd/l4a 0 2024-05-09T06:38:14.395 INFO:tasks.workunit.client.1.smithi193.stdout:4/256: mkdir d1a/d1f/d4b 0 2024-05-09T06:38:14.397 INFO:tasks.workunit.client.1.smithi193.stdout:3/183: dwrite d1/db/d1b/f3b [0,4194304] 0 2024-05-09T06:38:14.398 INFO:tasks.workunit.client.1.smithi193.stdout:3/184: fsync d1/db/f14 0 2024-05-09T06:38:14.400 INFO:tasks.workunit.client.1.smithi193.stdout:2/221: creat db/df/f50 x:0 0 0 2024-05-09T06:38:14.401 INFO:tasks.workunit.client.1.smithi193.stdout:2/222: chown db/l34 56910710 1 2024-05-09T06:38:14.402 INFO:tasks.workunit.client.1.smithi193.stdout:5/259: sync 2024-05-09T06:38:14.402 INFO:tasks.workunit.client.1.smithi193.stdout:2/223: chown db/df/d12/f13 1618837 1 2024-05-09T06:38:14.402 INFO:tasks.workunit.client.1.smithi193.stdout:4/257: dread f17 [0,4194304] 0 2024-05-09T06:38:14.403 INFO:tasks.workunit.client.0.smithi063.stdout:6/563: mknod d2/cb4 0 2024-05-09T06:38:14.403 INFO:tasks.workunit.client.1.smithi193.stdout:5/260: dread - d0/d1/da/d3a/f52 zero size 2024-05-09T06:38:14.404 INFO:tasks.workunit.client.1.smithi193.stdout:3/185: creat d1/d2d/f45 x:0 0 0 2024-05-09T06:38:14.404 INFO:tasks.workunit.client.1.smithi193.stdout:1/197: creat d4/f37 x:0 0 0 2024-05-09T06:38:14.405 INFO:tasks.workunit.client.0.smithi063.stdout:7/529: mknod d0/d15/d49/d4f/d66/cb8 0 2024-05-09T06:38:14.410 INFO:tasks.workunit.client.0.smithi063.stdout:3/571: symlink d7/d22/lb5 0 2024-05-09T06:38:14.412 INFO:tasks.workunit.client.1.smithi193.stdout:0/224: write d18/d1d/f23 [390501,108901] 0 2024-05-09T06:38:14.413 INFO:tasks.workunit.client.0.smithi063.stdout:8/601: link d9/da/d53/c98 d9/da/d18/dc7/d58/cd6 0 2024-05-09T06:38:14.414 INFO:tasks.workunit.client.1.smithi193.stdout:2/224: symlink db/l51 0 2024-05-09T06:38:14.416 INFO:tasks.workunit.client.1.smithi193.stdout:4/258: mknod d1a/d22/c4c 0 2024-05-09T06:38:14.416 INFO:tasks.workunit.client.0.smithi063.stdout:2/595: dwrite dc/dd/f23 [0,4194304] 0 2024-05-09T06:38:14.418 INFO:tasks.workunit.client.0.smithi063.stdout:9/562: link d0/f1f d0/d4/d26/d39/d3e/fc6 0 2024-05-09T06:38:14.420 INFO:tasks.workunit.client.1.smithi193.stdout:5/261: creat d0/d1/d4e/f57 x:0 0 0 2024-05-09T06:38:14.421 INFO:tasks.workunit.client.0.smithi063.stdout:4/534: creat d7/d3d/faa x:0 0 0 2024-05-09T06:38:14.423 INFO:tasks.workunit.client.0.smithi063.stdout:3/572: symlink d7/d4c/lb6 0 2024-05-09T06:38:14.425 INFO:tasks.workunit.client.1.smithi193.stdout:4/259: dwrite d1a/d24/d37/f4a [0,4194304] 0 2024-05-09T06:38:14.427 INFO:tasks.workunit.client.1.smithi193.stdout:7/219: write d6/dd/ff [10545869,3596] 0 2024-05-09T06:38:14.428 INFO:tasks.workunit.client.0.smithi063.stdout:5/525: dwrite d8/d17/d1e/f5f [0,4194304] 0 2024-05-09T06:38:14.429 INFO:tasks.workunit.client.0.smithi063.stdout:0/599: dwrite dd/d1a/d2c/d51/d72/d9e/f80 [0,4194304] 0 2024-05-09T06:38:14.429 INFO:tasks.workunit.client.1.smithi193.stdout:7/220: write d6/d17/f42 [2410577,66200] 0 2024-05-09T06:38:14.434 INFO:tasks.workunit.client.0.smithi063.stdout:7/530: dwrite d0/d15/d4a/d68/f9e [0,4194304] 0 2024-05-09T06:38:14.435 INFO:tasks.workunit.client.0.smithi063.stdout:0/600: dread - dd/d60/d44/d90/faa zero size 2024-05-09T06:38:14.435 INFO:tasks.workunit.client.1.smithi193.stdout:4/260: dwrite d1a/d24/d37/f44 [0,4194304] 0 2024-05-09T06:38:14.436 INFO:tasks.workunit.client.0.smithi063.stdout:9/563: creat d0/d4/d26/d39/d48/d80/d84/fc7 x:0 0 0 2024-05-09T06:38:14.438 INFO:tasks.workunit.client.1.smithi193.stdout:4/261: write d1a/f23 [4403932,117492] 0 2024-05-09T06:38:14.439 INFO:tasks.workunit.client.0.smithi063.stdout:9/564: write d0/d4/d26/d39/d3e/f8d [832533,17827] 0 2024-05-09T06:38:14.439 INFO:tasks.workunit.client.1.smithi193.stdout:4/262: readlink d1a/d1f/l2b 0 2024-05-09T06:38:14.441 INFO:tasks.workunit.client.1.smithi193.stdout:4/263: write f10 [2998437,11198] 0 2024-05-09T06:38:14.444 INFO:tasks.workunit.client.0.smithi063.stdout:7/531: dwrite d0/f18 [0,4194304] 0 2024-05-09T06:38:14.445 INFO:tasks.workunit.client.1.smithi193.stdout:1/198: dwrite d4/d16/f10 [0,4194304] 0 2024-05-09T06:38:14.446 INFO:tasks.workunit.client.0.smithi063.stdout:3/573: mkdir d7/d70/d98/d2d/d36/db7 0 2024-05-09T06:38:14.446 INFO:tasks.workunit.client.1.smithi193.stdout:1/199: fsync d4/ff 0 2024-05-09T06:38:14.449 INFO:tasks.workunit.client.1.smithi193.stdout:4/264: dread d1a/d24/d30/f34 [0,4194304] 0 2024-05-09T06:38:14.450 INFO:tasks.workunit.client.0.smithi063.stdout:5/526: creat d8/d35/fad x:0 0 0 2024-05-09T06:38:14.452 INFO:tasks.workunit.client.1.smithi193.stdout:7/221: symlink d6/d17/l4b 0 2024-05-09T06:38:14.452 INFO:tasks.workunit.client.1.smithi193.stdout:2/225: symlink db/d25/d38/d40/l52 0 2024-05-09T06:38:14.455 INFO:tasks.workunit.client.1.smithi193.stdout:2/226: write f1 [8725934,26427] 0 2024-05-09T06:38:14.455 INFO:tasks.workunit.client.1.smithi193.stdout:8/227: sync 2024-05-09T06:38:14.455 INFO:tasks.workunit.client.1.smithi193.stdout:6/196: sync 2024-05-09T06:38:14.455 INFO:tasks.workunit.client.1.smithi193.stdout:9/234: sync 2024-05-09T06:38:14.456 INFO:tasks.workunit.client.1.smithi193.stdout:3/186: getdents d1 0 2024-05-09T06:38:14.460 INFO:tasks.workunit.client.1.smithi193.stdout:2/227: fdatasync db/d25/f48 0 2024-05-09T06:38:14.466 INFO:tasks.workunit.client.1.smithi193.stdout:1/200: mknod d4/d16/de/c38 0 2024-05-09T06:38:14.470 INFO:tasks.workunit.client.0.smithi063.stdout:7/532: write d0/d15/d4a/d68/f75 [6656427,56891] 0 2024-05-09T06:38:14.470 INFO:tasks.workunit.client.0.smithi063.stdout:3/574: dread - d7/d70/d98/d5b/f9b zero size 2024-05-09T06:38:14.470 INFO:tasks.workunit.client.0.smithi063.stdout:3/575: chown d7/d22/d28 22 1 2024-05-09T06:38:14.472 INFO:tasks.workunit.client.1.smithi193.stdout:4/265: dwrite d1a/d24/d37/f4a [0,4194304] 0 2024-05-09T06:38:14.472 INFO:tasks.workunit.client.0.smithi063.stdout:0/601: read dd/d1a/d2c/d51/d72/d9e/f2b [3978517,14965] 0 2024-05-09T06:38:14.473 INFO:tasks.workunit.client.1.smithi193.stdout:9/235: unlink dd/df/c42 0 2024-05-09T06:38:14.474 INFO:tasks.workunit.client.1.smithi193.stdout:4/266: chown d1a/d1f/c21 484856345 1 2024-05-09T06:38:14.476 INFO:tasks.workunit.client.1.smithi193.stdout:1/201: write d4/f26 [636083,110997] 0 2024-05-09T06:38:14.476 INFO:tasks.workunit.client.0.smithi063.stdout:9/565: symlink d0/d4/d26/d39/d3e/lc8 0 2024-05-09T06:38:14.477 INFO:tasks.workunit.client.1.smithi193.stdout:3/187: symlink d1/db/d1b/l46 0 2024-05-09T06:38:14.477 INFO:tasks.workunit.client.0.smithi063.stdout:9/566: write d0/d8/da6/db0/fb4 [569259,47183] 0 2024-05-09T06:38:14.479 INFO:tasks.workunit.client.1.smithi193.stdout:5/262: dread d0/d1/f6 [0,4194304] 0 2024-05-09T06:38:14.480 INFO:tasks.workunit.client.1.smithi193.stdout:6/197: creat d1/d25/d2c/d3c/d3e/f3f x:0 0 0 2024-05-09T06:38:14.480 INFO:tasks.workunit.client.1.smithi193.stdout:4/267: read f7 [5839515,124261] 0 2024-05-09T06:38:14.487 INFO:tasks.workunit.client.1.smithi193.stdout:4/268: chown d1a/d24/d30/f34 23994566 1 2024-05-09T06:38:14.487 INFO:tasks.workunit.client.0.smithi063.stdout:3/576: rmdir d7/da/d2b/d66/d74 39 2024-05-09T06:38:14.488 INFO:tasks.workunit.client.1.smithi193.stdout:4/269: readlink l15 0 2024-05-09T06:38:14.491 INFO:tasks.workunit.client.1.smithi193.stdout:3/188: dwrite d1/db/d1b/f3b [0,4194304] 0 2024-05-09T06:38:14.492 INFO:tasks.workunit.client.1.smithi193.stdout:4/270: write d1a/d24/f41 [625867,68937] 0 2024-05-09T06:38:14.492 INFO:tasks.workunit.client.0.smithi063.stdout:2/596: dwrite dc/dd/d45/d5a/f66 [0,4194304] 0 2024-05-09T06:38:14.492 INFO:tasks.workunit.client.0.smithi063.stdout:8/602: dread f4 [4194304,4194304] 0 2024-05-09T06:38:14.493 INFO:tasks.workunit.client.1.smithi193.stdout:5/263: chown d0/d1/da/f23 10 1 2024-05-09T06:38:14.493 INFO:tasks.workunit.client.1.smithi193.stdout:7/222: creat d6/d8/d29/d2f/f4c x:0 0 0 2024-05-09T06:38:14.495 INFO:tasks.workunit.client.1.smithi193.stdout:3/189: chown d1/db/cc 227 1 2024-05-09T06:38:14.497 INFO:tasks.workunit.client.0.smithi063.stdout:0/602: mkdir dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0 0 2024-05-09T06:38:14.499 INFO:tasks.workunit.client.0.smithi063.stdout:9/567: read - d0/d4/d57/d60/d76/f8f zero size 2024-05-09T06:38:14.501 INFO:tasks.workunit.client.1.smithi193.stdout:6/198: creat d1/de/d11/d2d/f40 x:0 0 0 2024-05-09T06:38:14.504 INFO:tasks.workunit.client.1.smithi193.stdout:6/199: truncate d1/de/d11/f1d 923838 0 2024-05-09T06:38:14.505 INFO:tasks.workunit.client.1.smithi193.stdout:6/200: rename d1/de/d11 to d1/de/d11/d18/d41 22 2024-05-09T06:38:14.506 INFO:tasks.workunit.client.1.smithi193.stdout:5/264: rmdir d0 39 2024-05-09T06:38:14.507 INFO:tasks.workunit.client.1.smithi193.stdout:7/223: dwrite d6/d8/f44 [0,4194304] 0 2024-05-09T06:38:14.509 INFO:tasks.workunit.client.0.smithi063.stdout:5/527: link d8/c50 d8/d17/d1e/cae 0 2024-05-09T06:38:14.510 INFO:tasks.workunit.client.0.smithi063.stdout:5/528: truncate d8/d17/d1e/f81 1220689 0 2024-05-09T06:38:14.510 INFO:tasks.workunit.client.0.smithi063.stdout:8/603: mknod d9/da/d53/da3/cd7 0 2024-05-09T06:38:14.513 INFO:tasks.workunit.client.0.smithi063.stdout:0/603: fsync dd/f12 0 2024-05-09T06:38:14.515 INFO:tasks.workunit.client.1.smithi193.stdout:4/271: creat d1a/d22/d43/f4d x:0 0 0 2024-05-09T06:38:14.517 INFO:tasks.workunit.client.1.smithi193.stdout:5/265: readlink d0/d13/l51 0 2024-05-09T06:38:14.519 INFO:tasks.workunit.client.0.smithi063.stdout:3/577: rename d7/da/d21/d30/d44/l56 to d7/d22/d28/dab/lb8 0 2024-05-09T06:38:14.522 INFO:tasks.workunit.client.1.smithi193.stdout:3/190: creat d1/d44/f47 x:0 0 0 2024-05-09T06:38:14.522 INFO:tasks.workunit.client.0.smithi063.stdout:2/597: symlink dc/d36/laa 0 2024-05-09T06:38:14.523 INFO:tasks.workunit.client.1.smithi193.stdout:3/191: fsync d1/f1d 0 2024-05-09T06:38:14.524 INFO:tasks.workunit.client.1.smithi193.stdout:7/224: mkdir d6/d8/d1a/d36/d4d 0 2024-05-09T06:38:14.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.546+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 --> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0038105ec0 con 0x7f002006d720 2024-05-09T06:38:14.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.548+0000 7f0034ff9700 1 -- 172.21.15.63:0/1326672302 <== mgr.24415 v2:172.21.15.193:6828/1432929717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f0038105ec0 con 0x7f002006d720 2024-05-09T06:38:14.528 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:38:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:38:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:38:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:38:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-05-09T06:38:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "1/23 daemons upgraded", 2024-05-09T06:38:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-05-09T06:38:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:38:14.529 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:38:14.529 INFO:tasks.workunit.client.1.smithi193.stdout:1/202: dread d4/f32 [0,4194304] 0 2024-05-09T06:38:14.530 INFO:tasks.workunit.client.1.smithi193.stdout:0/225: dwrite f13 [0,4194304] 0 2024-05-09T06:38:14.530 INFO:tasks.workunit.client.0.smithi063.stdout:8/604: mknod d9/da/d53/da3/cd8 0 2024-05-09T06:38:14.531 INFO:tasks.workunit.client.0.smithi063.stdout:5/529: dwrite d8/f8f [0,4194304] 0 2024-05-09T06:38:14.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.551+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f002006d720 msgr2=0x7f002006fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.551+0000 7f003dd11700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f002006d720 0x7f002006fbe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0028009be0 tx=0x7f0028009260 comp rx=0 tx=0).stop 2024-05-09T06:38:14.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.551+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0038103fe0 msgr2=0x7f0038196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.551+0000 7f003dd11700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0038103fe0 0x7f0038196b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f002c00b800 tx=0x7f002c00f510 comp rx=0 tx=0).stop 2024-05-09T06:38:14.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.552+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 shutdown_connections 2024-05-09T06:38:14.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.552+0000 7f003dd11700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f002006d720 0x7f002006fbe0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.552+0000 7f003dd11700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00381016b0 0x7f00381965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.552+0000 7f003dd11700 1 --2- 172.21.15.63:0/1326672302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0038103fe0 0x7f0038196b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.552+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 >> 172.21.15.63:0/1326672302 conn(0x7f00380fb0c0 msgr2=0x7f00380fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:14.533 INFO:tasks.workunit.client.1.smithi193.stdout:3/192: dread d1/db/f19 [0,4194304] 0 2024-05-09T06:38:14.533 INFO:tasks.workunit.client.0.smithi063.stdout:5/530: stat d8/l92 0 2024-05-09T06:38:14.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.553+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 shutdown_connections 2024-05-09T06:38:14.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.553+0000 7f003dd11700 1 -- 172.21.15.63:0/1326672302 wait complete. 2024-05-09T06:38:14.537 INFO:tasks.workunit.client.0.smithi063.stdout:0/604: symlink dd/d1a/d2c/d42/lc1 0 2024-05-09T06:38:14.543 INFO:tasks.workunit.client.1.smithi193.stdout:7/225: creat d6/d8/d1a/d36/f4e x:0 0 0 2024-05-09T06:38:14.545 INFO:tasks.workunit.client.1.smithi193.stdout:8/228: write d6/fd [1219995,70831] 0 2024-05-09T06:38:14.547 INFO:tasks.workunit.client.0.smithi063.stdout:4/535: truncate d7/df/d1a/f22 1977087 0 2024-05-09T06:38:14.548 INFO:tasks.workunit.client.1.smithi193.stdout:1/203: rmdir d4/d14/d15 39 2024-05-09T06:38:14.549 INFO:tasks.workunit.client.0.smithi063.stdout:4/536: chown d7/da3/d28/d31/d32/d34/f44 35497 1 2024-05-09T06:38:14.550 INFO:tasks.workunit.client.0.smithi063.stdout:0/605: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d31/f4a [0,4194304] 0 2024-05-09T06:38:14.552 INFO:tasks.workunit.client.0.smithi063.stdout:7/533: rename d0/d15/l87 to d0/d15/d49/d54/d9f/d97/d9b/d9d/lb9 0 2024-05-09T06:38:14.555 INFO:tasks.workunit.client.0.smithi063.stdout:2/598: creat dc/d32/d33/d4e/fab x:0 0 0 2024-05-09T06:38:14.562 INFO:tasks.workunit.client.1.smithi193.stdout:0/226: dwrite d18/d29/f45 [0,4194304] 0 2024-05-09T06:38:14.563 INFO:tasks.workunit.client.0.smithi063.stdout:3/578: dread d7/d22/d28/f3c [0,4194304] 0 2024-05-09T06:38:14.563 INFO:tasks.workunit.client.1.smithi193.stdout:3/193: rename d1/lf to d1/d20/l48 0 2024-05-09T06:38:14.565 INFO:tasks.workunit.client.0.smithi063.stdout:8/605: creat d9/da/d51/dc2/fd9 x:0 0 0 2024-05-09T06:38:14.566 INFO:tasks.workunit.client.0.smithi063.stdout:1/500: sync 2024-05-09T06:38:14.566 INFO:tasks.workunit.client.0.smithi063.stdout:8/606: write d9/da/d18/f1d [2003185,39677] 0 2024-05-09T06:38:14.573 INFO:tasks.workunit.client.1.smithi193.stdout:3/194: dwrite d1/db/f19 [4194304,4194304] 0 2024-05-09T06:38:14.576 INFO:tasks.workunit.client.1.smithi193.stdout:3/195: fdatasync d1/d3e/f37 0 2024-05-09T06:38:14.579 INFO:tasks.workunit.client.1.smithi193.stdout:4/272: write f12 [650032,27559] 0 2024-05-09T06:38:14.583 INFO:tasks.workunit.client.1.smithi193.stdout:6/201: getdents d1/de/d11/d20 0 2024-05-09T06:38:14.583 INFO:tasks.workunit.client.1.smithi193.stdout:7/226: dread d6/dd/f22 [0,4194304] 0 2024-05-09T06:38:14.583 INFO:tasks.workunit.client.1.smithi193.stdout:0/227: dread f17 [0,4194304] 0 2024-05-09T06:38:14.584 INFO:tasks.workunit.client.1.smithi193.stdout:0/228: stat fe 0 2024-05-09T06:38:14.586 INFO:tasks.workunit.client.0.smithi063.stdout:7/534: dread d0/d15/d49/d4f/d66/f71 [0,4194304] 0 2024-05-09T06:38:14.587 INFO:tasks.workunit.client.1.smithi193.stdout:7/227: write d6/d8/d1a/f3c [4759181,76003] 0 2024-05-09T06:38:14.589 INFO:tasks.workunit.client.0.smithi063.stdout:9/568: dwrite d0/d4/d26/f49 [0,4194304] 0 2024-05-09T06:38:14.589 INFO:tasks.workunit.client.1.smithi193.stdout:0/229: read d18/d29/f2b [177049,69904] 0 2024-05-09T06:38:14.591 INFO:tasks.workunit.client.1.smithi193.stdout:5/266: dwrite d0/d1/da/f15 [0,4194304] 0 2024-05-09T06:38:14.594 INFO:tasks.workunit.client.1.smithi193.stdout:3/196: mknod d1/db/d1b/c49 0 2024-05-09T06:38:14.596 INFO:tasks.workunit.client.1.smithi193.stdout:4/273: symlink d1a/d24/l4e 0 2024-05-09T06:38:14.598 INFO:tasks.workunit.client.1.smithi193.stdout:4/274: readlink d1a/d1f/l2b 0 2024-05-09T06:38:14.600 INFO:tasks.workunit.client.1.smithi193.stdout:4/275: dread - d1a/f1d zero size 2024-05-09T06:38:14.601 INFO:tasks.workunit.client.1.smithi193.stdout:0/230: dwrite d18/d27/f34 [0,4194304] 0 2024-05-09T06:38:14.604 INFO:tasks.workunit.client.1.smithi193.stdout:1/204: creat d4/d14/f39 x:0 0 0 2024-05-09T06:38:14.606 INFO:tasks.workunit.client.1.smithi193.stdout:4/276: dread f10 [0,4194304] 0 2024-05-09T06:38:14.608 INFO:tasks.workunit.client.1.smithi193.stdout:4/277: truncate d1a/f1d 976584 0 2024-05-09T06:38:14.614 INFO:tasks.workunit.client.1.smithi193.stdout:7/228: rename d6/d8/d1a/d33/f46 to d6/d8/d29/d2f/f4f 0 2024-05-09T06:38:14.615 INFO:tasks.workunit.client.1.smithi193.stdout:7/229: dread - d6/d8/d1a/d36/f3b zero size 2024-05-09T06:38:14.618 INFO:tasks.workunit.client.1.smithi193.stdout:5/267: unlink d0/f56 0 2024-05-09T06:38:14.621 INFO:tasks.workunit.client.1.smithi193.stdout:3/197: creat d1/d44/f4a x:0 0 0 2024-05-09T06:38:14.639 INFO:tasks.workunit.client.1.smithi193.stdout:5/268: creat d0/d1/d4e/f58 x:0 0 0 2024-05-09T06:38:14.640 INFO:tasks.workunit.client.1.smithi193.stdout:3/198: mknod d1/c4b 0 2024-05-09T06:38:14.643 INFO:tasks.workunit.client.1.smithi193.stdout:5/269: dread d0/d13/d2c/f4a [0,4194304] 0 2024-05-09T06:38:14.643 INFO:tasks.workunit.client.1.smithi193.stdout:1/205: mkdir d4/d16/d17/d1a/d3a 0 2024-05-09T06:38:14.644 INFO:tasks.workunit.client.1.smithi193.stdout:5/270: chown d0/d13/d2c/f4b 2864 1 2024-05-09T06:38:14.645 INFO:tasks.workunit.client.1.smithi193.stdout:6/202: link d1/de/d11/d2d/f40 d1/f42 0 2024-05-09T06:38:14.648 INFO:tasks.workunit.client.1.smithi193.stdout:4/278: rename cf to d1a/d22/d43/c4f 0 2024-05-09T06:38:14.649 INFO:tasks.workunit.client.1.smithi193.stdout:7/230: link d6/d8/d1a/d36/f3b d6/d8/d1a/d36/f50 0 2024-05-09T06:38:14.651 INFO:tasks.workunit.client.1.smithi193.stdout:5/271: dwrite d0/d1/da/f15 [0,4194304] 0 2024-05-09T06:38:14.651 INFO:tasks.workunit.client.0.smithi063.stdout:6/564: sync 2024-05-09T06:38:14.651 INFO:tasks.workunit.client.1.smithi193.stdout:7/231: write d6/d8/d1a/f3c [1601073,111917] 0 2024-05-09T06:38:14.652 INFO:tasks.workunit.client.1.smithi193.stdout:3/199: mknod d1/db/d1b/c4c 0 2024-05-09T06:38:14.657 INFO:tasks.workunit.client.0.smithi063.stdout:0/606: mknod dd/d1a/d2c/d51/cc2 0 2024-05-09T06:38:14.659 INFO:tasks.workunit.client.1.smithi193.stdout:4/279: mknod d1a/d24/c50 0 2024-05-09T06:38:14.664 INFO:tasks.workunit.client.0.smithi063.stdout:3/579: creat d7/da/d21/d30/fb9 x:0 0 0 2024-05-09T06:38:14.665 INFO:tasks.workunit.client.1.smithi193.stdout:0/231: getdents d18/d27 0 2024-05-09T06:38:14.665 INFO:tasks.workunit.client.0.smithi063.stdout:0/607: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/f88 [0,4194304] 0 2024-05-09T06:38:14.666 INFO:tasks.workunit.client.1.smithi193.stdout:0/232: dread - d18/d1d/f33 zero size 2024-05-09T06:38:14.667 INFO:tasks.workunit.client.1.smithi193.stdout:0/233: write d18/d1d/f30 [1068798,129622] 0 2024-05-09T06:38:14.668 INFO:tasks.workunit.client.1.smithi193.stdout:0/234: chown d18/d1d/c20 24459705 1 2024-05-09T06:38:14.670 INFO:tasks.workunit.client.1.smithi193.stdout:0/235: write d18/d27/f34 [3554404,29459] 0 2024-05-09T06:38:14.671 INFO:tasks.workunit.client.0.smithi063.stdout:1/501: creat d9/d2a/d35/d24/fac x:0 0 0 2024-05-09T06:38:14.673 INFO:tasks.workunit.client.0.smithi063.stdout:2/599: dread dc/dd/d2d/f77 [0,4194304] 0 2024-05-09T06:38:14.673 INFO:tasks.workunit.client.1.smithi193.stdout:3/200: rename d1/c39 to d1/d3e/c4d 0 2024-05-09T06:38:14.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.697+0000 7f2cb3745700 1 -- 172.21.15.63:0/1725426052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac0ffba0 msgr2=0x7f2cac0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.697+0000 7f2cb3745700 1 --2- 172.21.15.63:0/1725426052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac0ffba0 0x7f2cac0fffc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009910 tx=0x7f2ca8009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:14.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.697+0000 7f2cb3745700 1 -- 172.21.15.63:0/1725426052 shutdown_connections 2024-05-09T06:38:14.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.697+0000 7f2cb3745700 1 --2- 172.21.15.63:0/1725426052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac100da0 0x7f2cac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.697+0000 7f2cb3745700 1 --2- 172.21.15.63:0/1725426052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac0ffba0 0x7f2cac0fffc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.697+0000 7f2cb3745700 1 -- 172.21.15.63:0/1725426052 >> 172.21.15.63:0/1725426052 conn(0x7f2cac0fb120 msgr2=0x7f2cac0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:14.677 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:14 smithi063 ceph-mon[27171]: pgmap v10: 65 pgs: 65 active+clean; 1.3 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 166 MiB/s wr, 334 op/s 2024-05-09T06:38:14.678 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:14 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1247040273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:38:14.678 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:14 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/3291182600' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:38:14.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.697+0000 7f2cb3745700 1 -- 172.21.15.63:0/1725426052 shutdown_connections 2024-05-09T06:38:14.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.697+0000 7f2cb3745700 1 -- 172.21.15.63:0/1725426052 wait complete. 2024-05-09T06:38:14.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.697+0000 7f2cb3745700 1 Processor -- start 2024-05-09T06:38:14.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb3745700 1 -- start start 2024-05-09T06:38:14.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb3745700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac0ffba0 0x7f2cac196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb3745700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac100da0 0x7f2cac196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb3745700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac197290 con 0x7f2cac100da0 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb3745700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1973d0 con 0x7f2cac0ffba0 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb0ce0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac100da0 0x7f2cac196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb0ce0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac100da0 0x7f2cac196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56098/0 (socket says 172.21.15.63:56098) 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb0ce0700 1 -- 172.21.15.63:0/883758720 learned_addr learned my addr 172.21.15.63:0/883758720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb14e1700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac0ffba0 0x7f2cac196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb0ce0700 1 -- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac0ffba0 msgr2=0x7f2cac196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb0ce0700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac0ffba0 0x7f2cac196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb0ce0700 1 -- 172.21.15.63:0/883758720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80095f0 con 0x7f2cac100da0 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.698+0000 7f2cb0ce0700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac100da0 0x7f2cac196c70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00b4b0 tx=0x7f2c9c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.699+0000 7f2ca27fc700 1 -- 172.21.15.63:0/883758720 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c9c011800 con 0x7f2cac100da0 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.699+0000 7f2ca27fc700 1 -- 172.21.15.63:0/883758720 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c9c011e20 con 0x7f2cac100da0 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.699+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac19be30 con 0x7f2cac100da0 2024-05-09T06:38:14.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.699+0000 7f2ca27fc700 1 -- 172.21.15.63:0/883758720 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c9c00f2e0 con 0x7f2cac100da0 2024-05-09T06:38:14.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.699+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac19c3d0 con 0x7f2cac100da0 2024-05-09T06:38:14.682 INFO:tasks.workunit.client.1.smithi193.stdout:6/203: link d1/de/f2f d1/d25/d2c/d3c/d3e/f43 0 2024-05-09T06:38:14.682 INFO:tasks.workunit.client.1.smithi193.stdout:6/204: chown d1/d25/d2c/d3c/f3d 674 1 2024-05-09T06:38:14.682 INFO:tasks.workunit.client.1.smithi193.stdout:4/280: mknod d1a/d24/d37/c51 0 2024-05-09T06:38:14.682 INFO:tasks.workunit.client.1.smithi193.stdout:3/201: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:14.682 INFO:tasks.workunit.client.1.smithi193.stdout:6/205: chown d1/c8 12 1 2024-05-09T06:38:14.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.700+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac04ebb0 con 0x7f2cac100da0 2024-05-09T06:38:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.701+0000 7f2ca27fc700 1 -- 172.21.15.63:0/883758720 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f2c9c010380 con 0x7f2cac100da0 2024-05-09T06:38:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.701+0000 7f2ca27fc700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f2c9806d6f0 0x7f2c9806fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.701+0000 7f2ca27fc700 1 -- 172.21.15.63:0/883758720 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f2c9c08caa0 con 0x7f2cac100da0 2024-05-09T06:38:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.701+0000 7f2cb14e1700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f2c9806d6f0 0x7f2c9806fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.702+0000 7f2cb14e1700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f2c9806d6f0 0x7f2c9806fbb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c00 tx=0x7f2ca8005960 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:14.683 INFO:tasks.workunit.client.1.smithi193.stdout:3/202: chown d1/c2 0 1 2024-05-09T06:38:14.684 INFO:tasks.workunit.client.1.smithi193.stdout:3/203: chown d1/db/c2e 15471 1 2024-05-09T06:38:14.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:14.706+0000 7f2ca27fc700 1 -- 172.21.15.63:0/883758720 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f2c9c059f50 con 0x7f2cac100da0 2024-05-09T06:38:14.688 INFO:tasks.workunit.client.1.smithi193.stdout:5/272: dread d0/f7 [0,4194304] 0 2024-05-09T06:38:14.690 INFO:tasks.workunit.client.0.smithi063.stdout:6/565: symlink d2/d1f/d6c/lb5 0 2024-05-09T06:38:14.697 INFO:tasks.workunit.client.1.smithi193.stdout:7/232: fdatasync d6/d8/d29/d2f/f4f 0 2024-05-09T06:38:14.697 INFO:tasks.workunit.client.1.smithi193.stdout:7/233: read - d6/dd/f47 zero size 2024-05-09T06:38:14.702 INFO:tasks.workunit.client.0.smithi063.stdout:8/607: rmdir d9/d37 39 2024-05-09T06:38:14.705 INFO:tasks.workunit.client.1.smithi193.stdout:3/204: creat d1/d3e/f4e x:0 0 0 2024-05-09T06:38:14.707 INFO:tasks.workunit.client.0.smithi063.stdout:7/535: mknod d0/cba 0 2024-05-09T06:38:14.707 INFO:tasks.workunit.client.1.smithi193.stdout:5/273: symlink d0/d13/d2c/d37/d2b/d47/l59 0 2024-05-09T06:38:14.709 INFO:tasks.workunit.client.1.smithi193.stdout:1/206: getdents d4/d14 0 2024-05-09T06:38:14.711 INFO:tasks.workunit.client.0.smithi063.stdout:4/537: write d7/da3/d28/d31/d61/f6d [216469,76535] 0 2024-05-09T06:38:14.712 INFO:tasks.workunit.client.1.smithi193.stdout:7/234: fsync d6/f13 0 2024-05-09T06:38:14.713 INFO:tasks.workunit.client.0.smithi063.stdout:3/580: mknod d7/da/d2b/cba 0 2024-05-09T06:38:14.714 INFO:tasks.workunit.client.1.smithi193.stdout:7/235: write d6/dd/f47 [580327,128122] 0 2024-05-09T06:38:14.718 INFO:tasks.workunit.client.1.smithi193.stdout:6/206: mkdir d1/d44 0 2024-05-09T06:38:14.720 INFO:tasks.workunit.client.0.smithi063.stdout:0/608: dwrite dd/d1a/d50/f7c [0,4194304] 0 2024-05-09T06:38:14.721 INFO:tasks.workunit.client.0.smithi063.stdout:5/531: getdents d8/d65 0 2024-05-09T06:38:14.721 INFO:tasks.workunit.client.1.smithi193.stdout:0/236: truncate f13 2802204 0 2024-05-09T06:38:14.724 INFO:tasks.workunit.client.0.smithi063.stdout:9/569: creat d0/d4/d57/d60/fc9 x:0 0 0 2024-05-09T06:38:14.725 INFO:tasks.workunit.client.0.smithi063.stdout:6/566: creat d2/df/d91/fb6 x:0 0 0 2024-05-09T06:38:14.726 INFO:tasks.workunit.client.0.smithi063.stdout:9/570: write d0/d4/d57/f9b [196899,87760] 0 2024-05-09T06:38:14.727 INFO:tasks.workunit.client.1.smithi193.stdout:7/236: fdatasync d6/d8/d29/d2f/f4f 0 2024-05-09T06:38:14.730 INFO:tasks.workunit.client.0.smithi063.stdout:2/600: creat dc/dd/d2d/d60/fac x:0 0 0 2024-05-09T06:38:14.731 INFO:tasks.workunit.client.1.smithi193.stdout:0/237: creat d18/d29/f46 x:0 0 0 2024-05-09T06:38:14.732 INFO:tasks.workunit.client.1.smithi193.stdout:0/238: write f15 [1616403,100787] 0 2024-05-09T06:38:14.736 INFO:tasks.workunit.client.0.smithi063.stdout:0/609: rename dd/d1a/d2c/d51/d72/d9e/d23/f2f to dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fc3 0 2024-05-09T06:38:14.740 INFO:tasks.workunit.client.1.smithi193.stdout:1/207: rename d4/d16/d17/c30 to d4/d16/d17/d1a/d3a/c3b 0 2024-05-09T06:38:14.741 INFO:tasks.workunit.client.1.smithi193.stdout:1/208: chown d4/d16/d35 12 1 2024-05-09T06:38:14.743 INFO:tasks.workunit.client.0.smithi063.stdout:6/567: unlink d2/d4/f5 0 2024-05-09T06:38:14.743 INFO:tasks.workunit.client.1.smithi193.stdout:1/209: dread d4/f9 [0,4194304] 0 2024-05-09T06:38:14.743 INFO:tasks.workunit.client.0.smithi063.stdout:6/568: chown d2/d1f/d29/d3f/d5c/f90 701508 1 2024-05-09T06:38:14.744 INFO:tasks.workunit.client.1.smithi193.stdout:1/210: fsync d4/d16/f10 0 2024-05-09T06:38:14.744 INFO:tasks.workunit.client.0.smithi063.stdout:6/569: chown d2/d12/f53 307 1 2024-05-09T06:38:14.744 INFO:tasks.workunit.client.0.smithi063.stdout:3/581: truncate f3 1172425 0 2024-05-09T06:38:14.745 INFO:tasks.workunit.client.0.smithi063.stdout:6/570: dread - d2/df/d59/faf zero size 2024-05-09T06:38:14.746 INFO:tasks.workunit.client.1.smithi193.stdout:6/207: symlink d1/de/d11/d18/d36/d37/l45 0 2024-05-09T06:38:14.748 INFO:tasks.workunit.client.0.smithi063.stdout:9/571: unlink d0/d4/da/c59 0 2024-05-09T06:38:14.750 INFO:tasks.workunit.client.0.smithi063.stdout:9/572: write d0/d4/d57/d60/d76/fa9 [599242,96728] 0 2024-05-09T06:38:14.750 INFO:tasks.workunit.client.0.smithi063.stdout:5/532: write d8/f16 [2868473,69720] 0 2024-05-09T06:38:14.751 INFO:tasks.workunit.client.0.smithi063.stdout:2/601: mkdir dc/dd/d45/d5a/d6c/d8c/dad 0 2024-05-09T06:38:14.751 INFO:tasks.workunit.client.0.smithi063.stdout:5/533: stat d8/d19/d30/d46/f6c 0 2024-05-09T06:38:14.751 INFO:tasks.workunit.client.1.smithi193.stdout:5/274: truncate d0/d13/f3d 4080317 0 2024-05-09T06:38:14.752 INFO:tasks.workunit.client.1.smithi193.stdout:4/281: getdents d1a/d22/d43 0 2024-05-09T06:38:14.753 INFO:tasks.workunit.client.0.smithi063.stdout:5/534: fsync d8/d17/d1e/f8c 0 2024-05-09T06:38:14.753 INFO:tasks.workunit.client.0.smithi063.stdout:9/573: truncate d0/d8/d2d/f7b 1215279 0 2024-05-09T06:38:14.758 INFO:tasks.workunit.client.0.smithi063.stdout:5/535: dread d8/d17/f56 [0,4194304] 0 2024-05-09T06:38:14.758 INFO:tasks.workunit.client.0.smithi063.stdout:0/610: unlink dd/d60/l5d 0 2024-05-09T06:38:14.761 INFO:tasks.workunit.client.0.smithi063.stdout:0/611: write dd/d1a/d2c/d51/d72/d9e/d23/d31/fa2 [158768,98044] 0 2024-05-09T06:38:14.761 INFO:tasks.workunit.client.0.smithi063.stdout:4/538: creat d7/d3d/fab x:0 0 0 2024-05-09T06:38:14.764 INFO:tasks.workunit.client.0.smithi063.stdout:4/539: chown d7/d50/l52 237 1 2024-05-09T06:38:14.767 INFO:tasks.workunit.client.0.smithi063.stdout:2/602: dread fa [0,4194304] 0 2024-05-09T06:38:14.767 INFO:tasks.workunit.client.0.smithi063.stdout:5/536: dwrite d8/d17/f3d [0,4194304] 0 2024-05-09T06:38:14.773 INFO:tasks.workunit.client.0.smithi063.stdout:7/536: getdents d0/d15/d49/d54/d9f/d97/d9b 0 2024-05-09T06:38:14.774 INFO:tasks.workunit.client.0.smithi063.stdout:7/537: fdatasync d0/d15/d4a/fb5 0 2024-05-09T06:38:14.775 INFO:tasks.workunit.client.0.smithi063.stdout:7/538: chown d0/d15/d4a/f7e 1 1 2024-05-09T06:38:14.776 INFO:tasks.workunit.client.1.smithi193.stdout:7/237: link d6/d17/f42 d6/d8/d1a/d33/f51 0 2024-05-09T06:38:14.777 INFO:tasks.workunit.client.0.smithi063.stdout:7/539: read - d0/d15/d49/d54/d9f/d97/d9b/d9d/fa8 zero size 2024-05-09T06:38:14.781 INFO:tasks.workunit.client.1.smithi193.stdout:1/211: dwrite d4/fc [0,4194304] 0 2024-05-09T06:38:14.782 INFO:tasks.workunit.client.1.smithi193.stdout:0/239: dread fe [0,4194304] 0 2024-05-09T06:38:14.785 INFO:tasks.workunit.client.0.smithi063.stdout:3/582: mkdir d7/d70/d98/dbb 0 2024-05-09T06:38:14.788 INFO:tasks.workunit.client.0.smithi063.stdout:6/571: getdents d2/d4/d96 0 2024-05-09T06:38:14.789 INFO:tasks.workunit.client.0.smithi063.stdout:6/572: truncate d2/fa9 69139 0 2024-05-09T06:38:14.789 INFO:tasks.workunit.client.0.smithi063.stdout:6/573: stat d2/d12/c22 0 2024-05-09T06:38:14.791 INFO:tasks.workunit.client.0.smithi063.stdout:6/574: truncate d2/fa9 720989 0 2024-05-09T06:38:14.791 INFO:tasks.workunit.client.1.smithi193.stdout:5/275: symlink d0/d1/l5a 0 2024-05-09T06:38:14.791 INFO:tasks.workunit.client.1.smithi193.stdout:0/240: dwrite d18/d1d/f1f [4194304,4194304] 0 2024-05-09T06:38:14.793 INFO:tasks.workunit.client.1.smithi193.stdout:0/241: write d18/d1d/f44 [513055,51200] 0 2024-05-09T06:38:14.795 INFO:tasks.workunit.client.0.smithi063.stdout:5/537: creat d8/d17/faf x:0 0 0 2024-05-09T06:38:14.796 INFO:tasks.workunit.client.1.smithi193.stdout:3/205: link d1/d20/l30 d1/db/l4f 0 2024-05-09T06:38:14.798 INFO:tasks.workunit.client.1.smithi193.stdout:6/208: creat d1/d25/f46 x:0 0 0 2024-05-09T06:38:14.805 INFO:tasks.workunit.client.0.smithi063.stdout:8/608: rename d9/da/d18/dc7/d8f/fce to d9/da/d18/fda 0 2024-05-09T06:38:14.806 INFO:tasks.workunit.client.1.smithi193.stdout:0/242: mkdir d18/d29/d47 0 2024-05-09T06:38:14.808 INFO:tasks.workunit.client.0.smithi063.stdout:3/583: creat d7/d4c/d31/da4/fbc x:0 0 0 2024-05-09T06:38:14.812 INFO:tasks.workunit.client.0.smithi063.stdout:0/612: link dd/fe dd/d1a/d2c/d51/d72/d9e/d23/d89/fc4 0 2024-05-09T06:38:14.815 INFO:tasks.workunit.client.0.smithi063.stdout:6/575: creat d2/d1f/d6c/fb7 x:0 0 0 2024-05-09T06:38:14.816 INFO:tasks.workunit.client.1.smithi193.stdout:3/206: mknod d1/db/c50 0 2024-05-09T06:38:14.816 INFO:tasks.workunit.client.0.smithi063.stdout:6/576: write f1 [1603219,71312] 0 2024-05-09T06:38:14.818 INFO:tasks.workunit.client.0.smithi063.stdout:4/540: creat d7/da3/d8b/fac x:0 0 0 2024-05-09T06:38:14.819 INFO:tasks.workunit.client.0.smithi063.stdout:0/613: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/f88 [0,4194304] 0 2024-05-09T06:38:14.820 INFO:tasks.workunit.client.1.smithi193.stdout:0/243: dread f15 [0,4194304] 0 2024-05-09T06:38:14.820 INFO:tasks.workunit.client.0.smithi063.stdout:5/538: mkdir d8/d7e/d96/db0 0 2024-05-09T06:38:14.821 INFO:tasks.workunit.client.0.smithi063.stdout:0/614: write dd/d1a/d2c/d51/d72/d9e/d23/d31/f4a [1194282,113557] 0 2024-05-09T06:38:14.823 INFO:tasks.workunit.client.0.smithi063.stdout:0/615: fsync dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/f4e 0 2024-05-09T06:38:14.824 INFO:tasks.workunit.client.0.smithi063.stdout:2/603: creat dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72/d7b/fae x:0 0 0 2024-05-09T06:38:14.827 INFO:tasks.workunit.client.1.smithi193.stdout:4/282: dwrite d1a/d24/d37/f3c [0,4194304] 0 2024-05-09T06:38:14.829 INFO:tasks.workunit.client.1.smithi193.stdout:4/283: chown f5 1029445 1 2024-05-09T06:38:14.831 INFO:tasks.workunit.client.1.smithi193.stdout:5/276: rename d0/d1 to d0/d13/d2c/d37/d5b 0 2024-05-09T06:38:14.833 INFO:tasks.workunit.client.0.smithi063.stdout:9/574: rename d0/d4/da/l27 to d0/da5/lca 0 2024-05-09T06:38:14.840 INFO:tasks.workunit.client.1.smithi193.stdout:6/209: rename d1/d44 to d1/de/d11/d2d/d3a/d47 0 2024-05-09T06:38:14.841 INFO:tasks.workunit.client.1.smithi193.stdout:6/210: write d1/f2 [8998350,48088] 0 2024-05-09T06:38:14.842 INFO:tasks.workunit.client.1.smithi193.stdout:3/207: getdents d1/d20 0 2024-05-09T06:38:14.843 INFO:tasks.workunit.client.1.smithi193.stdout:3/208: chown d1/db/f18 11564104 1 2024-05-09T06:38:14.843 INFO:tasks.workunit.client.0.smithi063.stdout:6/577: mknod d2/d20/d4a/cb8 0 2024-05-09T06:38:14.846 INFO:tasks.workunit.client.1.smithi193.stdout:0/244: rename d18/l2c to d18/d1d/l48 0 2024-05-09T06:38:14.847 INFO:tasks.workunit.client.1.smithi193.stdout:0/245: fsync d18/d1d/f44 0 2024-05-09T06:38:14.848 INFO:tasks.workunit.client.1.smithi193.stdout:6/211: readlink d1/l14 0 2024-05-09T06:38:14.848 INFO:tasks.workunit.client.1.smithi193.stdout:4/284: dread f11 [0,4194304] 0 2024-05-09T06:38:14.851 INFO:tasks.workunit.client.1.smithi193.stdout:0/246: truncate d18/d29/f3c 505969 0 2024-05-09T06:38:14.852 INFO:tasks.workunit.client.1.smithi193.stdout:3/209: dwrite d1/db/f41 [0,4194304] 0 2024-05-09T06:38:14.854 INFO:tasks.workunit.client.0.smithi063.stdout:0/616: fsync f2 0 2024-05-09T06:38:14.861 INFO:tasks.workunit.client.0.smithi063.stdout:9/575: readlink d0/d4/da/l20 0 2024-05-09T06:38:14.863 INFO:tasks.workunit.client.1.smithi193.stdout:4/285: rename f12 to d1a/d1f/d4b/f52 0 2024-05-09T06:38:14.864 INFO:tasks.workunit.client.1.smithi193.stdout:4/286: stat d1a/d1f/f3e 0 2024-05-09T06:38:14.865 INFO:tasks.workunit.client.0.smithi063.stdout:8/609: truncate d9/da/d18/d24/f3c 5176734 0 2024-05-09T06:38:14.865 INFO:tasks.workunit.client.1.smithi193.stdout:4/287: chown d1a/d24/f3d 4 1 2024-05-09T06:38:14.867 INFO:tasks.workunit.client.1.smithi193.stdout:0/247: creat d18/d27/f49 x:0 0 0 2024-05-09T06:38:14.867 INFO:tasks.workunit.client.0.smithi063.stdout:7/540: write d0/d15/d4a/d50/f37 [3191988,14514] 0 2024-05-09T06:38:14.868 INFO:tasks.workunit.client.1.smithi193.stdout:0/248: fsync d18/d1d/f30 0 2024-05-09T06:38:14.868 INFO:tasks.workunit.client.1.smithi193.stdout:6/212: mkdir d1/d48 0 2024-05-09T06:38:14.869 INFO:tasks.workunit.client.1.smithi193.stdout:4/288: dread d1a/d24/f40 [0,4194304] 0 2024-05-09T06:38:14.870 INFO:tasks.workunit.client.0.smithi063.stdout:8/610: dwrite d9/da/d51/dc2/fcd [0,4194304] 0 2024-05-09T06:38:14.871 INFO:tasks.workunit.client.1.smithi193.stdout:4/289: truncate d1a/d24/f41 1520593 0 2024-05-09T06:38:14.871 INFO:tasks.workunit.client.0.smithi063.stdout:3/584: mknod d7/da/d2b/d66/d74/d7e/cbd 0 2024-05-09T06:38:14.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:14 smithi193 ceph-mon[35141]: pgmap v10: 65 pgs: 65 active+clean; 1.3 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 166 MiB/s wr, 334 op/s 2024-05-09T06:38:14.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:14 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1247040273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:38:14.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:14 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/3291182600' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:38:14.872 INFO:tasks.workunit.client.0.smithi063.stdout:8/611: write d9/da/d18/d46/f52 [635342,110689] 0 2024-05-09T06:38:14.874 INFO:tasks.workunit.client.0.smithi063.stdout:6/578: creat d2/d1f/fb9 x:0 0 0 2024-05-09T06:38:14.876 INFO:tasks.workunit.client.0.smithi063.stdout:5/539: mkdir d8/d19/d30/db1 0 2024-05-09T06:38:14.877 INFO:tasks.workunit.client.1.smithi193.stdout:3/210: rename d1/d20 to d1/d2d/d51 0 2024-05-09T06:38:14.881 INFO:tasks.workunit.client.1.smithi193.stdout:5/277: write d0/d13/f3d [2658526,64254] 0 2024-05-09T06:38:14.882 INFO:tasks.workunit.client.1.smithi193.stdout:5/278: stat d0/d13/d2c/d37/d5b/f2a 0 2024-05-09T06:38:14.884 INFO:tasks.workunit.client.1.smithi193.stdout:6/213: creat d1/d25/d2c/d3c/f49 x:0 0 0 2024-05-09T06:38:14.885 INFO:tasks.workunit.client.0.smithi063.stdout:9/576: creat d0/da5/fcb x:0 0 0 2024-05-09T06:38:14.886 INFO:tasks.workunit.client.0.smithi063.stdout:9/577: write d0/d4/d57/f9b [270888,3732] 0 2024-05-09T06:38:14.887 INFO:tasks.workunit.client.1.smithi193.stdout:4/290: unlink d1a/f36 0 2024-05-09T06:38:14.887 INFO:tasks.workunit.client.1.smithi193.stdout:4/291: chown d1a 27368416 1 2024-05-09T06:38:14.890 INFO:tasks.workunit.client.1.smithi193.stdout:3/211: creat d1/d3e/f52 x:0 0 0 2024-05-09T06:38:14.891 INFO:tasks.workunit.client.1.smithi193.stdout:3/212: fsync d1/db/d1b/f3b 0 2024-05-09T06:38:14.891 INFO:tasks.workunit.client.0.smithi063.stdout:7/541: symlink d0/d15/d4a/d68/lbb 0 2024-05-09T06:38:14.891 INFO:tasks.workunit.client.1.smithi193.stdout:3/213: readlink d1/db/d1b/l1c 0 2024-05-09T06:38:14.892 INFO:tasks.workunit.client.1.smithi193.stdout:0/249: creat d18/d29/d47/f4a x:0 0 0 2024-05-09T06:38:14.893 INFO:tasks.workunit.client.1.smithi193.stdout:3/214: write d1/db/d1b/f3b [2224691,99414] 0 2024-05-09T06:38:14.893 INFO:tasks.workunit.client.1.smithi193.stdout:0/250: truncate d18/d27/f34 4255679 0 2024-05-09T06:38:14.894 INFO:tasks.workunit.client.1.smithi193.stdout:0/251: dread - d18/f26 zero size 2024-05-09T06:38:14.894 INFO:tasks.workunit.client.1.smithi193.stdout:3/215: write d1/db/d2f/f33 [883941,28749] 0 2024-05-09T06:38:14.895 INFO:tasks.workunit.client.1.smithi193.stdout:5/279: rmdir d0/d13/d2c/d37 39 2024-05-09T06:38:14.896 INFO:tasks.workunit.client.0.smithi063.stdout:4/541: getdents d7/da3/d28 0 2024-05-09T06:38:14.897 INFO:tasks.workunit.client.1.smithi193.stdout:6/214: mknod d1/d25/d2c/d3c/d3e/c4a 0 2024-05-09T06:38:14.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/215: chown d1/de/d11/f1d 114 1 2024-05-09T06:38:14.898 INFO:tasks.workunit.client.0.smithi063.stdout:7/542: dwrite d0/d15/d49/f5c [0,4194304] 0 2024-05-09T06:38:14.899 INFO:tasks.workunit.client.0.smithi063.stdout:7/543: read - d0/d15/d49/d54/d9f/d97/d9b/fac zero size 2024-05-09T06:38:14.902 INFO:tasks.workunit.client.0.smithi063.stdout:2/604: write dc/f21 [8731426,40766] 0 2024-05-09T06:38:14.906 INFO:tasks.workunit.client.0.smithi063.stdout:1/502: sync 2024-05-09T06:38:14.907 INFO:tasks.workunit.client.0.smithi063.stdout:5/540: rmdir d8/d17/d18/d9c 39 2024-05-09T06:38:14.911 INFO:tasks.workunit.client.0.smithi063.stdout:0/617: link dd/d1a/d2c/f43 dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0/fc5 0 2024-05-09T06:38:14.911 INFO:tasks.workunit.client.0.smithi063.stdout:4/542: dread d7/da3/f3a [0,4194304] 0 2024-05-09T06:38:14.913 INFO:tasks.workunit.client.0.smithi063.stdout:4/543: write d7/da3/d28/d31/f94 [814396,124260] 0 2024-05-09T06:38:14.913 INFO:tasks.workunit.client.1.smithi193.stdout:0/252: creat d18/d29/f4b x:0 0 0 2024-05-09T06:38:14.915 INFO:tasks.workunit.client.0.smithi063.stdout:3/585: write d7/d22/f27 [135048,84113] 0 2024-05-09T06:38:14.916 INFO:tasks.workunit.client.1.smithi193.stdout:3/216: rename d1/db/c2e to d1/d2d/d51/c53 0 2024-05-09T06:38:14.917 INFO:tasks.workunit.client.0.smithi063.stdout:9/578: rmdir d0/d3b/da0 39 2024-05-09T06:38:14.925 INFO:tasks.workunit.client.1.smithi193.stdout:4/292: dwrite f7 [4194304,4194304] 0 2024-05-09T06:38:14.926 INFO:tasks.workunit.client.1.smithi193.stdout:0/253: rename d18/d29 to d18/d4c 0 2024-05-09T06:38:14.928 INFO:tasks.workunit.client.1.smithi193.stdout:0/254: write d18/d1d/f23 [860760,38295] 0 2024-05-09T06:38:14.929 INFO:tasks.workunit.client.1.smithi193.stdout:5/280: dwrite d0/f9 [4194304,4194304] 0 2024-05-09T06:38:14.934 INFO:tasks.workunit.client.1.smithi193.stdout:3/217: rename d1/db/f41 to d1/d2d/d51/f54 0 2024-05-09T06:38:14.934 INFO:tasks.workunit.client.0.smithi063.stdout:0/618: dread dd/d1a/d2c/d51/d72/d9e/f64 [0,4194304] 0 2024-05-09T06:38:14.935 INFO:tasks.workunit.client.1.smithi193.stdout:3/218: chown d1/db/cc 11610 1 2024-05-09T06:38:14.935 INFO:tasks.workunit.client.1.smithi193.stdout:6/216: write d1/de/d11/d18/f1e [3221679,57577] 0 2024-05-09T06:38:14.936 INFO:tasks.workunit.client.1.smithi193.stdout:6/217: fsync d1/d25/f46 0 2024-05-09T06:38:14.938 INFO:tasks.workunit.client.0.smithi063.stdout:7/544: rmdir d0/d15/d49/d54 39 2024-05-09T06:38:14.941 INFO:tasks.workunit.client.0.smithi063.stdout:6/579: creat d2/df/d59/d99/d9f/fba x:0 0 0 2024-05-09T06:38:14.942 INFO:tasks.workunit.client.1.smithi193.stdout:4/293: symlink d1a/l53 0 2024-05-09T06:38:14.942 INFO:tasks.workunit.client.1.smithi193.stdout:6/218: dread d1/de/d11/f1d [0,4194304] 0 2024-05-09T06:38:14.942 INFO:tasks.workunit.client.0.smithi063.stdout:2/605: rmdir dc/d6e 39 2024-05-09T06:38:14.943 INFO:tasks.workunit.client.0.smithi063.stdout:2/606: write dc/f25 [2969628,111236] 0 2024-05-09T06:38:14.944 INFO:tasks.workunit.client.0.smithi063.stdout:1/503: truncate d9/d46/f54 507611 0 2024-05-09T06:38:14.947 INFO:tasks.workunit.client.0.smithi063.stdout:5/541: symlink d8/d65/d9e/lb2 0 2024-05-09T06:38:14.948 INFO:tasks.workunit.client.0.smithi063.stdout:5/542: dread - d8/f68 zero size 2024-05-09T06:38:14.950 INFO:tasks.workunit.client.1.smithi193.stdout:0/255: mknod d18/d4c/d47/c4d 0 2024-05-09T06:38:14.952 INFO:tasks.workunit.client.1.smithi193.stdout:3/219: rmdir d1/db/d1b 39 2024-05-09T06:38:14.953 INFO:tasks.workunit.client.1.smithi193.stdout:3/220: stat d1/d2d/d51/f29 0 2024-05-09T06:38:14.958 INFO:tasks.workunit.client.1.smithi193.stdout:4/294: creat d1a/d1f/d4b/f54 x:0 0 0 2024-05-09T06:38:14.961 INFO:tasks.workunit.client.1.smithi193.stdout:0/256: creat d18/f4e x:0 0 0 2024-05-09T06:38:14.963 INFO:tasks.workunit.client.1.smithi193.stdout:0/257: write d18/d4c/f4b [476590,82432] 0 2024-05-09T06:38:14.963 INFO:tasks.workunit.client.0.smithi063.stdout:8/612: creat d9/da/d18/d24/fdb x:0 0 0 2024-05-09T06:38:14.964 INFO:tasks.workunit.client.1.smithi193.stdout:3/221: symlink d1/d2d/l55 0 2024-05-09T06:38:14.964 INFO:tasks.workunit.client.1.smithi193.stdout:3/222: dread - d1/d2d/d51/f29 zero size 2024-05-09T06:38:14.966 INFO:tasks.workunit.client.1.smithi193.stdout:3/223: write d1/d2d/d51/f29 [258444,122178] 0 2024-05-09T06:38:14.967 INFO:tasks.workunit.client.0.smithi063.stdout:9/579: write d0/d4/da/f21 [388636,51256] 0 2024-05-09T06:38:14.968 INFO:tasks.workunit.client.1.smithi193.stdout:4/295: mknod d1a/d1f/c55 0 2024-05-09T06:38:14.971 INFO:tasks.workunit.client.1.smithi193.stdout:5/281: creat d0/f5c x:0 0 0 2024-05-09T06:38:14.971 INFO:tasks.workunit.client.1.smithi193.stdout:2/228: sync 2024-05-09T06:38:14.971 INFO:tasks.workunit.client.0.smithi063.stdout:2/607: rmdir dc/d32/d33/d4e 39 2024-05-09T06:38:14.973 INFO:tasks.workunit.client.0.smithi063.stdout:9/580: dwrite d0/d4/d26/fae [0,4194304] 0 2024-05-09T06:38:14.975 INFO:tasks.workunit.client.1.smithi193.stdout:4/296: dwrite d1a/f1b [0,4194304] 0 2024-05-09T06:38:14.978 INFO:tasks.workunit.client.1.smithi193.stdout:2/229: write db/f28 [42631,33315] 0 2024-05-09T06:38:14.986 INFO:tasks.workunit.client.1.smithi193.stdout:0/258: rename f13 to d18/f4f 0 2024-05-09T06:38:14.988 INFO:tasks.workunit.client.1.smithi193.stdout:2/230: dwrite db/f39 [0,4194304] 0 2024-05-09T06:38:14.988 INFO:tasks.workunit.client.1.smithi193.stdout:3/224: symlink d1/db/l56 0 2024-05-09T06:38:14.989 INFO:tasks.workunit.client.0.smithi063.stdout:4/544: truncate d7/df/d57/f93 3996838 0 2024-05-09T06:38:14.994 INFO:tasks.workunit.client.1.smithi193.stdout:3/225: symlink d1/d44/l57 0 2024-05-09T06:38:14.994 INFO:tasks.workunit.client.0.smithi063.stdout:7/545: rmdir d0/d15/d49/d54/d9f/d97/d9b/d9d 39 2024-05-09T06:38:14.995 INFO:tasks.workunit.client.0.smithi063.stdout:8/613: dread - d9/da/d18/dc7/d58/d79/f96 zero size 2024-05-09T06:38:14.995 INFO:tasks.workunit.client.0.smithi063.stdout:7/546: chown d0/d15/f58 3834047 1 2024-05-09T06:38:14.996 INFO:tasks.workunit.client.0.smithi063.stdout:8/614: fsync d9/da/d18/d46/f97 0 2024-05-09T06:38:14.996 INFO:tasks.workunit.client.1.smithi193.stdout:3/226: chown d1/c40 124750 1 2024-05-09T06:38:14.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/547: readlink d0/d15/d89/d98/l99 0 2024-05-09T06:38:14.997 INFO:tasks.workunit.client.1.smithi193.stdout:0/259: dwrite d18/d1d/f3d [0,4194304] 0 2024-05-09T06:38:14.997 INFO:tasks.workunit.client.0.smithi063.stdout:7/548: readlink d0/d15/d4a/d50/l59 0 2024-05-09T06:38:14.998 INFO:tasks.workunit.client.1.smithi193.stdout:0/260: chown d18/d4c/f3f 33692 1 2024-05-09T06:38:14.998 INFO:tasks.workunit.client.0.smithi063.stdout:7/549: dread - d0/d15/d49/d4f/d66/d67/d6b/f85 zero size 2024-05-09T06:38:14.999 INFO:tasks.workunit.client.0.smithi063.stdout:7/550: fdatasync d0/d15/d49/d54/d9f/d97/d9b/fac 0 2024-05-09T06:38:15.000 INFO:tasks.workunit.client.0.smithi063.stdout:7/551: write d0/d15/d4a/d68/f9e [2293566,82527] 0 2024-05-09T06:38:15.005 INFO:tasks.workunit.client.0.smithi063.stdout:3/586: dwrite d7/d70/d98/d2d/f5d [0,4194304] 0 2024-05-09T06:38:15.007 INFO:tasks.workunit.client.0.smithi063.stdout:2/608: truncate dc/dd/f20 1413735 0 2024-05-09T06:38:15.009 INFO:tasks.workunit.client.1.smithi193.stdout:0/261: dwrite d18/d4c/f46 [0,4194304] 0 2024-05-09T06:38:15.009 INFO:tasks.workunit.client.0.smithi063.stdout:9/581: symlink d0/d8/da6/lcc 0 2024-05-09T06:38:15.010 INFO:tasks.workunit.client.0.smithi063.stdout:9/582: read - d0/d4/d57/fa1 zero size 2024-05-09T06:38:15.012 INFO:tasks.workunit.client.0.smithi063.stdout:9/583: write d0/f71 [3952920,116242] 0 2024-05-09T06:38:15.012 INFO:tasks.workunit.client.1.smithi193.stdout:4/297: creat d1a/d22/f56 x:0 0 0 2024-05-09T06:38:15.013 INFO:tasks.workunit.client.1.smithi193.stdout:4/298: readlink d1a/l53 0 2024-05-09T06:38:15.013 INFO:tasks.workunit.client.0.smithi063.stdout:1/504: rename d9/d2a/d35/d1e/c72 to d9/d2a/d3c/cad 0 2024-05-09T06:38:15.014 INFO:tasks.workunit.client.1.smithi193.stdout:4/299: truncate d1a/f23 5276161 0 2024-05-09T06:38:15.015 INFO:tasks.workunit.client.1.smithi193.stdout:4/300: chown f7 162 1 2024-05-09T06:38:15.018 INFO:tasks.workunit.client.1.smithi193.stdout:0/262: symlink d18/d27/l50 0 2024-05-09T06:38:15.024 INFO:tasks.workunit.client.1.smithi193.stdout:2/231: truncate db/f39 3704895 0 2024-05-09T06:38:15.026 INFO:tasks.workunit.client.1.smithi193.stdout:4/301: symlink d1a/d24/d30/l57 0 2024-05-09T06:38:15.026 INFO:tasks.workunit.client.1.smithi193.stdout:2/232: read db/df/d12/f24 [60742,41875] 0 2024-05-09T06:38:15.029 INFO:tasks.workunit.client.0.smithi063.stdout:6/580: creat d2/d1f/d29/d3b/d4e/fbb x:0 0 0 2024-05-09T06:38:15.029 INFO:tasks.workunit.client.1.smithi193.stdout:3/227: creat d1/db/f58 x:0 0 0 2024-05-09T06:38:15.031 INFO:tasks.workunit.client.1.smithi193.stdout:3/228: write d1/d3e/f4e [263692,20099] 0 2024-05-09T06:38:15.033 INFO:tasks.workunit.client.0.smithi063.stdout:7/552: unlink d0/d15/d49/d54/d9f/d97/d9b/fa3 0 2024-05-09T06:38:15.035 INFO:tasks.workunit.client.1.smithi193.stdout:4/302: symlink d1a/l58 0 2024-05-09T06:38:15.037 INFO:tasks.workunit.client.0.smithi063.stdout:3/587: creat d7/d70/d98/d5b/fbe x:0 0 0 2024-05-09T06:38:15.042 INFO:tasks.workunit.client.0.smithi063.stdout:9/584: unlink d0/d4/d26/d39/d3e/f5c 0 2024-05-09T06:38:15.043 INFO:tasks.workunit.client.0.smithi063.stdout:2/609: dwrite dc/dd/d45/f53 [0,4194304] 0 2024-05-09T06:38:15.048 INFO:tasks.workunit.client.0.smithi063.stdout:5/543: link d8/d19/d30/d46/f6c d8/d17/d18/fb3 0 2024-05-09T06:38:15.053 INFO:tasks.workunit.client.0.smithi063.stdout:0/619: link dd/d1a/d2c/d51/d72/d9e/c40 dd/d1a/d2c/d42/cc6 0 2024-05-09T06:38:15.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/615: dwrite d9/da/d34/f87 [0,4194304] 0 2024-05-09T06:38:15.055 INFO:tasks.workunit.client.1.smithi193.stdout:0/263: getdents d18 0 2024-05-09T06:38:15.056 INFO:tasks.workunit.client.0.smithi063.stdout:0/620: fdatasync dd/d1a/d2c/d51/d72/d9e/f55 0 2024-05-09T06:38:15.057 INFO:tasks.workunit.client.1.smithi193.stdout:2/233: creat db/f53 x:0 0 0 2024-05-09T06:38:15.061 INFO:tasks.workunit.client.1.smithi193.stdout:3/229: rename d1/d3e to d1/db/d59 0 2024-05-09T06:38:15.062 INFO:tasks.workunit.client.1.smithi193.stdout:3/230: stat d1/db/ld 0 2024-05-09T06:38:15.063 INFO:tasks.workunit.client.0.smithi063.stdout:0/621: dread dd/d1a/f9f [0,4194304] 0 2024-05-09T06:38:15.069 INFO:tasks.workunit.client.0.smithi063.stdout:7/553: mknod d0/d15/d49/d54/cbc 0 2024-05-09T06:38:15.069 INFO:tasks.workunit.client.0.smithi063.stdout:1/505: write d9/d2a/d35/d1e/d9d/f82 [773083,117811] 0 2024-05-09T06:38:15.070 INFO:tasks.workunit.client.1.smithi193.stdout:3/231: dwrite d1/db/f14 [0,4194304] 0 2024-05-09T06:38:15.072 INFO:tasks.workunit.client.1.smithi193.stdout:4/303: rename d1a/d24/d37 to d1a/d1f/d4b/d59 0 2024-05-09T06:38:15.072 INFO:tasks.workunit.client.0.smithi063.stdout:3/588: mknod d7/d4c/d25/cbf 0 2024-05-09T06:38:15.078 INFO:tasks.workunit.client.1.smithi193.stdout:2/234: mkdir db/df/d54 0 2024-05-09T06:38:15.079 INFO:tasks.workunit.client.1.smithi193.stdout:2/235: chown db/d25/l3a 790832 1 2024-05-09T06:38:15.079 INFO:tasks.workunit.client.0.smithi063.stdout:4/545: dwrite d7/df/d1a/f22 [0,4194304] 0 2024-05-09T06:38:15.081 INFO:tasks.workunit.client.1.smithi193.stdout:3/232: chown d1/db/d59/c4d 199 1 2024-05-09T06:38:15.083 INFO:tasks.workunit.client.1.smithi193.stdout:4/304: dread d1a/d1f/f33 [0,4194304] 0 2024-05-09T06:38:15.084 INFO:tasks.workunit.client.1.smithi193.stdout:2/236: mknod db/d37/c55 0 2024-05-09T06:38:15.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.107+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2cac1089a0 con 0x7f2cac100da0 2024-05-09T06:38:15.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.107+0000 7f2ca27fc700 1 -- 172.21.15.63:0/883758720 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f2c9c059ae0 con 0x7f2cac100da0 2024-05-09T06:38:15.087 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:38:15.089 INFO:tasks.workunit.client.1.smithi193.stdout:0/264: dwrite f17 [0,4194304] 0 2024-05-09T06:38:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f2c9806d6f0 msgr2=0x7f2c9806fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f2c9806d6f0 0x7f2c9806fbb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c00 tx=0x7f2ca8005960 comp rx=0 tx=0).stop 2024-05-09T06:38:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac100da0 msgr2=0x7f2cac196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac100da0 0x7f2cac196c70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00b4b0 tx=0x7f2c9c00b7c0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 shutdown_connections 2024-05-09T06:38:15.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f2c9806d6f0 0x7f2c9806fbb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac0ffba0 0x7f2cac196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 --2- 172.21.15.63:0/883758720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cac100da0 0x7f2cac196c70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:15.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 >> 172.21.15.63:0/883758720 conn(0x7f2cac0fb120 msgr2=0x7f2cac0685d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:15.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 shutdown_connections 2024-05-09T06:38:15.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:15.110+0000 7f2cb3745700 1 -- 172.21.15.63:0/883758720 wait complete. 2024-05-09T06:38:15.093 INFO:tasks.workunit.client.1.smithi193.stdout:4/305: symlink d1a/d24/d30/l5a 0 2024-05-09T06:38:15.095 INFO:tasks.workunit.client.1.smithi193.stdout:2/237: mkdir db/d4b/d56 0 2024-05-09T06:38:15.097 INFO:tasks.workunit.client.0.smithi063.stdout:6/581: symlink d2/d1f/d29/d3b/d4e/lbc 0 2024-05-09T06:38:15.097 INFO:tasks.workunit.client.1.smithi193.stdout:2/238: write db/df/d12/f13 [1356362,61581] 0 2024-05-09T06:38:15.098 INFO:tasks.workunit.client.1.smithi193.stdout:2/239: readlink db/d4b/l4c 0 2024-05-09T06:38:15.100 INFO:tasks.workunit.client.1.smithi193.stdout:0/265: mknod d18/d4c/d47/c51 0 2024-05-09T06:38:15.102 INFO:tasks.workunit.client.0.smithi063.stdout:7/554: creat d0/d15/d49/d54/fbd x:0 0 0 2024-05-09T06:38:15.104 INFO:tasks.workunit.client.1.smithi193.stdout:4/306: mkdir d1a/d24/d30/d5b 0 2024-05-09T06:38:15.105 INFO:tasks.workunit.client.0.smithi063.stdout:6/582: dread d2/d1f/d29/d3f/d5c/fb0 [0,4194304] 0 2024-05-09T06:38:15.107 INFO:tasks.workunit.client.0.smithi063.stdout:6/583: fsync d2/df/f50 0 2024-05-09T06:38:15.107 INFO:tasks.workunit.client.1.smithi193.stdout:0/266: dwrite d18/f4e [0,4194304] 0 2024-05-09T06:38:15.108 INFO:tasks.workunit.client.1.smithi193.stdout:2/240: creat db/d25/f57 x:0 0 0 2024-05-09T06:38:15.108 INFO:tasks.workunit.client.0.smithi063.stdout:8/616: dwrite d9/da/d18/d24/f62 [0,4194304] 0 2024-05-09T06:38:15.108 INFO:tasks.workunit.client.0.smithi063.stdout:6/584: chown d2/d4/d7c/d42 7600 1 2024-05-09T06:38:15.109 INFO:tasks.workunit.client.1.smithi193.stdout:0/267: read d18/d1d/f1f [5207012,119478] 0 2024-05-09T06:38:15.110 INFO:tasks.workunit.client.0.smithi063.stdout:6/585: readlink d2/d4/d7c/d42/l5f 0 2024-05-09T06:38:15.112 INFO:tasks.workunit.client.0.smithi063.stdout:2/610: creat dc/dd/d45/d5a/d6c/d8c/dad/faf x:0 0 0 2024-05-09T06:38:15.113 INFO:tasks.workunit.client.1.smithi193.stdout:4/307: mknod d1a/c5c 0 2024-05-09T06:38:15.113 INFO:tasks.workunit.client.1.smithi193.stdout:4/308: write d1a/d1f/f2c [1708128,33978] 0 2024-05-09T06:38:15.114 INFO:tasks.workunit.client.0.smithi063.stdout:1/506: dwrite d9/d2a/d35/d1e/fa1 [0,4194304] 0 2024-05-09T06:38:15.115 INFO:tasks.workunit.client.0.smithi063.stdout:4/546: fdatasync d7/da3/d28/f7d 0 2024-05-09T06:38:15.116 INFO:tasks.workunit.client.0.smithi063.stdout:4/547: readlink d7/df/l5c 0 2024-05-09T06:38:15.118 INFO:tasks.workunit.client.0.smithi063.stdout:5/544: symlink d8/d19/d30/lb4 0 2024-05-09T06:38:15.120 INFO:tasks.workunit.client.1.smithi193.stdout:2/241: creat db/d25/d38/d40/f58 x:0 0 0 2024-05-09T06:38:15.123 INFO:tasks.workunit.client.1.smithi193.stdout:0/268: creat d18/f52 x:0 0 0 2024-05-09T06:38:15.128 INFO:tasks.workunit.client.1.smithi193.stdout:0/269: dread d18/d4c/f4b [0,4194304] 0 2024-05-09T06:38:15.143 INFO:tasks.workunit.client.1.smithi193.stdout:0/270: rmdir d18/d27 39 2024-05-09T06:38:15.144 INFO:tasks.workunit.client.1.smithi193.stdout:2/242: dread f1 [4194304,4194304] 0 2024-05-09T06:38:15.144 INFO:tasks.workunit.client.1.smithi193.stdout:0/271: write d18/d1d/f2d [574208,38224] 0 2024-05-09T06:38:15.145 INFO:tasks.workunit.client.0.smithi063.stdout:8/617: symlink d9/da/d53/da3/ldc 0 2024-05-09T06:38:15.145 INFO:tasks.workunit.client.1.smithi193.stdout:2/243: chown db/l34 193359 1 2024-05-09T06:38:15.145 INFO:tasks.workunit.client.0.smithi063.stdout:6/586: dread - d2/d1f/d29/d3f/d5c/d64/f6e zero size 2024-05-09T06:38:15.151 INFO:tasks.workunit.client.1.smithi193.stdout:4/309: getdents d1a/d1f 0 2024-05-09T06:38:15.152 INFO:tasks.workunit.client.1.smithi193.stdout:4/310: chown d1a/d1f/f2c 978 1 2024-05-09T06:38:15.153 INFO:tasks.workunit.client.0.smithi063.stdout:5/545: dread d8/d19/d30/d46/f62 [0,4194304] 0 2024-05-09T06:38:15.153 INFO:tasks.workunit.client.0.smithi063.stdout:2/611: dwrite dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f86 [4194304,4194304] 0 2024-05-09T06:38:15.154 INFO:tasks.workunit.client.0.smithi063.stdout:3/589: rename d7/d4c/d31 to d7/da/d2b/d66/dc0 0 2024-05-09T06:38:15.155 INFO:tasks.workunit.client.0.smithi063.stdout:5/546: write d8/d19/d30/d46/f94 [985107,57101] 0 2024-05-09T06:38:15.155 INFO:tasks.workunit.client.1.smithi193.stdout:2/244: rename db/l1c to db/d25/d38/d40/l59 0 2024-05-09T06:38:15.156 INFO:tasks.workunit.client.1.smithi193.stdout:2/245: fsync db/f53 0 2024-05-09T06:38:15.157 INFO:tasks.workunit.client.1.smithi193.stdout:2/246: chown db/df/c19 0 1 2024-05-09T06:38:15.158 INFO:tasks.workunit.client.0.smithi063.stdout:8/618: creat d9/da/d51/fdd x:0 0 0 2024-05-09T06:38:15.159 INFO:tasks.workunit.client.1.smithi193.stdout:0/272: link f15 d18/d27/f53 0 2024-05-09T06:38:15.160 INFO:tasks.workunit.client.0.smithi063.stdout:0/622: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fc3 [0,4194304] 0 2024-05-09T06:38:15.164 INFO:tasks.workunit.client.1.smithi193.stdout:0/273: mkdir d18/d4c/d47/d54 0 2024-05-09T06:38:15.165 INFO:tasks.workunit.client.1.smithi193.stdout:2/247: dwrite db/f44 [0,4194304] 0 2024-05-09T06:38:15.166 INFO:tasks.workunit.client.1.smithi193.stdout:0/274: write d18/d4c/f38 [681664,23468] 0 2024-05-09T06:38:15.169 INFO:tasks.workunit.client.1.smithi193.stdout:4/311: mkdir d1a/d22/d5d 0 2024-05-09T06:38:15.171 INFO:tasks.workunit.client.0.smithi063.stdout:4/548: mkdir d7/df/d35/d5a/dad 0 2024-05-09T06:38:15.172 INFO:tasks.workunit.client.0.smithi063.stdout:4/549: read d7/da3/d28/d31/f94 [932652,99980] 0 2024-05-09T06:38:15.173 INFO:tasks.workunit.client.0.smithi063.stdout:4/550: dread - d7/df/d1a/f96 zero size 2024-05-09T06:38:15.174 INFO:tasks.workunit.client.1.smithi193.stdout:9/236: sync 2024-05-09T06:38:15.174 INFO:tasks.workunit.client.1.smithi193.stdout:8/229: sync 2024-05-09T06:38:15.175 INFO:tasks.workunit.client.1.smithi193.stdout:9/237: write dd/d1e/f27 [855804,105883] 0 2024-05-09T06:38:15.177 INFO:tasks.workunit.client.1.smithi193.stdout:2/248: mknod db/d25/d38/d40/c5a 0 2024-05-09T06:38:15.178 INFO:tasks.workunit.client.1.smithi193.stdout:4/312: dread d1a/f23 [4194304,4194304] 0 2024-05-09T06:38:15.180 INFO:tasks.workunit.client.1.smithi193.stdout:0/275: dwrite d18/d1d/f33 [0,4194304] 0 2024-05-09T06:38:15.181 INFO:tasks.workunit.client.0.smithi063.stdout:5/547: creat d8/d17/d75/fb5 x:0 0 0 2024-05-09T06:38:15.181 INFO:tasks.workunit.client.1.smithi193.stdout:9/238: write dd/df/d15/d16/d26/f4b [1032661,25447] 0 2024-05-09T06:38:15.183 INFO:tasks.workunit.client.1.smithi193.stdout:8/230: creat d6/d1c/d27/f43 x:0 0 0 2024-05-09T06:38:15.183 INFO:tasks.workunit.client.1.smithi193.stdout:8/231: chown d6 81020003 1 2024-05-09T06:38:15.184 INFO:tasks.workunit.client.0.smithi063.stdout:8/619: rmdir d9/da/d45 39 2024-05-09T06:38:15.185 INFO:tasks.workunit.client.0.smithi063.stdout:9/585: sync 2024-05-09T06:38:15.186 INFO:tasks.workunit.client.1.smithi193.stdout:0/276: mkdir d18/d4c/d47/d55 0 2024-05-09T06:38:15.187 INFO:tasks.workunit.client.0.smithi063.stdout:0/623: creat dd/d1a/d2c/d51/d72/d9e/d23/d31/fc7 x:0 0 0 2024-05-09T06:38:15.188 INFO:tasks.workunit.client.1.smithi193.stdout:0/277: write d18/d1d/f2d [1634461,37130] 0 2024-05-09T06:38:15.191 INFO:tasks.workunit.client.0.smithi063.stdout:5/548: dwrite d8/d35/fad [0,4194304] 0 2024-05-09T06:38:15.192 INFO:tasks.workunit.client.1.smithi193.stdout:9/239: creat dd/df/d15/f56 x:0 0 0 2024-05-09T06:38:15.193 INFO:tasks.workunit.client.0.smithi063.stdout:4/551: mknod d7/df/d35/cae 0 2024-05-09T06:38:15.195 INFO:tasks.workunit.client.1.smithi193.stdout:8/232: creat d6/d1c/d18/d2f/f44 x:0 0 0 2024-05-09T06:38:15.196 INFO:tasks.workunit.client.0.smithi063.stdout:4/552: write d7/df/d57/f91 [332,9022] 0 2024-05-09T06:38:15.197 INFO:tasks.workunit.client.0.smithi063.stdout:8/620: dwrite d9/da/d51/f8b [0,4194304] 0 2024-05-09T06:38:15.197 INFO:tasks.workunit.client.0.smithi063.stdout:4/553: stat d7/df/f21 0 2024-05-09T06:38:15.199 INFO:tasks.workunit.client.0.smithi063.stdout:4/554: write d7/df/d35/d5a/f97 [724248,5965] 0 2024-05-09T06:38:15.200 INFO:tasks.workunit.client.0.smithi063.stdout:3/590: mknod d7/d70/d98/dbb/cc1 0 2024-05-09T06:38:15.201 INFO:tasks.workunit.client.0.smithi063.stdout:4/555: stat d7/da3/l83 0 2024-05-09T06:38:15.207 INFO:tasks.workunit.client.1.smithi193.stdout:0/278: mkdir d18/d1d/d56 0 2024-05-09T06:38:15.210 INFO:tasks.workunit.client.1.smithi193.stdout:9/240: mknod dd/d1e/c57 0 2024-05-09T06:38:15.213 INFO:tasks.workunit.client.0.smithi063.stdout:1/507: link d9/d2a/l73 d9/d46/lae 0 2024-05-09T06:38:15.214 INFO:tasks.workunit.client.1.smithi193.stdout:4/313: link d1a/d1f/c27 d1a/d1f/d4b/c5e 0 2024-05-09T06:38:15.214 INFO:tasks.workunit.client.0.smithi063.stdout:1/508: fdatasync d9/d2a/d3c/d4a/f7b 0 2024-05-09T06:38:15.217 INFO:tasks.workunit.client.1.smithi193.stdout:2/249: dread f8 [4194304,4194304] 0 2024-05-09T06:38:15.218 INFO:tasks.workunit.client.0.smithi063.stdout:9/586: mkdir d0/d4/d26/d39/d48/d80/d84/dcd 0 2024-05-09T06:38:15.219 INFO:tasks.workunit.client.1.smithi193.stdout:2/250: read db/d25/f48 [3387873,57620] 0 2024-05-09T06:38:15.220 INFO:tasks.workunit.client.1.smithi193.stdout:2/251: stat db/df/d12/c2e 0 2024-05-09T06:38:15.221 INFO:tasks.workunit.client.0.smithi063.stdout:5/549: rmdir d8/d53 39 2024-05-09T06:38:15.222 INFO:tasks.workunit.client.1.smithi193.stdout:4/314: dread d1a/d1f/d4b/f52 [0,4194304] 0 2024-05-09T06:38:15.223 INFO:tasks.workunit.client.1.smithi193.stdout:9/241: chown dd/f13 82918 1 2024-05-09T06:38:15.225 INFO:tasks.workunit.client.1.smithi193.stdout:8/233: symlink d6/d1c/l45 0 2024-05-09T06:38:15.227 INFO:tasks.workunit.client.1.smithi193.stdout:4/315: dread d1a/d24/f2e [0,4194304] 0 2024-05-09T06:38:15.227 INFO:tasks.workunit.client.0.smithi063.stdout:2/612: creat dc/d32/d33/fb0 x:0 0 0 2024-05-09T06:38:15.230 INFO:tasks.workunit.client.0.smithi063.stdout:7/555: rename d0/d15/d4a/f53 to d0/fbe 0 2024-05-09T06:38:15.231 INFO:tasks.workunit.client.0.smithi063.stdout:8/621: mknod d9/da/d18/dc7/d58/cde 0 2024-05-09T06:38:15.232 INFO:tasks.workunit.client.1.smithi193.stdout:2/252: mkdir db/d25/d38/d5b 0 2024-05-09T06:38:15.232 INFO:tasks.workunit.client.0.smithi063.stdout:2/613: dwrite dc/f2a [0,4194304] 0 2024-05-09T06:38:15.233 INFO:tasks.workunit.client.1.smithi193.stdout:0/279: truncate d18/f28 872926 0 2024-05-09T06:38:15.235 INFO:tasks.workunit.client.0.smithi063.stdout:4/556: chown d7/da3/d28/d31/d32/d34/c4e 6 1 2024-05-09T06:38:15.236 INFO:tasks.workunit.client.0.smithi063.stdout:6/587: getdents d2/d4 0 2024-05-09T06:38:15.237 INFO:tasks.workunit.client.1.smithi193.stdout:9/242: symlink dd/df/d15/d16/d39/d49/l58 0 2024-05-09T06:38:15.239 INFO:tasks.workunit.client.0.smithi063.stdout:7/556: dwrite d0/d15/f76 [0,4194304] 0 2024-05-09T06:38:15.240 INFO:tasks.workunit.client.1.smithi193.stdout:8/234: symlink d6/d1c/d1b/d3a/l46 0 2024-05-09T06:38:15.243 INFO:tasks.workunit.client.0.smithi063.stdout:2/614: dwrite dc/dd/d2d/fa0 [4194304,4194304] 0 2024-05-09T06:38:15.248 INFO:tasks.workunit.client.0.smithi063.stdout:2/615: read dc/d36/d43/d46/f70 [126433,66740] 0 2024-05-09T06:38:15.249 INFO:tasks.workunit.client.1.smithi193.stdout:2/253: mknod db/d25/d38/c5c 0 2024-05-09T06:38:15.250 INFO:tasks.workunit.client.1.smithi193.stdout:2/254: dread - db/f53 zero size 2024-05-09T06:38:15.251 INFO:tasks.workunit.client.0.smithi063.stdout:5/550: mkdir d8/d35/db6 0 2024-05-09T06:38:15.254 INFO:tasks.workunit.client.1.smithi193.stdout:0/280: unlink f15 0 2024-05-09T06:38:15.257 INFO:tasks.workunit.client.1.smithi193.stdout:2/255: dwrite db/df/d12/f17 [0,4194304] 0 2024-05-09T06:38:15.259 INFO:tasks.workunit.client.1.smithi193.stdout:9/243: mkdir dd/d1e/d1f/d48/d55/d59 0 2024-05-09T06:38:15.260 INFO:tasks.workunit.client.1.smithi193.stdout:9/244: chown dd/d1e/d1f/d48/d55 139 1 2024-05-09T06:38:15.261 INFO:tasks.workunit.client.1.smithi193.stdout:8/235: chown d6/c19 7 1 2024-05-09T06:38:15.262 INFO:tasks.workunit.client.1.smithi193.stdout:2/256: dread f1 [4194304,4194304] 0 2024-05-09T06:38:15.264 INFO:tasks.workunit.client.1.smithi193.stdout:2/257: readlink db/df/l11 0 2024-05-09T06:38:15.267 INFO:tasks.workunit.client.1.smithi193.stdout:4/316: truncate d1a/d24/f2e 846438 0 2024-05-09T06:38:15.267 INFO:tasks.workunit.client.0.smithi063.stdout:1/509: rmdir d9/d2a/d2e 39 2024-05-09T06:38:15.269 INFO:tasks.workunit.client.1.smithi193.stdout:8/236: creat d6/d1c/d1b/d3a/f47 x:0 0 0 2024-05-09T06:38:15.271 INFO:tasks.workunit.client.1.smithi193.stdout:8/237: chown d6/d1c 0 1 2024-05-09T06:38:15.271 INFO:tasks.workunit.client.1.smithi193.stdout:9/245: dwrite dd/df/d15/f56 [0,4194304] 0 2024-05-09T06:38:15.275 INFO:tasks.workunit.client.1.smithi193.stdout:2/258: dwrite db/d25/d38/d40/f58 [0,4194304] 0 2024-05-09T06:38:15.276 INFO:tasks.workunit.client.0.smithi063.stdout:0/624: truncate dd/d1a/d2c/d51/d72/d9e/f80 1282371 0 2024-05-09T06:38:15.279 INFO:tasks.workunit.client.1.smithi193.stdout:4/317: readlink l14 0 2024-05-09T06:38:15.279 INFO:tasks.workunit.client.1.smithi193.stdout:4/318: readlink d1a/d1f/l2b 0 2024-05-09T06:38:15.284 INFO:tasks.workunit.client.0.smithi063.stdout:5/551: mknod d8/d17/cb7 0 2024-05-09T06:38:15.286 INFO:tasks.workunit.client.1.smithi193.stdout:0/281: link d18/d1d/f30 d18/f57 0 2024-05-09T06:38:15.289 INFO:tasks.workunit.client.1.smithi193.stdout:9/246: creat dd/d1e/d1f/d48/f5a x:0 0 0 2024-05-09T06:38:15.290 INFO:tasks.workunit.client.0.smithi063.stdout:4/557: symlink d7/df/d35/d5a/dad/laf 0 2024-05-09T06:38:15.294 INFO:tasks.workunit.client.1.smithi193.stdout:8/238: symlink d6/d1c/d27/l48 0 2024-05-09T06:38:15.295 INFO:tasks.workunit.client.0.smithi063.stdout:6/588: creat d2/d4/d96/fbd x:0 0 0 2024-05-09T06:38:15.295 INFO:tasks.workunit.client.0.smithi063.stdout:6/589: stat d2/d4/c2c 0 2024-05-09T06:38:15.297 INFO:tasks.workunit.client.1.smithi193.stdout:4/319: dread d1a/f1b [0,4194304] 0 2024-05-09T06:38:15.299 INFO:tasks.workunit.client.0.smithi063.stdout:0/625: unlink dd/d1a/c49 0 2024-05-09T06:38:15.302 INFO:tasks.workunit.client.0.smithi063.stdout:9/587: creat d0/d4/d26/d39/d3e/fce x:0 0 0 2024-05-09T06:38:15.304 INFO:tasks.workunit.client.0.smithi063.stdout:2/616: symlink dc/lb1 0 2024-05-09T06:38:15.307 INFO:tasks.workunit.client.0.smithi063.stdout:4/558: mknod d7/df/d35/d5a/d60/d75/cb0 0 2024-05-09T06:38:15.309 INFO:tasks.workunit.client.1.smithi193.stdout:2/259: write db/d25/f27 [1138252,55064] 0 2024-05-09T06:38:15.309 INFO:tasks.workunit.client.0.smithi063.stdout:8/622: dwrite d9/f29 [0,4194304] 0 2024-05-09T06:38:15.310 INFO:tasks.workunit.client.1.smithi193.stdout:2/260: stat db/df/d12/c4d 0 2024-05-09T06:38:15.314 INFO:tasks.workunit.client.0.smithi063.stdout:7/557: rename d0/d15/d4a/d68/f75 to d0/d15/d49/fbf 0 2024-05-09T06:38:15.314 INFO:tasks.workunit.client.0.smithi063.stdout:7/558: dread - d0/f81 zero size 2024-05-09T06:38:15.318 INFO:tasks.workunit.client.1.smithi193.stdout:9/247: write dd/d1e/d1f/f22 [463700,45476] 0 2024-05-09T06:38:15.320 INFO:tasks.workunit.client.1.smithi193.stdout:7/238: sync 2024-05-09T06:38:15.320 INFO:tasks.workunit.client.1.smithi193.stdout:1/212: sync 2024-05-09T06:38:15.320 INFO:tasks.workunit.client.0.smithi063.stdout:9/588: write d0/d4/d26/d47/f2b [2458863,66151] 0 2024-05-09T06:38:15.320 INFO:tasks.workunit.client.0.smithi063.stdout:9/589: chown d0/f97 510621 1 2024-05-09T06:38:15.321 INFO:tasks.workunit.client.1.smithi193.stdout:2/261: truncate db/df/d12/f24 635525 0 2024-05-09T06:38:15.322 INFO:tasks.workunit.client.1.smithi193.stdout:0/282: truncate d18/d1d/f2d 2730788 0 2024-05-09T06:38:15.325 INFO:tasks.workunit.client.0.smithi063.stdout:9/590: dread d0/d4/d26/d39/d3e/fc6 [0,4194304] 0 2024-05-09T06:38:15.325 INFO:tasks.workunit.client.0.smithi063.stdout:6/590: mknod d2/d20/d95/cbe 0 2024-05-09T06:38:15.326 INFO:tasks.workunit.client.0.smithi063.stdout:9/591: chown d0/d8/d2d/f7b 355 1 2024-05-09T06:38:15.326 INFO:tasks.workunit.client.1.smithi193.stdout:8/239: write d6/d1c/d1b/f2b [245829,50623] 0 2024-05-09T06:38:15.326 INFO:tasks.workunit.client.0.smithi063.stdout:9/592: chown d0/d4/d26/d39/f53 1385 1 2024-05-09T06:38:15.329 INFO:tasks.workunit.client.0.smithi063.stdout:8/623: creat d9/da/d53/fdf x:0 0 0 2024-05-09T06:38:15.333 INFO:tasks.workunit.client.0.smithi063.stdout:1/510: link d9/d2a/d39/f8c d9/d2a/d35/d1e/faf 0 2024-05-09T06:38:15.333 INFO:tasks.workunit.client.1.smithi193.stdout:7/239: dwrite d6/dd/f47 [0,4194304] 0 2024-05-09T06:38:15.334 INFO:tasks.workunit.client.1.smithi193.stdout:1/213: dwrite d4/d16/de/f12 [4194304,4194304] 0 2024-05-09T06:38:15.337 INFO:tasks.workunit.client.1.smithi193.stdout:7/240: write d6/dd/f3f [180914,35285] 0 2024-05-09T06:38:15.339 INFO:tasks.workunit.client.1.smithi193.stdout:2/262: creat db/d4b/f5d x:0 0 0 2024-05-09T06:38:15.340 INFO:tasks.workunit.client.1.smithi193.stdout:1/214: read d4/d16/d17/d1a/f1f [972219,104793] 0 2024-05-09T06:38:15.342 INFO:tasks.workunit.client.0.smithi063.stdout:5/552: rename d8/d17/d18/f58 to d8/d65/d9e/fb8 0 2024-05-09T06:38:15.346 INFO:tasks.workunit.client.0.smithi063.stdout:0/626: getdents dd/d1a/d2c/d51/d72/d9e/d23/dbb 0 2024-05-09T06:38:15.346 INFO:tasks.workunit.client.1.smithi193.stdout:9/248: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:15.347 INFO:tasks.workunit.client.0.smithi063.stdout:0/627: fdatasync dd/d1a/d2c/d51/d76/d97/fa5 0 2024-05-09T06:38:15.348 INFO:tasks.workunit.client.0.smithi063.stdout:7/559: mknod d0/d15/d89/d98/cc0 0 2024-05-09T06:38:15.349 INFO:tasks.workunit.client.0.smithi063.stdout:7/560: fsync d0/d15/d49/f5f 0 2024-05-09T06:38:15.359 INFO:tasks.workunit.client.0.smithi063.stdout:8/624: creat d9/da/d53/db2/fe0 x:0 0 0 2024-05-09T06:38:15.361 INFO:tasks.workunit.client.1.smithi193.stdout:8/240: creat d6/d1c/d1b/d29/f49 x:0 0 0 2024-05-09T06:38:15.361 INFO:tasks.workunit.client.1.smithi193.stdout:0/283: write d18/d4c/f2b [1540923,54201] 0 2024-05-09T06:38:15.363 INFO:tasks.workunit.client.1.smithi193.stdout:8/241: truncate d6/d1c/f3d 379344 0 2024-05-09T06:38:15.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/553: mknod d8/d19/d30/d74/cb9 0 2024-05-09T06:38:15.369 INFO:tasks.workunit.client.0.smithi063.stdout:7/561: unlink d0/d15/d4a/d50/l2a 0 2024-05-09T06:38:15.371 INFO:tasks.workunit.client.0.smithi063.stdout:6/591: mkdir d2/dbf 0 2024-05-09T06:38:15.371 INFO:tasks.workunit.client.0.smithi063.stdout:3/591: sync 2024-05-09T06:38:15.372 INFO:tasks.workunit.client.1.smithi193.stdout:7/241: write d6/d8/d1a/d33/f51 [4164161,79253] 0 2024-05-09T06:38:15.373 INFO:tasks.workunit.client.0.smithi063.stdout:9/593: mkdir d0/d3b/da0/dcf 0 2024-05-09T06:38:15.375 INFO:tasks.workunit.client.0.smithi063.stdout:8/625: mkdir d9/da/d18/d46/de1 0 2024-05-09T06:38:15.376 INFO:tasks.workunit.client.0.smithi063.stdout:1/511: mknod d9/d2a/d35/d1e/d9d/cb0 0 2024-05-09T06:38:15.377 INFO:tasks.workunit.client.0.smithi063.stdout:5/554: creat d8/d7e/fba x:0 0 0 2024-05-09T06:38:15.379 INFO:tasks.workunit.client.0.smithi063.stdout:6/592: dwrite d2/d1f/d29/d3f/f58 [0,4194304] 0 2024-05-09T06:38:15.380 INFO:tasks.workunit.client.0.smithi063.stdout:7/562: creat d0/d15/d49/d54/fc1 x:0 0 0 2024-05-09T06:38:15.381 INFO:tasks.workunit.client.1.smithi193.stdout:9/249: mkdir dd/d1e/d1f/d43/d5b 0 2024-05-09T06:38:15.381 INFO:tasks.workunit.client.0.smithi063.stdout:3/592: creat d7/d22/d28/fc2 x:0 0 0 2024-05-09T06:38:15.386 INFO:tasks.workunit.client.0.smithi063.stdout:9/594: symlink d0/d4/d26/d39/d48/ld0 0 2024-05-09T06:38:15.387 INFO:tasks.workunit.client.1.smithi193.stdout:7/242: dread d6/d8/d1a/f27 [0,4194304] 0 2024-05-09T06:38:15.387 INFO:tasks.workunit.client.0.smithi063.stdout:8/626: dwrite d9/da/d51/f88 [0,4194304] 0 2024-05-09T06:38:15.389 INFO:tasks.workunit.client.0.smithi063.stdout:2/617: rename dc/dd/d2d/f82 to dc/dd/d2d/d60/fb2 0 2024-05-09T06:38:15.391 INFO:tasks.workunit.client.1.smithi193.stdout:1/215: rename d4/d14/d15/d2b to d4/d3c 0 2024-05-09T06:38:15.392 INFO:tasks.workunit.client.1.smithi193.stdout:1/216: fsync d4/d16/de/f12 0 2024-05-09T06:38:15.393 INFO:tasks.workunit.client.0.smithi063.stdout:1/512: mknod d9/d2a/d39/d5d/cb1 0 2024-05-09T06:38:15.395 INFO:tasks.workunit.client.1.smithi193.stdout:9/250: creat dd/d1e/d3c/f5c x:0 0 0 2024-05-09T06:38:15.399 INFO:tasks.workunit.client.1.smithi193.stdout:2/263: truncate f1 5849243 0 2024-05-09T06:38:15.401 INFO:tasks.workunit.client.0.smithi063.stdout:6/593: symlink d2/d4/d7c/lc0 0 2024-05-09T06:38:15.405 INFO:tasks.workunit.client.0.smithi063.stdout:7/563: unlink d0/d15/d49/d4f/d66/f91 0 2024-05-09T06:38:15.405 INFO:tasks.workunit.client.1.smithi193.stdout:7/243: dwrite d6/f2e [0,4194304] 0 2024-05-09T06:38:15.406 INFO:tasks.workunit.client.1.smithi193.stdout:1/217: truncate d4/d16/fb 3172127 0 2024-05-09T06:38:15.407 INFO:tasks.workunit.client.0.smithi063.stdout:9/595: dread d0/d43/f77 [0,4194304] 0 2024-05-09T06:38:15.408 INFO:tasks.workunit.client.1.smithi193.stdout:0/284: truncate d18/d4c/f3f 136996 0 2024-05-09T06:38:15.420 INFO:tasks.workunit.client.0.smithi063.stdout:9/596: dread d0/d8/d2d/f52 [0,4194304] 0 2024-05-09T06:38:15.421 INFO:tasks.workunit.client.0.smithi063.stdout:5/555: creat d8/d17/da2/fbb x:0 0 0 2024-05-09T06:38:15.426 INFO:tasks.workunit.client.0.smithi063.stdout:6/594: unlink d2/f7 0 2024-05-09T06:38:15.427 INFO:tasks.workunit.client.0.smithi063.stdout:6/595: write d2/d1f/fb9 [822578,85861] 0 2024-05-09T06:38:15.428 INFO:tasks.workunit.client.0.smithi063.stdout:1/513: dread d9/d2a/d35/d24/f53 [0,4194304] 0 2024-05-09T06:38:15.428 INFO:tasks.workunit.client.1.smithi193.stdout:7/244: creat d6/d17/f52 x:0 0 0 2024-05-09T06:38:15.430 INFO:tasks.workunit.client.1.smithi193.stdout:9/251: creat dd/d1e/d1f/d43/d5b/f5d x:0 0 0 2024-05-09T06:38:15.436 INFO:tasks.workunit.client.1.smithi193.stdout:7/245: dwrite d6/d8/d29/f2d [0,4194304] 0 2024-05-09T06:38:15.436 INFO:tasks.workunit.client.1.smithi193.stdout:0/285: rename d18/d1d/f24 to d18/d4c/d47/f58 0 2024-05-09T06:38:15.438 INFO:tasks.workunit.client.1.smithi193.stdout:9/252: dread f0 [4194304,4194304] 0 2024-05-09T06:38:15.441 INFO:tasks.workunit.client.1.smithi193.stdout:9/253: truncate dd/d1e/d1f/d48/f53 690347 0 2024-05-09T06:38:15.443 INFO:tasks.workunit.client.1.smithi193.stdout:1/218: symlink d4/d16/d17/l3d 0 2024-05-09T06:38:15.443 INFO:tasks.workunit.client.0.smithi063.stdout:8/627: creat d9/da/d18/d46/de1/fe2 x:0 0 0 2024-05-09T06:38:15.444 INFO:tasks.workunit.client.1.smithi193.stdout:1/219: write d4/d16/de/f28 [699681,111993] 0 2024-05-09T06:38:15.447 INFO:tasks.workunit.client.0.smithi063.stdout:0/628: rename dd/d1a/d2c/d51/d72/d9e/d23/d31/c34 to dd/cc8 0 2024-05-09T06:38:15.454 INFO:tasks.workunit.client.0.smithi063.stdout:1/514: creat d9/d2a/d3c/d4a/fb2 x:0 0 0 2024-05-09T06:38:15.455 INFO:tasks.workunit.client.1.smithi193.stdout:7/246: mkdir d6/dd/d53 0 2024-05-09T06:38:15.456 INFO:tasks.workunit.client.0.smithi063.stdout:7/564: dread d0/d15/d4a/d50/f1a [0,4194304] 0 2024-05-09T06:38:15.458 INFO:tasks.workunit.client.1.smithi193.stdout:0/286: mkdir d18/d4c/d47/d59 0 2024-05-09T06:38:15.461 INFO:tasks.workunit.client.1.smithi193.stdout:2/264: dwrite db/f1e [0,4194304] 0 2024-05-09T06:38:15.465 INFO:tasks.workunit.client.1.smithi193.stdout:9/254: rmdir dd/d1e/d1f/d43 39 2024-05-09T06:38:15.468 INFO:tasks.workunit.client.1.smithi193.stdout:1/220: mkdir d4/d3c/d3e 0 2024-05-09T06:38:15.470 INFO:tasks.workunit.client.0.smithi063.stdout:1/515: write d9/d2a/d35/d1e/f3a [4878076,126129] 0 2024-05-09T06:38:15.477 INFO:tasks.workunit.client.0.smithi063.stdout:9/597: creat d0/d4/fd1 x:0 0 0 2024-05-09T06:38:15.478 INFO:tasks.workunit.client.0.smithi063.stdout:4/559: sync 2024-05-09T06:38:15.479 INFO:tasks.workunit.client.0.smithi063.stdout:8/628: dread d9/d3b/fbd [0,4194304] 0 2024-05-09T06:38:15.480 INFO:tasks.workunit.client.1.smithi193.stdout:7/247: unlink d6/d8/d29/f2d 0 2024-05-09T06:38:15.480 INFO:tasks.workunit.client.0.smithi063.stdout:9/598: truncate d0/d4/d26/d39/d3e/fce 2327 0 2024-05-09T06:38:15.480 INFO:tasks.workunit.client.0.smithi063.stdout:9/599: stat d0/d8/d2d/c88 0 2024-05-09T06:38:15.482 INFO:tasks.workunit.client.0.smithi063.stdout:3/593: rename d7/da/d21/d30/d44/d48/l49 to d7/da/d21/lc3 0 2024-05-09T06:38:15.484 INFO:tasks.workunit.client.0.smithi063.stdout:0/629: symlink dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0/lc9 0 2024-05-09T06:38:15.485 INFO:tasks.workunit.client.1.smithi193.stdout:2/265: mkdir db/d25/d5e 0 2024-05-09T06:38:15.487 INFO:tasks.workunit.client.1.smithi193.stdout:1/221: mkdir d4/d16/d17/d1a/d3a/d3f 0 2024-05-09T06:38:15.489 INFO:tasks.workunit.client.0.smithi063.stdout:7/565: symlink d0/d15/d49/d54/d9f/d97/d9b/lc2 0 2024-05-09T06:38:15.491 INFO:tasks.workunit.client.1.smithi193.stdout:7/248: symlink d6/d8/d29/d2f/l54 0 2024-05-09T06:38:15.491 INFO:tasks.workunit.client.1.smithi193.stdout:7/249: fdatasync d6/d8/d1a/d33/f3e 0 2024-05-09T06:38:15.493 INFO:tasks.workunit.client.0.smithi063.stdout:4/560: creat d7/d3d/d73/fb1 x:0 0 0 2024-05-09T06:38:15.496 INFO:tasks.workunit.client.1.smithi193.stdout:9/255: symlink dd/d1e/d1f/d43/l5e 0 2024-05-09T06:38:15.497 INFO:tasks.workunit.client.0.smithi063.stdout:3/594: creat d7/d4c/fc4 x:0 0 0 2024-05-09T06:38:15.499 INFO:tasks.workunit.client.1.smithi193.stdout:0/287: dwrite d18/f25 [0,4194304] 0 2024-05-09T06:38:15.499 INFO:tasks.workunit.client.1.smithi193.stdout:1/222: unlink d4/d14/d15/c1c 0 2024-05-09T06:38:15.505 INFO:tasks.workunit.client.1.smithi193.stdout:2/266: dwrite db/f21 [0,4194304] 0 2024-05-09T06:38:15.505 INFO:tasks.workunit.client.0.smithi063.stdout:1/516: fsync d9/d2a/d35/d1e/d9d/f33 0 2024-05-09T06:38:15.508 INFO:tasks.workunit.client.0.smithi063.stdout:7/566: creat d0/d15/d49/d4f/d66/d67/d6b/fc3 x:0 0 0 2024-05-09T06:38:15.511 INFO:tasks.workunit.client.1.smithi193.stdout:0/288: symlink d18/d1d/l5a 0 2024-05-09T06:38:15.513 INFO:tasks.workunit.client.0.smithi063.stdout:4/561: rmdir d7 39 2024-05-09T06:38:15.516 INFO:tasks.workunit.client.0.smithi063.stdout:3/595: rmdir d7/da/d2b/d66/d74 39 2024-05-09T06:38:15.517 INFO:tasks.workunit.client.0.smithi063.stdout:8/629: dwrite d9/da/d18/dc7/f65 [0,4194304] 0 2024-05-09T06:38:15.520 INFO:tasks.workunit.client.1.smithi193.stdout:7/250: rename d6/d17/f43 to d6/d8/f55 0 2024-05-09T06:38:15.521 INFO:tasks.workunit.client.1.smithi193.stdout:0/289: rename d18/d1d to d18/d1d/d56/d5b 22 2024-05-09T06:38:15.522 INFO:tasks.workunit.client.1.smithi193.stdout:0/290: stat d18/d1d/f23 0 2024-05-09T06:38:15.522 INFO:tasks.workunit.client.1.smithi193.stdout:7/251: write d6/d17/f52 [25486,56360] 0 2024-05-09T06:38:15.523 INFO:tasks.workunit.client.0.smithi063.stdout:1/517: fsync d9/d46/f6e 0 2024-05-09T06:38:15.524 INFO:tasks.workunit.client.0.smithi063.stdout:7/567: symlink d0/d15/d49/d4f/d66/d67/d6b/lc4 0 2024-05-09T06:38:15.526 INFO:tasks.workunit.client.0.smithi063.stdout:0/630: dwrite dd/d1a/f2e [0,4194304] 0 2024-05-09T06:38:15.527 INFO:tasks.workunit.client.0.smithi063.stdout:6/596: rename d2/d1f/d29/d3f/d5c/d64/c81 to d2/d1f/d29/d3b/d4e/cc1 0 2024-05-09T06:38:15.528 INFO:tasks.workunit.client.0.smithi063.stdout:0/631: chown dd/d1a/d2c/d51/d72 0 1 2024-05-09T06:38:15.529 INFO:tasks.workunit.client.1.smithi193.stdout:2/267: rmdir db/d25/d5e 0 2024-05-09T06:38:15.530 INFO:tasks.workunit.client.0.smithi063.stdout:0/632: write dd/d1a/d2c/d51/d72/d9e/d23/d89/f8a [2453537,31941] 0 2024-05-09T06:38:15.531 INFO:tasks.workunit.client.0.smithi063.stdout:2/618: sync 2024-05-09T06:38:15.534 INFO:tasks.workunit.client.1.smithi193.stdout:1/223: dread d4/d16/f25 [0,4194304] 0 2024-05-09T06:38:15.538 INFO:tasks.workunit.client.1.smithi193.stdout:9/256: dread dd/df/d15/d16/f2a [0,4194304] 0 2024-05-09T06:38:15.539 INFO:tasks.workunit.client.0.smithi063.stdout:3/596: mknod d7/d70/d98/d5b/cc5 0 2024-05-09T06:38:15.540 INFO:tasks.workunit.client.1.smithi193.stdout:0/291: dread d18/d4c/d47/f58 [0,4194304] 0 2024-05-09T06:38:15.543 INFO:tasks.workunit.client.0.smithi063.stdout:4/562: mknod d7/df/d35/cb2 0 2024-05-09T06:38:15.544 INFO:tasks.workunit.client.1.smithi193.stdout:9/257: creat dd/df/d31/f5f x:0 0 0 2024-05-09T06:38:15.546 INFO:tasks.workunit.client.1.smithi193.stdout:7/252: link d6/d8/d1a/d36/l49 d6/d8/d1a/d36/d4d/l56 0 2024-05-09T06:38:15.549 INFO:tasks.workunit.client.0.smithi063.stdout:9/600: rename d0/d4/d57/d60/fb2 to d0/d3b/da0/dcf/fd2 0 2024-05-09T06:38:15.550 INFO:tasks.workunit.client.0.smithi063.stdout:9/601: readlink d0/d4/l50 0 2024-05-09T06:38:15.551 INFO:tasks.workunit.client.1.smithi193.stdout:1/224: mknod d4/d3c/d3e/c40 0 2024-05-09T06:38:15.552 INFO:tasks.workunit.client.0.smithi063.stdout:7/568: dwrite d0/d15/f9a [0,4194304] 0 2024-05-09T06:38:15.552 INFO:tasks.workunit.client.1.smithi193.stdout:1/225: dread d4/f32 [4194304,4194304] 0 2024-05-09T06:38:15.553 INFO:tasks.workunit.client.0.smithi063.stdout:0/633: creat dd/d1a/d2c/d51/d72/fca x:0 0 0 2024-05-09T06:38:15.556 INFO:tasks.workunit.client.0.smithi063.stdout:8/630: rename d9/da/d18/dc7/lb7 to d9/da/d18/dc7/le3 0 2024-05-09T06:38:15.558 INFO:tasks.workunit.client.0.smithi063.stdout:9/602: dwrite d0/f71 [0,4194304] 0 2024-05-09T06:38:15.558 INFO:tasks.workunit.client.1.smithi193.stdout:9/258: creat dd/df/d15/d16/d26/f60 x:0 0 0 2024-05-09T06:38:15.559 INFO:tasks.workunit.client.0.smithi063.stdout:8/631: write d9/da/d18/dc7/d58/f8e [3283559,110214] 0 2024-05-09T06:38:15.559 INFO:tasks.workunit.client.1.smithi193.stdout:9/259: fsync dd/f18 0 2024-05-09T06:38:15.560 INFO:tasks.workunit.client.1.smithi193.stdout:7/253: truncate d6/d17/f23 1665101 0 2024-05-09T06:38:15.561 INFO:tasks.workunit.client.1.smithi193.stdout:7/254: chown d6/f2e 566720 1 2024-05-09T06:38:15.564 INFO:tasks.workunit.client.1.smithi193.stdout:7/255: read d6/d8/d1a/f3c [3040306,49423] 0 2024-05-09T06:38:15.565 INFO:tasks.workunit.client.1.smithi193.stdout:7/256: chown d6/d8/d1a/d36/f3b 913 1 2024-05-09T06:38:15.566 INFO:tasks.workunit.client.1.smithi193.stdout:2/268: link db/l30 db/d25/d38/d5b/l5f 0 2024-05-09T06:38:15.570 INFO:tasks.workunit.client.1.smithi193.stdout:9/260: stat dd/df/d15/d16/d26/d37/f4d 0 2024-05-09T06:38:15.575 INFO:tasks.workunit.client.1.smithi193.stdout:1/226: creat d4/d16/d17/d1a/d3a/d3f/f41 x:0 0 0 2024-05-09T06:38:15.576 INFO:tasks.workunit.client.1.smithi193.stdout:1/227: write d4/d16/de/f12 [3850529,38500] 0 2024-05-09T06:38:15.576 INFO:tasks.workunit.client.1.smithi193.stdout:3/233: sync 2024-05-09T06:38:15.576 INFO:tasks.workunit.client.1.smithi193.stdout:6/219: sync 2024-05-09T06:38:15.576 INFO:tasks.workunit.client.1.smithi193.stdout:5/282: sync 2024-05-09T06:38:15.578 INFO:tasks.workunit.client.1.smithi193.stdout:1/228: stat d4/c7 0 2024-05-09T06:38:15.578 INFO:tasks.workunit.client.1.smithi193.stdout:0/292: truncate f8 2263337 0 2024-05-09T06:38:15.579 INFO:tasks.workunit.client.1.smithi193.stdout:0/293: fdatasync d18/f25 0 2024-05-09T06:38:15.580 INFO:tasks.workunit.client.1.smithi193.stdout:0/294: stat d18/c2a 0 2024-05-09T06:38:15.582 INFO:tasks.workunit.client.1.smithi193.stdout:2/269: mkdir db/df/d12/d60 0 2024-05-09T06:38:15.587 INFO:tasks.workunit.client.1.smithi193.stdout:5/283: stat d0/d13/d2c/d37/c2d 0 2024-05-09T06:38:15.588 INFO:tasks.workunit.client.1.smithi193.stdout:6/220: mknod d1/d25/d2c/d3c/c4b 0 2024-05-09T06:38:15.591 INFO:tasks.workunit.client.1.smithi193.stdout:1/229: rmdir d4/d3c 39 2024-05-09T06:38:15.592 INFO:tasks.workunit.client.1.smithi193.stdout:4/320: sync 2024-05-09T06:38:15.592 INFO:tasks.workunit.client.1.smithi193.stdout:8/242: sync 2024-05-09T06:38:15.595 INFO:tasks.workunit.client.1.smithi193.stdout:8/243: write d6/f20 [1001885,25306] 0 2024-05-09T06:38:15.597 INFO:tasks.workunit.client.1.smithi193.stdout:0/295: unlink d18/f52 0 2024-05-09T06:38:15.597 INFO:tasks.workunit.client.1.smithi193.stdout:6/221: dwrite d1/f23 [0,4194304] 0 2024-05-09T06:38:15.599 INFO:tasks.workunit.client.1.smithi193.stdout:7/257: dwrite d6/d8/d1a/d36/f50 [0,4194304] 0 2024-05-09T06:38:15.600 INFO:tasks.workunit.client.1.smithi193.stdout:2/270: mkdir db/d25/d38/d61 0 2024-05-09T06:38:15.600 INFO:tasks.workunit.client.1.smithi193.stdout:6/222: dread - d1/d25/f46 zero size 2024-05-09T06:38:15.600 INFO:tasks.workunit.client.1.smithi193.stdout:7/258: chown d6/d8/f35 854526507 1 2024-05-09T06:38:15.603 INFO:tasks.workunit.client.1.smithi193.stdout:3/234: mknod d1/db/c5a 0 2024-05-09T06:38:15.606 INFO:tasks.workunit.client.1.smithi193.stdout:5/284: mkdir d0/d13/d2c/d37/d5b/d4e/d5d 0 2024-05-09T06:38:15.607 INFO:tasks.workunit.client.1.smithi193.stdout:5/285: write d0/d13/d2c/d37/d5b/d4e/f58 [760772,72868] 0 2024-05-09T06:38:15.616 INFO:tasks.workunit.client.1.smithi193.stdout:2/271: mknod db/d25/d38/c62 0 2024-05-09T06:38:15.616 INFO:tasks.workunit.client.1.smithi193.stdout:2/272: chown db/d4b/d56 6187 1 2024-05-09T06:38:15.618 INFO:tasks.workunit.client.1.smithi193.stdout:7/259: creat d6/d8/d1a/d36/f57 x:0 0 0 2024-05-09T06:38:15.619 INFO:tasks.workunit.client.1.smithi193.stdout:7/260: write d6/d8/d1a/d36/f50 [1405065,61689] 0 2024-05-09T06:38:15.622 INFO:tasks.workunit.client.1.smithi193.stdout:7/261: read d6/d8/fa [1190062,97944] 0 2024-05-09T06:38:15.622 INFO:tasks.workunit.client.1.smithi193.stdout:7/262: stat d6/f30 0 2024-05-09T06:38:15.623 INFO:tasks.workunit.client.1.smithi193.stdout:7/263: fdatasync d6/d17/f42 0 2024-05-09T06:38:15.625 INFO:tasks.workunit.client.1.smithi193.stdout:5/286: symlink d0/d13/d2c/d37/d2b/l5e 0 2024-05-09T06:38:15.626 INFO:tasks.workunit.client.1.smithi193.stdout:5/287: readlink d0/d13/d2c/d37/d5b/l48 0 2024-05-09T06:38:15.626 INFO:tasks.workunit.client.1.smithi193.stdout:5/288: chown d0/d13/l51 30348 1 2024-05-09T06:38:15.631 INFO:tasks.workunit.client.1.smithi193.stdout:0/296: mknod d18/d4c/d47/d54/c5c 0 2024-05-09T06:38:15.633 INFO:tasks.workunit.client.0.smithi063.stdout:7/569: symlink d0/d15/d49/d54/d9f/lc5 0 2024-05-09T06:38:15.634 INFO:tasks.workunit.client.1.smithi193.stdout:9/261: getdents dd/d1e/d1f 0 2024-05-09T06:38:15.635 INFO:tasks.workunit.client.1.smithi193.stdout:9/262: write dd/df/d15/d16/d26/f35 [374624,97634] 0 2024-05-09T06:38:15.638 INFO:tasks.workunit.client.0.smithi063.stdout:7/570: dwrite d0/d15/f48 [0,4194304] 0 2024-05-09T06:38:15.639 INFO:tasks.workunit.client.1.smithi193.stdout:3/235: dread d1/db/d1b/f3b [0,4194304] 0 2024-05-09T06:38:15.640 INFO:tasks.workunit.client.0.smithi063.stdout:9/603: mknod d0/d3b/da0/cd3 0 2024-05-09T06:38:15.643 INFO:tasks.workunit.client.0.smithi063.stdout:4/563: dread d7/df/d35/f7f [0,4194304] 0 2024-05-09T06:38:15.643 INFO:tasks.workunit.client.0.smithi063.stdout:3/597: mknod d7/d70/d98/d2d/cc6 0 2024-05-09T06:38:15.644 INFO:tasks.workunit.client.0.smithi063.stdout:4/564: readlink d7/df/d57/l87 0 2024-05-09T06:38:15.647 INFO:tasks.workunit.client.0.smithi063.stdout:6/597: dread d2/d1f/d29/f47 [0,4194304] 0 2024-05-09T06:38:15.648 INFO:tasks.workunit.client.1.smithi193.stdout:1/230: creat d4/d16/f42 x:0 0 0 2024-05-09T06:38:15.653 INFO:tasks.workunit.client.0.smithi063.stdout:9/604: mkdir d0/d4/d57/d60/d76/dd4 0 2024-05-09T06:38:15.654 INFO:tasks.workunit.client.1.smithi193.stdout:5/289: symlink d0/d13/d2c/d37/d2b/d47/l5f 0 2024-05-09T06:38:15.654 INFO:tasks.workunit.client.1.smithi193.stdout:6/223: write d1/de/d11/f1d [653985,13141] 0 2024-05-09T06:38:15.655 INFO:tasks.workunit.client.0.smithi063.stdout:8/632: mkdir d9/da/d45/de4 0 2024-05-09T06:38:15.655 INFO:tasks.workunit.client.1.smithi193.stdout:6/224: chown d1/de/d11/d2d/d3a/f3b 251 1 2024-05-09T06:38:15.656 INFO:tasks.workunit.client.0.smithi063.stdout:8/633: fsync d9/da/d18/d24/f42 0 2024-05-09T06:38:15.660 INFO:tasks.workunit.client.1.smithi193.stdout:8/244: truncate d6/f21 393637 0 2024-05-09T06:38:15.660 INFO:tasks.workunit.client.1.smithi193.stdout:2/273: mkdir db/df/d54/d63 0 2024-05-09T06:38:15.662 INFO:tasks.workunit.client.0.smithi063.stdout:4/565: read d7/d3d/f49 [5460,74101] 0 2024-05-09T06:38:15.662 INFO:tasks.workunit.client.1.smithi193.stdout:9/263: mkdir dd/d1e/d1f/d61 0 2024-05-09T06:38:15.663 INFO:tasks.workunit.client.1.smithi193.stdout:6/225: dwrite d1/de/d11/d18/f2a [0,4194304] 0 2024-05-09T06:38:15.663 INFO:tasks.workunit.client.0.smithi063.stdout:2/619: link dc/dd/l29 dc/dd/d45/d5a/lb3 0 2024-05-09T06:38:15.664 INFO:tasks.workunit.client.0.smithi063.stdout:0/634: write dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0/fc5 [2427775,43510] 0 2024-05-09T06:38:15.665 INFO:tasks.workunit.client.1.smithi193.stdout:3/236: rename d1/d44/l57 to d1/d2d/l5b 0 2024-05-09T06:38:15.666 INFO:tasks.workunit.client.0.smithi063.stdout:5/556: sync 2024-05-09T06:38:15.668 INFO:tasks.workunit.client.1.smithi193.stdout:6/226: dread d1/f6 [0,4194304] 0 2024-05-09T06:38:15.673 INFO:tasks.workunit.client.0.smithi063.stdout:7/571: mknod d0/d15/d49/d54/d9f/d97/d9b/d9d/cc6 0 2024-05-09T06:38:15.674 INFO:tasks.workunit.client.0.smithi063.stdout:7/572: readlink d0/d15/d4a/d50/l55 0 2024-05-09T06:38:15.674 INFO:tasks.workunit.client.0.smithi063.stdout:5/557: dwrite d8/d7e/fa7 [0,4194304] 0 2024-05-09T06:38:15.674 INFO:tasks.workunit.client.1.smithi193.stdout:3/237: dwrite d1/db/f58 [0,4194304] 0 2024-05-09T06:38:15.675 INFO:tasks.workunit.client.0.smithi063.stdout:5/558: fsync d8/d65/f83 0 2024-05-09T06:38:15.675 INFO:tasks.workunit.client.1.smithi193.stdout:3/238: dread - d1/f1d zero size 2024-05-09T06:38:15.676 INFO:tasks.workunit.client.0.smithi063.stdout:9/605: truncate d0/d43/f4b 1308835 0 2024-05-09T06:38:15.676 INFO:tasks.workunit.client.1.smithi193.stdout:3/239: readlink d1/d2d/l55 0 2024-05-09T06:38:15.677 INFO:tasks.workunit.client.1.smithi193.stdout:7/264: creat d6/dd/d53/f58 x:0 0 0 2024-05-09T06:38:15.680 INFO:tasks.workunit.client.1.smithi193.stdout:4/321: link d1a/d24/f2e d1a/d22/d5d/f5f 0 2024-05-09T06:38:15.681 INFO:tasks.workunit.client.1.smithi193.stdout:7/265: write d6/d8/d1a/d36/f3b [288505,69085] 0 2024-05-09T06:38:15.681 INFO:tasks.workunit.client.0.smithi063.stdout:8/634: creat d9/da/d51/fe5 x:0 0 0 2024-05-09T06:38:15.684 INFO:tasks.workunit.client.1.smithi193.stdout:5/290: mkdir d0/d13/d2c/d37/d2b/d47/d60 0 2024-05-09T06:38:15.686 INFO:tasks.workunit.client.1.smithi193.stdout:5/291: dread - d0/d13/d2c/f4b zero size 2024-05-09T06:38:15.687 INFO:tasks.workunit.client.0.smithi063.stdout:3/598: creat d7/d22/d9c/fc7 x:0 0 0 2024-05-09T06:38:15.688 INFO:tasks.workunit.client.0.smithi063.stdout:9/606: dread d0/d4/d26/d39/d3e/f8d [0,4194304] 0 2024-05-09T06:38:15.688 INFO:tasks.workunit.client.0.smithi063.stdout:3/599: fsync d7/da/d21/d30/fa3 0 2024-05-09T06:38:15.689 INFO:tasks.workunit.client.1.smithi193.stdout:0/297: write d18/d27/f34 [894871,111677] 0 2024-05-09T06:38:15.689 INFO:tasks.workunit.client.0.smithi063.stdout:9/607: write d0/d4/f4d [1355598,105896] 0 2024-05-09T06:38:15.694 INFO:tasks.workunit.client.0.smithi063.stdout:2/620: creat dc/dd/d45/d5a/d6c/fb4 x:0 0 0 2024-05-09T06:38:15.697 INFO:tasks.workunit.client.0.smithi063.stdout:0/635: rmdir dd/d60 39 2024-05-09T06:38:15.704 INFO:tasks.workunit.client.0.smithi063.stdout:1/518: sync 2024-05-09T06:38:15.707 INFO:tasks.workunit.client.1.smithi193.stdout:3/240: symlink d1/db/d59/l5c 0 2024-05-09T06:38:15.707 INFO:tasks.workunit.client.1.smithi193.stdout:7/266: dread d6/d17/f37 [0,4194304] 0 2024-05-09T06:38:15.713 INFO:tasks.workunit.client.0.smithi063.stdout:3/600: mknod d7/d22/cc8 0 2024-05-09T06:38:15.713 INFO:tasks.workunit.client.0.smithi063.stdout:3/601: write d7/da/d2b/d66/f95 [1143779,75146] 0 2024-05-09T06:38:15.713 INFO:tasks.workunit.client.0.smithi063.stdout:5/559: write d8/d7e/fa7 [4736965,108222] 0 2024-05-09T06:38:15.714 INFO:tasks.workunit.client.0.smithi063.stdout:9/608: read - d0/d4/d57/d60/f70 zero size 2024-05-09T06:38:15.715 INFO:tasks.workunit.client.1.smithi193.stdout:5/292: dread d0/d13/d2c/d37/d5b/da/f23 [0,4194304] 0 2024-05-09T06:38:15.715 INFO:tasks.workunit.client.0.smithi063.stdout:3/602: readlink d7/da/d2b/d66/dc0/l4f 0 2024-05-09T06:38:15.715 INFO:tasks.workunit.client.0.smithi063.stdout:9/609: chown d0/f71 4 1 2024-05-09T06:38:15.717 INFO:tasks.workunit.client.1.smithi193.stdout:0/298: symlink d18/d4c/l5d 0 2024-05-09T06:38:15.717 INFO:tasks.workunit.client.1.smithi193.stdout:0/299: readlink d18/l40 0 2024-05-09T06:38:15.719 INFO:tasks.workunit.client.0.smithi063.stdout:0/636: mkdir dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dcb 0 2024-05-09T06:38:15.724 INFO:tasks.workunit.client.0.smithi063.stdout:8/635: symlink d9/da/d45/de4/le6 0 2024-05-09T06:38:15.725 INFO:tasks.workunit.client.1.smithi193.stdout:6/227: mkdir d1/d25/d4c 0 2024-05-09T06:38:15.727 INFO:tasks.workunit.client.1.smithi193.stdout:2/274: truncate db/d25/f27 936210 0 2024-05-09T06:38:15.728 INFO:tasks.workunit.client.1.smithi193.stdout:1/231: creat d4/d16/d17/f43 x:0 0 0 2024-05-09T06:38:15.728 INFO:tasks.workunit.client.0.smithi063.stdout:5/560: symlink d8/d7e/lbc 0 2024-05-09T06:38:15.729 INFO:tasks.workunit.client.0.smithi063.stdout:5/561: write d8/d17/f24 [241083,52727] 0 2024-05-09T06:38:15.730 INFO:tasks.workunit.client.1.smithi193.stdout:7/267: symlink d6/dd/d53/l59 0 2024-05-09T06:38:15.730 INFO:tasks.workunit.client.1.smithi193.stdout:9/264: dwrite dd/d1e/d1f/f22 [0,4194304] 0 2024-05-09T06:38:15.730 INFO:tasks.workunit.client.1.smithi193.stdout:1/232: readlink d4/d16/d17/l3d 0 2024-05-09T06:38:15.732 INFO:tasks.workunit.client.0.smithi063.stdout:4/566: rename d7/df/d35/d5a/d60/d75/cb0 to d7/d3d/cb3 0 2024-05-09T06:38:15.734 INFO:tasks.workunit.client.0.smithi063.stdout:4/567: truncate d7/d3d/d73/fb1 331792 0 2024-05-09T06:38:15.739 INFO:tasks.workunit.client.1.smithi193.stdout:3/241: dwrite d1/d2d/d51/f54 [0,4194304] 0 2024-05-09T06:38:15.740 INFO:tasks.workunit.client.1.smithi193.stdout:3/242: chown d1/db/l1e 208126388 1 2024-05-09T06:38:15.741 INFO:tasks.workunit.client.1.smithi193.stdout:0/300: unlink d18/d4c/f3c 0 2024-05-09T06:38:15.744 INFO:tasks.workunit.client.0.smithi063.stdout:1/519: symlink d9/d2a/d2e/lb3 0 2024-05-09T06:38:15.749 INFO:tasks.workunit.client.0.smithi063.stdout:5/562: mkdir d8/d65/dbd 0 2024-05-09T06:38:15.752 INFO:tasks.workunit.client.0.smithi063.stdout:6/598: rename d2/d1f/d29/f63 to d2/df/d59/d99/fc2 0 2024-05-09T06:38:15.754 INFO:tasks.workunit.client.0.smithi063.stdout:3/603: mknod d7/da/d21/d30/d44/cc9 0 2024-05-09T06:38:15.755 INFO:tasks.workunit.client.0.smithi063.stdout:5/563: dwrite d8/d65/f83 [0,4194304] 0 2024-05-09T06:38:15.758 INFO:tasks.workunit.client.0.smithi063.stdout:4/568: mkdir d7/da3/d28/d31/d32/d34/db4 0 2024-05-09T06:38:15.759 INFO:tasks.workunit.client.1.smithi193.stdout:9/265: creat dd/df/d15/d16/d26/f62 x:0 0 0 2024-05-09T06:38:15.760 INFO:tasks.workunit.client.1.smithi193.stdout:4/322: write d1a/d22/d5d/f5f [894359,105496] 0 2024-05-09T06:38:15.762 INFO:tasks.workunit.client.0.smithi063.stdout:2/621: creat dc/dd/fb5 x:0 0 0 2024-05-09T06:38:15.763 INFO:tasks.workunit.client.0.smithi063.stdout:0/637: write dd/d1a/f9f [1168070,112006] 0 2024-05-09T06:38:15.765 INFO:tasks.workunit.client.1.smithi193.stdout:5/293: creat d0/d13/d2c/d37/d5b/d4e/d5d/f61 x:0 0 0 2024-05-09T06:38:15.770 INFO:tasks.workunit.client.1.smithi193.stdout:3/243: rename d1/db/d59/c2c to d1/c5d 0 2024-05-09T06:38:15.770 INFO:tasks.workunit.client.1.smithi193.stdout:3/244: fsync d1/f1d 0 2024-05-09T06:38:15.772 INFO:tasks.workunit.client.1.smithi193.stdout:3/245: write d1/db/f58 [1707458,119564] 0 2024-05-09T06:38:15.772 INFO:tasks.workunit.client.0.smithi063.stdout:1/520: creat d9/d2a/d35/d1e/d9d/d3f/fb4 x:0 0 0 2024-05-09T06:38:15.775 INFO:tasks.workunit.client.1.smithi193.stdout:7/268: write d6/d8/d29/d2f/f4f [2344576,37240] 0 2024-05-09T06:38:15.775 INFO:tasks.workunit.client.0.smithi063.stdout:9/610: dwrite d0/d4/d26/d47/f7a [0,4194304] 0 2024-05-09T06:38:15.777 INFO:tasks.workunit.client.0.smithi063.stdout:9/611: fdatasync d0/d4/d26/f49 0 2024-05-09T06:38:15.778 INFO:tasks.workunit.client.0.smithi063.stdout:7/573: rename d0/ce to d0/d15/d49/d4f/d66/d67/cc7 0 2024-05-09T06:38:15.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:15 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:38:15.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:15 smithi193 ceph-mon[35141]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:15.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:15 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/883758720' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:38:15.783 INFO:tasks.workunit.client.0.smithi063.stdout:3/604: truncate d7/da/d2b/d66/d74/faf 877635 0 2024-05-09T06:38:15.784 INFO:tasks.workunit.client.1.smithi193.stdout:1/233: link d4/f32 d4/d14/f44 0 2024-05-09T06:38:15.784 INFO:tasks.workunit.client.0.smithi063.stdout:3/605: chown d7/d70/lad 23 1 2024-05-09T06:38:15.789 INFO:tasks.workunit.client.1.smithi193.stdout:4/323: mkdir d1a/d1f/d4b/d59/d60 0 2024-05-09T06:38:15.790 INFO:tasks.workunit.client.1.smithi193.stdout:4/324: chown d1a/d24/d30/l57 8869949 1 2024-05-09T06:38:15.791 INFO:tasks.workunit.client.1.smithi193.stdout:4/325: write d1a/f1d [419972,107173] 0 2024-05-09T06:38:15.793 INFO:tasks.workunit.client.0.smithi063.stdout:1/521: unlink d9/d46/d57/f65 0 2024-05-09T06:38:15.799 INFO:tasks.workunit.client.0.smithi063.stdout:2/622: rename dc/d32/d33/l37 to dc/dd/d45/d5a/lb6 0 2024-05-09T06:38:15.800 INFO:tasks.workunit.client.1.smithi193.stdout:6/228: creat d1/f4d x:0 0 0 2024-05-09T06:38:15.802 INFO:tasks.workunit.client.1.smithi193.stdout:7/269: symlink d6/d17/l5a 0 2024-05-09T06:38:15.805 INFO:tasks.workunit.client.1.smithi193.stdout:2/275: rename db/df/c19 to db/df/c64 0 2024-05-09T06:38:15.807 INFO:tasks.workunit.client.0.smithi063.stdout:8/636: dwrite d9/da/d34/fac [0,4194304] 0 2024-05-09T06:38:15.807 INFO:tasks.workunit.client.1.smithi193.stdout:1/234: creat d4/d16/d17/d1a/d3a/d3f/f45 x:0 0 0 2024-05-09T06:38:15.808 INFO:tasks.workunit.client.1.smithi193.stdout:9/266: symlink dd/l63 0 2024-05-09T06:38:15.809 INFO:tasks.workunit.client.1.smithi193.stdout:7/270: dwrite d6/d8/d1a/d36/f3b [0,4194304] 0 2024-05-09T06:38:15.813 INFO:tasks.workunit.client.0.smithi063.stdout:1/522: unlink d9/d2a/d39/f8c 0 2024-05-09T06:38:15.815 INFO:tasks.workunit.client.0.smithi063.stdout:9/612: mkdir d0/d4/d57/dba/dd5 0 2024-05-09T06:38:15.817 INFO:tasks.workunit.client.0.smithi063.stdout:4/569: write d7/d3d/f49 [443610,52393] 0 2024-05-09T06:38:15.817 INFO:tasks.workunit.client.0.smithi063.stdout:3/606: rename d7/da/d2b/d66/d74/faf to d7/da/d2b/d66/d74/d7e/fca 0 2024-05-09T06:38:15.817 INFO:tasks.workunit.client.1.smithi193.stdout:6/229: dread - d1/de/d11/f1f zero size 2024-05-09T06:38:15.820 INFO:tasks.workunit.client.0.smithi063.stdout:7/574: mkdir d0/d15/d4a/dc8 0 2024-05-09T06:38:15.820 INFO:tasks.workunit.client.0.smithi063.stdout:5/564: dwrite d8/d19/d30/d46/f6c [0,4194304] 0 2024-05-09T06:38:15.823 INFO:tasks.workunit.client.1.smithi193.stdout:0/301: truncate d18/d1d/f3d 3115557 0 2024-05-09T06:38:15.823 INFO:tasks.workunit.client.0.smithi063.stdout:8/637: fsync d9/da/d34/fac 0 2024-05-09T06:38:15.825 INFO:tasks.workunit.client.1.smithi193.stdout:0/302: write d18/d4c/f46 [515830,18672] 0 2024-05-09T06:38:15.825 INFO:tasks.workunit.client.0.smithi063.stdout:8/638: write d9/da/d34/f87 [971853,95042] 0 2024-05-09T06:38:15.829 INFO:tasks.workunit.client.0.smithi063.stdout:1/523: unlink d9/d2a/f74 0 2024-05-09T06:38:15.830 INFO:tasks.workunit.client.0.smithi063.stdout:1/524: write d9/d2a/d35/d1e/fa1 [74656,110653] 0 2024-05-09T06:38:15.833 INFO:tasks.workunit.client.1.smithi193.stdout:0/303: dread d18/d4c/f46 [0,4194304] 0 2024-05-09T06:38:15.835 INFO:tasks.workunit.client.1.smithi193.stdout:5/294: rename d0/d13/d2c/d37/d5b/l3b to d0/d13/l62 0 2024-05-09T06:38:15.837 INFO:tasks.workunit.client.1.smithi193.stdout:9/267: dwrite dd/df/d15/d16/d26/f62 [0,4194304] 0 2024-05-09T06:38:15.837 INFO:tasks.workunit.client.0.smithi063.stdout:9/613: dread d0/f5b [0,4194304] 0 2024-05-09T06:38:15.838 INFO:tasks.workunit.client.1.smithi193.stdout:2/276: creat db/d25/d38/d61/f65 x:0 0 0 2024-05-09T06:38:15.842 INFO:tasks.workunit.client.1.smithi193.stdout:1/235: symlink d4/d16/d35/l46 0 2024-05-09T06:38:15.847 INFO:tasks.workunit.client.0.smithi063.stdout:3/607: symlink d7/d4c/d25/lcb 0 2024-05-09T06:38:15.850 INFO:tasks.workunit.client.0.smithi063.stdout:4/570: symlink d7/df/d35/d5a/dad/lb5 0 2024-05-09T06:38:15.852 INFO:tasks.workunit.client.1.smithi193.stdout:7/271: link d6/d8/d1a/d36/f4e d6/d8/d1a/d36/f5b 0 2024-05-09T06:38:15.853 INFO:tasks.workunit.client.0.smithi063.stdout:3/608: dwrite d7/da/d21/d30/fa3 [0,4194304] 0 2024-05-09T06:38:15.853 INFO:tasks.workunit.client.0.smithi063.stdout:0/638: getdents dd/d1a/d2c/d51/d72/d9e/d23 0 2024-05-09T06:38:15.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:15 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:38:15.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:15 smithi063 ceph-mon[27171]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:15.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:15 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/883758720' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:38:15.856 INFO:tasks.workunit.client.1.smithi193.stdout:0/304: mkdir d18/d4c/d47/d5e 0 2024-05-09T06:38:15.860 INFO:tasks.workunit.client.0.smithi063.stdout:4/571: dwrite d7/df/d77/f8e [0,4194304] 0 2024-05-09T06:38:15.861 INFO:tasks.workunit.client.0.smithi063.stdout:4/572: chown d7/df/d1a/f22 540 1 2024-05-09T06:38:15.862 INFO:tasks.workunit.client.0.smithi063.stdout:6/599: sync 2024-05-09T06:38:15.864 INFO:tasks.workunit.client.1.smithi193.stdout:1/236: chown d4/c2c 6880 1 2024-05-09T06:38:15.865 INFO:tasks.workunit.client.1.smithi193.stdout:1/237: write d4/f26 [884829,95324] 0 2024-05-09T06:38:15.868 INFO:tasks.workunit.client.0.smithi063.stdout:6/600: dwrite d2/d1f/d29/f84 [0,4194304] 0 2024-05-09T06:38:15.870 INFO:tasks.workunit.client.1.smithi193.stdout:7/272: mknod d6/d8/d29/d2f/c5c 0 2024-05-09T06:38:15.871 INFO:tasks.workunit.client.0.smithi063.stdout:0/639: dread dd/d1a/f25 [0,4194304] 0 2024-05-09T06:38:15.873 INFO:tasks.workunit.client.0.smithi063.stdout:2/623: rename dc/dd/l29 to dc/d36/d43/d46/lb7 0 2024-05-09T06:38:15.875 INFO:tasks.workunit.client.0.smithi063.stdout:2/624: write dc/f21 [2695639,91468] 0 2024-05-09T06:38:15.876 INFO:tasks.workunit.client.0.smithi063.stdout:9/614: write d0/d4/d26/f33 [974382,107721] 0 2024-05-09T06:38:15.877 INFO:tasks.workunit.client.1.smithi193.stdout:7/273: dwrite d6/d8/d1a/d36/f3b [0,4194304] 0 2024-05-09T06:38:15.878 INFO:tasks.workunit.client.1.smithi193.stdout:7/274: readlink d6/d8/d29/d2f/l54 0 2024-05-09T06:38:15.879 INFO:tasks.workunit.client.1.smithi193.stdout:2/277: creat db/d25/d38/d5b/f66 x:0 0 0 2024-05-09T06:38:15.879 INFO:tasks.workunit.client.0.smithi063.stdout:3/609: unlink d7/d22/d28/f3f 0 2024-05-09T06:38:15.882 INFO:tasks.workunit.client.1.smithi193.stdout:9/268: link dd/df/d15/d16/d26/f32 dd/df/d15/d16/d39/d49/f64 0 2024-05-09T06:38:15.887 INFO:tasks.workunit.client.0.smithi063.stdout:4/573: creat d7/da3/d28/d31/d61/fb6 x:0 0 0 2024-05-09T06:38:15.892 INFO:tasks.workunit.client.0.smithi063.stdout:7/575: rename d0/d15/d49/d4f/d66/d67/d86/la1 to d0/d15/d49/d54/lc9 0 2024-05-09T06:38:15.893 INFO:tasks.workunit.client.0.smithi063.stdout:6/601: dread d2/d1f/d29/f47 [0,4194304] 0 2024-05-09T06:38:15.893 INFO:tasks.workunit.client.0.smithi063.stdout:7/576: chown d0/d15/d4a/d50/f23 14017 1 2024-05-09T06:38:15.896 INFO:tasks.workunit.client.1.smithi193.stdout:3/246: rename d1/d2d/l5b to d1/db/l5e 0 2024-05-09T06:38:15.897 INFO:tasks.workunit.client.0.smithi063.stdout:2/625: symlink dc/d36/d43/lb8 0 2024-05-09T06:38:15.899 INFO:tasks.workunit.client.1.smithi193.stdout:7/275: dread d6/d8/f35 [0,4194304] 0 2024-05-09T06:38:15.899 INFO:tasks.workunit.client.0.smithi063.stdout:6/602: dwrite d2/d1f/d29/d3f/f58 [0,4194304] 0 2024-05-09T06:38:15.900 INFO:tasks.workunit.client.1.smithi193.stdout:9/269: read dd/d1e/d1f/f2f [122106,58861] 0 2024-05-09T06:38:15.902 INFO:tasks.workunit.client.0.smithi063.stdout:3/610: dread - d7/da/d2b/d66/d74/d7e/f7f zero size 2024-05-09T06:38:15.904 INFO:tasks.workunit.client.1.smithi193.stdout:4/326: rename l8 to d1a/d24/l61 0 2024-05-09T06:38:15.906 INFO:tasks.workunit.client.0.smithi063.stdout:5/565: sync 2024-05-09T06:38:15.906 INFO:tasks.workunit.client.0.smithi063.stdout:1/525: link l4 d9/d76/d94/lb5 0 2024-05-09T06:38:15.911 INFO:tasks.workunit.client.0.smithi063.stdout:8/639: dwrite d9/f7d [0,4194304] 0 2024-05-09T06:38:15.913 INFO:tasks.workunit.client.0.smithi063.stdout:8/640: dread - d9/da/d18/dc7/d58/d79/db8/fd4 zero size 2024-05-09T06:38:15.914 INFO:tasks.workunit.client.1.smithi193.stdout:9/270: creat dd/df/d15/d16/d39/d49/f65 x:0 0 0 2024-05-09T06:38:15.917 INFO:tasks.workunit.client.1.smithi193.stdout:0/305: truncate d18/d1d/f3d 1418860 0 2024-05-09T06:38:15.919 INFO:tasks.workunit.client.1.smithi193.stdout:0/306: write d18/d4c/f2b [1162556,42588] 0 2024-05-09T06:38:15.922 INFO:tasks.workunit.client.1.smithi193.stdout:1/238: dread d4/d14/f2f [4194304,4194304] 0 2024-05-09T06:38:15.923 INFO:tasks.workunit.client.1.smithi193.stdout:9/271: dread dd/df/d15/d16/d26/d37/f4d [0,4194304] 0 2024-05-09T06:38:15.924 INFO:tasks.workunit.client.1.smithi193.stdout:1/239: chown d4/f9 184471680 1 2024-05-09T06:38:15.927 INFO:tasks.workunit.client.0.smithi063.stdout:8/641: creat d9/da/d18/d46/fe7 x:0 0 0 2024-05-09T06:38:15.929 INFO:tasks.workunit.client.0.smithi063.stdout:5/566: dread d8/d17/f6d [0,4194304] 0 2024-05-09T06:38:15.931 INFO:tasks.workunit.client.1.smithi193.stdout:9/272: dwrite dd/df/d15/d16/d26/f35 [0,4194304] 0 2024-05-09T06:38:15.931 INFO:tasks.workunit.client.1.smithi193.stdout:9/273: fdatasync fa 0 2024-05-09T06:38:15.933 INFO:tasks.workunit.client.1.smithi193.stdout:9/274: write dd/df/d15/f56 [2511106,7516] 0 2024-05-09T06:38:15.935 INFO:tasks.workunit.client.0.smithi063.stdout:3/611: symlink d7/lcc 0 2024-05-09T06:38:15.936 INFO:tasks.workunit.client.1.smithi193.stdout:6/230: rename d1/de/d11/d20/f21 to d1/de/d11/d18/d36/d37/f4e 0 2024-05-09T06:38:15.937 INFO:tasks.workunit.client.1.smithi193.stdout:0/307: dread - d18/d27/f41 zero size 2024-05-09T06:38:15.941 INFO:tasks.workunit.client.0.smithi063.stdout:9/615: dwrite d0/d4/d26/f36 [0,4194304] 0 2024-05-09T06:38:15.942 INFO:tasks.workunit.client.0.smithi063.stdout:4/574: rename d7/da3/d8b/fac to d7/d3d/fb7 0 2024-05-09T06:38:15.942 INFO:tasks.workunit.client.0.smithi063.stdout:6/603: write d2/df/f14 [2152689,69361] 0 2024-05-09T06:38:15.942 INFO:tasks.workunit.client.1.smithi193.stdout:4/327: write d1a/f23 [3170114,30912] 0 2024-05-09T06:38:15.944 INFO:tasks.workunit.client.0.smithi063.stdout:0/640: getdents dd/d1a/d2c/d51/d72/d9e/d23/d31 0 2024-05-09T06:38:15.959 INFO:tasks.workunit.client.1.smithi193.stdout:5/295: rename d0/d13/f40 to d0/d13/d2c/d37/d2b/d47/f63 0 2024-05-09T06:38:15.962 INFO:tasks.workunit.client.1.smithi193.stdout:3/247: getdents d1 0 2024-05-09T06:38:15.963 INFO:tasks.workunit.client.1.smithi193.stdout:3/248: fsync d1/d2d/f45 0 2024-05-09T06:38:15.963 INFO:tasks.workunit.client.1.smithi193.stdout:4/328: creat d1a/d24/f62 x:0 0 0 2024-05-09T06:38:15.964 INFO:tasks.workunit.client.0.smithi063.stdout:5/567: creat d8/d7e/fbe x:0 0 0 2024-05-09T06:38:15.964 INFO:tasks.workunit.client.0.smithi063.stdout:6/604: chown d2/d1f/d29/d3f/d5c/d64/c68 445 1 2024-05-09T06:38:15.964 INFO:tasks.workunit.client.1.smithi193.stdout:4/329: write d1a/f1d [1882010,21867] 0 2024-05-09T06:38:15.965 INFO:tasks.workunit.client.0.smithi063.stdout:7/577: rename d0/d15/f9a to d0/d15/d49/d4f/d66/d67/d86/fca 0 2024-05-09T06:38:15.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/616: mknod d0/d4/d26/d39/d48/d80/d84/dc2/cd6 0 2024-05-09T06:38:15.966 INFO:tasks.workunit.client.1.smithi193.stdout:4/330: fdatasync d1a/d1f/d4b/d59/f44 0 2024-05-09T06:38:15.966 INFO:tasks.workunit.client.1.smithi193.stdout:7/276: getdents d6/dd 0 2024-05-09T06:38:15.967 INFO:tasks.workunit.client.0.smithi063.stdout:0/641: write dd/d1a/f35 [3429777,125324] 0 2024-05-09T06:38:15.967 INFO:tasks.workunit.client.1.smithi193.stdout:7/277: stat d6/d8/fa 0 2024-05-09T06:38:15.968 INFO:tasks.workunit.client.0.smithi063.stdout:8/642: symlink d9/le8 0 2024-05-09T06:38:15.970 INFO:tasks.workunit.client.0.smithi063.stdout:1/526: getdents d9/d2a/d3c/d4c/d83 0 2024-05-09T06:38:15.970 INFO:tasks.workunit.client.0.smithi063.stdout:4/575: dwrite d7/d3d/faa [0,4194304] 0 2024-05-09T06:38:15.971 INFO:tasks.workunit.client.0.smithi063.stdout:4/576: stat d7/da3/d28/d31/d32/d34 0 2024-05-09T06:38:15.971 INFO:tasks.workunit.client.1.smithi193.stdout:9/275: rename dd/df/d15/d16/d39 to dd/d1e/d66 0 2024-05-09T06:38:15.973 INFO:tasks.workunit.client.0.smithi063.stdout:4/577: truncate d7/da3/d28/d31/d71/d98/fa0 216953 0 2024-05-09T06:38:15.973 INFO:tasks.workunit.client.1.smithi193.stdout:6/231: fsync d1/de/d11/d18/d36/d37/f4e 0 2024-05-09T06:38:15.976 INFO:tasks.workunit.client.1.smithi193.stdout:5/296: truncate d0/d13/d2c/d38/f3e 1273000 0 2024-05-09T06:38:15.978 INFO:tasks.workunit.client.1.smithi193.stdout:1/240: creat d4/d16/f47 x:0 0 0 2024-05-09T06:38:15.979 INFO:tasks.workunit.client.1.smithi193.stdout:5/297: truncate d0/d13/d2c/d37/d5b/d4e/f57 807576 0 2024-05-09T06:38:15.979 INFO:tasks.workunit.client.1.smithi193.stdout:1/241: fdatasync d4/d16/de/f12 0 2024-05-09T06:38:15.980 INFO:tasks.workunit.client.1.smithi193.stdout:1/242: chown d4/d16/d17/f29 224917625 1 2024-05-09T06:38:15.982 INFO:tasks.workunit.client.1.smithi193.stdout:3/249: creat d1/d44/f5f x:0 0 0 2024-05-09T06:38:15.982 INFO:tasks.workunit.client.1.smithi193.stdout:5/298: truncate d0/d13/d2c/d37/d5b/d4e/f58 1096601 0 2024-05-09T06:38:15.982 INFO:tasks.workunit.client.1.smithi193.stdout:3/250: dread - d1/db/f3f zero size 2024-05-09T06:38:15.982 INFO:tasks.workunit.client.1.smithi193.stdout:5/299: fsync d0/d13/f3d 0 2024-05-09T06:38:15.983 INFO:tasks.workunit.client.0.smithi063.stdout:5/568: stat d8/d17/d18/f51 0 2024-05-09T06:38:15.984 INFO:tasks.workunit.client.1.smithi193.stdout:5/300: truncate d0/f5c 926541 0 2024-05-09T06:38:15.984 INFO:tasks.workunit.client.1.smithi193.stdout:3/251: chown d1/db/f34 672 1 2024-05-09T06:38:15.984 INFO:tasks.workunit.client.1.smithi193.stdout:3/252: chown d1/db/c5a 62 1 2024-05-09T06:38:15.985 INFO:tasks.workunit.client.0.smithi063.stdout:2/626: rename dc/d36/f52 to dc/d36/d43/d46/d49/fb9 0 2024-05-09T06:38:15.988 INFO:tasks.workunit.client.0.smithi063.stdout:2/627: truncate dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72/d7b/fae 349247 0 2024-05-09T06:38:15.989 INFO:tasks.workunit.client.0.smithi063.stdout:9/617: chown d0/d4/d57/d60/cbf 2 1 2024-05-09T06:38:15.991 INFO:tasks.workunit.client.0.smithi063.stdout:7/578: unlink d0/d15/d49/d4f/d66/c77 0 2024-05-09T06:38:15.992 INFO:tasks.workunit.client.1.smithi193.stdout:9/276: mknod dd/d1e/c67 0 2024-05-09T06:38:15.993 INFO:tasks.workunit.client.0.smithi063.stdout:8/643: write d9/da/d34/fd5 [153500,100060] 0 2024-05-09T06:38:15.994 INFO:tasks.workunit.client.1.smithi193.stdout:6/232: mkdir d1/de/d11/d18/d36/d37/d4f 0 2024-05-09T06:38:15.995 INFO:tasks.workunit.client.1.smithi193.stdout:6/233: dread - d1/de/f38 zero size 2024-05-09T06:38:15.996 INFO:tasks.workunit.client.0.smithi063.stdout:8/644: dread d9/da/d51/f8b [0,4194304] 0 2024-05-09T06:38:15.999 INFO:tasks.workunit.client.1.smithi193.stdout:1/243: rmdir d4/d16 39 2024-05-09T06:38:16.000 INFO:tasks.workunit.client.1.smithi193.stdout:1/244: chown d4/d14/f2f 156 1 2024-05-09T06:38:16.000 INFO:tasks.workunit.client.1.smithi193.stdout:4/331: dread d1a/d24/d30/f34 [0,4194304] 0 2024-05-09T06:38:16.001 INFO:tasks.workunit.client.0.smithi063.stdout:5/569: truncate d8/f27 3861402 0 2024-05-09T06:38:16.001 INFO:tasks.workunit.client.1.smithi193.stdout:4/332: dread - d1a/d1f/d4b/d59/f42 zero size 2024-05-09T06:38:16.004 INFO:tasks.workunit.client.0.smithi063.stdout:6/605: write d2/fd [1015911,127541] 0 2024-05-09T06:38:16.005 INFO:tasks.workunit.client.1.smithi193.stdout:5/301: read d0/d13/d2c/d37/d5b/da/f26 [3277668,120517] 0 2024-05-09T06:38:16.005 INFO:tasks.workunit.client.1.smithi193.stdout:3/253: creat d1/d44/f60 x:0 0 0 2024-05-09T06:38:16.008 INFO:tasks.workunit.client.0.smithi063.stdout:1/527: readlink d9/d2a/d35/d1e/l6d 0 2024-05-09T06:38:16.008 INFO:tasks.workunit.client.1.smithi193.stdout:9/277: truncate f0 5717069 0 2024-05-09T06:38:16.009 INFO:tasks.workunit.client.1.smithi193.stdout:9/278: fsync dd/df/f10 0 2024-05-09T06:38:16.012 INFO:tasks.workunit.client.1.smithi193.stdout:4/333: creat d1a/d1f/d4b/d59/f63 x:0 0 0 2024-05-09T06:38:16.013 INFO:tasks.workunit.client.0.smithi063.stdout:8/645: mkdir d9/da/d53/da3/de9 0 2024-05-09T06:38:16.013 INFO:tasks.workunit.client.1.smithi193.stdout:1/245: dread d4/ff [0,4194304] 0 2024-05-09T06:38:16.014 INFO:tasks.workunit.client.1.smithi193.stdout:5/302: mknod d0/d13/d2c/d37/c64 0 2024-05-09T06:38:16.018 INFO:tasks.workunit.client.1.smithi193.stdout:3/254: symlink d1/d44/l61 0 2024-05-09T06:38:16.019 INFO:tasks.workunit.client.1.smithi193.stdout:1/246: dread d4/f26 [0,4194304] 0 2024-05-09T06:38:16.019 INFO:tasks.workunit.client.0.smithi063.stdout:6/606: symlink d2/d1f/d29/d3f/lc3 0 2024-05-09T06:38:16.021 INFO:tasks.workunit.client.0.smithi063.stdout:2/628: mknod dc/d6e/d7f/cba 0 2024-05-09T06:38:16.022 INFO:tasks.workunit.client.1.smithi193.stdout:3/255: dread d1/db/f34 [0,4194304] 0 2024-05-09T06:38:16.025 INFO:tasks.workunit.client.0.smithi063.stdout:9/618: mknod d0/d4/d57/dba/dd5/cd7 0 2024-05-09T06:38:16.027 INFO:tasks.workunit.client.0.smithi063.stdout:2/629: dread dc/d32/d33/f4c [0,4194304] 0 2024-05-09T06:38:16.028 INFO:tasks.workunit.client.0.smithi063.stdout:7/579: mkdir d0/dcb 0 2024-05-09T06:38:16.029 INFO:tasks.workunit.client.1.smithi193.stdout:3/256: dwrite d1/d44/f47 [0,4194304] 0 2024-05-09T06:38:16.029 INFO:tasks.workunit.client.0.smithi063.stdout:7/580: write d0/d15/d49/d54/f60 [4137221,126782] 0 2024-05-09T06:38:16.032 INFO:tasks.workunit.client.0.smithi063.stdout:1/528: mknod d9/d2a/d35/d24/cb6 0 2024-05-09T06:38:16.035 INFO:tasks.workunit.client.0.smithi063.stdout:4/578: link d7/da3/d28/d31/d32/d34/f44 d7/da3/da2/fb8 0 2024-05-09T06:38:16.038 INFO:tasks.workunit.client.0.smithi063.stdout:8/646: unlink d9/da/d53/db2/fd0 0 2024-05-09T06:38:16.039 INFO:tasks.workunit.client.1.smithi193.stdout:5/303: mknod d0/d13/d2c/d37/d5b/da/d3a/c65 0 2024-05-09T06:38:16.040 INFO:tasks.workunit.client.0.smithi063.stdout:0/642: dwrite dd/d60/d44/f61 [0,4194304] 0 2024-05-09T06:38:16.040 INFO:tasks.workunit.client.0.smithi063.stdout:5/570: creat d8/d17/da3/fbf x:0 0 0 2024-05-09T06:38:16.041 INFO:tasks.workunit.client.0.smithi063.stdout:0/643: readlink dd/d1a/d2c/d42/l81 0 2024-05-09T06:38:16.044 INFO:tasks.workunit.client.1.smithi193.stdout:1/247: stat d4/d16/de/f28 0 2024-05-09T06:38:16.045 INFO:tasks.workunit.client.1.smithi193.stdout:7/278: getdents d6/d8/d29 0 2024-05-09T06:38:16.046 INFO:tasks.workunit.client.1.smithi193.stdout:5/304: dwrite d0/d13/f3d [0,4194304] 0 2024-05-09T06:38:16.047 INFO:tasks.workunit.client.0.smithi063.stdout:9/619: mkdir d0/d4/d26/d39/d48/d80/d84/dd8 0 2024-05-09T06:38:16.048 INFO:tasks.workunit.client.1.smithi193.stdout:5/305: write d0/d13/d2c/d37/d5b/f21 [3228541,32917] 0 2024-05-09T06:38:16.052 INFO:tasks.workunit.client.1.smithi193.stdout:3/257: symlink d1/d2d/d51/l62 0 2024-05-09T06:38:16.053 INFO:tasks.workunit.client.0.smithi063.stdout:7/581: creat d0/d15/fcc x:0 0 0 2024-05-09T06:38:16.053 INFO:tasks.workunit.client.1.smithi193.stdout:3/258: dread - d1/db/f3f zero size 2024-05-09T06:38:16.054 INFO:tasks.workunit.client.1.smithi193.stdout:7/279: dread d6/d17/f37 [0,4194304] 0 2024-05-09T06:38:16.055 INFO:tasks.workunit.client.1.smithi193.stdout:7/280: chown d6/d8/fa 37 1 2024-05-09T06:38:16.057 INFO:tasks.workunit.client.0.smithi063.stdout:1/529: rmdir d9/d46/d57 39 2024-05-09T06:38:16.060 INFO:tasks.workunit.client.1.smithi193.stdout:3/259: dwrite d1/db/d59/f37 [0,4194304] 0 2024-05-09T06:38:16.063 INFO:tasks.workunit.client.0.smithi063.stdout:4/579: dread d7/da3/d28/f7d [0,4194304] 0 2024-05-09T06:38:16.064 INFO:tasks.workunit.client.1.smithi193.stdout:6/234: getdents d1/d25 0 2024-05-09T06:38:16.065 INFO:tasks.workunit.client.1.smithi193.stdout:6/235: dread - d1/d25/d2c/d3c/f3d zero size 2024-05-09T06:38:16.070 INFO:tasks.workunit.client.1.smithi193.stdout:5/306: read d0/f8 [556282,26046] 0 2024-05-09T06:38:16.073 INFO:tasks.workunit.client.0.smithi063.stdout:6/607: creat d2/d20/d26/fc4 x:0 0 0 2024-05-09T06:38:16.074 INFO:tasks.workunit.client.1.smithi193.stdout:4/334: truncate d1a/d1f/f20 742991 0 2024-05-09T06:38:16.074 INFO:tasks.workunit.client.0.smithi063.stdout:6/608: chown d2/d1f/d29/d3b/d4e/lbc 863 1 2024-05-09T06:38:16.074 INFO:tasks.workunit.client.1.smithi193.stdout:9/279: fdatasync f0 0 2024-05-09T06:38:16.075 INFO:tasks.workunit.client.1.smithi193.stdout:9/280: chown dd/df/d31/l3b 6165 1 2024-05-09T06:38:16.075 INFO:tasks.workunit.client.0.smithi063.stdout:4/580: dread d7/f39 [0,4194304] 0 2024-05-09T06:38:16.076 INFO:tasks.workunit.client.1.smithi193.stdout:7/281: write d6/f16 [3686517,93974] 0 2024-05-09T06:38:16.079 INFO:tasks.workunit.client.1.smithi193.stdout:3/260: rmdir d1/db/d59 39 2024-05-09T06:38:16.081 INFO:tasks.workunit.client.0.smithi063.stdout:0/644: write dd/d1a/d2c/d51/d72/d9e/f67 [688734,60587] 0 2024-05-09T06:38:16.082 INFO:tasks.workunit.client.0.smithi063.stdout:7/582: dread d0/d15/d49/d4f/d66/d67/d6b/f6e [0,4194304] 0 2024-05-09T06:38:16.083 INFO:tasks.workunit.client.0.smithi063.stdout:7/583: read d0/d15/d49/f65 [552865,99636] 0 2024-05-09T06:38:16.084 INFO:tasks.workunit.client.1.smithi193.stdout:1/248: dwrite d4/d16/f27 [0,4194304] 0 2024-05-09T06:38:16.085 INFO:tasks.workunit.client.1.smithi193.stdout:6/236: creat d1/d25/d2c/f50 x:0 0 0 2024-05-09T06:38:16.086 INFO:tasks.workunit.client.0.smithi063.stdout:8/647: truncate d9/f13 1005364 0 2024-05-09T06:38:16.088 INFO:tasks.workunit.client.1.smithi193.stdout:1/249: chown d4/d16/d35/l46 2092852 1 2024-05-09T06:38:16.089 INFO:tasks.workunit.client.1.smithi193.stdout:3/261: dwrite d1/d2d/d51/f54 [0,4194304] 0 2024-05-09T06:38:16.090 INFO:tasks.workunit.client.0.smithi063.stdout:5/571: symlink d8/d35/db6/lc0 0 2024-05-09T06:38:16.091 INFO:tasks.workunit.client.1.smithi193.stdout:5/307: rename d0/d13/d2c/d37/d5b/da to d0/d13/d2c/d37/d2b/d47/d66 0 2024-05-09T06:38:16.095 INFO:tasks.workunit.client.1.smithi193.stdout:4/335: creat d1a/f64 x:0 0 0 2024-05-09T06:38:16.096 INFO:tasks.workunit.client.1.smithi193.stdout:4/336: fdatasync d1a/d24/f2e 0 2024-05-09T06:38:16.096 INFO:tasks.workunit.client.1.smithi193.stdout:4/337: fdatasync d1a/d1f/d4b/f54 0 2024-05-09T06:38:16.098 INFO:tasks.workunit.client.0.smithi063.stdout:6/609: truncate d2/d4/d7c/d7e/f87 651812 0 2024-05-09T06:38:16.099 INFO:tasks.workunit.client.1.smithi193.stdout:4/338: dread d1a/d1f/d4b/d59/f3c [0,4194304] 0 2024-05-09T06:38:16.099 INFO:tasks.workunit.client.0.smithi063.stdout:2/630: rmdir dc/d36/d43/d46/d63/d79/da3 0 2024-05-09T06:38:16.100 INFO:tasks.workunit.client.1.smithi193.stdout:4/339: write d1a/d1f/f2c [2030381,116358] 0 2024-05-09T06:38:16.101 INFO:tasks.workunit.client.1.smithi193.stdout:7/282: fdatasync d6/dd/f40 0 2024-05-09T06:38:16.102 INFO:tasks.workunit.client.0.smithi063.stdout:1/530: rename d9/d2a/d35/d1e/d9d/d3f/c95 to d9/d2a/d35/d24/cb7 0 2024-05-09T06:38:16.107 INFO:tasks.workunit.client.0.smithi063.stdout:7/584: rmdir d0/d15/d49/d4f/d66 39 2024-05-09T06:38:16.107 INFO:tasks.workunit.client.1.smithi193.stdout:4/340: dwrite d1a/f1b [0,4194304] 0 2024-05-09T06:38:16.109 INFO:tasks.workunit.client.1.smithi193.stdout:4/341: write d1a/f1b [493448,96556] 0 2024-05-09T06:38:16.109 INFO:tasks.workunit.client.1.smithi193.stdout:1/250: write d4/d14/f2f [1804483,60591] 0 2024-05-09T06:38:16.109 INFO:tasks.workunit.client.0.smithi063.stdout:8/648: chown d9/da/d18/dc7/d8f/lbe 375 1 2024-05-09T06:38:16.116 INFO:tasks.workunit.client.0.smithi063.stdout:4/581: creat d7/df/d35/da5/fb9 x:0 0 0 2024-05-09T06:38:16.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/531: dread d9/d2a/d2e/f81 [0,4194304] 0 2024-05-09T06:38:16.123 INFO:tasks.workunit.client.1.smithi193.stdout:9/281: rename dd/d1e/d1f/d48/d55/d59 to dd/d1e/d1f/d48/d55/d41/d68 0 2024-05-09T06:38:16.124 INFO:tasks.workunit.client.0.smithi063.stdout:3/612: sync 2024-05-09T06:38:16.125 INFO:tasks.workunit.client.0.smithi063.stdout:3/613: chown d7/da/d2b/d66/d74/d7e/l7d 333516 1 2024-05-09T06:38:16.129 INFO:tasks.workunit.client.1.smithi193.stdout:6/237: mknod d1/c51 0 2024-05-09T06:38:16.132 INFO:tasks.workunit.client.1.smithi193.stdout:4/342: creat d1a/f65 x:0 0 0 2024-05-09T06:38:16.132 INFO:tasks.workunit.client.1.smithi193.stdout:4/343: dread - d1a/d22/f31 zero size 2024-05-09T06:38:16.133 INFO:tasks.workunit.client.0.smithi063.stdout:1/532: truncate d9/d2a/d35/f8f 47013 0 2024-05-09T06:38:16.134 INFO:tasks.workunit.client.0.smithi063.stdout:1/533: dread - d9/d2a/d2e/f7d zero size 2024-05-09T06:38:16.134 INFO:tasks.workunit.client.0.smithi063.stdout:4/582: dread d7/d50/f9d [0,4194304] 0 2024-05-09T06:38:16.135 INFO:tasks.workunit.client.1.smithi193.stdout:1/251: creat d4/d16/d35/f48 x:0 0 0 2024-05-09T06:38:16.135 INFO:tasks.workunit.client.0.smithi063.stdout:8/649: write d9/d3b/f40 [2801467,54917] 0 2024-05-09T06:38:16.135 INFO:tasks.workunit.client.0.smithi063.stdout:1/534: stat d9/d2a/d3c/d4a 0 2024-05-09T06:38:16.136 INFO:tasks.workunit.client.0.smithi063.stdout:5/572: link d8/d53/f93 d8/d17/fc1 0 2024-05-09T06:38:16.138 INFO:tasks.workunit.client.1.smithi193.stdout:3/262: mkdir d1/db/d59/d63 0 2024-05-09T06:38:16.140 INFO:tasks.workunit.client.0.smithi063.stdout:3/614: creat d7/d70/d98/d2d/d36/fcd x:0 0 0 2024-05-09T06:38:16.140 INFO:tasks.workunit.client.0.smithi063.stdout:2/631: write dc/dd/f4f [2913868,117716] 0 2024-05-09T06:38:16.141 INFO:tasks.workunit.client.1.smithi193.stdout:5/308: mkdir d0/d13/d2c/d4d/d67 0 2024-05-09T06:38:16.141 INFO:tasks.workunit.client.0.smithi063.stdout:3/615: dread - d7/da/d2b/d66/dc0/fb3 zero size 2024-05-09T06:38:16.142 INFO:tasks.workunit.client.1.smithi193.stdout:9/282: fdatasync dd/df/d15/d16/d26/f32 0 2024-05-09T06:38:16.143 INFO:tasks.workunit.client.1.smithi193.stdout:9/283: chown dd/df/d15/d16/f50 5562295 1 2024-05-09T06:38:16.144 INFO:tasks.workunit.client.1.smithi193.stdout:7/283: mknod d6/d8/d1a/c5d 0 2024-05-09T06:38:16.145 INFO:tasks.workunit.client.0.smithi063.stdout:8/650: creat d9/da/d34/fea x:0 0 0 2024-05-09T06:38:16.145 INFO:tasks.workunit.client.1.smithi193.stdout:6/238: creat d1/de/d11/d2d/f52 x:0 0 0 2024-05-09T06:38:16.148 INFO:tasks.workunit.client.0.smithi063.stdout:1/535: mkdir d9/d2a/d3c/d4a/db8 0 2024-05-09T06:38:16.148 INFO:tasks.workunit.client.1.smithi193.stdout:4/344: write f6 [315828,15378] 0 2024-05-09T06:38:16.149 INFO:tasks.workunit.client.1.smithi193.stdout:1/252: chown d4/d14/f39 5 1 2024-05-09T06:38:16.150 INFO:tasks.workunit.client.1.smithi193.stdout:4/345: fdatasync d1a/d1f/d4b/f54 0 2024-05-09T06:38:16.151 INFO:tasks.workunit.client.1.smithi193.stdout:7/284: dwrite d6/f16 [4194304,4194304] 0 2024-05-09T06:38:16.152 INFO:tasks.workunit.client.1.smithi193.stdout:7/285: fdatasync d6/d8/d1a/d36/f57 0 2024-05-09T06:38:16.153 INFO:tasks.workunit.client.1.smithi193.stdout:3/263: mknod d1/d2d/c64 0 2024-05-09T06:38:16.154 INFO:tasks.workunit.client.0.smithi063.stdout:2/632: mkdir dc/d36/d43/d46/d49/dbb 0 2024-05-09T06:38:16.158 INFO:tasks.workunit.client.1.smithi193.stdout:7/286: dread d6/dd/f47 [0,4194304] 0 2024-05-09T06:38:16.159 INFO:tasks.workunit.client.0.smithi063.stdout:8/651: chown d9/da/l12 38460109 1 2024-05-09T06:38:16.161 INFO:tasks.workunit.client.0.smithi063.stdout:4/583: getdents d7/da3/d28/d31/d7b/d81 0 2024-05-09T06:38:16.162 INFO:tasks.workunit.client.0.smithi063.stdout:4/584: dread - d7/df/d35/d5a/d60/f6e zero size 2024-05-09T06:38:16.163 INFO:tasks.workunit.client.0.smithi063.stdout:1/536: symlink d9/d10/lb9 0 2024-05-09T06:38:16.163 INFO:tasks.workunit.client.0.smithi063.stdout:1/537: chown d9/d2a/d35/d1e/d9d/l6b 58 1 2024-05-09T06:38:16.164 INFO:tasks.workunit.client.1.smithi193.stdout:9/284: symlink dd/d1e/d66/l69 0 2024-05-09T06:38:16.164 INFO:tasks.workunit.client.0.smithi063.stdout:1/538: chown d9/d2a/d35/d24/f26 24370 1 2024-05-09T06:38:16.165 INFO:tasks.workunit.client.1.smithi193.stdout:6/239: creat d1/de/f53 x:0 0 0 2024-05-09T06:38:16.166 INFO:tasks.workunit.client.1.smithi193.stdout:6/240: write d1/de/d11/d18/f29 [1329708,24364] 0 2024-05-09T06:38:16.167 INFO:tasks.workunit.client.1.smithi193.stdout:4/346: creat d1a/d1f/d4b/d59/f66 x:0 0 0 2024-05-09T06:38:16.168 INFO:tasks.workunit.client.0.smithi063.stdout:8/652: creat d9/da/d18/d24/d71/feb x:0 0 0 2024-05-09T06:38:16.168 INFO:tasks.workunit.client.0.smithi063.stdout:8/653: stat d9/da/d18/ca0 0 2024-05-09T06:38:16.169 INFO:tasks.workunit.client.1.smithi193.stdout:8/245: sync 2024-05-09T06:38:16.171 INFO:tasks.workunit.client.0.smithi063.stdout:5/573: write d8/f21 [3401046,79022] 0 2024-05-09T06:38:16.172 INFO:tasks.workunit.client.1.smithi193.stdout:3/264: creat d1/d2d/d51/f65 x:0 0 0 2024-05-09T06:38:16.173 INFO:tasks.workunit.client.1.smithi193.stdout:3/265: readlink d1/l15 0 2024-05-09T06:38:16.174 INFO:tasks.workunit.client.0.smithi063.stdout:1/539: unlink d9/d2a/d2e/f81 0 2024-05-09T06:38:16.174 INFO:tasks.workunit.client.1.smithi193.stdout:3/266: chown d1/db 30 1 2024-05-09T06:38:16.174 INFO:tasks.workunit.client.1.smithi193.stdout:6/241: dwrite d1/de/f53 [0,4194304] 0 2024-05-09T06:38:16.175 INFO:tasks.workunit.client.0.smithi063.stdout:1/540: write d9/d2a/d3c/d4a/f67 [254250,125670] 0 2024-05-09T06:38:16.177 INFO:tasks.workunit.client.1.smithi193.stdout:7/287: mkdir d6/dd/d5e 0 2024-05-09T06:38:16.179 INFO:tasks.workunit.client.1.smithi193.stdout:9/285: fsync dd/df/f1d 0 2024-05-09T06:38:16.181 INFO:tasks.workunit.client.0.smithi063.stdout:4/585: mknod d7/da6/cba 0 2024-05-09T06:38:16.182 INFO:tasks.workunit.client.1.smithi193.stdout:9/286: dread - dd/df/d31/f4c zero size 2024-05-09T06:38:16.183 INFO:tasks.workunit.client.0.smithi063.stdout:1/541: dwrite d9/d2a/d35/d1e/d9d/d3f/d59/d62/fa5 [0,4194304] 0 2024-05-09T06:38:16.183 INFO:tasks.workunit.client.0.smithi063.stdout:5/574: rmdir d8/d19/d30/d46 39 2024-05-09T06:38:16.184 INFO:tasks.workunit.client.1.smithi193.stdout:7/288: dwrite d6/f13 [0,4194304] 0 2024-05-09T06:38:16.184 INFO:tasks.workunit.client.0.smithi063.stdout:5/575: write d8/d65/f80 [5247409,87693] 0 2024-05-09T06:38:16.186 INFO:tasks.workunit.client.0.smithi063.stdout:5/576: dread - d8/d17/d75/fb5 zero size 2024-05-09T06:38:16.187 INFO:tasks.workunit.client.1.smithi193.stdout:8/246: creat d6/d1c/d1b/f4a x:0 0 0 2024-05-09T06:38:16.189 INFO:tasks.workunit.client.1.smithi193.stdout:8/247: fsync d6/d1c/d18/f26 0 2024-05-09T06:38:16.195 INFO:tasks.workunit.client.0.smithi063.stdout:8/654: rename d9/da/d18/dc7/d58/d79 to d9/d41/dec 0 2024-05-09T06:38:16.196 INFO:tasks.workunit.client.1.smithi193.stdout:3/267: creat d1/db/d2f/f66 x:0 0 0 2024-05-09T06:38:16.197 INFO:tasks.workunit.client.1.smithi193.stdout:6/242: creat d1/d25/d2c/d3c/f54 x:0 0 0 2024-05-09T06:38:16.198 INFO:tasks.workunit.client.1.smithi193.stdout:6/243: stat d1/f42 0 2024-05-09T06:38:16.200 INFO:tasks.workunit.client.0.smithi063.stdout:4/586: dread d7/da3/da2/fb8 [0,4194304] 0 2024-05-09T06:38:16.200 INFO:tasks.workunit.client.0.smithi063.stdout:2/633: getdents dc/d36/d43/d46/d63 0 2024-05-09T06:38:16.204 INFO:tasks.workunit.client.0.smithi063.stdout:8/655: creat d9/d41/dec/da1/fed x:0 0 0 2024-05-09T06:38:16.205 INFO:tasks.workunit.client.1.smithi193.stdout:5/309: getdents d0/d13/d2c/d4d 0 2024-05-09T06:38:16.205 INFO:tasks.workunit.client.1.smithi193.stdout:9/287: dwrite dd/f18 [0,4194304] 0 2024-05-09T06:38:16.206 INFO:tasks.workunit.client.0.smithi063.stdout:2/634: dwrite dc/d32/d33/fb0 [0,4194304] 0 2024-05-09T06:38:16.207 INFO:tasks.workunit.client.1.smithi193.stdout:9/288: write dd/d1e/d3c/f5c [615228,24343] 0 2024-05-09T06:38:16.208 INFO:tasks.workunit.client.1.smithi193.stdout:6/244: creat d1/de/d11/d2d/d3a/f55 x:0 0 0 2024-05-09T06:38:16.210 INFO:tasks.workunit.client.0.smithi063.stdout:8/656: mknod d9/da/cee 0 2024-05-09T06:38:16.211 INFO:tasks.workunit.client.1.smithi193.stdout:8/248: fdatasync d6/f21 0 2024-05-09T06:38:16.213 INFO:tasks.workunit.client.0.smithi063.stdout:2/635: mknod dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72/d90/cbc 0 2024-05-09T06:38:16.214 INFO:tasks.workunit.client.1.smithi193.stdout:3/268: symlink d1/db/l67 0 2024-05-09T06:38:16.215 INFO:tasks.workunit.client.1.smithi193.stdout:3/269: chown d1/db/d59/f43 129637430 1 2024-05-09T06:38:16.216 INFO:tasks.workunit.client.0.smithi063.stdout:1/542: rename d9/d2a/d35/c2d to d9/d46/cba 0 2024-05-09T06:38:16.217 INFO:tasks.workunit.client.1.smithi193.stdout:5/310: write d0/d13/d2c/d37/d2b/d47/d66/f23 [307771,103391] 0 2024-05-09T06:38:16.218 INFO:tasks.workunit.client.1.smithi193.stdout:5/311: chown d0/d13/d2c/d4d/l50 818 1 2024-05-09T06:38:16.219 INFO:tasks.workunit.client.0.smithi063.stdout:2/636: dwrite dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72/d7b/fae [0,4194304] 0 2024-05-09T06:38:16.220 INFO:tasks.workunit.client.1.smithi193.stdout:6/245: mknod d1/de/d11/d18/c56 0 2024-05-09T06:38:16.226 INFO:tasks.workunit.client.0.smithi063.stdout:2/637: truncate dc/dd/f64 2034344 0 2024-05-09T06:38:16.228 INFO:tasks.workunit.client.1.smithi193.stdout:6/246: dwrite d1/de/d11/d2d/d3a/f55 [0,4194304] 0 2024-05-09T06:38:16.230 INFO:tasks.workunit.client.1.smithi193.stdout:9/289: mknod dd/df/c6a 0 2024-05-09T06:38:16.235 INFO:tasks.workunit.client.0.smithi063.stdout:5/577: dwrite d8/f66 [0,4194304] 0 2024-05-09T06:38:16.237 INFO:tasks.workunit.client.1.smithi193.stdout:7/289: dwrite d6/d8/d1a/d36/f5b [0,4194304] 0 2024-05-09T06:38:16.237 INFO:tasks.workunit.client.0.smithi063.stdout:8/657: getdents d9/da/d18/d46 0 2024-05-09T06:38:16.238 INFO:tasks.workunit.client.1.smithi193.stdout:8/249: link d6/f20 d6/d1c/d1b/d29/f4b 0 2024-05-09T06:38:16.240 INFO:tasks.workunit.client.1.smithi193.stdout:8/250: write d6/d1c/d18/d2f/f39 [546854,2149] 0 2024-05-09T06:38:16.246 INFO:tasks.workunit.client.1.smithi193.stdout:9/290: dread dd/d1e/d1f/f2f [0,4194304] 0 2024-05-09T06:38:16.247 INFO:tasks.workunit.client.1.smithi193.stdout:9/291: chown dd/df/d15/d16/d26 44 1 2024-05-09T06:38:16.248 INFO:tasks.workunit.client.0.smithi063.stdout:8/658: creat d9/d41/fef x:0 0 0 2024-05-09T06:38:16.251 INFO:tasks.workunit.client.1.smithi193.stdout:7/290: creat d6/d8/d1a/d36/f5f x:0 0 0 2024-05-09T06:38:16.253 INFO:tasks.workunit.client.0.smithi063.stdout:5/578: dwrite d8/d53/fab [0,4194304] 0 2024-05-09T06:38:16.255 INFO:tasks.workunit.client.1.smithi193.stdout:7/291: dread d6/d17/f42 [0,4194304] 0 2024-05-09T06:38:16.257 INFO:tasks.workunit.client.0.smithi063.stdout:8/659: mknod d9/da/d45/cf0 0 2024-05-09T06:38:16.258 INFO:tasks.workunit.client.1.smithi193.stdout:7/292: dread d6/d17/f52 [0,4194304] 0 2024-05-09T06:38:16.258 INFO:tasks.workunit.client.0.smithi063.stdout:2/638: write dc/dd/d45/d5a/d6a/fa4 [2103988,106497] 0 2024-05-09T06:38:16.259 INFO:tasks.workunit.client.0.smithi063.stdout:5/579: dwrite d8/d35/f64 [0,4194304] 0 2024-05-09T06:38:16.259 INFO:tasks.workunit.client.1.smithi193.stdout:3/270: rename d1/db/l21 to d1/db/l68 0 2024-05-09T06:38:16.263 INFO:tasks.workunit.client.1.smithi193.stdout:6/247: creat d1/d48/f57 x:0 0 0 2024-05-09T06:38:16.264 INFO:tasks.workunit.client.1.smithi193.stdout:8/251: fsync d6/f20 0 2024-05-09T06:38:16.265 INFO:tasks.workunit.client.0.smithi063.stdout:2/639: dwrite dc/d6e/d7f/fa1 [0,4194304] 0 2024-05-09T06:38:16.266 INFO:tasks.workunit.client.0.smithi063.stdout:2/640: readlink dc/dd/l28 0 2024-05-09T06:38:16.267 INFO:tasks.workunit.client.1.smithi193.stdout:9/292: creat dd/d1e/d1f/d48/d55/d41/d68/f6b x:0 0 0 2024-05-09T06:38:16.267 INFO:tasks.workunit.client.0.smithi063.stdout:2/641: write dc/dd/d2d/d60/fa2 [573815,109478] 0 2024-05-09T06:38:16.272 INFO:tasks.workunit.client.0.smithi063.stdout:5/580: fdatasync d8/d35/f7b 0 2024-05-09T06:38:16.273 INFO:tasks.workunit.client.0.smithi063.stdout:5/581: dread - d8/d53/fa5 zero size 2024-05-09T06:38:16.275 INFO:tasks.workunit.client.1.smithi193.stdout:7/293: chown d6/d8/d1a/d36/l49 457845 1 2024-05-09T06:38:16.275 INFO:tasks.workunit.client.0.smithi063.stdout:5/582: write d8/d35/f41 [1685492,61246] 0 2024-05-09T06:38:16.275 INFO:tasks.workunit.client.0.smithi063.stdout:8/660: mkdir d9/da/d18/dc7/dd3/df1 0 2024-05-09T06:38:16.276 INFO:tasks.workunit.client.1.smithi193.stdout:3/271: creat d1/db/d59/f69 x:0 0 0 2024-05-09T06:38:16.277 INFO:tasks.workunit.client.1.smithi193.stdout:3/272: read - d1/d44/f4a zero size 2024-05-09T06:38:16.278 INFO:tasks.workunit.client.1.smithi193.stdout:3/273: write d1/db/d59/f37 [4041535,98187] 0 2024-05-09T06:38:16.280 INFO:tasks.workunit.client.1.smithi193.stdout:6/248: mkdir d1/d25/d2c/d3c/d3e/d58 0 2024-05-09T06:38:16.280 INFO:tasks.workunit.client.1.smithi193.stdout:6/249: chown d1/de/d11/l32 6531281 1 2024-05-09T06:38:16.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/252: symlink d6/d1c/d1b/d3a/l4c 0 2024-05-09T06:38:16.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/583: mkdir d8/d35/db6/dc2 0 2024-05-09T06:38:16.284 INFO:tasks.workunit.client.1.smithi193.stdout:9/293: symlink dd/d1e/d1f/d48/l6c 0 2024-05-09T06:38:16.284 INFO:tasks.workunit.client.0.smithi063.stdout:8/661: dwrite d9/da/d45/fa4 [0,4194304] 0 2024-05-09T06:38:16.285 INFO:tasks.workunit.client.0.smithi063.stdout:2/642: fdatasync dc/dd/f64 0 2024-05-09T06:38:16.291 INFO:tasks.workunit.client.1.smithi193.stdout:9/294: dwrite dd/d1e/d66/d49/f51 [0,4194304] 0 2024-05-09T06:38:16.292 INFO:tasks.workunit.client.1.smithi193.stdout:9/295: readlink dd/df/d31/l3d 0 2024-05-09T06:38:16.295 INFO:tasks.workunit.client.1.smithi193.stdout:3/274: dread d1/db/f34 [0,4194304] 0 2024-05-09T06:38:16.298 INFO:tasks.workunit.client.1.smithi193.stdout:8/253: creat d6/d1c/d27/f4d x:0 0 0 2024-05-09T06:38:16.302 INFO:tasks.workunit.client.1.smithi193.stdout:6/250: symlink d1/d25/d2c/d3c/d3e/d58/l59 0 2024-05-09T06:38:16.303 INFO:tasks.workunit.client.1.smithi193.stdout:9/296: dwrite dd/df/d15/d16/d26/f35 [0,4194304] 0 2024-05-09T06:38:16.303 INFO:tasks.workunit.client.0.smithi063.stdout:5/584: creat d8/d17/d18/fc3 x:0 0 0 2024-05-09T06:38:16.306 INFO:tasks.workunit.client.0.smithi063.stdout:8/662: getdents d9/da/d18 0 2024-05-09T06:38:16.307 INFO:tasks.workunit.client.1.smithi193.stdout:3/275: truncate d1/db/d1b/f3b 2541677 0 2024-05-09T06:38:16.307 INFO:tasks.workunit.client.0.smithi063.stdout:8/663: truncate d9/d41/dec/dc5/fd1 498605 0 2024-05-09T06:38:16.308 INFO:tasks.workunit.client.0.smithi063.stdout:5/585: symlink d8/d65/dbd/lc4 0 2024-05-09T06:38:16.309 INFO:tasks.workunit.client.1.smithi193.stdout:9/297: chown l8 350678446 1 2024-05-09T06:38:16.311 INFO:tasks.workunit.client.1.smithi193.stdout:3/276: chown d1/db/l56 15905 1 2024-05-09T06:38:16.311 INFO:tasks.workunit.client.0.smithi063.stdout:5/586: creat d8/d7e/d96/fc5 x:0 0 0 2024-05-09T06:38:16.312 INFO:tasks.workunit.client.1.smithi193.stdout:8/254: dwrite d6/d1c/f14 [0,4194304] 0 2024-05-09T06:38:16.312 INFO:tasks.workunit.client.1.smithi193.stdout:3/277: chown d1/l2b 132471 1 2024-05-09T06:38:16.315 INFO:tasks.workunit.client.1.smithi193.stdout:8/255: write d6/d1c/f1e [1045828,109906] 0 2024-05-09T06:38:16.316 INFO:tasks.workunit.client.0.smithi063.stdout:8/664: dwrite d9/da/d34/f59 [4194304,4194304] 0 2024-05-09T06:38:16.317 INFO:tasks.workunit.client.1.smithi193.stdout:3/278: chown d1/d2d/d51/f54 0 1 2024-05-09T06:38:16.317 INFO:tasks.workunit.client.1.smithi193.stdout:6/251: dwrite d1/de/d11/d2d/f52 [0,4194304] 0 2024-05-09T06:38:16.318 INFO:tasks.workunit.client.1.smithi193.stdout:8/256: readlink d6/d1c/d27/l2d 0 2024-05-09T06:38:16.319 INFO:tasks.workunit.client.0.smithi063.stdout:9/620: sync 2024-05-09T06:38:16.319 INFO:tasks.workunit.client.1.smithi193.stdout:6/252: chown d1/d25/d2c/d3c/d3e/d58 6373941 1 2024-05-09T06:38:16.320 INFO:tasks.workunit.client.0.smithi063.stdout:8/665: unlink d9/da/d45/l6f 0 2024-05-09T06:38:16.321 INFO:tasks.workunit.client.0.smithi063.stdout:9/621: write d0/d4/d57/d60/d76/f8f [994157,36543] 0 2024-05-09T06:38:16.322 INFO:tasks.workunit.client.1.smithi193.stdout:3/279: creat d1/d2d/d51/f6a x:0 0 0 2024-05-09T06:38:16.322 INFO:tasks.workunit.client.1.smithi193.stdout:6/253: truncate d1/fd 278717 0 2024-05-09T06:38:16.323 INFO:tasks.workunit.client.1.smithi193.stdout:3/280: chown d1/db/f3f 118778 1 2024-05-09T06:38:16.323 INFO:tasks.workunit.client.1.smithi193.stdout:6/254: read d1/de/f53 [2755947,41148] 0 2024-05-09T06:38:16.331 INFO:tasks.workunit.client.1.smithi193.stdout:3/281: symlink d1/db/d1b/l6b 0 2024-05-09T06:38:16.331 INFO:tasks.workunit.client.1.smithi193.stdout:8/257: rmdir d6/d1c/d1b/d3a 39 2024-05-09T06:38:16.334 INFO:tasks.workunit.client.1.smithi193.stdout:6/255: creat d1/d25/d2c/d3c/f5a x:0 0 0 2024-05-09T06:38:16.334 INFO:tasks.workunit.client.1.smithi193.stdout:8/258: dread - d6/d1c/d1b/d29/f41 zero size 2024-05-09T06:38:16.336 INFO:tasks.workunit.client.1.smithi193.stdout:3/282: symlink d1/db/d2f/l6c 0 2024-05-09T06:38:16.337 INFO:tasks.workunit.client.0.smithi063.stdout:8/666: link d9/da/d45/de4/le6 d9/da/d45/de4/lf2 0 2024-05-09T06:38:16.338 INFO:tasks.workunit.client.0.smithi063.stdout:8/667: chown d9/da/d45/cf0 0 1 2024-05-09T06:38:16.341 INFO:tasks.workunit.client.0.smithi063.stdout:8/668: read d9/da/d34/f87 [331845,93957] 0 2024-05-09T06:38:16.342 INFO:tasks.workunit.client.1.smithi193.stdout:8/259: dwrite d6/d1c/d18/f26 [0,4194304] 0 2024-05-09T06:38:16.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/283: creat d1/f6d x:0 0 0 2024-05-09T06:38:16.344 INFO:tasks.workunit.client.0.smithi063.stdout:2/643: truncate dc/f1b 6577672 0 2024-05-09T06:38:16.345 INFO:tasks.workunit.client.1.smithi193.stdout:7/294: dwrite d6/d8/d1a/f3c [0,4194304] 0 2024-05-09T06:38:16.350 INFO:tasks.workunit.client.0.smithi063.stdout:8/669: creat d9/da/d45/de4/ff3 x:0 0 0 2024-05-09T06:38:16.353 INFO:tasks.workunit.client.0.smithi063.stdout:2/644: unlink dc/dd/d45/d5a/l81 0 2024-05-09T06:38:16.355 INFO:tasks.workunit.client.1.smithi193.stdout:8/260: symlink d6/d1c/d18/d2f/d40/l4e 0 2024-05-09T06:38:16.356 INFO:tasks.workunit.client.0.smithi063.stdout:8/670: fdatasync d9/d3b/f68 0 2024-05-09T06:38:16.356 INFO:tasks.workunit.client.1.smithi193.stdout:8/261: write d6/d1c/d1b/f30 [951783,28122] 0 2024-05-09T06:38:16.357 INFO:tasks.workunit.client.1.smithi193.stdout:8/262: readlink d6/d1c/d18/l35 0 2024-05-09T06:38:16.358 INFO:tasks.workunit.client.0.smithi063.stdout:5/587: write d8/d19/f48 [387366,12836] 0 2024-05-09T06:38:16.359 INFO:tasks.workunit.client.0.smithi063.stdout:2/645: mkdir dc/dd/d45/d5a/d6c/d8f/dbd 0 2024-05-09T06:38:16.361 INFO:tasks.workunit.client.0.smithi063.stdout:9/622: write d0/d4/da/f1c [5530846,4082] 0 2024-05-09T06:38:16.361 INFO:tasks.workunit.client.1.smithi193.stdout:7/295: mkdir d6/d8/d1a/d36/d4d/d60 0 2024-05-09T06:38:16.363 INFO:tasks.workunit.client.0.smithi063.stdout:8/671: creat d9/da/d18/d46/ff4 x:0 0 0 2024-05-09T06:38:16.364 INFO:tasks.workunit.client.0.smithi063.stdout:8/672: chown d9/da/d18/l28 7915556 1 2024-05-09T06:38:16.364 INFO:tasks.workunit.client.1.smithi193.stdout:3/284: creat d1/db/f6e x:0 0 0 2024-05-09T06:38:16.365 INFO:tasks.workunit.client.0.smithi063.stdout:5/588: chown d8/d19/d30/f43 469 1 2024-05-09T06:38:16.367 INFO:tasks.workunit.client.0.smithi063.stdout:2/646: mknod dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72/d90/cbe 0 2024-05-09T06:38:16.369 INFO:tasks.workunit.client.0.smithi063.stdout:9/623: symlink d0/d4/d57/ld9 0 2024-05-09T06:38:16.369 INFO:tasks.workunit.client.1.smithi193.stdout:6/256: write d1/de/d11/d20/f27 [126552,72863] 0 2024-05-09T06:38:16.370 INFO:tasks.workunit.client.0.smithi063.stdout:9/624: write d0/d4/d7d/f99 [38767,42031] 0 2024-05-09T06:38:16.371 INFO:tasks.workunit.client.0.smithi063.stdout:8/673: creat d9/d41/dec/db8/ff5 x:0 0 0 2024-05-09T06:38:16.373 INFO:tasks.workunit.client.0.smithi063.stdout:5/589: symlink d8/d35/db6/lc6 0 2024-05-09T06:38:16.375 INFO:tasks.workunit.client.0.smithi063.stdout:5/590: write d8/f68 [802097,94470] 0 2024-05-09T06:38:16.376 INFO:tasks.workunit.client.0.smithi063.stdout:2/647: mkdir dc/dd/d45/d5a/d6c/d8f/dbf 0 2024-05-09T06:38:16.378 INFO:tasks.workunit.client.1.smithi193.stdout:7/296: dread d6/f30 [0,4194304] 0 2024-05-09T06:38:16.378 INFO:tasks.workunit.client.1.smithi193.stdout:3/285: link d1/db/d59/c4d d1/db/d59/c6f 0 2024-05-09T06:38:16.379 INFO:tasks.workunit.client.0.smithi063.stdout:9/625: dread d0/d4/da/f21 [0,4194304] 0 2024-05-09T06:38:16.379 INFO:tasks.workunit.client.1.smithi193.stdout:3/286: dread - d1/db/f6e zero size 2024-05-09T06:38:16.381 INFO:tasks.workunit.client.0.smithi063.stdout:5/591: dread d8/f68 [0,4194304] 0 2024-05-09T06:38:16.385 INFO:tasks.workunit.client.0.smithi063.stdout:2/648: fsync dc/dd/f2f 0 2024-05-09T06:38:16.386 INFO:tasks.workunit.client.1.smithi193.stdout:3/287: dwrite d1/d2d/d51/f54 [4194304,4194304] 0 2024-05-09T06:38:16.386 INFO:tasks.workunit.client.0.smithi063.stdout:2/649: fdatasync dc/d32/d33/f41 0 2024-05-09T06:38:16.387 INFO:tasks.workunit.client.0.smithi063.stdout:5/592: dwrite d8/d7e/d96/fc5 [0,4194304] 0 2024-05-09T06:38:16.388 INFO:tasks.workunit.client.0.smithi063.stdout:2/650: fdatasync dc/dd/d45/d5a/d6a/fa4 0 2024-05-09T06:38:16.390 INFO:tasks.workunit.client.0.smithi063.stdout:9/626: symlink d0/d4/d26/d39/d48/lda 0 2024-05-09T06:38:16.390 INFO:tasks.workunit.client.0.smithi063.stdout:2/651: dread f3 [0,4194304] 0 2024-05-09T06:38:16.391 INFO:tasks.workunit.client.1.smithi193.stdout:3/288: creat d1/db/d1b/f70 x:0 0 0 2024-05-09T06:38:16.392 INFO:tasks.workunit.client.1.smithi193.stdout:3/289: chown d1/d2d/f45 3713 1 2024-05-09T06:38:16.394 INFO:tasks.workunit.client.0.smithi063.stdout:0/645: sync 2024-05-09T06:38:16.402 INFO:tasks.workunit.client.0.smithi063.stdout:5/593: unlink d8/d19/d30/l9b 0 2024-05-09T06:38:16.403 INFO:tasks.workunit.client.0.smithi063.stdout:9/627: rmdir d0/d4/d26 39 2024-05-09T06:38:16.404 INFO:tasks.workunit.client.1.smithi193.stdout:6/257: truncate d1/de/d11/f22 68031 0 2024-05-09T06:38:16.405 INFO:tasks.workunit.client.1.smithi193.stdout:3/290: getdents d1/db/d2f 0 2024-05-09T06:38:16.405 INFO:tasks.workunit.client.1.smithi193.stdout:6/258: write d1/f4d [749107,110248] 0 2024-05-09T06:38:16.408 INFO:tasks.workunit.client.0.smithi063.stdout:8/674: dwrite d9/d3b/f5c [0,4194304] 0 2024-05-09T06:38:16.409 INFO:tasks.workunit.client.0.smithi063.stdout:8/675: stat d9/da/d53/fa2 0 2024-05-09T06:38:16.412 INFO:tasks.workunit.client.1.smithi193.stdout:3/291: dwrite d1/d2d/d51/f54 [4194304,4194304] 0 2024-05-09T06:38:16.413 INFO:tasks.workunit.client.1.smithi193.stdout:3/292: fsync d1/d44/f5f 0 2024-05-09T06:38:16.414 INFO:tasks.workunit.client.1.smithi193.stdout:6/259: unlink d1/de/d11/d18/f1b 0 2024-05-09T06:38:16.418 INFO:tasks.workunit.client.0.smithi063.stdout:9/628: rename d0/d4/d57/d60/f70 to d0/d4/d57/dba/dd5/fdb 0 2024-05-09T06:38:16.419 INFO:tasks.workunit.client.1.smithi193.stdout:3/293: mknod d1/db/d59/c71 0 2024-05-09T06:38:16.420 INFO:tasks.workunit.client.0.smithi063.stdout:2/652: mknod dc/d32/d33/d4e/cc0 0 2024-05-09T06:38:16.422 INFO:tasks.workunit.client.1.smithi193.stdout:6/260: rename d1/de/d11/d18/d36/d37/f4e to d1/de/d11/d18/d36/d37/f5b 0 2024-05-09T06:38:16.425 INFO:tasks.workunit.client.0.smithi063.stdout:8/676: creat d9/da/d45/de4/ff6 x:0 0 0 2024-05-09T06:38:16.427 INFO:tasks.workunit.client.0.smithi063.stdout:9/629: dwrite d0/d3b/da0/f9c [0,4194304] 0 2024-05-09T06:38:16.432 INFO:tasks.workunit.client.1.smithi193.stdout:6/261: link d1/de/d11/d18/f1c d1/de/d11/f5c 0 2024-05-09T06:38:16.436 INFO:tasks.workunit.client.0.smithi063.stdout:2/653: rmdir dc/dd 39 2024-05-09T06:38:16.442 INFO:tasks.workunit.client.1.smithi193.stdout:6/262: creat d1/de/d11/f5d x:0 0 0 2024-05-09T06:38:16.443 INFO:tasks.workunit.client.0.smithi063.stdout:8/677: creat d9/d41/dec/da1/ff7 x:0 0 0 2024-05-09T06:38:16.444 INFO:tasks.workunit.client.1.smithi193.stdout:3/294: getdents d1/d2d 0 2024-05-09T06:38:16.445 INFO:tasks.workunit.client.0.smithi063.stdout:9/630: creat d0/d4/d7d/fdc x:0 0 0 2024-05-09T06:38:16.447 INFO:tasks.workunit.client.0.smithi063.stdout:2/654: mkdir dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72/d90/dc1 0 2024-05-09T06:38:16.448 INFO:tasks.workunit.client.0.smithi063.stdout:2/655: stat dc/d6e/d7f 0 2024-05-09T06:38:16.449 INFO:tasks.workunit.client.0.smithi063.stdout:5/594: truncate d8/d65/f83 4147172 0 2024-05-09T06:38:16.450 INFO:tasks.workunit.client.1.smithi193.stdout:3/295: dwrite d1/db/f6e [0,4194304] 0 2024-05-09T06:38:16.451 INFO:tasks.workunit.client.1.smithi193.stdout:3/296: fsync d1/db/d59/f43 0 2024-05-09T06:38:16.454 INFO:tasks.workunit.client.0.smithi063.stdout:8/678: unlink d9/da/d18/dc7/d58/fb4 0 2024-05-09T06:38:16.457 INFO:tasks.workunit.client.0.smithi063.stdout:0/646: write dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/f57 [47081,85502] 0 2024-05-09T06:38:16.457 INFO:tasks.workunit.client.1.smithi193.stdout:3/297: symlink d1/db/d59/l72 0 2024-05-09T06:38:16.459 INFO:tasks.workunit.client.0.smithi063.stdout:2/656: rmdir dc/d36/d43/d46/d63 39 2024-05-09T06:38:16.462 INFO:tasks.workunit.client.1.smithi193.stdout:3/298: dread d1/db/d59/f37 [0,4194304] 0 2024-05-09T06:38:16.464 INFO:tasks.workunit.client.1.smithi193.stdout:3/299: truncate d1/f1d 12901 0 2024-05-09T06:38:16.468 INFO:tasks.workunit.client.0.smithi063.stdout:0/647: creat dd/d60/d44/d90/da4/fcc x:0 0 0 2024-05-09T06:38:16.470 INFO:tasks.workunit.client.0.smithi063.stdout:0/648: dread dd/d1a/d2c/d51/d72/d9e/d23/d46/f82 [0,4194304] 0 2024-05-09T06:38:16.471 INFO:tasks.workunit.client.1.smithi193.stdout:3/300: dwrite d1/d2d/d51/f6a [0,4194304] 0 2024-05-09T06:38:16.472 INFO:tasks.workunit.client.1.smithi193.stdout:6/263: dread d1/de/d11/d18/f1e [0,4194304] 0 2024-05-09T06:38:16.473 INFO:tasks.workunit.client.1.smithi193.stdout:3/301: truncate d1/f11 727777 0 2024-05-09T06:38:16.474 INFO:tasks.workunit.client.0.smithi063.stdout:5/595: mknod d8/d35/db6/dc2/cc7 0 2024-05-09T06:38:16.475 INFO:tasks.workunit.client.0.smithi063.stdout:5/596: chown d8/c15 220572200 1 2024-05-09T06:38:16.477 INFO:tasks.workunit.client.1.smithi193.stdout:6/264: mknod d1/de/d11/d2d/d3a/d47/c5e 0 2024-05-09T06:38:16.478 INFO:tasks.workunit.client.1.smithi193.stdout:3/302: dread d1/d2d/d51/f54 [4194304,4194304] 0 2024-05-09T06:38:16.479 INFO:tasks.workunit.client.1.smithi193.stdout:3/303: stat d1/d44/f60 0 2024-05-09T06:38:16.480 INFO:tasks.workunit.client.1.smithi193.stdout:3/304: dread - d1/d2d/d51/f65 zero size 2024-05-09T06:38:16.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/597: dwrite d8/d53/fab [0,4194304] 0 2024-05-09T06:38:16.483 INFO:tasks.workunit.client.0.smithi063.stdout:9/631: rename d0/d4/d26/d39/d48/d80/d84/dc2/lc5 to d0/d3b/d63/ldd 0 2024-05-09T06:38:16.485 INFO:tasks.workunit.client.0.smithi063.stdout:8/679: write d9/da/f4e [1548545,63487] 0 2024-05-09T06:38:16.487 INFO:tasks.workunit.client.0.smithi063.stdout:2/657: rmdir dc/d36/d43/d46/d63/d79/d7d 39 2024-05-09T06:38:16.490 INFO:tasks.workunit.client.0.smithi063.stdout:6/610: sync 2024-05-09T06:38:16.490 INFO:tasks.workunit.client.0.smithi063.stdout:3/616: sync 2024-05-09T06:38:16.490 INFO:tasks.workunit.client.0.smithi063.stdout:1/543: sync 2024-05-09T06:38:16.490 INFO:tasks.workunit.client.0.smithi063.stdout:4/587: sync 2024-05-09T06:38:16.490 INFO:tasks.workunit.client.0.smithi063.stdout:7/585: sync 2024-05-09T06:38:16.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/617: chown d7/d22/d28 244 1 2024-05-09T06:38:16.493 INFO:tasks.workunit.client.0.smithi063.stdout:0/649: getdents dd/d1a/d2c/d51/d72/d9e/d23/dbb 0 2024-05-09T06:38:16.494 INFO:tasks.workunit.client.0.smithi063.stdout:9/632: creat d0/d3b/da0/fde x:0 0 0 2024-05-09T06:38:16.494 INFO:tasks.workunit.client.0.smithi063.stdout:3/618: write d7/da/d21/f6b [1988018,4089] 0 2024-05-09T06:38:16.495 INFO:tasks.workunit.client.0.smithi063.stdout:8/680: mkdir d9/d3b/df8 0 2024-05-09T06:38:16.495 INFO:tasks.workunit.client.0.smithi063.stdout:9/633: write d0/d4/d7d/f87 [1100067,131047] 0 2024-05-09T06:38:16.502 INFO:tasks.workunit.client.0.smithi063.stdout:1/544: creat d9/d2a/d3c/d4c/fbb x:0 0 0 2024-05-09T06:38:16.502 INFO:tasks.workunit.client.0.smithi063.stdout:2/658: dread dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72/d7b/fae [0,4194304] 0 2024-05-09T06:38:16.502 INFO:tasks.workunit.client.0.smithi063.stdout:6/611: symlink d2/d1f/d29/lc5 0 2024-05-09T06:38:16.503 INFO:tasks.workunit.client.1.smithi193.stdout:3/305: write d1/db/d1b/f3b [2016757,78776] 0 2024-05-09T06:38:16.503 INFO:tasks.workunit.client.0.smithi063.stdout:6/612: read d2/d1f/d29/f84 [2881277,58319] 0 2024-05-09T06:38:16.504 INFO:tasks.workunit.client.1.smithi193.stdout:3/306: chown d1/d2d/d51/f6a 0 1 2024-05-09T06:38:16.504 INFO:tasks.workunit.client.1.smithi193.stdout:3/307: read - d1/db/d1b/f70 zero size 2024-05-09T06:38:16.505 INFO:tasks.workunit.client.0.smithi063.stdout:3/619: write d7/da/d21/f4d [1495228,44679] 0 2024-05-09T06:38:16.507 INFO:tasks.workunit.client.0.smithi063.stdout:7/586: write d0/d15/d49/f5b [87805,2919] 0 2024-05-09T06:38:16.509 INFO:tasks.workunit.client.1.smithi193.stdout:3/308: dread d1/d44/f47 [0,4194304] 0 2024-05-09T06:38:16.509 INFO:tasks.workunit.client.1.smithi193.stdout:6/265: dread d1/de/d11/d18/f24 [0,4194304] 0 2024-05-09T06:38:16.513 INFO:tasks.workunit.client.0.smithi063.stdout:0/650: write f2 [2353824,49862] 0 2024-05-09T06:38:16.521 INFO:tasks.workunit.client.0.smithi063.stdout:9/634: mkdir d0/d4/d26/d39/d48/d80/d84/dd8/ddf 0 2024-05-09T06:38:16.522 INFO:tasks.workunit.client.0.smithi063.stdout:5/598: getdents d8/d17 0 2024-05-09T06:38:16.522 INFO:tasks.workunit.client.1.smithi193.stdout:6/266: dwrite d1/f23 [12582912,4194304] 0 2024-05-09T06:38:16.523 INFO:tasks.workunit.client.0.smithi063.stdout:2/659: truncate dc/f1b 6244683 0 2024-05-09T06:38:16.526 INFO:tasks.workunit.client.0.smithi063.stdout:9/635: fsync d0/d4/d57/d60/d76/f8f 0 2024-05-09T06:38:16.526 INFO:tasks.workunit.client.0.smithi063.stdout:6/613: truncate d2/d12/f53 926003 0 2024-05-09T06:38:16.527 INFO:tasks.workunit.client.0.smithi063.stdout:4/588: dwrite d7/df/d35/f36 [0,4194304] 0 2024-05-09T06:38:16.527 INFO:tasks.workunit.client.0.smithi063.stdout:7/587: symlink d0/d15/d49/d4f/d66/d67/lcd 0 2024-05-09T06:38:16.527 INFO:tasks.workunit.client.1.smithi193.stdout:3/309: dread d1/db/f18 [0,4194304] 0 2024-05-09T06:38:16.530 INFO:tasks.workunit.client.1.smithi193.stdout:6/267: dwrite d1/de/ff [4194304,4194304] 0 2024-05-09T06:38:16.533 INFO:tasks.workunit.client.0.smithi063.stdout:3/620: dread d7/da/d2b/d66/d74/d7e/fa6 [0,4194304] 0 2024-05-09T06:38:16.534 INFO:tasks.workunit.client.1.smithi193.stdout:6/268: creat d1/de/d11/d2d/d3a/d47/f5f x:0 0 0 2024-05-09T06:38:16.535 INFO:tasks.workunit.client.1.smithi193.stdout:3/310: dwrite d1/db/f6e [0,4194304] 0 2024-05-09T06:38:16.536 INFO:tasks.workunit.client.0.smithi063.stdout:7/588: chown d0/d15/d4a/d50/l7c 0 1 2024-05-09T06:38:16.537 INFO:tasks.workunit.client.0.smithi063.stdout:4/589: mkdir d7/da3/d28/d31/d71/dbb 0 2024-05-09T06:38:16.540 INFO:tasks.workunit.client.0.smithi063.stdout:2/660: symlink dc/d36/d43/lc2 0 2024-05-09T06:38:16.541 INFO:tasks.workunit.client.0.smithi063.stdout:3/621: write d7/f11 [3424854,84820] 0 2024-05-09T06:38:16.541 INFO:tasks.workunit.client.0.smithi063.stdout:6/614: unlink d2/d1f/d73/d88/d98/lab 0 2024-05-09T06:38:16.542 INFO:tasks.workunit.client.1.smithi193.stdout:6/269: dwrite d1/de/d11/d2d/f2e [0,4194304] 0 2024-05-09T06:38:16.545 INFO:tasks.workunit.client.0.smithi063.stdout:4/590: readlink d7/da3/d28/d31/d32/d34/l4d 0 2024-05-09T06:38:16.545 INFO:tasks.workunit.client.1.smithi193.stdout:6/270: read - d1/de/d11/d2d/d3a/f3b zero size 2024-05-09T06:38:16.545 INFO:tasks.workunit.client.0.smithi063.stdout:6/615: readlink d2/d1f/l31 0 2024-05-09T06:38:16.549 INFO:tasks.workunit.client.0.smithi063.stdout:6/616: symlink d2/d1f/d29/lc6 0 2024-05-09T06:38:16.549 INFO:tasks.workunit.client.0.smithi063.stdout:1/545: dread d9/f23 [4194304,4194304] 0 2024-05-09T06:38:16.551 INFO:tasks.workunit.client.1.smithi193.stdout:6/271: mknod d1/de/c60 0 2024-05-09T06:38:16.551 INFO:tasks.workunit.client.1.smithi193.stdout:6/272: chown d1/d48 125619962 1 2024-05-09T06:38:16.560 INFO:tasks.workunit.client.0.smithi063.stdout:5/599: dwrite d8/d17/f3d [4194304,4194304] 0 2024-05-09T06:38:16.561 INFO:tasks.workunit.client.0.smithi063.stdout:6/617: mknod d2/df/d59/cc7 0 2024-05-09T06:38:16.561 INFO:tasks.workunit.client.1.smithi193.stdout:6/273: creat d1/de/d11/d18/f61 x:0 0 0 2024-05-09T06:38:16.562 INFO:tasks.workunit.client.0.smithi063.stdout:0/651: write dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/f9b [642204,57812] 0 2024-05-09T06:38:16.563 INFO:tasks.workunit.client.0.smithi063.stdout:6/618: chown d2/df/f50 105153518 1 2024-05-09T06:38:16.565 INFO:tasks.workunit.client.0.smithi063.stdout:5/600: chown d8/d19/d30/d74/d77/c91 95170354 1 2024-05-09T06:38:16.566 INFO:tasks.workunit.client.1.smithi193.stdout:6/274: creat d1/d25/d2c/d3c/d3e/d58/f62 x:0 0 0 2024-05-09T06:38:16.566 INFO:tasks.workunit.client.0.smithi063.stdout:0/652: write dd/d1a/f2e [1983386,6150] 0 2024-05-09T06:38:16.571 INFO:tasks.workunit.client.0.smithi063.stdout:7/589: write d0/d15/f20 [55592,125196] 0 2024-05-09T06:38:16.572 INFO:tasks.workunit.client.1.smithi193.stdout:6/275: symlink d1/d25/d4c/l63 0 2024-05-09T06:38:16.575 INFO:tasks.workunit.client.1.smithi193.stdout:6/276: chown d1/de/d11/d18/d36/d37/l45 9736 1 2024-05-09T06:38:16.578 INFO:tasks.workunit.client.0.smithi063.stdout:9/636: dwrite d0/d4/d26/d39/f55 [0,4194304] 0 2024-05-09T06:38:16.580 INFO:tasks.workunit.client.0.smithi063.stdout:9/637: chown d0/d4/d26/d47/f7a 1 1 2024-05-09T06:38:16.589 INFO:tasks.workunit.client.1.smithi193.stdout:6/277: dread d1/d25/f30 [0,4194304] 0 2024-05-09T06:38:16.590 INFO:tasks.workunit.client.1.smithi193.stdout:6/278: write d1/d25/d2c/f50 [302748,44643] 0 2024-05-09T06:38:16.600 INFO:tasks.workunit.client.1.smithi193.stdout:0/308: sync 2024-05-09T06:38:16.600 INFO:tasks.workunit.client.1.smithi193.stdout:2/278: sync 2024-05-09T06:38:16.604 INFO:tasks.workunit.client.1.smithi193.stdout:6/279: mknod d1/de/d11/d18/d36/c64 0 2024-05-09T06:38:16.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:16 smithi063 ceph-mon[27171]: pgmap v11: 65 pgs: 65 active+clean; 1.3 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 166 MiB/s wr, 334 op/s 2024-05-09T06:38:16.606 INFO:tasks.workunit.client.1.smithi193.stdout:6/280: chown d1/de/d11/d2d/f34 0 1 2024-05-09T06:38:16.606 INFO:tasks.workunit.client.1.smithi193.stdout:0/309: dwrite d18/d4c/f45 [4194304,4194304] 0 2024-05-09T06:38:16.607 INFO:tasks.workunit.client.1.smithi193.stdout:0/310: chown d18/d4c/l5d 105389 1 2024-05-09T06:38:16.609 INFO:tasks.workunit.client.1.smithi193.stdout:2/279: mkdir db/df/d54/d67 0 2024-05-09T06:38:16.611 INFO:tasks.workunit.client.1.smithi193.stdout:0/311: mkdir d18/d4c/d5f 0 2024-05-09T06:38:16.613 INFO:tasks.workunit.client.1.smithi193.stdout:2/280: symlink db/df/d54/l68 0 2024-05-09T06:38:16.634 INFO:tasks.workunit.client.0.smithi063.stdout:4/591: creat d7/da3/d8b/fbc x:0 0 0 2024-05-09T06:38:16.635 INFO:tasks.workunit.client.0.smithi063.stdout:1/546: mkdir d9/d2a/d3c/dbc 0 2024-05-09T06:38:16.635 INFO:tasks.workunit.client.0.smithi063.stdout:4/592: write d7/da3/d28/d31/d71/d98/fa0 [89199,77875] 0 2024-05-09T06:38:16.638 INFO:tasks.workunit.client.0.smithi063.stdout:3/622: write d7/da/d2b/d66/d74/d7e/fca [953218,88496] 0 2024-05-09T06:38:16.646 INFO:tasks.workunit.client.0.smithi063.stdout:2/661: creat dc/d32/d33/fc3 x:0 0 0 2024-05-09T06:38:16.648 INFO:tasks.workunit.client.0.smithi063.stdout:5/601: mkdir d8/dc8 0 2024-05-09T06:38:16.649 INFO:tasks.workunit.client.0.smithi063.stdout:5/602: fdatasync d8/f66 0 2024-05-09T06:38:16.650 INFO:tasks.workunit.client.0.smithi063.stdout:5/603: truncate d8/d17/da3/fbf 376185 0 2024-05-09T06:38:16.651 INFO:tasks.workunit.client.0.smithi063.stdout:1/547: mknod d9/d46/d57/cbd 0 2024-05-09T06:38:16.652 INFO:tasks.workunit.client.0.smithi063.stdout:4/593: creat d7/da3/d28/d31/d32/d34/db4/fbd x:0 0 0 2024-05-09T06:38:16.653 INFO:tasks.workunit.client.0.smithi063.stdout:2/662: truncate f6 1023705 0 2024-05-09T06:38:16.654 INFO:tasks.workunit.client.0.smithi063.stdout:4/594: truncate d7/da3/d28/d31/d32/f8c 4575155 0 2024-05-09T06:38:16.654 INFO:tasks.workunit.client.0.smithi063.stdout:2/663: chown dc/d36/d43/d46/d63/d79/d7d/d8d 7891 1 2024-05-09T06:38:16.656 INFO:tasks.workunit.client.0.smithi063.stdout:9/638: creat d0/d4/d57/d60/fe0 x:0 0 0 2024-05-09T06:38:16.657 INFO:tasks.workunit.client.0.smithi063.stdout:5/604: mkdir d8/d17/d75/dc9 0 2024-05-09T06:38:16.661 INFO:tasks.workunit.client.0.smithi063.stdout:3/623: creat d7/da/fce x:0 0 0 2024-05-09T06:38:16.661 INFO:tasks.workunit.client.0.smithi063.stdout:4/595: read - d7/df/d35/d5a/f6f zero size 2024-05-09T06:38:16.662 INFO:tasks.workunit.client.0.smithi063.stdout:3/624: truncate d7/d22/d28/fc2 636296 0 2024-05-09T06:38:16.663 INFO:tasks.workunit.client.0.smithi063.stdout:3/625: chown d7/da/d21/d30/c5e 16093 1 2024-05-09T06:38:16.663 INFO:tasks.workunit.client.0.smithi063.stdout:3/626: chown d7/da/d21/f2c 7007016 1 2024-05-09T06:38:16.664 INFO:tasks.workunit.client.0.smithi063.stdout:3/627: write d7/d70/d98/d5b/f9b [819460,124453] 0 2024-05-09T06:38:16.665 INFO:tasks.workunit.client.0.smithi063.stdout:5/605: rename d8/d17/d1e to d8/d53/dca 0 2024-05-09T06:38:16.667 INFO:tasks.workunit.client.0.smithi063.stdout:2/664: chown dc/dd/f20 22574 1 2024-05-09T06:38:16.667 INFO:tasks.workunit.client.0.smithi063.stdout:2/665: chown dc 1073593943 1 2024-05-09T06:38:16.669 INFO:tasks.workunit.client.0.smithi063.stdout:9/639: link d0/d4/d57/d60/d76/f8f d0/d4/d57/d60/d76/fe1 0 2024-05-09T06:38:16.670 INFO:tasks.workunit.client.0.smithi063.stdout:4/596: creat d7/da3/d28/d31/d7b/fbe x:0 0 0 2024-05-09T06:38:16.672 INFO:tasks.workunit.client.0.smithi063.stdout:3/628: mkdir d7/d4c/dcf 0 2024-05-09T06:38:16.673 INFO:tasks.workunit.client.0.smithi063.stdout:1/548: creat d9/fbe x:0 0 0 2024-05-09T06:38:16.678 INFO:tasks.workunit.client.0.smithi063.stdout:5/606: symlink d8/d19/d84/lcb 0 2024-05-09T06:38:16.678 INFO:tasks.workunit.client.0.smithi063.stdout:4/597: read d7/df/d57/f93 [1851555,105292] 0 2024-05-09T06:38:16.680 INFO:tasks.workunit.client.0.smithi063.stdout:2/666: mknod dc/dd/d2d/cc4 0 2024-05-09T06:38:16.682 INFO:tasks.workunit.client.0.smithi063.stdout:1/549: read d9/d46/f54 [249674,61902] 0 2024-05-09T06:38:16.683 INFO:tasks.workunit.client.0.smithi063.stdout:6/619: dwrite d2/d12/f24 [0,4194304] 0 2024-05-09T06:38:16.683 INFO:tasks.workunit.client.0.smithi063.stdout:5/607: chown d8/d53/dca/l33 103004570 1 2024-05-09T06:38:16.683 INFO:tasks.workunit.client.0.smithi063.stdout:3/629: dwrite d7/d22/f27 [0,4194304] 0 2024-05-09T06:38:16.685 INFO:tasks.workunit.client.0.smithi063.stdout:9/640: getdents d0/d4 0 2024-05-09T06:38:16.685 INFO:tasks.workunit.client.0.smithi063.stdout:4/598: mkdir d7/da3/d28/d31/dbf 0 2024-05-09T06:38:16.685 INFO:tasks.workunit.client.0.smithi063.stdout:7/590: dwrite d0/fbe [4194304,4194304] 0 2024-05-09T06:38:16.688 INFO:tasks.workunit.client.0.smithi063.stdout:5/608: chown d8/d53/l60 32778494 1 2024-05-09T06:38:16.688 INFO:tasks.workunit.client.0.smithi063.stdout:9/641: chown d0/d4/l50 30184 1 2024-05-09T06:38:16.689 INFO:tasks.workunit.client.0.smithi063.stdout:2/667: read dc/f9d [106683,53435] 0 2024-05-09T06:38:16.690 INFO:tasks.workunit.client.0.smithi063.stdout:6/620: symlink d2/d12/lc8 0 2024-05-09T06:38:16.692 INFO:tasks.workunit.client.0.smithi063.stdout:6/621: stat d2/d4 0 2024-05-09T06:38:16.695 INFO:tasks.workunit.client.0.smithi063.stdout:4/599: write d7/df/d35/d5a/d60/f6e [438065,97168] 0 2024-05-09T06:38:16.700 INFO:tasks.workunit.client.0.smithi063.stdout:5/609: getdents d8/d17/d75/dc9 0 2024-05-09T06:38:16.700 INFO:tasks.workunit.client.0.smithi063.stdout:6/622: creat d2/d1f/d29/d3f/fc9 x:0 0 0 2024-05-09T06:38:16.701 INFO:tasks.workunit.client.0.smithi063.stdout:4/600: rmdir d7/d50 39 2024-05-09T06:38:16.704 INFO:tasks.workunit.client.0.smithi063.stdout:3/630: dwrite d7/da/d2b/d66/dc0/da4/fbc [0,4194304] 0 2024-05-09T06:38:16.704 INFO:tasks.workunit.client.0.smithi063.stdout:5/610: dread d8/d17/d18/f51 [0,4194304] 0 2024-05-09T06:38:16.705 INFO:tasks.workunit.client.0.smithi063.stdout:1/550: dwrite d9/d2a/d35/d1e/d9d/f82 [0,4194304] 0 2024-05-09T06:38:16.708 INFO:tasks.workunit.client.0.smithi063.stdout:8/681: sync 2024-05-09T06:38:16.710 INFO:tasks.workunit.client.0.smithi063.stdout:9/642: rename d0/d4/da/f1c to d0/d4/d26/d39/d48/d80/d84/dd8/ddf/fe2 0 2024-05-09T06:38:16.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/631: mknod d7/d70/d98/d5b/cd0 0 2024-05-09T06:38:16.713 INFO:tasks.workunit.client.0.smithi063.stdout:7/591: link d0/d15/d4a/c8e d0/d15/d49/d4f/d66/cce 0 2024-05-09T06:38:16.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/623: creat d2/d20/d26/fca x:0 0 0 2024-05-09T06:38:16.715 INFO:tasks.workunit.client.0.smithi063.stdout:9/643: fsync d0/f97 0 2024-05-09T06:38:16.716 INFO:tasks.workunit.client.0.smithi063.stdout:8/682: creat d9/da/d18/dc7/d58/ff9 x:0 0 0 2024-05-09T06:38:16.719 INFO:tasks.workunit.client.0.smithi063.stdout:6/624: write d2/d1f/fb9 [1775439,36668] 0 2024-05-09T06:38:16.720 INFO:tasks.workunit.client.0.smithi063.stdout:3/632: creat d7/d4c/d25/fd1 x:0 0 0 2024-05-09T06:38:16.721 INFO:tasks.workunit.client.0.smithi063.stdout:1/551: dwrite d9/d2a/d35/d24/fac [0,4194304] 0 2024-05-09T06:38:16.724 INFO:tasks.workunit.client.0.smithi063.stdout:6/625: write d2/d4/d7c/d7e/f87 [906917,106254] 0 2024-05-09T06:38:16.726 INFO:tasks.workunit.client.0.smithi063.stdout:1/552: truncate d9/d2a/d2e/f7d 1021780 0 2024-05-09T06:38:16.730 INFO:tasks.workunit.client.0.smithi063.stdout:4/601: dwrite d7/d50/f9d [0,4194304] 0 2024-05-09T06:38:16.731 INFO:tasks.workunit.client.0.smithi063.stdout:3/633: creat d7/da/d2b/d66/fd2 x:0 0 0 2024-05-09T06:38:16.733 INFO:tasks.workunit.client.0.smithi063.stdout:6/626: truncate d2/d1f/d29/d3f/d5c/d64/f83 580251 0 2024-05-09T06:38:16.737 INFO:tasks.workunit.client.0.smithi063.stdout:6/627: symlink d2/d4/d7c/d42/lcb 0 2024-05-09T06:38:16.739 INFO:tasks.workunit.client.0.smithi063.stdout:2/668: truncate dc/dd/d45/f53 2065101 0 2024-05-09T06:38:16.742 INFO:tasks.workunit.client.0.smithi063.stdout:4/602: read d7/f62 [2159500,121293] 0 2024-05-09T06:38:16.751 INFO:tasks.workunit.client.0.smithi063.stdout:6/628: chown d2/d1f/d29/d3b/d4e/cc1 3695 1 2024-05-09T06:38:16.752 INFO:tasks.workunit.client.0.smithi063.stdout:3/634: dwrite d7/da/d21/d30/d44/d48/f6f [0,4194304] 0 2024-05-09T06:38:16.753 INFO:tasks.workunit.client.0.smithi063.stdout:5/611: dwrite d8/d53/dca/f42 [0,4194304] 0 2024-05-09T06:38:16.755 INFO:tasks.workunit.client.0.smithi063.stdout:8/683: dread d9/da/d51/dc2/fcd [0,4194304] 0 2024-05-09T06:38:16.760 INFO:tasks.workunit.client.0.smithi063.stdout:9/644: dread d0/d4/d26/fae [0,4194304] 0 2024-05-09T06:38:16.760 INFO:tasks.workunit.client.0.smithi063.stdout:3/635: symlink d7/d70/ld3 0 2024-05-09T06:38:16.763 INFO:tasks.workunit.client.0.smithi063.stdout:7/592: dwrite d0/ff [0,4194304] 0 2024-05-09T06:38:16.764 INFO:tasks.workunit.client.0.smithi063.stdout:8/684: symlink d9/d41/lfa 0 2024-05-09T06:38:16.764 INFO:tasks.workunit.client.0.smithi063.stdout:2/669: dread dc/dd/d45/f8a [0,4194304] 0 2024-05-09T06:38:16.765 INFO:tasks.workunit.client.0.smithi063.stdout:1/553: dread d9/d10/f1c [4194304,4194304] 0 2024-05-09T06:38:16.769 INFO:tasks.workunit.client.0.smithi063.stdout:6/629: rename d2/d12/f53 to d2/d4/d7c/fcc 0 2024-05-09T06:38:16.770 INFO:tasks.workunit.client.0.smithi063.stdout:5/612: link d8/d7e/l85 d8/d35/lcc 0 2024-05-09T06:38:16.771 INFO:tasks.workunit.client.0.smithi063.stdout:8/685: read d9/da/d18/dc7/d58/f8e [196978,17529] 0 2024-05-09T06:38:16.773 INFO:tasks.workunit.client.0.smithi063.stdout:0/653: sync 2024-05-09T06:38:16.777 INFO:tasks.workunit.client.0.smithi063.stdout:4/603: dwrite f5 [4194304,4194304] 0 2024-05-09T06:38:16.777 INFO:tasks.workunit.client.0.smithi063.stdout:6/630: symlink d2/d4/lcd 0 2024-05-09T06:38:16.780 INFO:tasks.workunit.client.0.smithi063.stdout:6/631: write d2/d1f/d6c/f7d [126883,47864] 0 2024-05-09T06:38:16.781 INFO:tasks.workunit.client.0.smithi063.stdout:6/632: readlink d2/d20/d4a/l77 0 2024-05-09T06:38:16.783 INFO:tasks.workunit.client.0.smithi063.stdout:0/654: fsync dd/d1a/d50/f7a 0 2024-05-09T06:38:16.784 INFO:tasks.workunit.client.0.smithi063.stdout:8/686: fdatasync d9/f13 0 2024-05-09T06:38:16.787 INFO:tasks.workunit.client.0.smithi063.stdout:3/636: rename d7/da/f14 to d7/d22/d28/dab/fd4 0 2024-05-09T06:38:16.789 INFO:tasks.workunit.client.0.smithi063.stdout:6/633: dwrite d2/d4/f1c [0,4194304] 0 2024-05-09T06:38:16.792 INFO:tasks.workunit.client.0.smithi063.stdout:2/670: write dc/d32/d33/d4e/f88 [259813,111389] 0 2024-05-09T06:38:16.794 INFO:tasks.workunit.client.0.smithi063.stdout:8/687: creat d9/d3b/ffb x:0 0 0 2024-05-09T06:38:16.795 INFO:tasks.workunit.client.0.smithi063.stdout:5/613: rename d8/d19/d30/d46/f62 to d8/d35/fcd 0 2024-05-09T06:38:16.797 INFO:tasks.workunit.client.0.smithi063.stdout:1/554: write d9/d2a/d3c/d4c/f56 [949780,40433] 0 2024-05-09T06:38:16.797 INFO:tasks.workunit.client.0.smithi063.stdout:2/671: readlink dc/dd/d2d/d60/l69 0 2024-05-09T06:38:16.798 INFO:tasks.workunit.client.0.smithi063.stdout:6/634: mknod d2/d1f/d29/d3f/d5c/d64/cce 0 2024-05-09T06:38:16.799 INFO:tasks.workunit.client.0.smithi063.stdout:9/645: dwrite d0/f24 [0,4194304] 0 2024-05-09T06:38:16.799 INFO:tasks.workunit.client.0.smithi063.stdout:0/655: read dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fc3 [7936717,92009] 0 2024-05-09T06:38:16.800 INFO:tasks.workunit.client.0.smithi063.stdout:8/688: mknod d9/da/d18/dc7/d8f/cfc 0 2024-05-09T06:38:16.802 INFO:tasks.workunit.client.0.smithi063.stdout:1/555: chown d9/d2a/d35/d1e/d9d/d3f/d59/d62/fa5 2561900 1 2024-05-09T06:38:16.806 INFO:tasks.workunit.client.0.smithi063.stdout:6/635: mkdir d2/d20/d26/d7f/dcf 0 2024-05-09T06:38:16.807 INFO:tasks.workunit.client.0.smithi063.stdout:1/556: rmdir d9/d2a/d2e 39 2024-05-09T06:38:16.810 INFO:tasks.workunit.client.0.smithi063.stdout:8/689: rename d9/da/d18/d46/ff4 to d9/d41/dec/da1/ffd 0 2024-05-09T06:38:16.814 INFO:tasks.workunit.client.0.smithi063.stdout:9/646: dwrite d0/d8/d2d/faf [0,4194304] 0 2024-05-09T06:38:16.817 INFO:tasks.workunit.client.0.smithi063.stdout:6/636: read d2/d1f/d29/d3f/d5c/d64/f6f [1602255,97267] 0 2024-05-09T06:38:16.817 INFO:tasks.workunit.client.0.smithi063.stdout:8/690: unlink d9/da/d53/db2/cb6 0 2024-05-09T06:38:16.818 INFO:tasks.workunit.client.0.smithi063.stdout:0/656: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d46/fa7 [0,4194304] 0 2024-05-09T06:38:16.819 INFO:tasks.workunit.client.0.smithi063.stdout:4/604: dwrite d7/df/d35/d78/f86 [0,4194304] 0 2024-05-09T06:38:16.821 INFO:tasks.workunit.client.0.smithi063.stdout:5/614: write d8/d19/d30/f43 [2119821,45994] 0 2024-05-09T06:38:16.823 INFO:tasks.workunit.client.0.smithi063.stdout:0/657: mkdir dd/d60/dcd 0 2024-05-09T06:38:16.823 INFO:tasks.workunit.client.0.smithi063.stdout:9/647: truncate d0/d4/d26/d47/f5d 8757143 0 2024-05-09T06:38:16.824 INFO:tasks.workunit.client.0.smithi063.stdout:6/637: symlink d2/df/d91/ld0 0 2024-05-09T06:38:16.826 INFO:tasks.workunit.client.0.smithi063.stdout:3/637: dwrite d7/d4c/f54 [0,4194304] 0 2024-05-09T06:38:16.826 INFO:tasks.workunit.client.0.smithi063.stdout:1/557: getdents d9/d2a/d35/d1e/d9d/d3f/d59 0 2024-05-09T06:38:16.828 INFO:tasks.workunit.client.0.smithi063.stdout:9/648: unlink d0/c61 0 2024-05-09T06:38:16.830 INFO:tasks.workunit.client.0.smithi063.stdout:5/615: unlink d8/d19/d30/d74/ca8 0 2024-05-09T06:38:16.831 INFO:tasks.workunit.client.0.smithi063.stdout:4/605: creat d7/da3/d28/d31/d71/dbb/fc0 x:0 0 0 2024-05-09T06:38:16.833 INFO:tasks.workunit.client.0.smithi063.stdout:6/638: rename d2/d12/ca4 to d2/df/d91/cd1 0 2024-05-09T06:38:16.834 INFO:tasks.workunit.client.0.smithi063.stdout:3/638: mkdir d7/da/d2b/d66/d74/d7e/dd5 0 2024-05-09T06:38:16.835 INFO:tasks.workunit.client.0.smithi063.stdout:1/558: mknod d9/d2a/d39/d5d/d75/cbf 0 2024-05-09T06:38:16.836 INFO:tasks.workunit.client.0.smithi063.stdout:4/606: rmdir d7/da3/da2 39 2024-05-09T06:38:16.836 INFO:tasks.workunit.client.0.smithi063.stdout:6/639: truncate d2/d20/f9c 741221 0 2024-05-09T06:38:16.837 INFO:tasks.workunit.client.0.smithi063.stdout:0/658: rename dd/d60/d44/d90/da4/fcc to dd/d60/d44/d90/fce 0 2024-05-09T06:38:16.839 INFO:tasks.workunit.client.0.smithi063.stdout:8/691: getdents d9/d41/dec 0 2024-05-09T06:38:16.839 INFO:tasks.workunit.client.0.smithi063.stdout:1/559: chown d9/d2a/d35/f29 648 1 2024-05-09T06:38:16.839 INFO:tasks.workunit.client.0.smithi063.stdout:5/616: dwrite d8/d19/f48 [0,4194304] 0 2024-05-09T06:38:16.839 INFO:tasks.workunit.client.0.smithi063.stdout:9/649: creat d0/d4/d26/d39/d3e/d66/fe3 x:0 0 0 2024-05-09T06:38:16.840 INFO:tasks.workunit.client.0.smithi063.stdout:4/607: dread - d7/da3/d8b/fbc zero size 2024-05-09T06:38:16.841 INFO:tasks.workunit.client.0.smithi063.stdout:6/640: mkdir d2/df/dd2 0 2024-05-09T06:38:16.842 INFO:tasks.workunit.client.0.smithi063.stdout:9/650: write d0/d4/d57/fa1 [732424,73555] 0 2024-05-09T06:38:16.843 INFO:tasks.workunit.client.0.smithi063.stdout:3/639: creat d7/da/d21/d30/d44/fd6 x:0 0 0 2024-05-09T06:38:16.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/659: read dd/d1a/d2c/d51/d72/d9e/d23/d33/f88 [3349097,123859] 0 2024-05-09T06:38:16.845 INFO:tasks.workunit.client.0.smithi063.stdout:4/608: mknod d7/da3/d28/d31/d71/cc1 0 2024-05-09T06:38:16.846 INFO:tasks.workunit.client.0.smithi063.stdout:6/641: write d2/fa9 [72964,40856] 0 2024-05-09T06:38:16.846 INFO:tasks.workunit.client.0.smithi063.stdout:1/560: creat d9/d46/fc0 x:0 0 0 2024-05-09T06:38:16.847 INFO:tasks.workunit.client.0.smithi063.stdout:4/609: chown d7/da3/d8b 4137963 1 2024-05-09T06:38:16.847 INFO:tasks.workunit.client.0.smithi063.stdout:0/660: chown dd/d1a/d2c/d51/d76/d97/fa5 1294 1 2024-05-09T06:38:16.848 INFO:tasks.workunit.client.0.smithi063.stdout:4/610: fsync d7/d3d/fa7 0 2024-05-09T06:38:16.851 INFO:tasks.workunit.client.0.smithi063.stdout:0/661: creat dd/d1a/d2c/d42/fcf x:0 0 0 2024-05-09T06:38:16.853 INFO:tasks.workunit.client.0.smithi063.stdout:0/662: truncate dd/d60/d44/d90/fce 644193 0 2024-05-09T06:38:16.858 INFO:tasks.workunit.client.0.smithi063.stdout:0/663: creat dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fd0 x:0 0 0 2024-05-09T06:38:16.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:16 smithi193 ceph-mon[35141]: pgmap v11: 65 pgs: 65 active+clean; 1.3 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 166 MiB/s wr, 334 op/s 2024-05-09T06:38:16.873 INFO:tasks.workunit.client.0.smithi063.stdout:0/664: dread dd/d1a/d2c/d51/d72/d9e/d23/d89/fc4 [0,4194304] 0 2024-05-09T06:38:16.874 INFO:tasks.workunit.client.0.smithi063.stdout:0/665: chown dd/d1a/c1c 171287862 1 2024-05-09T06:38:16.875 INFO:tasks.workunit.client.0.smithi063.stdout:0/666: chown dd/d1a/d2c/d51/d72/d9e/c32 0 1 2024-05-09T06:38:16.876 INFO:tasks.workunit.client.0.smithi063.stdout:0/667: dread - dd/d1a/d2c/d42/fcf zero size 2024-05-09T06:38:16.878 INFO:tasks.workunit.client.0.smithi063.stdout:0/668: symlink dd/d1a/d50/ld1 0 2024-05-09T06:38:16.880 INFO:tasks.workunit.client.0.smithi063.stdout:0/669: mkdir dd/d1a/d2c/d51/d72/d9e/d23/d31/dd2 0 2024-05-09T06:38:16.921 INFO:tasks.workunit.client.0.smithi063.stdout:0/670: getdents dd/d1a/d2c/d51/d72/d9e/d23/d89/da8 0 2024-05-09T06:38:16.922 INFO:tasks.workunit.client.0.smithi063.stdout:8/692: getdents d9/da/d53/db2 0 2024-05-09T06:38:16.924 INFO:tasks.workunit.client.0.smithi063.stdout:8/693: write d9/da/d18/d24/fdb [1009109,98680] 0 2024-05-09T06:38:16.925 INFO:tasks.workunit.client.0.smithi063.stdout:0/671: creat dd/d1a/d2c/d51/d72/fd3 x:0 0 0 2024-05-09T06:38:16.933 INFO:tasks.workunit.client.0.smithi063.stdout:4/611: truncate d7/df/f21 7347427 0 2024-05-09T06:38:16.934 INFO:tasks.workunit.client.0.smithi063.stdout:0/672: chown dd/d1a/d2c/d42/c95 97138 1 2024-05-09T06:38:16.936 INFO:tasks.workunit.client.0.smithi063.stdout:8/694: mknod d9/da/d53/da3/de9/cfe 0 2024-05-09T06:38:16.941 INFO:tasks.workunit.client.0.smithi063.stdout:5/617: dwrite d8/d53/f93 [0,4194304] 0 2024-05-09T06:38:16.943 INFO:tasks.workunit.client.0.smithi063.stdout:1/561: dwrite d9/d46/f54 [0,4194304] 0 2024-05-09T06:38:16.944 INFO:tasks.workunit.client.0.smithi063.stdout:4/612: chown d7/df/d35/d5a/dad/laf 687 1 2024-05-09T06:38:16.945 INFO:tasks.workunit.client.0.smithi063.stdout:9/651: dwrite d0/d3b/da0/f9c [4194304,4194304] 0 2024-05-09T06:38:16.946 INFO:tasks.workunit.client.0.smithi063.stdout:3/640: dwrite d7/d70/d98/d5b/f78 [0,4194304] 0 2024-05-09T06:38:16.948 INFO:tasks.workunit.client.0.smithi063.stdout:8/695: stat d9/da/d53/da3/cd8 0 2024-05-09T06:38:16.948 INFO:tasks.workunit.client.0.smithi063.stdout:1/562: stat d9/d2a/d3c/la3 0 2024-05-09T06:38:16.952 INFO:tasks.workunit.client.0.smithi063.stdout:5/618: fsync d8/d19/d30/f61 0 2024-05-09T06:38:16.952 INFO:tasks.workunit.client.0.smithi063.stdout:8/696: chown d9/da/d18/d24/f27 1026205 1 2024-05-09T06:38:16.952 INFO:tasks.workunit.client.0.smithi063.stdout:0/673: rmdir dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75 39 2024-05-09T06:38:16.955 INFO:tasks.workunit.client.0.smithi063.stdout:1/563: write d9/d76/d94/fa7 [603625,113047] 0 2024-05-09T06:38:16.955 INFO:tasks.workunit.client.0.smithi063.stdout:3/641: symlink d7/d22/d9c/ld7 0 2024-05-09T06:38:16.961 INFO:tasks.workunit.client.0.smithi063.stdout:3/642: chown d7/da/d21/d30/fb9 2663927 1 2024-05-09T06:38:16.961 INFO:tasks.workunit.client.0.smithi063.stdout:4/613: dwrite d7/da3/d28/d31/d61/fb6 [0,4194304] 0 2024-05-09T06:38:16.964 INFO:tasks.workunit.client.0.smithi063.stdout:9/652: link d0/d4/d26/c38 d0/d4/d7d/d94/ce4 0 2024-05-09T06:38:16.965 INFO:tasks.workunit.client.0.smithi063.stdout:4/614: write d7/d3d/faa [11554,65634] 0 2024-05-09T06:38:16.967 INFO:tasks.workunit.client.0.smithi063.stdout:7/593: sync 2024-05-09T06:38:16.969 INFO:tasks.workunit.client.0.smithi063.stdout:8/697: dread d9/da/d45/fa4 [0,4194304] 0 2024-05-09T06:38:16.972 INFO:tasks.workunit.client.0.smithi063.stdout:0/674: dwrite dd/d60/d44/d90/faa [0,4194304] 0 2024-05-09T06:38:16.972 INFO:tasks.workunit.client.0.smithi063.stdout:4/615: mknod d7/da3/d28/cc2 0 2024-05-09T06:38:16.975 INFO:tasks.workunit.client.0.smithi063.stdout:9/653: mknod d0/d4/d26/d39/d48/d80/d84/dc2/ce5 0 2024-05-09T06:38:16.975 INFO:tasks.workunit.client.0.smithi063.stdout:1/564: link d9/d2a/d35/d1e/d9d/d3f/l77 d9/d2a/d35/d1e/d9d/d3f/d59/d62/d7e/lc1 0 2024-05-09T06:38:16.977 INFO:tasks.workunit.client.0.smithi063.stdout:7/594: creat d0/d15/d4a/dc8/fcf x:0 0 0 2024-05-09T06:38:16.978 INFO:tasks.workunit.client.0.smithi063.stdout:5/619: truncate d8/d17/f1c 313798 0 2024-05-09T06:38:16.981 INFO:tasks.workunit.client.0.smithi063.stdout:5/620: dread d8/d17/da3/fbf [0,4194304] 0 2024-05-09T06:38:16.981 INFO:tasks.workunit.client.0.smithi063.stdout:0/675: read dd/d1a/f78 [128527,66430] 0 2024-05-09T06:38:16.982 INFO:tasks.workunit.client.0.smithi063.stdout:5/621: write d8/d17/f3d [2183922,8352] 0 2024-05-09T06:38:16.983 INFO:tasks.workunit.client.0.smithi063.stdout:5/622: chown d8/d65 9 1 2024-05-09T06:38:16.983 INFO:tasks.workunit.client.0.smithi063.stdout:8/698: rename d9/da/d51/f8b to d9/da/d45/de4/fff 0 2024-05-09T06:38:16.984 INFO:tasks.workunit.client.0.smithi063.stdout:5/623: fsync d8/d53/dca/f5f 0 2024-05-09T06:38:16.984 INFO:tasks.workunit.client.0.smithi063.stdout:4/616: stat d7/da3/d28/d31/d61/c9b 0 2024-05-09T06:38:16.984 INFO:tasks.workunit.client.0.smithi063.stdout:3/643: write d7/d70/d98/d2d/f93 [666646,75413] 0 2024-05-09T06:38:16.985 INFO:tasks.workunit.client.0.smithi063.stdout:5/624: stat d8/d17/d18/f1f 0 2024-05-09T06:38:16.986 INFO:tasks.workunit.client.0.smithi063.stdout:4/617: stat d7/da3/d28/d31/d7b/d81/c8d 0 2024-05-09T06:38:16.986 INFO:tasks.workunit.client.0.smithi063.stdout:1/565: mkdir d9/d2a/d3c/d4c/d83/d88/dc2 0 2024-05-09T06:38:16.989 INFO:tasks.workunit.client.0.smithi063.stdout:0/676: creat dd/d60/fd4 x:0 0 0 2024-05-09T06:38:16.991 INFO:tasks.workunit.client.0.smithi063.stdout:8/699: readlink d9/d41/dec/lc6 0 2024-05-09T06:38:16.995 INFO:tasks.workunit.client.0.smithi063.stdout:3/644: mkdir d7/d22/d28/dab/dd8 0 2024-05-09T06:38:16.995 INFO:tasks.workunit.client.0.smithi063.stdout:3/645: stat d7/da/f29 0 2024-05-09T06:38:16.996 INFO:tasks.workunit.client.0.smithi063.stdout:3/646: write d7/da/d2b/d66/dc0/f6e [4754360,95261] 0 2024-05-09T06:38:17.002 INFO:tasks.workunit.client.0.smithi063.stdout:0/677: symlink dd/d60/d44/d90/da4/ld5 0 2024-05-09T06:38:17.003 INFO:tasks.workunit.client.0.smithi063.stdout:3/647: dread d7/d4c/f80 [0,4194304] 0 2024-05-09T06:38:17.004 INFO:tasks.workunit.client.0.smithi063.stdout:8/700: symlink d9/da/d53/dbc/l100 0 2024-05-09T06:38:17.006 INFO:tasks.workunit.client.0.smithi063.stdout:4/618: dread d7/da3/f65 [0,4194304] 0 2024-05-09T06:38:17.007 INFO:tasks.workunit.client.0.smithi063.stdout:9/654: truncate d0/d4/d26/fae 1700134 0 2024-05-09T06:38:17.007 INFO:tasks.workunit.client.0.smithi063.stdout:7/595: link d0/d15/d49/d4f/d66/d67/c88 d0/d15/cd0 0 2024-05-09T06:38:17.009 INFO:tasks.workunit.client.0.smithi063.stdout:0/678: mkdir dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/dd6 0 2024-05-09T06:38:17.011 INFO:tasks.workunit.client.0.smithi063.stdout:8/701: rename d9/da/d18/d24/d71/lab to d9/da/d18/dc7/dd3/df1/l101 0 2024-05-09T06:38:17.011 INFO:tasks.workunit.client.0.smithi063.stdout:7/596: fsync d0/d15/d89/fa9 0 2024-05-09T06:38:17.011 INFO:tasks.workunit.client.0.smithi063.stdout:9/655: stat d0/d4/d26/d39/d48/d80/d84/fc7 0 2024-05-09T06:38:17.012 INFO:tasks.workunit.client.0.smithi063.stdout:4/619: creat d7/df/d35/d5a/dad/fc3 x:0 0 0 2024-05-09T06:38:17.017 INFO:tasks.workunit.client.0.smithi063.stdout:9/656: fdatasync d0/d4/d26/d47/fa8 0 2024-05-09T06:38:17.018 INFO:tasks.workunit.client.0.smithi063.stdout:3/648: dwrite d7/da/d2b/d66/dc0/fb3 [0,4194304] 0 2024-05-09T06:38:17.019 INFO:tasks.workunit.client.0.smithi063.stdout:5/625: dwrite d8/f68 [0,4194304] 0 2024-05-09T06:38:17.021 INFO:tasks.workunit.client.0.smithi063.stdout:9/657: stat d0/f24 0 2024-05-09T06:38:17.024 INFO:tasks.workunit.client.0.smithi063.stdout:6/642: sync 2024-05-09T06:38:17.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/672: sync 2024-05-09T06:38:17.027 INFO:tasks.workunit.client.0.smithi063.stdout:9/658: mknod d0/ce6 0 2024-05-09T06:38:17.028 INFO:tasks.workunit.client.0.smithi063.stdout:3/649: symlink d7/da/d21/d30/d44/d48/ld9 0 2024-05-09T06:38:17.029 INFO:tasks.workunit.client.0.smithi063.stdout:1/566: dwrite d9/d2a/d35/d1e/d9d/f33 [0,4194304] 0 2024-05-09T06:38:17.030 INFO:tasks.workunit.client.0.smithi063.stdout:2/673: rmdir dc/d36 39 2024-05-09T06:38:17.031 INFO:tasks.workunit.client.0.smithi063.stdout:4/620: dwrite d7/df/f12 [0,4194304] 0 2024-05-09T06:38:17.031 INFO:tasks.workunit.client.0.smithi063.stdout:5/626: mknod d8/d19/cce 0 2024-05-09T06:38:17.035 INFO:tasks.workunit.client.0.smithi063.stdout:7/597: write d0/d15/d4a/d68/f6f [249628,38047] 0 2024-05-09T06:38:17.036 INFO:tasks.workunit.client.0.smithi063.stdout:9/659: fsync d0/d4/d26/d39/d48/d80/fc1 0 2024-05-09T06:38:17.040 INFO:tasks.workunit.client.0.smithi063.stdout:2/674: fsync f6 0 2024-05-09T06:38:17.041 INFO:tasks.workunit.client.0.smithi063.stdout:8/702: truncate d9/f7d 553428 0 2024-05-09T06:38:17.044 INFO:tasks.workunit.client.0.smithi063.stdout:7/598: stat d0/d15/f25 0 2024-05-09T06:38:17.044 INFO:tasks.workunit.client.0.smithi063.stdout:5/627: chown d8/d17/d18/f38 21683 1 2024-05-09T06:38:17.044 INFO:tasks.workunit.client.0.smithi063.stdout:3/650: truncate d7/d22/d28/f8c 1107998 0 2024-05-09T06:38:17.045 INFO:tasks.workunit.client.0.smithi063.stdout:4/621: link d7/fb d7/d50/fc4 0 2024-05-09T06:38:17.046 INFO:tasks.workunit.client.0.smithi063.stdout:9/660: mkdir d0/d4/d26/d47/de7 0 2024-05-09T06:38:17.048 INFO:tasks.workunit.client.0.smithi063.stdout:8/703: mkdir d9/da/d53/dbc/d102 0 2024-05-09T06:38:17.048 INFO:tasks.workunit.client.0.smithi063.stdout:0/679: dwrite dd/f15 [4194304,4194304] 0 2024-05-09T06:38:17.049 INFO:tasks.workunit.client.0.smithi063.stdout:1/567: read d9/d2a/d35/d1e/d9d/d3f/d59/d62/fa5 [1742119,44761] 0 2024-05-09T06:38:17.050 INFO:tasks.workunit.client.0.smithi063.stdout:6/643: dwrite d2/d1f/d6c/f71 [0,4194304] 0 2024-05-09T06:38:17.050 INFO:tasks.workunit.client.0.smithi063.stdout:9/661: dread - d0/d4/d57/d60/fe0 zero size 2024-05-09T06:38:17.052 INFO:tasks.workunit.client.0.smithi063.stdout:5/628: read d8/d19/f1a [3688630,3203] 0 2024-05-09T06:38:17.052 INFO:tasks.workunit.client.0.smithi063.stdout:7/599: rmdir d0/d15/d49/d54/d9f 39 2024-05-09T06:38:17.056 INFO:tasks.workunit.client.0.smithi063.stdout:7/600: chown d0/d15/d49/d4f/d66/d67/cb3 14 1 2024-05-09T06:38:17.056 INFO:tasks.workunit.client.0.smithi063.stdout:4/622: mknod d7/da3/d28/d31/d7b/d81/cc5 0 2024-05-09T06:38:17.058 INFO:tasks.workunit.client.0.smithi063.stdout:9/662: dread d0/d4/d7d/f87 [0,4194304] 0 2024-05-09T06:38:17.063 INFO:tasks.workunit.client.0.smithi063.stdout:8/704: rename d9/d41/c48 to d9/da/d18/d24/d71/c103 0 2024-05-09T06:38:17.064 INFO:tasks.workunit.client.0.smithi063.stdout:6/644: fdatasync d2/f4d 0 2024-05-09T06:38:17.071 INFO:tasks.workunit.client.0.smithi063.stdout:9/663: unlink d0/d4/d57/dba/dd5/cd7 0 2024-05-09T06:38:17.072 INFO:tasks.workunit.client.0.smithi063.stdout:8/705: fsync d9/da/d18/d24/d71/fa6 0 2024-05-09T06:38:17.074 INFO:tasks.workunit.client.0.smithi063.stdout:0/680: dread dd/d60/d44/f61 [0,4194304] 0 2024-05-09T06:38:17.075 INFO:tasks.workunit.client.0.smithi063.stdout:0/681: fsync dd/d1a/d2c/d51/d72/d9e/d23/d31/fc7 0 2024-05-09T06:38:17.076 INFO:tasks.workunit.client.0.smithi063.stdout:5/629: link d8/d35/f64 d8/d17/d75/fcf 0 2024-05-09T06:38:17.077 INFO:tasks.workunit.client.0.smithi063.stdout:3/651: getdents d7/da/d2b/d66/d74 0 2024-05-09T06:38:17.078 INFO:tasks.workunit.client.0.smithi063.stdout:3/652: dread - d7/d4c/d25/fa0 zero size 2024-05-09T06:38:17.079 INFO:tasks.workunit.client.0.smithi063.stdout:7/601: dread d0/f10 [0,4194304] 0 2024-05-09T06:38:17.083 INFO:tasks.workunit.client.0.smithi063.stdout:9/664: rename d0/d4/d7d/f99 to d0/d4/d57/fe8 0 2024-05-09T06:38:17.085 INFO:tasks.workunit.client.0.smithi063.stdout:4/623: creat d7/df/fc6 x:0 0 0 2024-05-09T06:38:17.086 INFO:tasks.workunit.client.0.smithi063.stdout:5/630: fsync d8/d53/dca/f23 0 2024-05-09T06:38:17.086 INFO:tasks.workunit.client.1.smithi193.stdout:1/253: sync 2024-05-09T06:38:17.087 INFO:tasks.workunit.client.0.smithi063.stdout:8/706: rmdir d9/da 39 2024-05-09T06:38:17.087 INFO:tasks.workunit.client.0.smithi063.stdout:5/631: dread - d8/d17/d75/f7f zero size 2024-05-09T06:38:17.089 INFO:tasks.workunit.client.0.smithi063.stdout:3/653: mknod d7/da/d2b/d66/dc0/da4/cda 0 2024-05-09T06:38:17.089 INFO:tasks.workunit.client.0.smithi063.stdout:1/568: write d9/d2a/d35/d1e/faf [513760,33401] 0 2024-05-09T06:38:17.093 INFO:tasks.workunit.client.0.smithi063.stdout:7/602: dread d0/d15/d49/f5e [0,4194304] 0 2024-05-09T06:38:17.094 INFO:tasks.workunit.client.0.smithi063.stdout:5/632: dwrite d8/d19/f48 [0,4194304] 0 2024-05-09T06:38:17.094 INFO:tasks.workunit.client.1.smithi193.stdout:1/254: symlink d4/d14/l49 0 2024-05-09T06:38:17.095 INFO:tasks.workunit.client.0.smithi063.stdout:6/645: mkdir d2/d89/dd3 0 2024-05-09T06:38:17.097 INFO:tasks.workunit.client.0.smithi063.stdout:1/569: dwrite d9/d46/f54 [0,4194304] 0 2024-05-09T06:38:17.099 INFO:tasks.workunit.client.0.smithi063.stdout:9/665: creat d0/d8/da6/db0/db5/fe9 x:0 0 0 2024-05-09T06:38:17.104 INFO:tasks.workunit.client.0.smithi063.stdout:1/570: dwrite d9/d2a/d3c/d4c/f78 [0,4194304] 0 2024-05-09T06:38:17.106 INFO:tasks.workunit.client.1.smithi193.stdout:1/255: truncate d4/d16/f10 3568491 0 2024-05-09T06:38:17.107 INFO:tasks.workunit.client.1.smithi193.stdout:1/256: fdatasync d4/d16/de/f12 0 2024-05-09T06:38:17.108 INFO:tasks.workunit.client.1.smithi193.stdout:1/257: chown d4/d16/f27 11 1 2024-05-09T06:38:17.113 INFO:tasks.workunit.client.1.smithi193.stdout:1/258: mkdir d4/d16/d17/d1a/d3a/d4a 0 2024-05-09T06:38:17.115 INFO:tasks.workunit.client.0.smithi063.stdout:7/603: creat d0/d15/fd1 x:0 0 0 2024-05-09T06:38:17.117 INFO:tasks.workunit.client.0.smithi063.stdout:4/624: write d7/d50/fc4 [109013,24875] 0 2024-05-09T06:38:17.117 INFO:tasks.workunit.client.1.smithi193.stdout:1/259: symlink d4/d16/d17/d1a/l4b 0 2024-05-09T06:38:17.119 INFO:tasks.workunit.client.0.smithi063.stdout:6/646: mknod d2/d4/d96/cd4 0 2024-05-09T06:38:17.121 INFO:tasks.workunit.client.0.smithi063.stdout:9/666: chown d0/d8/d2d/c95 25863097 1 2024-05-09T06:38:17.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/571: mkdir d9/d2a/d3c/d4c/dc3 0 2024-05-09T06:38:17.122 INFO:tasks.workunit.client.0.smithi063.stdout:0/682: link dd/d1a/d2c/d51/d72/d9e/d23/d31/f5e dd/d1a/d2c/d51/fd7 0 2024-05-09T06:38:17.124 INFO:tasks.workunit.client.1.smithi193.stdout:1/260: dwrite d4/d16/d17/f43 [0,4194304] 0 2024-05-09T06:38:17.125 INFO:tasks.workunit.client.0.smithi063.stdout:5/633: creat d8/d17/d75/dc9/fd0 x:0 0 0 2024-05-09T06:38:17.126 INFO:tasks.workunit.client.1.smithi193.stdout:1/261: write d4/d16/d35/f48 [104070,24802] 0 2024-05-09T06:38:17.127 INFO:tasks.workunit.client.0.smithi063.stdout:4/625: rmdir d7/da3/d28/d31/d7b 39 2024-05-09T06:38:17.128 INFO:tasks.workunit.client.0.smithi063.stdout:1/572: write d9/d2a/d35/f29 [1860343,10926] 0 2024-05-09T06:38:17.130 INFO:tasks.workunit.client.0.smithi063.stdout:6/647: dwrite d2/d4/f1c [0,4194304] 0 2024-05-09T06:38:17.130 INFO:tasks.workunit.client.0.smithi063.stdout:1/573: stat d9/d2a/d35/d24/f26 0 2024-05-09T06:38:17.133 INFO:tasks.workunit.client.0.smithi063.stdout:6/648: dread - d2/d20/d26/fc4 zero size 2024-05-09T06:38:17.133 INFO:tasks.workunit.client.0.smithi063.stdout:8/707: link d9/da/d18/d24/d71/feb d9/da/d53/dbc/d102/f104 0 2024-05-09T06:38:17.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/675: sync 2024-05-09T06:38:17.134 INFO:tasks.workunit.client.0.smithi063.stdout:8/708: stat d9/d3b 0 2024-05-09T06:38:17.135 INFO:tasks.workunit.client.0.smithi063.stdout:1/574: fdatasync d9/d2a/d3c/d4c/f78 0 2024-05-09T06:38:17.135 INFO:tasks.workunit.client.0.smithi063.stdout:3/654: creat d7/da/d2b/d66/fdb x:0 0 0 2024-05-09T06:38:17.135 INFO:tasks.workunit.client.1.smithi193.stdout:1/262: mknod d4/d16/d17/c4c 0 2024-05-09T06:38:17.135 INFO:tasks.workunit.client.1.smithi193.stdout:7/297: sync 2024-05-09T06:38:17.135 INFO:tasks.workunit.client.1.smithi193.stdout:5/312: sync 2024-05-09T06:38:17.136 INFO:tasks.workunit.client.1.smithi193.stdout:8/263: sync 2024-05-09T06:38:17.136 INFO:tasks.workunit.client.1.smithi193.stdout:2/281: sync 2024-05-09T06:38:17.136 INFO:tasks.workunit.client.1.smithi193.stdout:4/347: sync 2024-05-09T06:38:17.136 INFO:tasks.workunit.client.1.smithi193.stdout:3/311: sync 2024-05-09T06:38:17.136 INFO:tasks.workunit.client.1.smithi193.stdout:0/312: sync 2024-05-09T06:38:17.136 INFO:tasks.workunit.client.1.smithi193.stdout:9/298: sync 2024-05-09T06:38:17.136 INFO:tasks.workunit.client.1.smithi193.stdout:6/281: sync 2024-05-09T06:38:17.137 INFO:tasks.workunit.client.0.smithi063.stdout:8/709: truncate d9/da/d53/db2/fb5 798401 0 2024-05-09T06:38:17.139 INFO:tasks.workunit.client.1.smithi193.stdout:1/263: write d4/d16/f27 [5103302,71467] 0 2024-05-09T06:38:17.141 INFO:tasks.workunit.client.1.smithi193.stdout:6/282: write d1/d25/d2c/d3c/d3e/f3f [793397,127175] 0 2024-05-09T06:38:17.142 INFO:tasks.workunit.client.0.smithi063.stdout:0/683: creat dd/d1a/d2c/d42/fd8 x:0 0 0 2024-05-09T06:38:17.143 INFO:tasks.workunit.client.0.smithi063.stdout:7/604: mkdir d0/d15/d49/d54/d9f/dd2 0 2024-05-09T06:38:17.146 INFO:tasks.workunit.client.1.smithi193.stdout:3/312: fdatasync d1/db/f34 0 2024-05-09T06:38:17.147 INFO:tasks.workunit.client.0.smithi063.stdout:0/684: dwrite dd/d1a/d2c/d51/d72/d9e/f55 [0,4194304] 0 2024-05-09T06:38:17.148 INFO:tasks.workunit.client.1.smithi193.stdout:1/264: symlink d4/d16/de/l4d 0 2024-05-09T06:38:17.148 INFO:tasks.workunit.client.1.smithi193.stdout:7/298: chown d6/d8/d29/d2f/l54 833298 1 2024-05-09T06:38:17.153 INFO:tasks.workunit.client.0.smithi063.stdout:3/655: mknod d7/da/d21/d30/cdc 0 2024-05-09T06:38:17.153 INFO:tasks.workunit.client.0.smithi063.stdout:0/685: dread dd/d1a/d50/f86 [0,4194304] 0 2024-05-09T06:38:17.154 INFO:tasks.workunit.client.0.smithi063.stdout:3/656: chown d7/da/d21/l6d 32159 1 2024-05-09T06:38:17.154 INFO:tasks.workunit.client.0.smithi063.stdout:1/575: chown d9/d2a/d35/d1e/l38 3214719 1 2024-05-09T06:38:17.155 INFO:tasks.workunit.client.0.smithi063.stdout:1/576: chown d9/fe 0 1 2024-05-09T06:38:17.156 INFO:tasks.workunit.client.0.smithi063.stdout:8/710: write d9/da/d45/fa4 [438286,127099] 0 2024-05-09T06:38:17.159 INFO:tasks.workunit.client.0.smithi063.stdout:7/605: rmdir d0/d15/d49/d4f 39 2024-05-09T06:38:17.163 INFO:tasks.workunit.client.0.smithi063.stdout:5/634: write d8/d17/d18/f51 [73858,106458] 0 2024-05-09T06:38:17.164 INFO:tasks.workunit.client.0.smithi063.stdout:5/635: fdatasync d8/d17/faf 0 2024-05-09T06:38:17.166 INFO:tasks.workunit.client.1.smithi193.stdout:9/299: dwrite dd/f11 [0,4194304] 0 2024-05-09T06:38:17.166 INFO:tasks.workunit.client.1.smithi193.stdout:5/313: creat d0/d13/d2c/d37/f68 x:0 0 0 2024-05-09T06:38:17.167 INFO:tasks.workunit.client.0.smithi063.stdout:5/636: dread d8/d19/f48 [0,4194304] 0 2024-05-09T06:38:17.168 INFO:tasks.workunit.client.0.smithi063.stdout:2/676: write dc/dd/d2d/d60/fb2 [2613310,102547] 0 2024-05-09T06:38:17.169 INFO:tasks.workunit.client.1.smithi193.stdout:4/348: dwrite d1a/d24/d30/f49 [0,4194304] 0 2024-05-09T06:38:17.176 INFO:tasks.workunit.client.0.smithi063.stdout:4/626: dread d7/df/f21 [0,4194304] 0 2024-05-09T06:38:17.177 INFO:tasks.workunit.client.0.smithi063.stdout:4/627: fsync d7/df/d35/d5a/fa4 0 2024-05-09T06:38:17.178 INFO:tasks.workunit.client.1.smithi193.stdout:4/349: fdatasync d1a/d22/f31 0 2024-05-09T06:38:17.180 INFO:tasks.workunit.client.1.smithi193.stdout:1/265: truncate d4/d14/f1b 210366 0 2024-05-09T06:38:17.180 INFO:tasks.workunit.client.1.smithi193.stdout:2/282: creat db/df/f69 x:0 0 0 2024-05-09T06:38:17.181 INFO:tasks.workunit.client.0.smithi063.stdout:3/657: rename d7/da/d2b/d66/d74/d7e/dd5 to d7/da/d2b/d66/dc0/da4/db0/ddd 0 2024-05-09T06:38:17.183 INFO:tasks.workunit.client.1.smithi193.stdout:4/350: write d1a/f64 [24221,54552] 0 2024-05-09T06:38:17.183 INFO:tasks.workunit.client.0.smithi063.stdout:8/711: truncate d9/da/d18/dc7/d8f/f89 153383 0 2024-05-09T06:38:17.184 INFO:tasks.workunit.client.1.smithi193.stdout:0/313: creat d18/f60 x:0 0 0 2024-05-09T06:38:17.184 INFO:tasks.workunit.client.0.smithi063.stdout:8/712: read d9/da/d18/f22 [29973,29519] 0 2024-05-09T06:38:17.186 INFO:tasks.workunit.client.1.smithi193.stdout:7/299: dread d6/f2e [0,4194304] 0 2024-05-09T06:38:17.186 INFO:tasks.workunit.client.1.smithi193.stdout:8/264: link d6/d1c/d18/f37 d6/d1c/d1b/d29/f4f 0 2024-05-09T06:38:17.186 INFO:tasks.workunit.client.1.smithi193.stdout:9/300: dwrite dd/d1e/d1f/d48/d55/d41/d68/f6b [0,4194304] 0 2024-05-09T06:38:17.186 INFO:tasks.workunit.client.1.smithi193.stdout:5/314: dwrite d0/f9 [0,4194304] 0 2024-05-09T06:38:17.187 INFO:tasks.workunit.client.1.smithi193.stdout:1/266: write d4/d16/d17/f29 [4702039,69452] 0 2024-05-09T06:38:17.188 INFO:tasks.workunit.client.1.smithi193.stdout:4/351: chown d1a/d1f/l2f 1849228 1 2024-05-09T06:38:17.191 INFO:tasks.workunit.client.1.smithi193.stdout:1/267: fsync d4/d16/d17/d1a/f1e 0 2024-05-09T06:38:17.191 INFO:tasks.workunit.client.1.smithi193.stdout:1/268: chown d4/d16/d17 10 1 2024-05-09T06:38:17.191 INFO:tasks.workunit.client.0.smithi063.stdout:6/649: mknod d2/dbf/cd5 0 2024-05-09T06:38:17.192 INFO:tasks.workunit.client.1.smithi193.stdout:8/265: fsync d6/d1c/d18/d2f/f39 0 2024-05-09T06:38:17.193 INFO:tasks.workunit.client.1.smithi193.stdout:4/352: write d1a/d1f/d4b/d59/f44 [95541,125355] 0 2024-05-09T06:38:17.196 INFO:tasks.workunit.client.1.smithi193.stdout:9/301: read dd/d1e/d1f/f22 [4071277,92505] 0 2024-05-09T06:38:17.197 INFO:tasks.workunit.client.1.smithi193.stdout:9/302: write dd/d1e/d1f/d48/f5a [663428,54288] 0 2024-05-09T06:38:17.202 INFO:tasks.workunit.client.0.smithi063.stdout:1/577: symlink d9/d46/lc4 0 2024-05-09T06:38:17.203 INFO:tasks.workunit.client.0.smithi063.stdout:1/578: fsync d9/d2a/d35/d1e/faf 0 2024-05-09T06:38:17.204 INFO:tasks.workunit.client.1.smithi193.stdout:6/283: link d1/de/f10 d1/de/d11/d18/f65 0 2024-05-09T06:38:17.205 INFO:tasks.workunit.client.1.smithi193.stdout:6/284: stat d1/de/d11/d2d/c35 0 2024-05-09T06:38:17.209 INFO:tasks.workunit.client.1.smithi193.stdout:0/314: fsync d18/f4f 0 2024-05-09T06:38:17.210 INFO:tasks.workunit.client.1.smithi193.stdout:0/315: chown d18/d1d/f1e 23 1 2024-05-09T06:38:17.211 INFO:tasks.workunit.client.0.smithi063.stdout:9/667: getdents d0/d4/d26/d39/d48 0 2024-05-09T06:38:17.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/606: symlink d0/dcb/ld3 0 2024-05-09T06:38:17.216 INFO:tasks.workunit.client.1.smithi193.stdout:3/313: truncate d1/f9 3273717 0 2024-05-09T06:38:17.219 INFO:tasks.workunit.client.1.smithi193.stdout:5/315: dwrite d0/d13/d2c/f4f [0,4194304] 0 2024-05-09T06:38:17.221 INFO:tasks.workunit.client.1.smithi193.stdout:4/353: mkdir d1a/d67 0 2024-05-09T06:38:17.229 INFO:tasks.workunit.client.1.smithi193.stdout:9/303: dread dd/d1e/d1f/f21 [0,4194304] 0 2024-05-09T06:38:17.230 INFO:tasks.workunit.client.1.smithi193.stdout:2/283: unlink f1 0 2024-05-09T06:38:17.230 INFO:tasks.workunit.client.1.smithi193.stdout:9/304: chown dd/d1e/d66/d49/l58 5762437 1 2024-05-09T06:38:17.231 INFO:tasks.workunit.client.1.smithi193.stdout:9/305: write dd/d1e/d66/f45 [57719,67852] 0 2024-05-09T06:38:17.234 INFO:tasks.workunit.client.0.smithi063.stdout:4/628: creat d7/da3/d28/d31/d7b/fc7 x:0 0 0 2024-05-09T06:38:17.235 INFO:tasks.workunit.client.1.smithi193.stdout:0/316: truncate fe 6360730 0 2024-05-09T06:38:17.236 INFO:tasks.workunit.client.1.smithi193.stdout:6/285: dread d1/de/d11/d18/f29 [0,4194304] 0 2024-05-09T06:38:17.237 INFO:tasks.workunit.client.1.smithi193.stdout:7/300: creat d6/dd/d5e/f61 x:0 0 0 2024-05-09T06:38:17.237 INFO:tasks.workunit.client.1.smithi193.stdout:6/286: write d1/d25/d2c/d3c/f3d [328398,74124] 0 2024-05-09T06:38:17.238 INFO:tasks.workunit.client.0.smithi063.stdout:1/579: symlink d9/d10/lc5 0 2024-05-09T06:38:17.240 INFO:tasks.workunit.client.0.smithi063.stdout:4/629: read d7/df/d57/f91 [1221,81420] 0 2024-05-09T06:38:17.241 INFO:tasks.workunit.client.0.smithi063.stdout:8/713: write d9/da/d53/dbc/d102/f104 [646102,89036] 0 2024-05-09T06:38:17.242 INFO:tasks.workunit.client.0.smithi063.stdout:8/714: fsync d9/d3b/f40 0 2024-05-09T06:38:17.242 INFO:tasks.workunit.client.0.smithi063.stdout:3/658: symlink d7/d70/d98/lde 0 2024-05-09T06:38:17.244 INFO:tasks.workunit.client.0.smithi063.stdout:5/637: truncate d8/d53/dca/f42 2386472 0 2024-05-09T06:38:17.245 INFO:tasks.workunit.client.0.smithi063.stdout:4/630: dread d7/da3/d28/f7d [0,4194304] 0 2024-05-09T06:38:17.245 INFO:tasks.workunit.client.1.smithi193.stdout:8/266: mknod d6/d1c/d1b/d3a/c50 0 2024-05-09T06:38:17.246 INFO:tasks.workunit.client.0.smithi063.stdout:4/631: fdatasync d7/df/d1a/f96 0 2024-05-09T06:38:17.247 INFO:tasks.workunit.client.0.smithi063.stdout:6/650: write d2/d1f/d29/f47 [3480655,77217] 0 2024-05-09T06:38:17.248 INFO:tasks.workunit.client.0.smithi063.stdout:6/651: chown d2/d4 1179598639 1 2024-05-09T06:38:17.250 INFO:tasks.workunit.client.0.smithi063.stdout:8/715: dwrite d9/d41/dec/db8/ff5 [0,4194304] 0 2024-05-09T06:38:17.251 INFO:tasks.workunit.client.0.smithi063.stdout:8/716: fsync d9/da/d51/fdd 0 2024-05-09T06:38:17.255 INFO:tasks.workunit.client.1.smithi193.stdout:8/267: dwrite d6/d1c/d1b/f30 [0,4194304] 0 2024-05-09T06:38:17.258 INFO:tasks.workunit.client.1.smithi193.stdout:8/268: write d6/d1c/f12 [1191528,96497] 0 2024-05-09T06:38:17.258 INFO:tasks.workunit.client.0.smithi063.stdout:2/677: link dc/dd/l2c dc/dd/d45/d5a/d6c/d8f/dbf/lc5 0 2024-05-09T06:38:17.259 INFO:tasks.workunit.client.1.smithi193.stdout:5/316: dwrite d0/d13/d2c/d37/d2b/d47/d66/f26 [0,4194304] 0 2024-05-09T06:38:17.261 INFO:tasks.workunit.client.0.smithi063.stdout:0/686: getdents dd/d1a/d50 0 2024-05-09T06:38:17.263 INFO:tasks.workunit.client.0.smithi063.stdout:0/687: dread - dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/fbd zero size 2024-05-09T06:38:17.268 INFO:tasks.workunit.client.0.smithi063.stdout:0/688: chown dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0/fc5 60649980 1 2024-05-09T06:38:17.268 INFO:tasks.workunit.client.1.smithi193.stdout:2/284: mkdir db/d25/d38/d6a 0 2024-05-09T06:38:17.270 INFO:tasks.workunit.client.0.smithi063.stdout:1/580: creat d9/d2a/d39/d5d/fc6 x:0 0 0 2024-05-09T06:38:17.270 INFO:tasks.workunit.client.0.smithi063.stdout:9/668: dwrite d0/d4/f6a [0,4194304] 0 2024-05-09T06:38:17.271 INFO:tasks.workunit.client.0.smithi063.stdout:7/607: write d0/d15/d4a/d50/f1a [43929,127056] 0 2024-05-09T06:38:17.272 INFO:tasks.workunit.client.0.smithi063.stdout:1/581: chown d9/d2a/d35/d1e/l6d 8 1 2024-05-09T06:38:17.274 INFO:tasks.workunit.client.0.smithi063.stdout:5/638: rmdir d8/d19/d30/d74/d77 39 2024-05-09T06:38:17.276 INFO:tasks.workunit.client.1.smithi193.stdout:4/354: dwrite fb [0,4194304] 0 2024-05-09T06:38:17.283 INFO:tasks.workunit.client.1.smithi193.stdout:1/269: link d4/d16/d35/l46 d4/d16/de/l4e 0 2024-05-09T06:38:17.284 INFO:tasks.workunit.client.0.smithi063.stdout:8/717: chown d9/c94 47314 1 2024-05-09T06:38:17.284 INFO:tasks.workunit.client.1.smithi193.stdout:1/270: chown d4/d16/de/c34 71803381 1 2024-05-09T06:38:17.284 INFO:tasks.workunit.client.0.smithi063.stdout:2/678: mknod dc/d6e/cc6 0 2024-05-09T06:38:17.284 INFO:tasks.workunit.client.1.smithi193.stdout:8/269: read d6/d1c/f12 [381498,61319] 0 2024-05-09T06:38:17.285 INFO:tasks.workunit.client.1.smithi193.stdout:8/270: chown d6/d1c/c15 42 1 2024-05-09T06:38:17.294 INFO:tasks.workunit.client.0.smithi063.stdout:0/689: creat dd/d1a/d2c/d51/d72/d9e/d23/d31/fd9 x:0 0 0 2024-05-09T06:38:17.295 INFO:tasks.workunit.client.1.smithi193.stdout:5/317: creat d0/d13/d2c/d37/d5b/d4e/d5d/f69 x:0 0 0 2024-05-09T06:38:17.296 INFO:tasks.workunit.client.0.smithi063.stdout:0/690: write dd/d1a/d2c/d51/d72/d9e/d23/d31/fd9 [385058,115393] 0 2024-05-09T06:38:17.297 INFO:tasks.workunit.client.0.smithi063.stdout:3/659: symlink d7/da/d2b/d66/d74/d7e/d97/ldf 0 2024-05-09T06:38:17.298 INFO:tasks.workunit.client.0.smithi063.stdout:0/691: write dd/d1a/d2c/d42/fcf [985059,17667] 0 2024-05-09T06:38:17.299 INFO:tasks.workunit.client.0.smithi063.stdout:1/582: unlink d9/d2a/d35/d24/cb6 0 2024-05-09T06:38:17.300 INFO:tasks.workunit.client.0.smithi063.stdout:5/639: read d8/d19/f48 [3362533,126144] 0 2024-05-09T06:38:17.302 INFO:tasks.workunit.client.1.smithi193.stdout:5/318: dwrite d0/d13/d2c/d37/d5b/d4e/d5d/f61 [0,4194304] 0 2024-05-09T06:38:17.304 INFO:tasks.workunit.client.1.smithi193.stdout:7/301: fsync d6/d8/f55 0 2024-05-09T06:38:17.304 INFO:tasks.workunit.client.0.smithi063.stdout:4/632: dread f6 [0,4194304] 0 2024-05-09T06:38:17.306 INFO:tasks.workunit.client.1.smithi193.stdout:6/287: creat d1/de/d11/d18/d36/d37/d4f/f66 x:0 0 0 2024-05-09T06:38:17.307 INFO:tasks.workunit.client.0.smithi063.stdout:8/718: chown d9/da/d18/dc7/dd3/df1/l101 50393026 1 2024-05-09T06:38:17.307 INFO:tasks.workunit.client.1.smithi193.stdout:7/302: rename d6 to d6/d8/d1a/d36/d4d/d60/d62 22 2024-05-09T06:38:17.308 INFO:tasks.workunit.client.1.smithi193.stdout:4/355: symlink d1a/l68 0 2024-05-09T06:38:17.308 INFO:tasks.workunit.client.1.smithi193.stdout:7/303: fsync d6/d8/d1a/f3c 0 2024-05-09T06:38:17.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/319: dread d0/d13/d2c/d37/d5b/d4e/d5d/f61 [0,4194304] 0 2024-05-09T06:38:17.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/320: fsync d0/d13/d2c/f3f 0 2024-05-09T06:38:17.310 INFO:tasks.workunit.client.1.smithi193.stdout:3/314: creat d1/db/f73 x:0 0 0 2024-05-09T06:38:17.311 INFO:tasks.workunit.client.1.smithi193.stdout:3/315: read d1/d44/f47 [3901373,6079] 0 2024-05-09T06:38:17.311 INFO:tasks.workunit.client.1.smithi193.stdout:5/321: chown d0/d13/d2c/d37/d5b/d4e/f58 46212 1 2024-05-09T06:38:17.312 INFO:tasks.workunit.client.0.smithi063.stdout:2/679: dread dc/f2a [4194304,4194304] 0 2024-05-09T06:38:17.312 INFO:tasks.workunit.client.0.smithi063.stdout:9/669: mkdir d0/d8/dea 0 2024-05-09T06:38:17.312 INFO:tasks.workunit.client.0.smithi063.stdout:9/670: write d0/f1f [1298283,74174] 0 2024-05-09T06:38:17.313 INFO:tasks.workunit.client.1.smithi193.stdout:1/271: chown d4/d14/d15/c33 11420 1 2024-05-09T06:38:17.315 INFO:tasks.workunit.client.1.smithi193.stdout:8/271: mknod d6/d1c/d1b/d29/c51 0 2024-05-09T06:38:17.316 INFO:tasks.workunit.client.0.smithi063.stdout:3/660: creat d7/d22/d28/dab/fe0 x:0 0 0 2024-05-09T06:38:17.318 INFO:tasks.workunit.client.0.smithi063.stdout:0/692: mkdir dd/d1a/d2c/d51/d72/d9e/d23/d31/dda 0 2024-05-09T06:38:17.319 INFO:tasks.workunit.client.1.smithi193.stdout:2/285: creat db/d25/d38/f6b x:0 0 0 2024-05-09T06:38:17.319 INFO:tasks.workunit.client.0.smithi063.stdout:7/608: dwrite d0/d15/d4a/d50/f2e [4194304,4194304] 0 2024-05-09T06:38:17.319 INFO:tasks.workunit.client.0.smithi063.stdout:6/652: truncate d2/d1f/d29/d3f/d5c/f9d 129235 0 2024-05-09T06:38:17.322 INFO:tasks.workunit.client.0.smithi063.stdout:1/583: truncate d9/d2a/d39/f5e 635917 0 2024-05-09T06:38:17.324 INFO:tasks.workunit.client.1.smithi193.stdout:0/317: creat d18/f61 x:0 0 0 2024-05-09T06:38:17.328 INFO:tasks.workunit.client.1.smithi193.stdout:2/286: dwrite db/d25/d38/d61/f65 [0,4194304] 0 2024-05-09T06:38:17.329 INFO:tasks.workunit.client.1.smithi193.stdout:2/287: stat db/d25/c43 0 2024-05-09T06:38:17.332 INFO:tasks.workunit.client.0.smithi063.stdout:2/680: mkdir dc/d6e/d7f/dc7 0 2024-05-09T06:38:17.333 INFO:tasks.workunit.client.1.smithi193.stdout:6/288: mknod d1/de/c67 0 2024-05-09T06:38:17.336 INFO:tasks.workunit.client.0.smithi063.stdout:3/661: chown f1 83 1 2024-05-09T06:38:17.339 INFO:tasks.workunit.client.0.smithi063.stdout:0/693: creat dd/d1a/d2c/d51/d72/fdb x:0 0 0 2024-05-09T06:38:17.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/316: fsync d1/db/d59/f37 0 2024-05-09T06:38:17.346 INFO:tasks.workunit.client.0.smithi063.stdout:5/640: symlink d8/d17/d18/d9c/ld1 0 2024-05-09T06:38:17.346 INFO:tasks.workunit.client.1.smithi193.stdout:5/322: mkdir d0/d13/d2c/d37/d2b/d6a 0 2024-05-09T06:38:17.348 INFO:tasks.workunit.client.1.smithi193.stdout:1/272: stat d4/d3c/f36 0 2024-05-09T06:38:17.348 INFO:tasks.workunit.client.0.smithi063.stdout:4/633: dwrite d7/da3/f3a [0,4194304] 0 2024-05-09T06:38:17.349 INFO:tasks.workunit.client.1.smithi193.stdout:1/273: read d4/d16/d35/f48 [41954,8139] 0 2024-05-09T06:38:17.350 INFO:tasks.workunit.client.1.smithi193.stdout:8/272: rmdir d6/d1c/d18/d2f 39 2024-05-09T06:38:17.350 INFO:tasks.workunit.client.1.smithi193.stdout:1/274: fdatasync d4/f37 0 2024-05-09T06:38:17.355 INFO:tasks.workunit.client.1.smithi193.stdout:0/318: readlink d18/d1d/l48 0 2024-05-09T06:38:17.357 INFO:tasks.workunit.client.0.smithi063.stdout:3/662: mknod d7/da/d2b/d66/dc0/da4/ce1 0 2024-05-09T06:38:17.357 INFO:tasks.workunit.client.1.smithi193.stdout:9/306: link dd/c4a dd/df/d15/d16/d26/c6d 0 2024-05-09T06:38:17.358 INFO:tasks.workunit.client.1.smithi193.stdout:8/273: dwrite d6/d1c/d1b/d29/f49 [0,4194304] 0 2024-05-09T06:38:17.359 INFO:tasks.workunit.client.0.smithi063.stdout:2/681: dread dc/f1c [0,4194304] 0 2024-05-09T06:38:17.359 INFO:tasks.workunit.client.1.smithi193.stdout:2/288: read f8 [9043445,6622] 0 2024-05-09T06:38:17.361 INFO:tasks.workunit.client.1.smithi193.stdout:2/289: chown db/d25/f48 341612 1 2024-05-09T06:38:17.361 INFO:tasks.workunit.client.1.smithi193.stdout:8/274: truncate d6/d1c/d1b/f4a 882433 0 2024-05-09T06:38:17.366 INFO:tasks.workunit.client.0.smithi063.stdout:7/609: write d0/d15/d49/d4f/d66/f71 [100052,60336] 0 2024-05-09T06:38:17.368 INFO:tasks.workunit.client.0.smithi063.stdout:1/584: mkdir d9/d46/dc7 0 2024-05-09T06:38:17.369 INFO:tasks.workunit.client.1.smithi193.stdout:6/289: creat d1/de/d11/f68 x:0 0 0 2024-05-09T06:38:17.369 INFO:tasks.workunit.client.0.smithi063.stdout:1/585: stat d9/d2a/d35/d1e/d9d/d3f/c6a 0 2024-05-09T06:38:17.369 INFO:tasks.workunit.client.0.smithi063.stdout:1/586: dread - d9/d2a/d35/d24/faa zero size 2024-05-09T06:38:17.370 INFO:tasks.workunit.client.0.smithi063.stdout:5/641: rename d8/d17/l73 to d8/d53/ld2 0 2024-05-09T06:38:17.371 INFO:tasks.workunit.client.0.smithi063.stdout:1/587: fdatasync d9/d2a/d3c/d4a/fb2 0 2024-05-09T06:38:17.372 INFO:tasks.workunit.client.0.smithi063.stdout:4/634: dread d7/df/d57/f91 [0,4194304] 0 2024-05-09T06:38:17.372 INFO:tasks.workunit.client.0.smithi063.stdout:6/653: dread d2/d4/f3c [0,4194304] 0 2024-05-09T06:38:17.374 INFO:tasks.workunit.client.1.smithi193.stdout:4/356: symlink d1a/d22/d5d/l69 0 2024-05-09T06:38:17.379 INFO:tasks.workunit.client.1.smithi193.stdout:5/323: creat d0/d13/d2c/d37/d5b/d4e/f6b x:0 0 0 2024-05-09T06:38:17.382 INFO:tasks.workunit.client.0.smithi063.stdout:9/671: getdents d0/d3b 0 2024-05-09T06:38:17.383 INFO:tasks.workunit.client.0.smithi063.stdout:0/694: truncate dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/f93 82565 0 2024-05-09T06:38:17.385 INFO:tasks.workunit.client.1.smithi193.stdout:0/319: fsync d18/f26 0 2024-05-09T06:38:17.388 INFO:tasks.workunit.client.1.smithi193.stdout:9/307: unlink l5 0 2024-05-09T06:38:17.390 INFO:tasks.workunit.client.1.smithi193.stdout:2/290: rename db/d4b/c4f to db/d4b/d56/c6c 0 2024-05-09T06:38:17.391 INFO:tasks.workunit.client.0.smithi063.stdout:4/635: dread d7/d50/fc4 [0,4194304] 0 2024-05-09T06:38:17.392 INFO:tasks.workunit.client.0.smithi063.stdout:3/663: symlink d7/d4c/dcf/le2 0 2024-05-09T06:38:17.392 INFO:tasks.workunit.client.1.smithi193.stdout:1/275: dread d4/d16/f10 [0,4194304] 0 2024-05-09T06:38:17.393 INFO:tasks.workunit.client.0.smithi063.stdout:3/664: chown d7/da/d2b/d66/dc0/da4/db0/ddd 130411 1 2024-05-09T06:38:17.395 INFO:tasks.workunit.client.1.smithi193.stdout:6/290: mkdir d1/d48/d69 0 2024-05-09T06:38:17.395 INFO:tasks.workunit.client.1.smithi193.stdout:1/276: dread - d4/d16/f42 zero size 2024-05-09T06:38:17.395 INFO:tasks.workunit.client.1.smithi193.stdout:1/277: fsync d4/d14/f2f 0 2024-05-09T06:38:17.399 INFO:tasks.workunit.client.0.smithi063.stdout:7/610: truncate d0/d15/d49/d4f/d66/fb7 2312136 0 2024-05-09T06:38:17.401 INFO:tasks.workunit.client.1.smithi193.stdout:1/278: dwrite d4/d16/f47 [0,4194304] 0 2024-05-09T06:38:17.403 INFO:tasks.workunit.client.1.smithi193.stdout:1/279: fsync d4/d16/d17/f29 0 2024-05-09T06:38:17.407 INFO:tasks.workunit.client.1.smithi193.stdout:0/320: unlink d18/d4c/d47/c4d 0 2024-05-09T06:38:17.408 INFO:tasks.workunit.client.0.smithi063.stdout:9/672: dwrite d0/d4/da/f21 [4194304,4194304] 0 2024-05-09T06:38:17.409 INFO:tasks.workunit.client.0.smithi063.stdout:9/673: fsync d0/f9f 0 2024-05-09T06:38:17.413 INFO:tasks.workunit.client.0.smithi063.stdout:4/636: creat d7/da3/da2/fc8 x:0 0 0 2024-05-09T06:38:17.414 INFO:tasks.workunit.client.0.smithi063.stdout:0/695: write dd/d1a/d2c/d51/d72/d9e/d23/d89/fc4 [3685261,5718] 0 2024-05-09T06:38:17.416 INFO:tasks.workunit.client.0.smithi063.stdout:4/637: chown d7/df/f21 1 1 2024-05-09T06:38:17.416 INFO:tasks.workunit.client.0.smithi063.stdout:3/665: rmdir d7/d70/d98/d5b 39 2024-05-09T06:38:17.416 INFO:tasks.workunit.client.1.smithi193.stdout:4/357: dread d1a/d1f/f2a [0,4194304] 0 2024-05-09T06:38:17.418 INFO:tasks.workunit.client.0.smithi063.stdout:0/696: chown dd/d1a/d2c/d51/d72/d9e/d23/d33/l38 3372 1 2024-05-09T06:38:17.418 INFO:tasks.workunit.client.0.smithi063.stdout:3/666: write d7/da/d21/f4d [1848343,44071] 0 2024-05-09T06:38:17.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/674: dwrite d0/f9f [0,4194304] 0 2024-05-09T06:38:17.421 INFO:tasks.workunit.client.0.smithi063.stdout:9/675: chown d0/d4/d26/f49 1628010308 1 2024-05-09T06:38:17.423 INFO:tasks.workunit.client.0.smithi063.stdout:1/588: truncate d9/d10/f1c 8043086 0 2024-05-09T06:38:17.425 INFO:tasks.workunit.client.1.smithi193.stdout:2/291: dwrite db/df/f4a [4194304,4194304] 0 2024-05-09T06:38:17.425 INFO:tasks.workunit.client.0.smithi063.stdout:1/589: chown d9/d2a/d3c/d4c/d83/d88/dc2 521 1 2024-05-09T06:38:17.426 INFO:tasks.workunit.client.1.smithi193.stdout:2/292: chown db/f39 0 1 2024-05-09T06:38:17.426 INFO:tasks.workunit.client.1.smithi193.stdout:7/304: getdents d6/dd/d5e 0 2024-05-09T06:38:17.427 INFO:tasks.workunit.client.1.smithi193.stdout:7/305: write d6/d17/f37 [5257067,128511] 0 2024-05-09T06:38:17.429 INFO:tasks.workunit.client.0.smithi063.stdout:0/697: dwrite dd/d1a/f35 [0,4194304] 0 2024-05-09T06:38:17.429 INFO:tasks.workunit.client.1.smithi193.stdout:7/306: write d6/dd/f3a [1328496,111533] 0 2024-05-09T06:38:17.433 INFO:tasks.workunit.client.0.smithi063.stdout:5/642: link d8/d35/c6e d8/d35/db6/dc2/cd3 0 2024-05-09T06:38:17.434 INFO:tasks.workunit.client.1.smithi193.stdout:3/317: link d1/c2 d1/d2d/d51/c74 0 2024-05-09T06:38:17.435 INFO:tasks.workunit.client.1.smithi193.stdout:3/318: dread - d1/db/d2f/f66 zero size 2024-05-09T06:38:17.436 INFO:tasks.workunit.client.1.smithi193.stdout:1/280: mkdir d4/d14/d15/d4f 0 2024-05-09T06:38:17.436 INFO:tasks.workunit.client.1.smithi193.stdout:1/281: read - d4/f37 zero size 2024-05-09T06:38:17.437 INFO:tasks.workunit.client.0.smithi063.stdout:0/698: dwrite dd/d1a/d50/f7c [0,4194304] 0 2024-05-09T06:38:17.439 INFO:tasks.workunit.client.0.smithi063.stdout:0/699: fdatasync dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fd0 0 2024-05-09T06:38:17.439 INFO:tasks.workunit.client.1.smithi193.stdout:0/321: unlink d18/d1d/c22 0 2024-05-09T06:38:17.441 INFO:tasks.workunit.client.0.smithi063.stdout:4/638: rmdir d7/da3/d28/d31/d7b 39 2024-05-09T06:38:17.445 INFO:tasks.workunit.client.0.smithi063.stdout:6/654: link d2/d1f/d29/d3f/f82 d2/d1f/fd6 0 2024-05-09T06:38:17.448 INFO:tasks.workunit.client.0.smithi063.stdout:7/611: write d0/d15/d4a/d50/f29 [1610709,70042] 0 2024-05-09T06:38:17.451 INFO:tasks.workunit.client.1.smithi193.stdout:2/293: mkdir db/d37/d6d 0 2024-05-09T06:38:17.452 INFO:tasks.workunit.client.0.smithi063.stdout:9/676: unlink d0/d8/d2d/f52 0 2024-05-09T06:38:17.454 INFO:tasks.workunit.client.0.smithi063.stdout:2/682: getdents dc/dd/d45/d5a/d6c/d8f/dbf 0 2024-05-09T06:38:17.455 INFO:tasks.workunit.client.0.smithi063.stdout:1/590: chown d9/d76/d94/c9c 37687401 1 2024-05-09T06:38:17.456 INFO:tasks.workunit.client.1.smithi193.stdout:1/282: rmdir d4/d16/d17/d1a/d3a/d3f 39 2024-05-09T06:38:17.458 INFO:tasks.workunit.client.1.smithi193.stdout:3/319: truncate d1/d2d/d51/f54 8682496 0 2024-05-09T06:38:17.459 INFO:tasks.workunit.client.1.smithi193.stdout:4/358: write f10 [2411842,64249] 0 2024-05-09T06:38:17.460 INFO:tasks.workunit.client.1.smithi193.stdout:3/320: dread d1/db/f34 [0,4194304] 0 2024-05-09T06:38:17.460 INFO:tasks.workunit.client.1.smithi193.stdout:0/322: symlink d18/d4c/d47/d54/l62 0 2024-05-09T06:38:17.462 INFO:tasks.workunit.client.1.smithi193.stdout:3/321: write d1/db/d59/f69 [623724,11934] 0 2024-05-09T06:38:17.462 INFO:tasks.workunit.client.1.smithi193.stdout:9/308: rename dd/d1e/d1f/d48/d55/d41/f4e to dd/df/f6e 0 2024-05-09T06:38:17.463 INFO:tasks.workunit.client.1.smithi193.stdout:0/323: rename d18/d4c/d47 to d18/d4c/d47/d5e/d63 22 2024-05-09T06:38:17.466 INFO:tasks.workunit.client.1.smithi193.stdout:6/291: creat d1/f6a x:0 0 0 2024-05-09T06:38:17.468 INFO:tasks.workunit.client.1.smithi193.stdout:7/307: mknod d6/d8/d1a/d36/d4d/d60/c63 0 2024-05-09T06:38:17.469 INFO:tasks.workunit.client.0.smithi063.stdout:9/677: symlink d0/d3b/da0/dcf/leb 0 2024-05-09T06:38:17.470 INFO:tasks.workunit.client.1.smithi193.stdout:5/324: getdents d0/d13/d2c/d37/d2b/d47 0 2024-05-09T06:38:17.471 INFO:tasks.workunit.client.0.smithi063.stdout:5/643: mknod d8/dc8/cd4 0 2024-05-09T06:38:17.474 INFO:tasks.workunit.client.0.smithi063.stdout:9/678: mknod d0/d4/d57/cec 0 2024-05-09T06:38:17.478 INFO:tasks.workunit.client.0.smithi063.stdout:1/591: mkdir d9/dc8 0 2024-05-09T06:38:17.478 INFO:tasks.workunit.client.1.smithi193.stdout:7/308: creat d6/d8/d1a/d36/d4d/f64 x:0 0 0 2024-05-09T06:38:17.482 INFO:tasks.workunit.client.1.smithi193.stdout:1/283: truncate d4/fc 3095814 0 2024-05-09T06:38:17.482 INFO:tasks.workunit.client.0.smithi063.stdout:9/679: truncate d0/d4/d26/d39/f6c 476566 0 2024-05-09T06:38:17.483 INFO:tasks.workunit.client.1.smithi193.stdout:5/325: write d0/d13/d2c/d38/f54 [1020455,32483] 0 2024-05-09T06:38:17.486 INFO:tasks.workunit.client.1.smithi193.stdout:7/309: dwrite d6/dd/f3f [0,4194304] 0 2024-05-09T06:38:17.486 INFO:tasks.workunit.client.1.smithi193.stdout:4/359: symlink d1a/d22/l6a 0 2024-05-09T06:38:17.487 INFO:tasks.workunit.client.1.smithi193.stdout:7/310: dread - d6/d8/d1a/d36/f5f zero size 2024-05-09T06:38:17.488 INFO:tasks.workunit.client.0.smithi063.stdout:1/592: creat d9/d2a/d3c/fc9 x:0 0 0 2024-05-09T06:38:17.489 INFO:tasks.workunit.client.0.smithi063.stdout:6/655: link d2/df/d59/l6d d2/df/d59/ld7 0 2024-05-09T06:38:17.490 INFO:tasks.workunit.client.1.smithi193.stdout:6/292: unlink d1/de/d11/d18/f1c 0 2024-05-09T06:38:17.493 INFO:tasks.workunit.client.0.smithi063.stdout:7/612: rename d0/d15/d49/d4f/c5d to d0/d15/d4a/cd4 0 2024-05-09T06:38:17.494 INFO:tasks.workunit.client.0.smithi063.stdout:0/700: link dd/d1a/d2c/d51/d72/d9e/c24 dd/d1a/cdc 0 2024-05-09T06:38:17.495 INFO:tasks.workunit.client.1.smithi193.stdout:1/284: rmdir d4/d16/d35 39 2024-05-09T06:38:17.496 INFO:tasks.workunit.client.0.smithi063.stdout:9/680: dread d0/d4/d26/f49 [0,4194304] 0 2024-05-09T06:38:17.496 INFO:tasks.workunit.client.1.smithi193.stdout:1/285: chown d4/d16/d17/d1a/f1f 12848288 1 2024-05-09T06:38:17.498 INFO:tasks.workunit.client.1.smithi193.stdout:5/326: mknod d0/d13/d2c/d37/d2b/d47/c6c 0 2024-05-09T06:38:17.499 INFO:tasks.workunit.client.1.smithi193.stdout:5/327: fsync d0/d13/d2c/d38/f54 0 2024-05-09T06:38:17.500 INFO:tasks.workunit.client.0.smithi063.stdout:2/683: rename dc/dd/d45/d5a/c93 to dc/d6e/cc8 0 2024-05-09T06:38:17.500 INFO:tasks.workunit.client.1.smithi193.stdout:5/328: dread - d0/d13/d2c/d37/f68 zero size 2024-05-09T06:38:17.502 INFO:tasks.workunit.client.1.smithi193.stdout:4/360: rename d1a/d1f/d4b/d59 to d1a/d6b 0 2024-05-09T06:38:17.503 INFO:tasks.workunit.client.0.smithi063.stdout:7/613: creat d0/d15/fd5 x:0 0 0 2024-05-09T06:38:17.506 INFO:tasks.workunit.client.1.smithi193.stdout:4/361: dread fb [0,4194304] 0 2024-05-09T06:38:17.508 INFO:tasks.workunit.client.1.smithi193.stdout:4/362: fsync d1a/d6b/f3c 0 2024-05-09T06:38:17.508 INFO:tasks.workunit.client.0.smithi063.stdout:9/681: symlink d0/d4/d57/dba/dd5/led 0 2024-05-09T06:38:17.510 INFO:tasks.workunit.client.1.smithi193.stdout:6/293: fdatasync d1/de/d11/d18/f2a 0 2024-05-09T06:38:17.512 INFO:tasks.workunit.client.1.smithi193.stdout:7/311: dwrite d6/f2e [0,4194304] 0 2024-05-09T06:38:17.512 INFO:tasks.workunit.client.1.smithi193.stdout:1/286: symlink d4/d3c/d3e/l50 0 2024-05-09T06:38:17.517 INFO:tasks.workunit.client.0.smithi063.stdout:5/644: rename d8/d7e to d8/d17/d75/dc9/dd5 0 2024-05-09T06:38:17.517 INFO:tasks.workunit.client.1.smithi193.stdout:4/363: dwrite d1a/d6b/f42 [0,4194304] 0 2024-05-09T06:38:17.518 INFO:tasks.workunit.client.1.smithi193.stdout:4/364: truncate d1a/f65 103610 0 2024-05-09T06:38:17.518 INFO:tasks.workunit.client.0.smithi063.stdout:6/656: write d2/df/f28 [4501499,33412] 0 2024-05-09T06:38:17.519 INFO:tasks.workunit.client.1.smithi193.stdout:4/365: fdatasync d1a/f1b 0 2024-05-09T06:38:17.520 INFO:tasks.workunit.client.1.smithi193.stdout:4/366: dread - d1a/d22/f31 zero size 2024-05-09T06:38:17.520 INFO:tasks.workunit.client.0.smithi063.stdout:2/684: symlink dc/d32/lc9 0 2024-05-09T06:38:17.525 INFO:tasks.workunit.client.0.smithi063.stdout:7/614: truncate d0/d15/d4a/d50/f96 415294 0 2024-05-09T06:38:17.527 INFO:tasks.workunit.client.0.smithi063.stdout:7/615: write d0/d15/d49/d4f/d66/d67/d6b/fc3 [328281,97383] 0 2024-05-09T06:38:17.528 INFO:tasks.workunit.client.0.smithi063.stdout:7/616: chown d0/d15/d4a/d50/l55 873833 1 2024-05-09T06:38:17.529 INFO:tasks.workunit.client.0.smithi063.stdout:0/701: getdents dd/d1a/d2c/d51/d72/d9e/d23/dbb 0 2024-05-09T06:38:17.530 INFO:tasks.workunit.client.0.smithi063.stdout:0/702: read - dd/d1a/d2c/d42/fd8 zero size 2024-05-09T06:38:17.530 INFO:tasks.workunit.client.0.smithi063.stdout:9/682: write d0/d3b/da0/dcf/fd2 [1417301,108468] 0 2024-05-09T06:38:17.532 INFO:tasks.workunit.client.1.smithi193.stdout:9/309: creat dd/d1e/d1f/f6f x:0 0 0 2024-05-09T06:38:17.534 INFO:tasks.workunit.client.0.smithi063.stdout:1/593: rename d9/d2a/d35/d1e/d9d/d3f/d59/d62/l86 to d9/d2a/d39/d5d/d75/lca 0 2024-05-09T06:38:17.539 INFO:tasks.workunit.client.0.smithi063.stdout:7/617: creat d0/d15/d4a/dc8/fd6 x:0 0 0 2024-05-09T06:38:17.541 INFO:tasks.workunit.client.1.smithi193.stdout:7/312: creat d6/d8/d1a/d36/f65 x:0 0 0 2024-05-09T06:38:17.543 INFO:tasks.workunit.client.0.smithi063.stdout:0/703: creat dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/fdd x:0 0 0 2024-05-09T06:38:17.545 INFO:tasks.workunit.client.0.smithi063.stdout:0/704: truncate dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/fbd 617996 0 2024-05-09T06:38:17.547 INFO:tasks.workunit.client.0.smithi063.stdout:6/657: rename d2/d1f/d73/d88/d98 to d2/df/d91/dd8 0 2024-05-09T06:38:17.549 INFO:tasks.workunit.client.0.smithi063.stdout:1/594: mknod d9/d2a/d3c/d4c/d83/ccb 0 2024-05-09T06:38:17.550 INFO:tasks.workunit.client.1.smithi193.stdout:5/329: mkdir d0/d6d 0 2024-05-09T06:38:17.555 INFO:tasks.workunit.client.0.smithi063.stdout:5/645: dread d8/f10 [0,4194304] 0 2024-05-09T06:38:17.555 INFO:tasks.workunit.client.0.smithi063.stdout:5/646: chown d8/c15 4131847 1 2024-05-09T06:38:17.557 INFO:tasks.workunit.client.1.smithi193.stdout:5/330: dwrite d0/d13/d2c/d37/d5b/d4e/f58 [0,4194304] 0 2024-05-09T06:38:17.559 INFO:tasks.workunit.client.1.smithi193.stdout:9/310: rename dd/df/f47 to dd/d1e/d66/d49/f70 0 2024-05-09T06:38:17.561 INFO:tasks.workunit.client.1.smithi193.stdout:6/294: fdatasync d1/de/d11/d18/d36/d37/f5b 0 2024-05-09T06:38:17.565 INFO:tasks.workunit.client.0.smithi063.stdout:7/618: dread d0/d15/d89/d98/fb6 [0,4194304] 0 2024-05-09T06:38:17.566 INFO:tasks.workunit.client.0.smithi063.stdout:7/619: truncate d0/d15/d89/fa9 1391147 0 2024-05-09T06:38:17.567 INFO:tasks.workunit.client.0.smithi063.stdout:7/620: truncate d0/d15/d4a/d68/f6f 898437 0 2024-05-09T06:38:17.568 INFO:tasks.workunit.client.0.smithi063.stdout:2/685: rename dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/d72 to dc/d36/d43/dca 0 2024-05-09T06:38:17.569 INFO:tasks.workunit.client.0.smithi063.stdout:7/621: chown d0/d15/d49/d54/d9f/lc5 7 1 2024-05-09T06:38:17.569 INFO:tasks.workunit.client.0.smithi063.stdout:9/683: truncate d0/d4/d26/d47/f2b 1649435 0 2024-05-09T06:38:17.570 INFO:tasks.workunit.client.0.smithi063.stdout:2/686: truncate dc/d32/d33/d4e/f88 1553851 0 2024-05-09T06:38:17.570 INFO:tasks.workunit.client.0.smithi063.stdout:9/684: fsync d0/d4/d57/f9b 0 2024-05-09T06:38:17.571 INFO:tasks.workunit.client.0.smithi063.stdout:2/687: dread - dc/dd/d45/d5a/d6c/fb4 zero size 2024-05-09T06:38:17.572 INFO:tasks.workunit.client.1.smithi193.stdout:4/367: unlink fb 0 2024-05-09T06:38:17.573 INFO:tasks.workunit.client.0.smithi063.stdout:0/705: write dd/d1a/d2c/d51/d72/d9e/d23/d46/f82 [771019,46325] 0 2024-05-09T06:38:17.576 INFO:tasks.workunit.client.0.smithi063.stdout:0/706: dread dd/fe [0,4194304] 0 2024-05-09T06:38:17.576 INFO:tasks.workunit.client.0.smithi063.stdout:0/707: dread - dd/d1a/d2c/d51/d72/fca zero size 2024-05-09T06:38:17.578 INFO:tasks.workunit.client.0.smithi063.stdout:0/708: chown dd/d1a/d2c/d51/d72/d9e/f64 7149 1 2024-05-09T06:38:17.585 INFO:tasks.workunit.client.0.smithi063.stdout:8/719: sync 2024-05-09T06:38:17.586 INFO:tasks.workunit.client.0.smithi063.stdout:8/720: dread - d9/da/f9a zero size 2024-05-09T06:38:17.603 INFO:tasks.workunit.client.0.smithi063.stdout:5/647: mkdir d8/d35/dd6 0 2024-05-09T06:38:17.607 INFO:tasks.workunit.client.1.smithi193.stdout:6/295: rename d1/d25/d2c/d3c/f5a to d1/d25/d4c/f6b 0 2024-05-09T06:38:17.609 INFO:tasks.workunit.client.0.smithi063.stdout:5/648: dwrite d8/d17/faa [0,4194304] 0 2024-05-09T06:38:17.613 INFO:tasks.workunit.client.0.smithi063.stdout:5/649: truncate d8/d17/d75/dc9/dd5/fbe 729244 0 2024-05-09T06:38:17.614 INFO:tasks.workunit.client.1.smithi193.stdout:1/287: creat d4/d14/f51 x:0 0 0 2024-05-09T06:38:17.614 INFO:tasks.workunit.client.0.smithi063.stdout:5/650: dread d8/f8e [0,4194304] 0 2024-05-09T06:38:17.618 INFO:tasks.workunit.client.1.smithi193.stdout:5/331: getdents d0/d13/d2c/d4d/d67 0 2024-05-09T06:38:17.622 INFO:tasks.workunit.client.1.smithi193.stdout:7/313: write d6/d17/f23 [2599205,115307] 0 2024-05-09T06:38:17.623 INFO:tasks.workunit.client.1.smithi193.stdout:7/314: chown d6/dd/d5e/f61 104 1 2024-05-09T06:38:17.623 INFO:tasks.workunit.client.1.smithi193.stdout:5/332: read d0/d13/d2c/d37/d2b/d47/f63 [302318,68945] 0 2024-05-09T06:38:17.623 INFO:tasks.workunit.client.1.smithi193.stdout:9/311: dread dd/d1e/d1f/f22 [0,4194304] 0 2024-05-09T06:38:17.625 INFO:tasks.workunit.client.1.smithi193.stdout:7/315: dread - d6/d8/d1a/d36/f65 zero size 2024-05-09T06:38:17.625 INFO:tasks.workunit.client.1.smithi193.stdout:9/312: chown dd/df/f1d 338943645 1 2024-05-09T06:38:17.626 INFO:tasks.workunit.client.1.smithi193.stdout:5/333: write d0/d13/d2c/d37/d5b/f21 [1203245,123127] 0 2024-05-09T06:38:17.628 INFO:tasks.workunit.client.1.smithi193.stdout:4/368: link d1a/d1f/f3e d1a/d67/f6c 0 2024-05-09T06:38:17.635 INFO:tasks.workunit.client.1.smithi193.stdout:9/313: mkdir dd/df/d15/d16/d26/d71 0 2024-05-09T06:38:17.636 INFO:tasks.workunit.client.0.smithi063.stdout:5/651: dread d8/f66 [0,4194304] 0 2024-05-09T06:38:17.638 INFO:tasks.workunit.client.1.smithi193.stdout:6/296: dwrite d1/de/d11/d18/f61 [0,4194304] 0 2024-05-09T06:38:17.640 INFO:tasks.workunit.client.1.smithi193.stdout:4/369: dread d1a/d1f/d4b/f52 [0,4194304] 0 2024-05-09T06:38:17.641 INFO:tasks.workunit.client.1.smithi193.stdout:7/316: dread d6/d8/f44 [0,4194304] 0 2024-05-09T06:38:17.641 INFO:tasks.workunit.client.1.smithi193.stdout:4/370: fdatasync d1a/d22/f56 0 2024-05-09T06:38:17.644 INFO:tasks.workunit.client.1.smithi193.stdout:5/334: symlink d0/d13/d2c/d37/d2b/d47/d60/l6e 0 2024-05-09T06:38:17.651 INFO:tasks.workunit.client.1.smithi193.stdout:4/371: mkdir d1a/d6d 0 2024-05-09T06:38:17.653 INFO:tasks.workunit.client.1.smithi193.stdout:9/314: dwrite dd/d1e/d1f/d48/f5a [0,4194304] 0 2024-05-09T06:38:17.654 INFO:tasks.workunit.client.1.smithi193.stdout:9/315: fdatasync dd/f11 0 2024-05-09T06:38:17.655 INFO:tasks.workunit.client.1.smithi193.stdout:9/316: dread - dd/d1e/d1f/f6f zero size 2024-05-09T06:38:17.655 INFO:tasks.workunit.client.1.smithi193.stdout:7/317: creat d6/d8/d29/d2f/f66 x:0 0 0 2024-05-09T06:38:17.657 INFO:tasks.workunit.client.1.smithi193.stdout:1/288: truncate d4/d16/f25 3102160 0 2024-05-09T06:38:17.657 INFO:tasks.workunit.client.1.smithi193.stdout:9/317: write dd/d1e/d1f/d48/f53 [1297420,75766] 0 2024-05-09T06:38:17.657 INFO:tasks.workunit.client.0.smithi063.stdout:3/667: sync 2024-05-09T06:38:17.657 INFO:tasks.workunit.client.0.smithi063.stdout:4/639: sync 2024-05-09T06:38:17.659 INFO:tasks.workunit.client.0.smithi063.stdout:3/668: write d7/da/d21/d30/f57 [1254626,9939] 0 2024-05-09T06:38:17.660 INFO:tasks.workunit.client.1.smithi193.stdout:4/372: mknod d1a/d1f/c6e 0 2024-05-09T06:38:17.660 INFO:tasks.workunit.client.1.smithi193.stdout:4/373: truncate d1a/f64 202855 0 2024-05-09T06:38:17.661 INFO:tasks.workunit.client.1.smithi193.stdout:4/374: chown d1a/d6b/f63 0 1 2024-05-09T06:38:17.662 INFO:tasks.workunit.client.0.smithi063.stdout:7/622: creat d0/d15/d89/d98/fd7 x:0 0 0 2024-05-09T06:38:17.665 INFO:tasks.workunit.client.1.smithi193.stdout:7/318: mknod d6/d8/d1a/d36/d4d/d60/c67 0 2024-05-09T06:38:17.668 INFO:tasks.workunit.client.1.smithi193.stdout:6/297: creat d1/de/d11/f6c x:0 0 0 2024-05-09T06:38:17.668 INFO:tasks.workunit.client.0.smithi063.stdout:7/623: dwrite d0/d15/f76 [0,4194304] 0 2024-05-09T06:38:17.669 INFO:tasks.workunit.client.0.smithi063.stdout:7/624: chown d0/d15/d49/f5e 0 1 2024-05-09T06:38:17.670 INFO:tasks.workunit.client.1.smithi193.stdout:1/289: rename d4/d16/d17/f43 to d4/d16/d17/d1a/f52 0 2024-05-09T06:38:17.670 INFO:tasks.workunit.client.0.smithi063.stdout:7/625: dread - d0/d15/d49/d54/fc1 zero size 2024-05-09T06:38:17.671 INFO:tasks.workunit.client.1.smithi193.stdout:1/290: dread - d4/d14/f51 zero size 2024-05-09T06:38:17.672 INFO:tasks.workunit.client.1.smithi193.stdout:4/375: mknod d1a/d22/d5d/c6f 0 2024-05-09T06:38:17.672 INFO:tasks.workunit.client.1.smithi193.stdout:7/319: dwrite d6/f2e [0,4194304] 0 2024-05-09T06:38:17.673 INFO:tasks.workunit.client.0.smithi063.stdout:1/595: creat d9/d2a/d2e/fcc x:0 0 0 2024-05-09T06:38:17.676 INFO:tasks.workunit.client.0.smithi063.stdout:0/709: unlink dd/d1a/d50/f6f 0 2024-05-09T06:38:17.677 INFO:tasks.workunit.client.0.smithi063.stdout:0/710: dread - dd/d1a/d2c/d51/d72/d9e/d23/d31/fc7 zero size 2024-05-09T06:38:17.678 INFO:tasks.workunit.client.1.smithi193.stdout:9/318: creat dd/d1e/d1f/d61/f72 x:0 0 0 2024-05-09T06:38:17.678 INFO:tasks.workunit.client.1.smithi193.stdout:9/319: read - dd/df/d31/f5f zero size 2024-05-09T06:38:17.681 INFO:tasks.workunit.client.1.smithi193.stdout:6/298: unlink d1/l14 0 2024-05-09T06:38:17.686 INFO:tasks.workunit.client.1.smithi193.stdout:4/376: rename d1a/d6b/f42 to d1a/d24/d30/d5b/f70 0 2024-05-09T06:38:17.687 INFO:tasks.workunit.client.0.smithi063.stdout:5/652: creat d8/d17/d75/dc9/fd7 x:0 0 0 2024-05-09T06:38:17.689 INFO:tasks.workunit.client.1.smithi193.stdout:6/299: chown d1/de/d11/d18/l39 672764 1 2024-05-09T06:38:17.691 INFO:tasks.workunit.client.0.smithi063.stdout:4/640: dread d7/da3/f65 [0,4194304] 0 2024-05-09T06:38:17.691 INFO:tasks.workunit.client.1.smithi193.stdout:1/291: fdatasync d4/d14/f2f 0 2024-05-09T06:38:17.693 INFO:tasks.workunit.client.0.smithi063.stdout:3/669: fsync d7/da/d2b/d66/d74/f92 0 2024-05-09T06:38:17.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/670: fsync d7/da/d21/d30/d44/d48/f6f 0 2024-05-09T06:38:17.698 INFO:tasks.workunit.client.1.smithi193.stdout:7/320: rename d6/d17/f42 to d6/dd/f68 0 2024-05-09T06:38:17.699 INFO:tasks.workunit.client.1.smithi193.stdout:7/321: chown d6/d8/d1a/d36/l49 529972627 1 2024-05-09T06:38:17.701 INFO:tasks.workunit.client.0.smithi063.stdout:9/685: symlink d0/lee 0 2024-05-09T06:38:17.702 INFO:tasks.workunit.client.0.smithi063.stdout:2/688: mknod dc/dd/d2d/ccb 0 2024-05-09T06:38:17.709 INFO:tasks.workunit.client.0.smithi063.stdout:1/596: rename d9/d76/d94/fa7 to d9/d2a/d3c/d4c/d83/d88/fcd 0 2024-05-09T06:38:17.709 INFO:tasks.workunit.client.1.smithi193.stdout:1/292: chown d4/d14/f2e 1 1 2024-05-09T06:38:17.710 INFO:tasks.workunit.client.0.smithi063.stdout:1/597: write d9/d2a/d35/d24/fa6 [879746,57074] 0 2024-05-09T06:38:17.716 INFO:tasks.workunit.client.1.smithi193.stdout:9/320: creat dd/f73 x:0 0 0 2024-05-09T06:38:17.717 INFO:tasks.workunit.client.1.smithi193.stdout:9/321: chown dd/df/d15/d16/d26/f62 1183054 1 2024-05-09T06:38:17.718 INFO:tasks.workunit.client.1.smithi193.stdout:9/322: chown dd/d1e/d66/d49/f51 1 1 2024-05-09T06:38:17.720 INFO:tasks.workunit.client.0.smithi063.stdout:4/641: dwrite d7/fb [0,4194304] 0 2024-05-09T06:38:17.722 INFO:tasks.workunit.client.0.smithi063.stdout:3/671: read d7/f16 [38173,40196] 0 2024-05-09T06:38:17.723 INFO:tasks.workunit.client.1.smithi193.stdout:7/322: mknod d6/d8/c69 0 2024-05-09T06:38:17.723 INFO:tasks.workunit.client.0.smithi063.stdout:6/658: getdents d2/df/d59/d99/d9f 0 2024-05-09T06:38:17.724 INFO:tasks.workunit.client.1.smithi193.stdout:6/300: creat d1/de/d11/d18/f6d x:0 0 0 2024-05-09T06:38:17.725 INFO:tasks.workunit.client.0.smithi063.stdout:6/659: chown d2/d1f/d29/d3f/f25 64 1 2024-05-09T06:38:17.725 INFO:tasks.workunit.client.0.smithi063.stdout:9/686: mkdir d0/d3b/da0/dcf/def 0 2024-05-09T06:38:17.732 INFO:tasks.workunit.client.0.smithi063.stdout:7/626: write d0/d15/f4e [2077766,58654] 0 2024-05-09T06:38:17.733 INFO:tasks.workunit.client.0.smithi063.stdout:5/653: rename d8/d17/d18/d9c/ld1 to d8/d19/d84/ld8 0 2024-05-09T06:38:17.734 INFO:tasks.workunit.client.1.smithi193.stdout:9/323: symlink dd/d1e/d1f/d43/l74 0 2024-05-09T06:38:17.735 INFO:tasks.workunit.client.1.smithi193.stdout:9/324: chown dd/df/d15/d16/d26/d37 363522793 1 2024-05-09T06:38:17.737 INFO:tasks.workunit.client.0.smithi063.stdout:1/598: fdatasync d9/f12 0 2024-05-09T06:38:17.738 INFO:tasks.workunit.client.1.smithi193.stdout:4/377: truncate f11 880281 0 2024-05-09T06:38:17.738 INFO:tasks.workunit.client.1.smithi193.stdout:4/378: readlink l4 0 2024-05-09T06:38:17.741 INFO:tasks.workunit.client.1.smithi193.stdout:6/301: rename d1/de/d11/d20/c2b to d1/de/d11/d18/d36/c6e 0 2024-05-09T06:38:17.743 INFO:tasks.workunit.client.1.smithi193.stdout:4/379: read d1a/d24/d30/f34 [3650009,81141] 0 2024-05-09T06:38:17.743 INFO:tasks.workunit.client.1.smithi193.stdout:1/293: getdents d4/d16 0 2024-05-09T06:38:17.744 INFO:tasks.workunit.client.1.smithi193.stdout:7/323: dread d6/dd/f47 [0,4194304] 0 2024-05-09T06:38:17.745 INFO:tasks.workunit.client.1.smithi193.stdout:9/325: mknod dd/df/c75 0 2024-05-09T06:38:17.746 INFO:tasks.workunit.client.0.smithi063.stdout:6/660: mkdir d2/df/d91/dd8/dd9 0 2024-05-09T06:38:17.747 INFO:tasks.workunit.client.1.smithi193.stdout:6/302: creat d1/d25/d2c/f6f x:0 0 0 2024-05-09T06:38:17.750 INFO:tasks.workunit.client.1.smithi193.stdout:4/380: fdatasync d1a/d1f/f33 0 2024-05-09T06:38:17.753 INFO:tasks.workunit.client.0.smithi063.stdout:3/672: dwrite d7/da/d2b/d66/dc0/fa2 [0,4194304] 0 2024-05-09T06:38:17.755 INFO:tasks.workunit.client.0.smithi063.stdout:8/721: getdents d9/da/d53/dbc/d102 0 2024-05-09T06:38:17.755 INFO:tasks.workunit.client.1.smithi193.stdout:4/381: chown d1a/c5c 423424 1 2024-05-09T06:38:17.755 INFO:tasks.workunit.client.1.smithi193.stdout:7/324: dwrite d6/dd/f3a [0,4194304] 0 2024-05-09T06:38:17.755 INFO:tasks.workunit.client.1.smithi193.stdout:4/382: chown d1a/f1b 2710 1 2024-05-09T06:38:17.759 INFO:tasks.workunit.client.1.smithi193.stdout:1/294: rename d4/d16/d17/d1a/d3a/c3b to d4/d3c/c53 0 2024-05-09T06:38:17.760 INFO:tasks.workunit.client.1.smithi193.stdout:1/295: fdatasync d4/f37 0 2024-05-09T06:38:17.764 INFO:tasks.workunit.client.1.smithi193.stdout:9/326: symlink dd/d1e/d66/d49/l76 0 2024-05-09T06:38:17.765 INFO:tasks.workunit.client.0.smithi063.stdout:0/711: truncate dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fc3 4551580 0 2024-05-09T06:38:17.769 INFO:tasks.workunit.client.0.smithi063.stdout:2/689: creat dc/d36/d43/d46/fcc x:0 0 0 2024-05-09T06:38:17.770 INFO:tasks.workunit.client.0.smithi063.stdout:6/661: rename d2/d1f/d29/d3f/d5c/d64/f8d to d2/d20/d26/d7f/dcf/fda 0 2024-05-09T06:38:17.771 INFO:tasks.workunit.client.0.smithi063.stdout:6/662: stat d2/df/d91/dd8 0 2024-05-09T06:38:17.772 INFO:tasks.workunit.client.1.smithi193.stdout:4/383: symlink d1a/d6d/l71 0 2024-05-09T06:38:17.772 INFO:tasks.workunit.client.1.smithi193.stdout:4/384: stat d1a/d24/d30/f32 0 2024-05-09T06:38:17.773 INFO:tasks.workunit.client.0.smithi063.stdout:3/673: truncate d7/da/d2b/f43 638201 0 2024-05-09T06:38:17.774 INFO:tasks.workunit.client.0.smithi063.stdout:3/674: chown d7/da/d21/d30/d44/d48/f6f 0 1 2024-05-09T06:38:17.775 INFO:tasks.workunit.client.0.smithi063.stdout:8/722: fsync d9/da/fb 0 2024-05-09T06:38:17.780 INFO:tasks.workunit.client.1.smithi193.stdout:9/327: fsync dd/d1e/d1f/f21 0 2024-05-09T06:38:17.780 INFO:tasks.workunit.client.1.smithi193.stdout:4/385: dwrite d1a/d24/f3d [0,4194304] 0 2024-05-09T06:38:17.781 INFO:tasks.workunit.client.1.smithi193.stdout:9/328: read - dd/f73 zero size 2024-05-09T06:38:17.783 INFO:tasks.workunit.client.0.smithi063.stdout:6/663: dread d2/d1f/d29/f47 [0,4194304] 0 2024-05-09T06:38:17.787 INFO:tasks.workunit.client.0.smithi063.stdout:0/712: fsync dd/d1a/f65 0 2024-05-09T06:38:17.789 INFO:tasks.workunit.client.1.smithi193.stdout:1/296: creat d4/d16/d17/f54 x:0 0 0 2024-05-09T06:38:17.789 INFO:tasks.workunit.client.1.smithi193.stdout:1/297: fdatasync d4/f37 0 2024-05-09T06:38:17.794 INFO:tasks.workunit.client.0.smithi063.stdout:9/687: rmdir d0/d3b/da0/dcf/def 0 2024-05-09T06:38:17.797 INFO:tasks.workunit.client.1.smithi193.stdout:4/386: creat d1a/d6b/f72 x:0 0 0 2024-05-09T06:38:17.798 INFO:tasks.workunit.client.1.smithi193.stdout:4/387: fdatasync d1a/d22/f31 0 2024-05-09T06:38:17.801 INFO:tasks.workunit.client.1.smithi193.stdout:4/388: dread d1a/d1f/f2a [0,4194304] 0 2024-05-09T06:38:17.802 INFO:tasks.workunit.client.1.smithi193.stdout:6/303: rename d1/de/d11/d18/f1e to d1/f70 0 2024-05-09T06:38:17.802 INFO:tasks.workunit.client.1.smithi193.stdout:6/304: fdatasync d1/f4d 0 2024-05-09T06:38:17.804 INFO:tasks.workunit.client.1.smithi193.stdout:4/389: dread d1a/d1f/d4b/f52 [0,4194304] 0 2024-05-09T06:38:17.810 INFO:tasks.workunit.client.1.smithi193.stdout:7/325: rename d6/dd/f3a to d6/dd/d53/f6a 0 2024-05-09T06:38:17.811 INFO:tasks.workunit.client.0.smithi063.stdout:8/723: creat d9/d41/dec/f105 x:0 0 0 2024-05-09T06:38:17.814 INFO:tasks.workunit.client.0.smithi063.stdout:6/664: creat d2/d4/d7c/fdb x:0 0 0 2024-05-09T06:38:17.815 INFO:tasks.workunit.client.1.smithi193.stdout:4/390: mkdir d1a/d24/d73 0 2024-05-09T06:38:17.816 INFO:tasks.workunit.client.0.smithi063.stdout:1/599: link d9/d10/f22 d9/d76/fce 0 2024-05-09T06:38:17.817 INFO:tasks.workunit.client.0.smithi063.stdout:1/600: chown d9/d2a/d3c/d4c/f92 107298 1 2024-05-09T06:38:17.819 INFO:tasks.workunit.client.0.smithi063.stdout:4/642: getdents d7/da3/da2 0 2024-05-09T06:38:17.820 INFO:tasks.workunit.client.1.smithi193.stdout:7/326: mkdir d6/dd/d5e/d6b 0 2024-05-09T06:38:17.821 INFO:tasks.workunit.client.1.smithi193.stdout:7/327: dread - d6/d8/d1a/d36/f65 zero size 2024-05-09T06:38:17.822 INFO:tasks.workunit.client.0.smithi063.stdout:9/688: unlink d0/d4/d7d/c90 0 2024-05-09T06:38:17.823 INFO:tasks.workunit.client.0.smithi063.stdout:9/689: write d0/d4/d26/f36 [2911854,48566] 0 2024-05-09T06:38:17.824 INFO:tasks.workunit.client.0.smithi063.stdout:2/690: mknod dc/dd/d45/d5a/d6a/ccd 0 2024-05-09T06:38:17.825 INFO:tasks.workunit.client.1.smithi193.stdout:4/391: mknod d1a/c74 0 2024-05-09T06:38:17.826 INFO:tasks.workunit.client.0.smithi063.stdout:5/654: getdents d8/d17/da3 0 2024-05-09T06:38:17.827 INFO:tasks.workunit.client.1.smithi193.stdout:4/392: write d1a/d6b/f63 [416019,6921] 0 2024-05-09T06:38:17.830 INFO:tasks.workunit.client.1.smithi193.stdout:1/298: write d4/f32 [2762940,40847] 0 2024-05-09T06:38:17.832 INFO:tasks.workunit.client.1.smithi193.stdout:1/299: write d4/d14/f44 [773783,60137] 0 2024-05-09T06:38:17.832 INFO:tasks.workunit.client.0.smithi063.stdout:3/675: unlink d7/f2f 0 2024-05-09T06:38:17.832 INFO:tasks.workunit.client.0.smithi063.stdout:5/655: dwrite d8/d17/d18/f2d [0,4194304] 0 2024-05-09T06:38:17.833 INFO:tasks.workunit.client.1.smithi193.stdout:7/328: dwrite d6/d8/f44 [0,4194304] 0 2024-05-09T06:38:17.834 INFO:tasks.workunit.client.1.smithi193.stdout:4/393: rename f6 to d1a/d22/d43/f75 0 2024-05-09T06:38:17.834 INFO:tasks.workunit.client.0.smithi063.stdout:3/676: write d7/d70/d98/d2d/f65 [983268,31412] 0 2024-05-09T06:38:17.836 INFO:tasks.workunit.client.1.smithi193.stdout:6/305: getdents d1/d25/d4c 0 2024-05-09T06:38:17.838 INFO:tasks.workunit.client.0.smithi063.stdout:3/677: readlink d7/d22/lb5 0 2024-05-09T06:38:17.838 INFO:tasks.workunit.client.0.smithi063.stdout:8/724: chown d9/da/d18/dc7/le3 4294 1 2024-05-09T06:38:17.839 INFO:tasks.workunit.client.1.smithi193.stdout:6/306: dread - d1/de/f38 zero size 2024-05-09T06:38:17.840 INFO:tasks.workunit.client.1.smithi193.stdout:6/307: chown d1/d25 0 1 2024-05-09T06:38:17.841 INFO:tasks.workunit.client.0.smithi063.stdout:7/627: link d0/d15/d49/d4f/d66/c7b d0/d15/d49/d54/d9f/d97/d9b/d9d/cd8 0 2024-05-09T06:38:17.842 INFO:tasks.workunit.client.0.smithi063.stdout:5/656: dwrite d8/d17/da2/fbb [0,4194304] 0 2024-05-09T06:38:17.843 INFO:tasks.workunit.client.1.smithi193.stdout:9/329: dwrite dd/d1e/d1f/f2f [0,4194304] 0 2024-05-09T06:38:17.844 INFO:tasks.workunit.client.1.smithi193.stdout:9/330: chown dd/d1e/d66/f45 405 1 2024-05-09T06:38:17.909 INFO:tasks.workunit.client.1.smithi193.stdout:4/394: stat f5 0 2024-05-09T06:38:17.911 INFO:tasks.workunit.client.0.smithi063.stdout:6/665: creat d2/d1f/d29/d3f/fdc x:0 0 0 2024-05-09T06:38:17.912 INFO:tasks.workunit.client.0.smithi063.stdout:6/666: read - d2/d1f/d29/fb1 zero size 2024-05-09T06:38:17.915 INFO:tasks.workunit.client.0.smithi063.stdout:1/601: creat d9/d2a/d39/d5d/d75/fcf x:0 0 0 2024-05-09T06:38:17.917 INFO:tasks.workunit.client.1.smithi193.stdout:6/308: mknod d1/d25/d2c/d3c/c71 0 2024-05-09T06:38:17.918 INFO:tasks.workunit.client.1.smithi193.stdout:6/309: stat d1/de/d11/f6c 0 2024-05-09T06:38:17.919 INFO:tasks.workunit.client.1.smithi193.stdout:9/331: mknod dd/d1e/d66/d49/c77 0 2024-05-09T06:38:17.919 INFO:tasks.workunit.client.1.smithi193.stdout:9/332: read - dd/f73 zero size 2024-05-09T06:38:17.920 INFO:tasks.workunit.client.1.smithi193.stdout:9/333: chown dd/df/c75 341589 1 2024-05-09T06:38:17.920 INFO:tasks.workunit.client.1.smithi193.stdout:9/334: readlink lc 0 2024-05-09T06:38:17.921 INFO:tasks.workunit.client.0.smithi063.stdout:2/691: read dc/d36/d43/d46/f70 [127858,67456] 0 2024-05-09T06:38:17.923 INFO:tasks.workunit.client.1.smithi193.stdout:1/300: truncate d4/d16/d17/d1a/f1e 1351517 0 2024-05-09T06:38:17.925 INFO:tasks.workunit.client.1.smithi193.stdout:9/335: dread dd/d1e/d66/d49/f51 [0,4194304] 0 2024-05-09T06:38:17.927 INFO:tasks.workunit.client.1.smithi193.stdout:6/310: dread - d1/de/d11/f5d zero size 2024-05-09T06:38:17.928 INFO:tasks.workunit.client.1.smithi193.stdout:4/395: mknod d1a/d24/d73/c76 0 2024-05-09T06:38:17.929 INFO:tasks.workunit.client.0.smithi063.stdout:3/678: dread d7/d70/d98/d5b/f8f [0,4194304] 0 2024-05-09T06:38:17.930 INFO:tasks.workunit.client.0.smithi063.stdout:7/628: creat d0/d15/d49/d54/d9f/fd9 x:0 0 0 2024-05-09T06:38:17.931 INFO:tasks.workunit.client.0.smithi063.stdout:6/667: write d2/f38 [1839729,66319] 0 2024-05-09T06:38:17.933 INFO:tasks.workunit.client.1.smithi193.stdout:9/336: creat dd/d1e/d1f/d48/d55/d41/d68/f78 x:0 0 0 2024-05-09T06:38:17.934 INFO:tasks.workunit.client.1.smithi193.stdout:9/337: fsync dd/f73 0 2024-05-09T06:38:17.936 INFO:tasks.workunit.client.1.smithi193.stdout:6/311: mknod d1/d25/d2c/d3c/d3e/d58/c72 0 2024-05-09T06:38:17.937 INFO:tasks.workunit.client.1.smithi193.stdout:4/396: creat d1a/d6b/f77 x:0 0 0 2024-05-09T06:38:17.941 INFO:tasks.workunit.client.0.smithi063.stdout:8/725: mkdir d9/d37/d106 0 2024-05-09T06:38:17.943 INFO:tasks.workunit.client.1.smithi193.stdout:1/301: creat d4/d16/d17/d1a/d3a/d3f/f55 x:0 0 0 2024-05-09T06:38:17.947 INFO:tasks.workunit.client.0.smithi063.stdout:7/629: mknod d0/d15/d4a/d68/cda 0 2024-05-09T06:38:17.947 INFO:tasks.workunit.client.1.smithi193.stdout:1/302: symlink d4/d16/de/l56 0 2024-05-09T06:38:17.948 INFO:tasks.workunit.client.0.smithi063.stdout:8/726: dwrite d9/da/d18/d24/fdb [0,4194304] 0 2024-05-09T06:38:17.949 INFO:tasks.workunit.client.1.smithi193.stdout:1/303: dread d4/f9 [0,4194304] 0 2024-05-09T06:38:17.954 INFO:tasks.workunit.client.0.smithi063.stdout:4/643: dwrite d7/df/f2b [0,4194304] 0 2024-05-09T06:38:17.954 INFO:tasks.workunit.client.1.smithi193.stdout:7/329: truncate d6/d8/d1a/d36/f3b 3429224 0 2024-05-09T06:38:17.954 INFO:tasks.workunit.client.0.smithi063.stdout:9/690: rename d0/d8/da6/db0/fb4 to d0/d4/d26/d39/d3e/ff0 0 2024-05-09T06:38:17.955 INFO:tasks.workunit.client.0.smithi063.stdout:4/644: stat d7/da3/f3a 0 2024-05-09T06:38:17.956 INFO:tasks.workunit.client.0.smithi063.stdout:2/692: mknod dc/d36/cce 0 2024-05-09T06:38:17.959 INFO:tasks.workunit.client.0.smithi063.stdout:3/679: symlink d7/d70/d98/d2d/d36/db7/le3 0 2024-05-09T06:38:17.962 INFO:tasks.workunit.client.0.smithi063.stdout:0/713: link dd/d1a/d2c/d51/d72/d9e/f80 dd/fde 0 2024-05-09T06:38:17.964 INFO:tasks.workunit.client.1.smithi193.stdout:6/312: creat d1/de/d11/f73 x:0 0 0 2024-05-09T06:38:17.965 INFO:tasks.workunit.client.1.smithi193.stdout:6/313: dread - d1/de/d11/f6c zero size 2024-05-09T06:38:17.965 INFO:tasks.workunit.client.0.smithi063.stdout:3/680: dwrite d7/da/d21/d30/fb9 [0,4194304] 0 2024-05-09T06:38:17.966 INFO:tasks.workunit.client.1.smithi193.stdout:6/314: chown d1/d25/d2c/d3c/d3e/c4a 24154 1 2024-05-09T06:38:17.967 INFO:tasks.workunit.client.1.smithi193.stdout:7/330: unlink d6/d8/d1a/d36/f5f 0 2024-05-09T06:38:17.969 INFO:tasks.workunit.client.1.smithi193.stdout:9/338: getdents dd/df/d15/d16/d26 0 2024-05-09T06:38:17.970 INFO:tasks.workunit.client.1.smithi193.stdout:9/339: chown dd/df/d15/d16/c1b 604936 1 2024-05-09T06:38:17.972 INFO:tasks.workunit.client.1.smithi193.stdout:6/315: symlink d1/d25/d2c/d3c/d3e/d58/l74 0 2024-05-09T06:38:17.974 INFO:tasks.workunit.client.0.smithi063.stdout:2/693: creat dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/fcf x:0 0 0 2024-05-09T06:38:17.975 INFO:tasks.workunit.client.0.smithi063.stdout:2/694: write dc/d32/d33/f61 [1490704,115166] 0 2024-05-09T06:38:17.976 INFO:tasks.workunit.client.0.smithi063.stdout:2/695: fsync dc/d32/d33/d4e/f88 0 2024-05-09T06:38:17.978 INFO:tasks.workunit.client.1.smithi193.stdout:9/340: rename dd/d1e/d3c/f5c to dd/df/d15/d16/d26/d37/f79 0 2024-05-09T06:38:17.979 INFO:tasks.workunit.client.0.smithi063.stdout:4/645: dread d7/d3d/f55 [0,4194304] 0 2024-05-09T06:38:17.982 INFO:tasks.workunit.client.0.smithi063.stdout:3/681: fsync d7/f16 0 2024-05-09T06:38:17.985 INFO:tasks.workunit.client.1.smithi193.stdout:9/341: mkdir dd/df/d15/d16/d7a 0 2024-05-09T06:38:17.986 INFO:tasks.workunit.client.1.smithi193.stdout:1/304: rmdir d4 39 2024-05-09T06:38:17.988 INFO:tasks.workunit.client.1.smithi193.stdout:6/316: write d1/de/d11/d2d/f34 [1793151,55934] 0 2024-05-09T06:38:17.989 INFO:tasks.workunit.client.1.smithi193.stdout:6/317: truncate d1/de/d11/f5d 242977 0 2024-05-09T06:38:17.995 INFO:tasks.workunit.client.0.smithi063.stdout:4/646: unlink f6 0 2024-05-09T06:38:17.997 INFO:tasks.workunit.client.1.smithi193.stdout:6/318: creat d1/d25/d2c/f75 x:0 0 0 2024-05-09T06:38:17.998 INFO:tasks.workunit.client.1.smithi193.stdout:6/319: stat d1/de/d11/d2d/d3a/d47/f5f 0 2024-05-09T06:38:18.000 INFO:tasks.workunit.client.0.smithi063.stdout:5/657: rename d8/c40 to d8/d19/cd9 0 2024-05-09T06:38:18.001 INFO:tasks.workunit.client.0.smithi063.stdout:6/668: write d2/d1f/d29/d3f/d5c/d64/f83 [861990,44749] 0 2024-05-09T06:38:18.001 INFO:tasks.workunit.client.1.smithi193.stdout:9/342: mknod dd/d1e/d1f/c7b 0 2024-05-09T06:38:18.003 INFO:tasks.workunit.client.0.smithi063.stdout:8/727: dwrite d9/da/f2f [0,4194304] 0 2024-05-09T06:38:18.010 INFO:tasks.workunit.client.1.smithi193.stdout:7/331: dread d6/d8/d1a/d36/f50 [0,4194304] 0 2024-05-09T06:38:18.012 INFO:tasks.workunit.client.1.smithi193.stdout:7/332: write d6/d8/d29/d2f/f4c [815691,99118] 0 2024-05-09T06:38:18.040 INFO:tasks.workunit.client.0.smithi063.stdout:9/691: creat d0/d4/d26/d39/d3e/ff1 x:0 0 0 2024-05-09T06:38:18.074 INFO:tasks.workunit.client.1.smithi193.stdout:6/320: rename d1/d48 to d1/d25/d2c/d76 0 2024-05-09T06:38:18.078 INFO:tasks.workunit.client.0.smithi063.stdout:1/602: rename d9/d46/d57 to d9/d2a/d3c/d4c/d83/d88/dc2/dd0 0 2024-05-09T06:38:18.081 INFO:tasks.workunit.client.1.smithi193.stdout:9/343: creat dd/df/d15/d16/d26/d37/f7c x:0 0 0 2024-05-09T06:38:18.082 INFO:tasks.workunit.client.0.smithi063.stdout:6/669: rmdir d2/d20/d26 39 2024-05-09T06:38:18.083 INFO:tasks.workunit.client.0.smithi063.stdout:6/670: fdatasync f1 0 2024-05-09T06:38:18.084 INFO:tasks.workunit.client.0.smithi063.stdout:6/671: read d2/d1f/d29/d3f/f25 [173265,70603] 0 2024-05-09T06:38:18.084 INFO:tasks.workunit.client.1.smithi193.stdout:8/275: sync 2024-05-09T06:38:18.085 INFO:tasks.workunit.client.0.smithi063.stdout:8/728: creat d9/da/d45/de4/f107 x:0 0 0 2024-05-09T06:38:18.085 INFO:tasks.workunit.client.1.smithi193.stdout:8/276: truncate d6/d1c/f1e 1683344 0 2024-05-09T06:38:18.086 INFO:tasks.workunit.client.0.smithi063.stdout:8/729: write f4 [5004677,72471] 0 2024-05-09T06:38:18.089 INFO:tasks.workunit.client.1.smithi193.stdout:6/321: readlink d1/de/d11/d18/l33 0 2024-05-09T06:38:18.090 INFO:tasks.workunit.client.1.smithi193.stdout:6/322: fsync d1/f6 0 2024-05-09T06:38:18.090 INFO:tasks.workunit.client.0.smithi063.stdout:6/672: dwrite d2/f38 [0,4194304] 0 2024-05-09T06:38:18.093 INFO:tasks.workunit.client.0.smithi063.stdout:6/673: dread d2/d4/f3c [4194304,4194304] 0 2024-05-09T06:38:18.094 INFO:tasks.workunit.client.0.smithi063.stdout:1/603: stat d9/d2a/d39/d5d/c9e 0 2024-05-09T06:38:18.095 INFO:tasks.workunit.client.0.smithi063.stdout:6/674: chown d2/d1f/d29/f84 23277749 1 2024-05-09T06:38:18.096 INFO:tasks.workunit.client.1.smithi193.stdout:9/344: creat dd/d1e/d3c/f7d x:0 0 0 2024-05-09T06:38:18.100 INFO:tasks.workunit.client.0.smithi063.stdout:1/604: dwrite d9/d10/f3e [0,4194304] 0 2024-05-09T06:38:18.118 INFO:tasks.workunit.client.1.smithi193.stdout:9/345: write dd/df/d15/d16/d26/d37/f79 [468644,24672] 0 2024-05-09T06:38:18.120 INFO:tasks.workunit.client.0.smithi063.stdout:6/675: chown d2/df/d91/cd1 2792 1 2024-05-09T06:38:18.125 INFO:tasks.workunit.client.1.smithi193.stdout:9/346: fdatasync dd/d1e/d1f/d48/f5a 0 2024-05-09T06:38:18.126 INFO:tasks.workunit.client.1.smithi193.stdout:9/347: truncate f0 6408168 0 2024-05-09T06:38:18.127 INFO:tasks.workunit.client.1.smithi193.stdout:8/277: creat d6/d1c/d1b/f52 x:0 0 0 2024-05-09T06:38:18.127 INFO:tasks.workunit.client.1.smithi193.stdout:9/348: chown dd/d1e/d1f/d48/d55/d41 8191144 1 2024-05-09T06:38:18.134 INFO:tasks.workunit.client.1.smithi193.stdout:9/349: write dd/d1e/d66/d49/f70 [1010607,92497] 0 2024-05-09T06:38:18.136 INFO:tasks.workunit.client.0.smithi063.stdout:6/676: dread d2/d20/d4a/f93 [0,4194304] 0 2024-05-09T06:38:18.138 INFO:tasks.workunit.client.1.smithi193.stdout:9/350: read dd/d1e/d66/d49/f51 [31775,3672] 0 2024-05-09T06:38:18.139 INFO:tasks.workunit.client.1.smithi193.stdout:9/351: write f0 [1641328,81075] 0 2024-05-09T06:38:18.140 INFO:tasks.workunit.client.0.smithi063.stdout:6/677: read - d2/d20/d26/fc4 zero size 2024-05-09T06:38:18.140 INFO:tasks.workunit.client.1.smithi193.stdout:8/278: dwrite d6/d1c/d18/d2f/f39 [0,4194304] 0 2024-05-09T06:38:18.144 INFO:tasks.workunit.client.1.smithi193.stdout:8/279: rmdir d6 39 2024-05-09T06:38:18.147 INFO:tasks.workunit.client.1.smithi193.stdout:9/352: dwrite dd/df/f10 [0,4194304] 0 2024-05-09T06:38:18.155 INFO:tasks.workunit.client.1.smithi193.stdout:8/280: getdents d6 0 2024-05-09T06:38:18.155 INFO:tasks.workunit.client.1.smithi193.stdout:8/281: read - d6/d1c/f34 zero size 2024-05-09T06:38:18.160 INFO:tasks.workunit.client.1.smithi193.stdout:8/282: dread d6/d1c/f1e [0,4194304] 0 2024-05-09T06:38:18.165 INFO:tasks.workunit.client.0.smithi063.stdout:1/605: creat d9/d2a/d35/d1e/fd1 x:0 0 0 2024-05-09T06:38:18.165 INFO:tasks.workunit.client.1.smithi193.stdout:8/283: read d6/f23 [1776364,96239] 0 2024-05-09T06:38:18.168 INFO:tasks.workunit.client.0.smithi063.stdout:1/606: dread - d9/d2a/d3c/d4c/d83/d88/dc2/dd0/f9b zero size 2024-05-09T06:38:18.173 INFO:tasks.workunit.client.0.smithi063.stdout:4/647: rmdir d7/df/d77 39 2024-05-09T06:38:18.174 INFO:tasks.workunit.client.0.smithi063.stdout:4/648: dread - d7/d3d/fab zero size 2024-05-09T06:38:18.176 INFO:tasks.workunit.client.0.smithi063.stdout:4/649: creat d7/d50/fc9 x:0 0 0 2024-05-09T06:38:18.179 INFO:tasks.workunit.client.0.smithi063.stdout:4/650: mkdir d7/d3d/dca 0 2024-05-09T06:38:18.180 INFO:tasks.workunit.client.0.smithi063.stdout:4/651: read - d7/df/d35/da5/fb9 zero size 2024-05-09T06:38:18.186 INFO:tasks.workunit.client.0.smithi063.stdout:4/652: link d7/c11 d7/df/d35/d5a/ccb 0 2024-05-09T06:38:18.190 INFO:tasks.workunit.client.0.smithi063.stdout:4/653: mkdir d7/da3/d28/d31/d32/d34/db4/dcc 0 2024-05-09T06:38:18.191 INFO:tasks.workunit.client.0.smithi063.stdout:3/682: sync 2024-05-09T06:38:18.194 INFO:tasks.workunit.client.0.smithi063.stdout:3/683: mkdir d7/da/de4 0 2024-05-09T06:38:18.196 INFO:tasks.workunit.client.0.smithi063.stdout:4/654: dwrite d7/fb [0,4194304] 0 2024-05-09T06:38:18.196 INFO:tasks.workunit.client.0.smithi063.stdout:3/684: write d7/da/d2b/d66/f95 [913720,129082] 0 2024-05-09T06:38:18.201 INFO:tasks.workunit.client.0.smithi063.stdout:4/655: creat d7/da3/d8b/fcd x:0 0 0 2024-05-09T06:38:18.207 INFO:tasks.workunit.client.0.smithi063.stdout:4/656: creat d7/da3/d28/d31/d32/fce x:0 0 0 2024-05-09T06:38:18.209 INFO:tasks.workunit.client.0.smithi063.stdout:3/685: rmdir d7/da/d2b/d66/dc0/da4/db0/ddd 0 2024-05-09T06:38:18.212 INFO:tasks.workunit.client.0.smithi063.stdout:3/686: stat d7/d22/d28/dab/lb8 0 2024-05-09T06:38:18.217 INFO:tasks.workunit.client.1.smithi193.stdout:7/333: truncate d6/d8/d1a/d36/f4e 1621117 0 2024-05-09T06:38:18.223 INFO:tasks.workunit.client.1.smithi193.stdout:6/323: write d1/de/d11/f5c [4041859,107085] 0 2024-05-09T06:38:18.224 INFO:tasks.workunit.client.1.smithi193.stdout:6/324: dread - d1/d25/d2c/f75 zero size 2024-05-09T06:38:18.228 INFO:tasks.workunit.client.1.smithi193.stdout:1/305: creat d4/f57 x:0 0 0 2024-05-09T06:38:18.237 INFO:tasks.workunit.client.1.smithi193.stdout:6/325: dread d1/de/d11/d2d/f34 [0,4194304] 0 2024-05-09T06:38:18.239 INFO:tasks.workunit.client.0.smithi063.stdout:7/630: rename d0/cba to d0/d15/d4a/d68/cdb 0 2024-05-09T06:38:18.239 INFO:tasks.workunit.client.1.smithi193.stdout:1/306: dread d4/d16/de/f12 [0,4194304] 0 2024-05-09T06:38:18.241 INFO:tasks.workunit.client.1.smithi193.stdout:1/307: truncate d4/d16/d17/f29 5317945 0 2024-05-09T06:38:18.241 INFO:tasks.workunit.client.1.smithi193.stdout:6/326: rename d1/d25/d2c/d3c/d3e/d58/l74 to d1/d25/d2c/d3c/d3e/d58/l77 0 2024-05-09T06:38:18.242 INFO:tasks.workunit.client.1.smithi193.stdout:1/308: read d4/d14/d15/f18 [3070925,42025] 0 2024-05-09T06:38:18.243 INFO:tasks.workunit.client.0.smithi063.stdout:0/714: rename dd/d1a/d2c/d51/d72/d9e/d23/d31/fd9 to dd/d1a/d2c/d51/d72/d9e/d23/d46/fdf 0 2024-05-09T06:38:18.246 INFO:tasks.workunit.client.1.smithi193.stdout:6/327: fsync d1/de/d11/d18/f24 0 2024-05-09T06:38:18.249 INFO:tasks.workunit.client.0.smithi063.stdout:7/631: creat d0/d15/d49/d54/d9f/d97/fdc x:0 0 0 2024-05-09T06:38:18.250 INFO:tasks.workunit.client.0.smithi063.stdout:2/696: rename dc/d36/d43/dca to dc/dd/d2d/d60/dd0 0 2024-05-09T06:38:18.252 INFO:tasks.workunit.client.1.smithi193.stdout:8/284: mkdir d6/d1c/d53 0 2024-05-09T06:38:18.254 INFO:tasks.workunit.client.0.smithi063.stdout:5/658: rename d8/d35/da9 to d8/dc8/dda 0 2024-05-09T06:38:18.256 INFO:tasks.workunit.client.0.smithi063.stdout:2/697: dread - dc/d36/d43/d46/f6b zero size 2024-05-09T06:38:18.257 INFO:tasks.workunit.client.1.smithi193.stdout:6/328: getdents d1/de/d11/d2d/d3a 0 2024-05-09T06:38:18.261 INFO:tasks.workunit.client.0.smithi063.stdout:5/659: creat d8/d17/d75/fdb x:0 0 0 2024-05-09T06:38:18.262 INFO:tasks.workunit.client.1.smithi193.stdout:8/285: dwrite d6/f23 [0,4194304] 0 2024-05-09T06:38:18.262 INFO:tasks.workunit.client.1.smithi193.stdout:9/353: rename dd/df/d15/d16/d26 to dd/df/d31/d7e 0 2024-05-09T06:38:18.265 INFO:tasks.workunit.client.0.smithi063.stdout:2/698: rmdir dc/d36/d43/d46/d63/d79 39 2024-05-09T06:38:18.267 INFO:tasks.workunit.client.1.smithi193.stdout:7/334: rename d6/d8/d29/d2f/f66 to d6/dd/f6c 0 2024-05-09T06:38:18.267 INFO:tasks.workunit.client.1.smithi193.stdout:6/329: dread d1/f70 [0,4194304] 0 2024-05-09T06:38:18.269 INFO:tasks.workunit.client.1.smithi193.stdout:6/330: write d1/d25/d2c/d3c/d3e/f3f [1969131,34853] 0 2024-05-09T06:38:18.270 INFO:tasks.workunit.client.1.smithi193.stdout:8/286: unlink d6/d1c/d1b/d29/f4b 0 2024-05-09T06:38:18.275 INFO:tasks.workunit.client.0.smithi063.stdout:9/692: rename d0/d4/d26/d47/f1d to d0/d4/d26/d39/d3e/ff2 0 2024-05-09T06:38:18.278 INFO:tasks.workunit.client.0.smithi063.stdout:9/693: fdatasync d0/d8/da6/db0/db5/fe9 0 2024-05-09T06:38:18.279 INFO:tasks.workunit.client.1.smithi193.stdout:7/335: rename d6/d8/d1a/d36/d4d/d60/c63 to d6/d8/d1a/d36/c6d 0 2024-05-09T06:38:18.279 INFO:tasks.workunit.client.0.smithi063.stdout:5/660: dread d8/d53/fab [0,4194304] 0 2024-05-09T06:38:18.279 INFO:tasks.workunit.client.0.smithi063.stdout:2/699: getdents dc/dd/d45/d5a/d6c/d8c/d9a 0 2024-05-09T06:38:18.280 INFO:tasks.workunit.client.0.smithi063.stdout:3/687: dwrite d7/d4c/d25/f72 [0,4194304] 0 2024-05-09T06:38:18.282 INFO:tasks.workunit.client.0.smithi063.stdout:3/688: readlink d7/da/d21/d30/d44/d48/l7a 0 2024-05-09T06:38:18.282 INFO:tasks.workunit.client.0.smithi063.stdout:8/730: rename d9/da/d18/dc7/d58/ff9 to d9/d41/f108 0 2024-05-09T06:38:18.283 INFO:tasks.workunit.client.1.smithi193.stdout:8/287: symlink d6/d1c/d27/l54 0 2024-05-09T06:38:18.285 INFO:tasks.workunit.client.1.smithi193.stdout:9/354: link dd/df/d31/f4c dd/d1e/d66/d49/f7f 0 2024-05-09T06:38:18.289 INFO:tasks.workunit.client.0.smithi063.stdout:5/661: mknod d8/d17/d75/cdc 0 2024-05-09T06:38:18.289 INFO:tasks.workunit.client.0.smithi063.stdout:3/689: dwrite d7/da/d2b/d66/dc0/fb3 [0,4194304] 0 2024-05-09T06:38:18.294 INFO:tasks.workunit.client.1.smithi193.stdout:9/355: creat dd/d1e/d66/f80 x:0 0 0 2024-05-09T06:38:18.294 INFO:tasks.workunit.client.0.smithi063.stdout:7/632: write d0/d15/d49/f65 [1865695,87246] 0 2024-05-09T06:38:18.298 INFO:tasks.workunit.client.1.smithi193.stdout:9/356: dread dd/df/f10 [0,4194304] 0 2024-05-09T06:38:18.303 INFO:tasks.workunit.client.0.smithi063.stdout:9/694: creat d0/d8/dea/ff3 x:0 0 0 2024-05-09T06:38:18.303 INFO:tasks.workunit.client.0.smithi063.stdout:9/695: write d0/d4/d57/f9b [248601,94010] 0 2024-05-09T06:38:18.308 INFO:tasks.workunit.client.0.smithi063.stdout:2/700: getdents dc/d36/d43/d46/d49/dbb 0 2024-05-09T06:38:18.309 INFO:tasks.workunit.client.1.smithi193.stdout:9/357: mknod dd/d1e/d1f/d43/c81 0 2024-05-09T06:38:18.311 INFO:tasks.workunit.client.0.smithi063.stdout:9/696: dwrite d0/d4/d7d/f82 [0,4194304] 0 2024-05-09T06:38:18.313 INFO:tasks.workunit.client.1.smithi193.stdout:6/331: dwrite d1/d25/d2c/d3c/d3e/f43 [0,4194304] 0 2024-05-09T06:38:18.314 INFO:tasks.workunit.client.0.smithi063.stdout:5/662: dread d8/d17/d75/fcf [0,4194304] 0 2024-05-09T06:38:18.315 INFO:tasks.workunit.client.1.smithi193.stdout:7/336: write d6/dd/f47 [5086334,47411] 0 2024-05-09T06:38:18.318 INFO:tasks.workunit.client.0.smithi063.stdout:3/690: symlink d7/d70/d98/le5 0 2024-05-09T06:38:18.321 INFO:tasks.workunit.client.0.smithi063.stdout:8/731: write d9/da/d18/f22 [731136,553] 0 2024-05-09T06:38:18.323 INFO:tasks.workunit.client.0.smithi063.stdout:8/732: truncate d9/da/d53/db2/fe0 54334 0 2024-05-09T06:38:18.327 INFO:tasks.workunit.client.0.smithi063.stdout:6/678: rename d2/df/d91/fb6 to d2/fdd 0 2024-05-09T06:38:18.328 INFO:tasks.workunit.client.0.smithi063.stdout:6/679: stat d2/fd 0 2024-05-09T06:38:18.328 INFO:tasks.workunit.client.1.smithi193.stdout:8/288: dwrite d6/d1c/d1b/d29/f4f [0,4194304] 0 2024-05-09T06:38:18.330 INFO:tasks.workunit.client.1.smithi193.stdout:9/358: rename dd/d1e/d1f/d61/f72 to dd/f82 0 2024-05-09T06:38:18.330 INFO:tasks.workunit.client.0.smithi063.stdout:8/733: dwrite d9/da/d53/db2/fb5 [0,4194304] 0 2024-05-09T06:38:18.330 INFO:tasks.workunit.client.1.smithi193.stdout:8/289: write d6/d1c/d27/f43 [98389,101418] 0 2024-05-09T06:38:18.331 INFO:tasks.workunit.client.1.smithi193.stdout:9/359: chown dd/d1e/d66/d49/f51 21029 1 2024-05-09T06:38:18.333 INFO:tasks.workunit.client.1.smithi193.stdout:9/360: write dd/d1e/d1f/d48/d55/d41/d68/f6b [2441361,2176] 0 2024-05-09T06:38:18.333 INFO:tasks.workunit.client.1.smithi193.stdout:9/361: readlink l8 0 2024-05-09T06:38:18.333 INFO:tasks.workunit.client.0.smithi063.stdout:8/734: stat d9/da/d18/dc7/dd3 0 2024-05-09T06:38:18.333 INFO:tasks.workunit.client.1.smithi193.stdout:9/362: readlink lb 0 2024-05-09T06:38:18.334 INFO:tasks.workunit.client.0.smithi063.stdout:7/633: mkdir d0/d15/d49/d4f/d66/ddd 0 2024-05-09T06:38:18.334 INFO:tasks.workunit.client.1.smithi193.stdout:9/363: write dd/f73 [630017,109322] 0 2024-05-09T06:38:18.336 INFO:tasks.workunit.client.0.smithi063.stdout:9/697: creat d0/d4/d57/ff4 x:0 0 0 2024-05-09T06:38:18.338 INFO:tasks.workunit.client.0.smithi063.stdout:5/663: symlink d8/d17/ldd 0 2024-05-09T06:38:18.340 INFO:tasks.workunit.client.1.smithi193.stdout:8/290: mkdir d6/d1c/d18/d2f/d40/d55 0 2024-05-09T06:38:18.341 INFO:tasks.workunit.client.1.smithi193.stdout:7/337: write d6/d8/f55 [444936,35126] 0 2024-05-09T06:38:18.342 INFO:tasks.workunit.client.0.smithi063.stdout:6/680: dwrite d2/d1f/d29/d3b/fac [0,4194304] 0 2024-05-09T06:38:18.345 INFO:tasks.workunit.client.0.smithi063.stdout:1/607: rename d9/d2a/d35/d1e/d9d/d3f/fb4 to d9/d2a/d3c/d4c/dc3/fd2 0 2024-05-09T06:38:18.345 INFO:tasks.workunit.client.0.smithi063.stdout:4/657: rename d7/da3/d28/d31 to d7/da3/d28/d31/d71/d98/dcf 22 2024-05-09T06:38:18.345 INFO:tasks.workunit.client.0.smithi063.stdout:7/634: dwrite d0/d15/d4a/d50/f2e [4194304,4194304] 0 2024-05-09T06:38:18.348 INFO:tasks.workunit.client.1.smithi193.stdout:9/364: unlink f0 0 2024-05-09T06:38:18.351 INFO:tasks.workunit.client.1.smithi193.stdout:6/332: truncate d1/de/d11/d20/f27 28592 0 2024-05-09T06:38:18.352 INFO:tasks.workunit.client.1.smithi193.stdout:6/333: dread - d1/de/d11/d2d/d3a/d47/f5f zero size 2024-05-09T06:38:18.357 INFO:tasks.workunit.client.0.smithi063.stdout:4/658: dread d7/d3d/f49 [0,4194304] 0 2024-05-09T06:38:18.358 INFO:tasks.workunit.client.0.smithi063.stdout:4/659: write d7/df/d35/d5a/dad/fc3 [34708,80958] 0 2024-05-09T06:38:18.372 INFO:tasks.workunit.client.1.smithi193.stdout:9/365: creat dd/df/d31/d7e/d37/f83 x:0 0 0 2024-05-09T06:38:18.373 INFO:tasks.workunit.client.1.smithi193.stdout:7/338: dwrite d6/d8/d1a/d33/f51 [0,4194304] 0 2024-05-09T06:38:18.376 INFO:tasks.workunit.client.1.smithi193.stdout:6/334: rmdir d1/d25/d2c/d3c/d3e/d58 39 2024-05-09T06:38:18.381 INFO:tasks.workunit.client.0.smithi063.stdout:9/698: mkdir d0/d4/d57/d60/df5 0 2024-05-09T06:38:18.381 INFO:tasks.workunit.client.0.smithi063.stdout:9/699: chown d0 89096 1 2024-05-09T06:38:18.382 INFO:tasks.workunit.client.0.smithi063.stdout:1/608: dread d9/d2a/d35/d1e/f27 [0,4194304] 0 2024-05-09T06:38:18.384 INFO:tasks.workunit.client.1.smithi193.stdout:6/335: write d1/de/d11/d2d/f34 [2647347,54890] 0 2024-05-09T06:38:18.385 INFO:tasks.workunit.client.1.smithi193.stdout:9/366: creat dd/df/d31/d7e/d71/f84 x:0 0 0 2024-05-09T06:38:18.388 INFO:tasks.workunit.client.0.smithi063.stdout:5/664: mkdir d8/d19/d30/d74/d77/dde 0 2024-05-09T06:38:18.388 INFO:tasks.workunit.client.1.smithi193.stdout:9/367: chown dd/df/d31/d7e/d37/l46 1623018 1 2024-05-09T06:38:18.389 INFO:tasks.workunit.client.1.smithi193.stdout:8/291: dwrite d6/d1c/f1e [0,4194304] 0 2024-05-09T06:38:18.389 INFO:tasks.workunit.client.0.smithi063.stdout:4/660: write d7/da3/da2/fb8 [502019,5140] 0 2024-05-09T06:38:18.391 INFO:tasks.workunit.client.0.smithi063.stdout:2/701: rename dc/dd/d2d/c5d to dc/dd/d45/cd1 0 2024-05-09T06:38:18.393 INFO:tasks.workunit.client.1.smithi193.stdout:7/339: creat d6/d8/f6e x:0 0 0 2024-05-09T06:38:18.394 INFO:tasks.workunit.client.1.smithi193.stdout:7/340: write d6/dd/f68 [2293337,113418] 0 2024-05-09T06:38:18.396 INFO:tasks.workunit.client.0.smithi063.stdout:7/635: truncate d0/d15/d49/d4f/d66/fb7 2851783 0 2024-05-09T06:38:18.396 INFO:tasks.workunit.client.1.smithi193.stdout:6/336: mkdir d1/de/d11/d20/d78 0 2024-05-09T06:38:18.398 INFO:tasks.workunit.client.0.smithi063.stdout:8/735: creat d9/da/d18/d24/f109 x:0 0 0 2024-05-09T06:38:18.398 INFO:tasks.workunit.client.1.smithi193.stdout:9/368: creat dd/d1e/d1f/d61/f85 x:0 0 0 2024-05-09T06:38:18.399 INFO:tasks.workunit.client.0.smithi063.stdout:8/736: fdatasync d9/da/d34/fac 0 2024-05-09T06:38:18.403 INFO:tasks.workunit.client.1.smithi193.stdout:7/341: rename d6/d8/d1a/d33/c48 to d6/dd/d53/c6f 0 2024-05-09T06:38:18.404 INFO:tasks.workunit.client.0.smithi063.stdout:3/691: link d7/da/d2b/d66/d74/d7e/d97/ldf d7/d70/d98/d2d/le6 0 2024-05-09T06:38:18.404 INFO:tasks.workunit.client.1.smithi193.stdout:7/342: chown d6/dd/f47 93645 1 2024-05-09T06:38:18.406 INFO:tasks.workunit.client.1.smithi193.stdout:6/337: symlink d1/de/d11/d18/d36/d37/d4f/l79 0 2024-05-09T06:38:18.407 INFO:tasks.workunit.client.0.smithi063.stdout:5/665: dread - d8/d53/dca/f76 zero size 2024-05-09T06:38:18.407 INFO:tasks.workunit.client.1.smithi193.stdout:8/292: mkdir d6/d1c/d53/d56 0 2024-05-09T06:38:18.409 INFO:tasks.workunit.client.0.smithi063.stdout:4/661: fsync d7/da3/f88 0 2024-05-09T06:38:18.409 INFO:tasks.workunit.client.1.smithi193.stdout:8/293: write d6/d1c/d1b/d29/f41 [970203,75082] 0 2024-05-09T06:38:18.410 INFO:tasks.workunit.client.0.smithi063.stdout:3/692: dwrite d7/da/d2b/d66/dc0/fa2 [0,4194304] 0 2024-05-09T06:38:18.411 INFO:tasks.workunit.client.1.smithi193.stdout:8/294: write d6/d1c/d1b/d3a/f47 [699145,129766] 0 2024-05-09T06:38:18.412 INFO:tasks.workunit.client.1.smithi193.stdout:8/295: stat d6/d1c/d18/f37 0 2024-05-09T06:38:18.413 INFO:tasks.workunit.client.0.smithi063.stdout:6/681: rename d2/d1f/d29/d3b/fac to d2/d20/d26/fde 0 2024-05-09T06:38:18.414 INFO:tasks.workunit.client.1.smithi193.stdout:7/343: mkdir d6/dd/d70 0 2024-05-09T06:38:18.416 INFO:tasks.workunit.client.0.smithi063.stdout:1/609: write d9/d2a/d35/f44 [897090,102613] 0 2024-05-09T06:38:18.417 INFO:tasks.workunit.client.1.smithi193.stdout:6/338: creat d1/de/d11/d2d/d3a/d47/f7a x:0 0 0 2024-05-09T06:38:18.419 INFO:tasks.workunit.client.0.smithi063.stdout:3/693: dwrite d7/d4c/d25/f72 [0,4194304] 0 2024-05-09T06:38:18.420 INFO:tasks.workunit.client.0.smithi063.stdout:3/694: chown d7/d4c/d25/lcb 433505 1 2024-05-09T06:38:18.423 INFO:tasks.workunit.client.1.smithi193.stdout:7/344: dwrite d6/d8/d29/d2f/f4f [0,4194304] 0 2024-05-09T06:38:18.424 INFO:tasks.workunit.client.1.smithi193.stdout:0/324: sync 2024-05-09T06:38:18.424 INFO:tasks.workunit.client.1.smithi193.stdout:4/397: sync 2024-05-09T06:38:18.424 INFO:tasks.workunit.client.1.smithi193.stdout:3/322: sync 2024-05-09T06:38:18.424 INFO:tasks.workunit.client.1.smithi193.stdout:2/294: sync 2024-05-09T06:38:18.424 INFO:tasks.workunit.client.1.smithi193.stdout:5/335: sync 2024-05-09T06:38:18.424 INFO:tasks.workunit.client.0.smithi063.stdout:9/700: dread d0/d4/d26/d39/f53 [0,4194304] 0 2024-05-09T06:38:18.426 INFO:tasks.workunit.client.1.smithi193.stdout:7/345: write d6/d17/f23 [2251435,88448] 0 2024-05-09T06:38:18.428 INFO:tasks.workunit.client.0.smithi063.stdout:8/737: fsync d9/da/f9f 0 2024-05-09T06:38:18.431 INFO:tasks.workunit.client.0.smithi063.stdout:3/695: dread d7/da/d2b/d66/d74/d7e/fca [0,4194304] 0 2024-05-09T06:38:18.433 INFO:tasks.workunit.client.1.smithi193.stdout:6/339: dread - d1/d25/d2c/d3c/d3e/d58/f62 zero size 2024-05-09T06:38:18.433 INFO:tasks.workunit.client.0.smithi063.stdout:9/701: dwrite d0/d4/d26/d39/d3e/fce [0,4194304] 0 2024-05-09T06:38:18.434 INFO:tasks.workunit.client.0.smithi063.stdout:4/662: creat d7/da3/d28/fd0 x:0 0 0 2024-05-09T06:38:18.436 INFO:tasks.workunit.client.0.smithi063.stdout:4/663: write f5 [458241,38377] 0 2024-05-09T06:38:18.436 INFO:tasks.workunit.client.1.smithi193.stdout:4/398: creat d1a/d24/f78 x:0 0 0 2024-05-09T06:38:18.437 INFO:tasks.workunit.client.1.smithi193.stdout:4/399: chown d1a/d6b/f44 154830 1 2024-05-09T06:38:18.438 INFO:tasks.workunit.client.1.smithi193.stdout:4/400: chown l2 850 1 2024-05-09T06:38:18.438 INFO:tasks.workunit.client.1.smithi193.stdout:0/325: mkdir d18/d4c/d64 0 2024-05-09T06:38:18.439 INFO:tasks.workunit.client.1.smithi193.stdout:0/326: write d18/f4e [3030455,112156] 0 2024-05-09T06:38:18.441 INFO:tasks.workunit.client.1.smithi193.stdout:8/296: dread d6/d1c/f12 [0,4194304] 0 2024-05-09T06:38:18.441 INFO:tasks.workunit.client.1.smithi193.stdout:4/401: dread d1a/f65 [0,4194304] 0 2024-05-09T06:38:18.442 INFO:tasks.workunit.client.1.smithi193.stdout:2/295: mknod db/df/d12/c6e 0 2024-05-09T06:38:18.443 INFO:tasks.workunit.client.1.smithi193.stdout:8/297: dread - d6/d1c/d18/d2f/f44 zero size 2024-05-09T06:38:18.444 INFO:tasks.workunit.client.0.smithi063.stdout:0/715: sync 2024-05-09T06:38:18.445 INFO:tasks.workunit.client.1.smithi193.stdout:8/298: fdatasync d6/d1c/d18/f26 0 2024-05-09T06:38:18.445 INFO:tasks.workunit.client.0.smithi063.stdout:9/702: dwrite d0/d4/d57/d60/d76/fa9 [0,4194304] 0 2024-05-09T06:38:18.446 INFO:tasks.workunit.client.1.smithi193.stdout:5/336: rename d0/d13/d2c/d38 to d0/d13/d2c/d37/d2b/d47/d6f 0 2024-05-09T06:38:18.447 INFO:tasks.workunit.client.1.smithi193.stdout:5/337: chown d0/f8 14185056 1 2024-05-09T06:38:18.448 INFO:tasks.workunit.client.1.smithi193.stdout:8/299: write d6/d1c/d1b/f2b [3025611,2073] 0 2024-05-09T06:38:18.448 INFO:tasks.workunit.client.1.smithi193.stdout:7/346: mknod d6/dd/c71 0 2024-05-09T06:38:18.450 INFO:tasks.workunit.client.1.smithi193.stdout:6/340: symlink d1/d25/d4c/l7b 0 2024-05-09T06:38:18.451 INFO:tasks.workunit.client.0.smithi063.stdout:2/702: dread dc/d36/d43/d46/d63/d79/d7d/f84 [4194304,4194304] 0 2024-05-09T06:38:18.451 INFO:tasks.workunit.client.0.smithi063.stdout:2/703: chown dc/d36/d43/lc2 348 1 2024-05-09T06:38:18.452 INFO:tasks.workunit.client.1.smithi193.stdout:9/369: write dd/d1e/d1f/f21 [530097,94532] 0 2024-05-09T06:38:18.453 INFO:tasks.workunit.client.0.smithi063.stdout:2/704: stat dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92 0 2024-05-09T06:38:18.458 INFO:tasks.workunit.client.0.smithi063.stdout:8/738: creat d9/da/d34/f10a x:0 0 0 2024-05-09T06:38:18.460 INFO:tasks.workunit.client.1.smithi193.stdout:9/370: dwrite dd/d1e/d1f/d48/d55/d41/d68/f6b [0,4194304] 0 2024-05-09T06:38:18.462 INFO:tasks.workunit.client.1.smithi193.stdout:4/402: mknod d1a/d6d/c79 0 2024-05-09T06:38:18.462 INFO:tasks.workunit.client.0.smithi063.stdout:3/696: creat d7/da/d21/d30/d44/d48/fe7 x:0 0 0 2024-05-09T06:38:18.464 INFO:tasks.workunit.client.1.smithi193.stdout:4/403: write d1a/d22/d43/f4d [55487,43919] 0 2024-05-09T06:38:18.465 INFO:tasks.workunit.client.0.smithi063.stdout:3/697: read d7/da/d2b/d66/dc0/da4/fbc [1973927,82409] 0 2024-05-09T06:38:18.466 INFO:tasks.workunit.client.1.smithi193.stdout:3/323: dread d1/db/d1b/f3b [0,4194304] 0 2024-05-09T06:38:18.471 INFO:tasks.workunit.client.1.smithi193.stdout:8/300: mkdir d6/d1c/d1b/d29/d57 0 2024-05-09T06:38:18.476 INFO:tasks.workunit.client.1.smithi193.stdout:6/341: unlink d1/f6a 0 2024-05-09T06:38:18.477 INFO:tasks.workunit.client.0.smithi063.stdout:7/636: creat d0/d15/d49/d54/d9f/d97/fde x:0 0 0 2024-05-09T06:38:18.479 INFO:tasks.workunit.client.1.smithi193.stdout:0/327: mkdir d18/d65 0 2024-05-09T06:38:18.482 INFO:tasks.workunit.client.1.smithi193.stdout:9/371: unlink dd/f18 0 2024-05-09T06:38:18.482 INFO:tasks.workunit.client.1.smithi193.stdout:7/347: write d6/d17/f52 [1123658,58432] 0 2024-05-09T06:38:18.482 INFO:tasks.workunit.client.0.smithi063.stdout:0/716: dread dd/d1a/d2c/d51/d72/d9e/d23/d89/fc4 [0,4194304] 0 2024-05-09T06:38:18.483 INFO:tasks.workunit.client.1.smithi193.stdout:1/309: sync 2024-05-09T06:38:18.483 INFO:tasks.workunit.client.0.smithi063.stdout:0/717: chown dd/d1a/d2c/d51/d72/d9e/d23/c3a 20 1 2024-05-09T06:38:18.485 INFO:tasks.workunit.client.1.smithi193.stdout:3/324: symlink d1/db/d2f/l75 0 2024-05-09T06:38:18.489 INFO:tasks.workunit.client.1.smithi193.stdout:8/301: unlink d6/c3c 0 2024-05-09T06:38:18.493 INFO:tasks.workunit.client.0.smithi063.stdout:4/664: link d7/df/d35/d78/f86 d7/da3/d28/d31/d32/d34/db4/fd1 0 2024-05-09T06:38:18.495 INFO:tasks.workunit.client.0.smithi063.stdout:1/610: rmdir d9/d46/dc7 0 2024-05-09T06:38:18.499 INFO:tasks.workunit.client.1.smithi193.stdout:3/325: dread d1/f4 [4194304,4194304] 0 2024-05-09T06:38:18.499 INFO:tasks.workunit.client.0.smithi063.stdout:9/703: mkdir d0/d4/d26/df6 0 2024-05-09T06:38:18.501 INFO:tasks.workunit.client.0.smithi063.stdout:2/705: getdents dc/dd/d2d/d60/dd0/d90/dc1 0 2024-05-09T06:38:18.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/372: chown dd/df/l34 4 1 2024-05-09T06:38:18.503 INFO:tasks.workunit.client.0.smithi063.stdout:8/739: link d9/da/d45/de4/ff3 d9/da/d51/f10b 0 2024-05-09T06:38:18.504 INFO:tasks.workunit.client.1.smithi193.stdout:5/338: dwrite d0/f29 [0,4194304] 0 2024-05-09T06:38:18.505 INFO:tasks.workunit.client.1.smithi193.stdout:7/348: fsync d6/f30 0 2024-05-09T06:38:18.506 INFO:tasks.workunit.client.1.smithi193.stdout:2/296: link db/l51 db/df/d54/d63/l6f 0 2024-05-09T06:38:18.507 INFO:tasks.workunit.client.1.smithi193.stdout:9/373: truncate dd/d1e/d66/d49/f7f 330020 0 2024-05-09T06:38:18.507 INFO:tasks.workunit.client.0.smithi063.stdout:3/698: mkdir d7/da/de4/de8 0 2024-05-09T06:38:18.508 INFO:tasks.workunit.client.1.smithi193.stdout:9/374: chown dd/d1e/c3e 6781 1 2024-05-09T06:38:18.508 INFO:tasks.workunit.client.0.smithi063.stdout:4/665: creat d7/da6/fd2 x:0 0 0 2024-05-09T06:38:18.510 INFO:tasks.workunit.client.1.smithi193.stdout:9/375: dread - dd/d1e/d66/d49/f65 zero size 2024-05-09T06:38:18.512 INFO:tasks.workunit.client.0.smithi063.stdout:4/666: dread d7/da3/f3a [0,4194304] 0 2024-05-09T06:38:18.513 INFO:tasks.workunit.client.0.smithi063.stdout:7/637: creat d0/d15/d49/d4f/d66/ddd/fdf x:0 0 0 2024-05-09T06:38:18.513 INFO:tasks.workunit.client.0.smithi063.stdout:7/638: chown d0/d15/d49 505693278 1 2024-05-09T06:38:18.514 INFO:tasks.workunit.client.0.smithi063.stdout:0/718: mkdir dd/d60/dcd/de0 0 2024-05-09T06:38:18.517 INFO:tasks.workunit.client.0.smithi063.stdout:9/704: read - d0/d4/d26/d39/d48/d80/fad zero size 2024-05-09T06:38:18.520 INFO:tasks.workunit.client.1.smithi193.stdout:0/328: mkdir d18/d4c/d64/d66 0 2024-05-09T06:38:18.521 INFO:tasks.workunit.client.0.smithi063.stdout:8/740: mkdir d9/da/d18/d46/d10c 0 2024-05-09T06:38:18.524 INFO:tasks.workunit.client.1.smithi193.stdout:6/342: write d1/de/d11/f1f [743475,61571] 0 2024-05-09T06:38:18.527 INFO:tasks.workunit.client.0.smithi063.stdout:4/667: creat d7/da6/fd3 x:0 0 0 2024-05-09T06:38:18.529 INFO:tasks.workunit.client.0.smithi063.stdout:8/741: dwrite d9/da/d18/d46/f97 [0,4194304] 0 2024-05-09T06:38:18.529 INFO:tasks.workunit.client.0.smithi063.stdout:7/639: creat d0/d15/d89/d98/fe0 x:0 0 0 2024-05-09T06:38:18.529 INFO:tasks.workunit.client.0.smithi063.stdout:1/611: dread d9/d2a/d35/d1e/f25 [4194304,4194304] 0 2024-05-09T06:38:18.530 INFO:tasks.workunit.client.1.smithi193.stdout:5/339: truncate d0/d13/d2c/d37/d5b/f32 915425 0 2024-05-09T06:38:18.532 INFO:tasks.workunit.client.0.smithi063.stdout:7/640: fdatasync d0/d15/d89/fa9 0 2024-05-09T06:38:18.533 INFO:tasks.workunit.client.1.smithi193.stdout:4/404: creat d1a/d22/f7a x:0 0 0 2024-05-09T06:38:18.534 INFO:tasks.workunit.client.1.smithi193.stdout:9/376: rmdir dd/df/d15 39 2024-05-09T06:38:18.535 INFO:tasks.workunit.client.0.smithi063.stdout:7/641: stat d0/d15/d4a/d50/f2e 0 2024-05-09T06:38:18.536 INFO:tasks.workunit.client.0.smithi063.stdout:7/642: fdatasync d0/f11 0 2024-05-09T06:38:18.537 INFO:tasks.workunit.client.1.smithi193.stdout:4/405: truncate d1a/d24/d30/f32 1076977 0 2024-05-09T06:38:18.538 INFO:tasks.workunit.client.1.smithi193.stdout:9/377: readlink dd/df/d31/d7e/d37/l46 0 2024-05-09T06:38:18.538 INFO:tasks.workunit.client.0.smithi063.stdout:4/668: dwrite d7/df/f2b [4194304,4194304] 0 2024-05-09T06:38:18.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/297: dwrite db/df/f10 [0,4194304] 0 2024-05-09T06:38:18.541 INFO:tasks.workunit.client.0.smithi063.stdout:0/719: mkdir dd/d1a/d2c/dba/de1 0 2024-05-09T06:38:18.541 INFO:tasks.workunit.client.0.smithi063.stdout:8/742: dwrite d9/da/d34/f10a [0,4194304] 0 2024-05-09T06:38:18.542 INFO:tasks.workunit.client.0.smithi063.stdout:2/706: dread dc/dd/f13 [0,4194304] 0 2024-05-09T06:38:18.542 INFO:tasks.workunit.client.1.smithi193.stdout:2/298: stat db/l34 0 2024-05-09T06:38:18.543 INFO:tasks.workunit.client.0.smithi063.stdout:0/720: stat dd/d1a/d2c/d51/d72/d9e/f55 0 2024-05-09T06:38:18.544 INFO:tasks.workunit.client.1.smithi193.stdout:2/299: chown db/df/d12/c6e 0 1 2024-05-09T06:38:18.546 INFO:tasks.workunit.client.0.smithi063.stdout:1/612: symlink d9/d2a/d35/d1e/d9d/d3f/d59/d62/ld3 0 2024-05-09T06:38:18.550 INFO:tasks.workunit.client.0.smithi063.stdout:3/699: link d7/da/d2b/d66/fd2 d7/d70/d98/fe9 0 2024-05-09T06:38:18.551 INFO:tasks.workunit.client.1.smithi193.stdout:1/310: link d4/f13 d4/d14/d15/f58 0 2024-05-09T06:38:18.552 INFO:tasks.workunit.client.1.smithi193.stdout:9/378: dread dd/d1e/d1f/d48/d55/d41/d68/f6b [0,4194304] 0 2024-05-09T06:38:18.554 INFO:tasks.workunit.client.1.smithi193.stdout:9/379: truncate dd/d1e/d1f/d48/f52 382461 0 2024-05-09T06:38:18.559 INFO:tasks.workunit.client.1.smithi193.stdout:9/380: dwrite dd/d1e/d66/f80 [0,4194304] 0 2024-05-09T06:38:18.560 INFO:tasks.workunit.client.0.smithi063.stdout:7/643: creat d0/d15/d49/d4f/d66/d67/d86/fe1 x:0 0 0 2024-05-09T06:38:18.561 INFO:tasks.workunit.client.1.smithi193.stdout:4/406: creat d1a/d6d/f7b x:0 0 0 2024-05-09T06:38:18.562 INFO:tasks.workunit.client.1.smithi193.stdout:4/407: read d1a/d1f/f2a [1741731,122310] 0 2024-05-09T06:38:18.563 INFO:tasks.workunit.client.0.smithi063.stdout:4/669: creat d7/d3d/d73/fd4 x:0 0 0 2024-05-09T06:38:18.563 INFO:tasks.workunit.client.0.smithi063.stdout:4/670: dread - d7/da6/fd3 zero size 2024-05-09T06:38:18.564 INFO:tasks.workunit.client.0.smithi063.stdout:5/666: sync 2024-05-09T06:38:18.564 INFO:tasks.workunit.client.0.smithi063.stdout:6/682: sync 2024-05-09T06:38:18.565 INFO:tasks.workunit.client.1.smithi193.stdout:8/302: rename d6/d1c/d1b/d29/c3e to d6/d1c/d18/d2f/d40/d55/c58 0 2024-05-09T06:38:18.570 INFO:tasks.workunit.client.1.smithi193.stdout:6/343: write d1/de/d11/d18/d36/d37/f5b [1035270,14376] 0 2024-05-09T06:38:18.570 INFO:tasks.workunit.client.0.smithi063.stdout:2/707: creat dc/d36/d43/d46/d63/d9e/fd2 x:0 0 0 2024-05-09T06:38:18.571 INFO:tasks.workunit.client.1.smithi193.stdout:6/344: chown d1/d25/d2c/d3c/f3d 11050080 1 2024-05-09T06:38:18.571 INFO:tasks.workunit.client.0.smithi063.stdout:2/708: write dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f86 [2001179,126490] 0 2024-05-09T06:38:18.572 INFO:tasks.workunit.client.1.smithi193.stdout:6/345: fdatasync d1/d25/d2c/d3c/f3d 0 2024-05-09T06:38:18.572 INFO:tasks.workunit.client.0.smithi063.stdout:2/709: fdatasync dc/d32/d33/fb0 0 2024-05-09T06:38:18.573 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:18 smithi063 ceph-mon[27171]: pgmap v12: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 70 MiB/s rd, 263 MiB/s wr, 520 op/s 2024-05-09T06:38:18.573 INFO:tasks.workunit.client.0.smithi063.stdout:0/721: mknod dd/d1a/d2c/d51/d76/d97/ce2 0 2024-05-09T06:38:18.574 INFO:tasks.workunit.client.1.smithi193.stdout:6/346: fdatasync d1/d25/d2c/d3c/d3e/f43 0 2024-05-09T06:38:18.576 INFO:tasks.workunit.client.0.smithi063.stdout:1/613: unlink d9/f12 0 2024-05-09T06:38:18.590 INFO:tasks.workunit.client.0.smithi063.stdout:8/743: dwrite d9/da/d18/d24/f84 [0,4194304] 0 2024-05-09T06:38:18.590 INFO:tasks.workunit.client.0.smithi063.stdout:8/744: fdatasync d9/d37/fa9 0 2024-05-09T06:38:18.596 INFO:tasks.workunit.client.0.smithi063.stdout:7/644: dwrite d0/f32 [0,4194304] 0 2024-05-09T06:38:18.597 INFO:tasks.workunit.client.0.smithi063.stdout:7/645: dread - d0/d15/d4a/fb5 zero size 2024-05-09T06:38:18.604 INFO:tasks.workunit.client.0.smithi063.stdout:5/667: write d8/d17/f6d [747395,90237] 0 2024-05-09T06:38:18.605 INFO:tasks.workunit.client.0.smithi063.stdout:6/683: fdatasync d2/d20/d26/d7f/dcf/fda 0 2024-05-09T06:38:18.606 INFO:tasks.workunit.client.0.smithi063.stdout:9/705: getdents d0/d4/da 0 2024-05-09T06:38:18.606 INFO:tasks.workunit.client.0.smithi063.stdout:6/684: chown d2/d20/f21 87 1 2024-05-09T06:38:18.606 INFO:tasks.workunit.client.0.smithi063.stdout:9/706: fdatasync d0/d4/d26/f49 0 2024-05-09T06:38:18.607 INFO:tasks.workunit.client.0.smithi063.stdout:9/707: chown d0/f9f 52745107 1 2024-05-09T06:38:18.607 INFO:tasks.workunit.client.0.smithi063.stdout:6/685: write d2/d1f/d29/d3f/d5c/d64/f83 [1603901,11133] 0 2024-05-09T06:38:18.608 INFO:tasks.workunit.client.0.smithi063.stdout:2/710: mknod dc/dd/d45/d5a/cd3 0 2024-05-09T06:38:18.610 INFO:tasks.workunit.client.0.smithi063.stdout:1/614: truncate d9/d2a/d35/d24/d68/f98 933961 0 2024-05-09T06:38:18.612 INFO:tasks.workunit.client.0.smithi063.stdout:8/745: creat d9/da/d34/f10d x:0 0 0 2024-05-09T06:38:18.613 INFO:tasks.workunit.client.0.smithi063.stdout:7/646: chown d0/d15/d49/d54/d9f/d97/d9b/d9d/lb9 1539846 1 2024-05-09T06:38:18.614 INFO:tasks.workunit.client.0.smithi063.stdout:4/671: symlink d7/ld5 0 2024-05-09T06:38:18.615 INFO:tasks.workunit.client.0.smithi063.stdout:4/672: truncate d7/df/d35/da5/fb9 295503 0 2024-05-09T06:38:18.617 INFO:tasks.workunit.client.0.smithi063.stdout:0/722: creat dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dcb/fe3 x:0 0 0 2024-05-09T06:38:18.618 INFO:tasks.workunit.client.0.smithi063.stdout:9/708: mkdir d0/d4/da/df7 0 2024-05-09T06:38:18.619 INFO:tasks.workunit.client.0.smithi063.stdout:3/700: link d7/d70/d98/d2d/le6 d7/d70/d98/d2d/d36/lea 0 2024-05-09T06:38:18.621 INFO:tasks.workunit.client.0.smithi063.stdout:8/746: truncate d9/da/d18/f1b 2094384 0 2024-05-09T06:38:18.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:18 smithi193 ceph-mon[35141]: pgmap v12: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 70 MiB/s rd, 263 MiB/s wr, 520 op/s 2024-05-09T06:38:18.622 INFO:tasks.workunit.client.0.smithi063.stdout:5/668: creat d8/d19/d30/d74/d77/dde/fdf x:0 0 0 2024-05-09T06:38:18.623 INFO:tasks.workunit.client.0.smithi063.stdout:7/647: symlink d0/d15/d49/d4f/d66/ddd/le2 0 2024-05-09T06:38:18.624 INFO:tasks.workunit.client.0.smithi063.stdout:6/686: fsync d2/d4/d7c/fcc 0 2024-05-09T06:38:18.626 INFO:tasks.workunit.client.0.smithi063.stdout:4/673: symlink d7/da3/d28/d31/d32/d34/db4/ld6 0 2024-05-09T06:38:18.626 INFO:tasks.workunit.client.0.smithi063.stdout:3/701: dwrite d7/da/d2b/d66/fdb [0,4194304] 0 2024-05-09T06:38:18.628 INFO:tasks.workunit.client.0.smithi063.stdout:9/709: chown d0/d3b/d63/ldd 0 1 2024-05-09T06:38:18.630 INFO:tasks.workunit.client.0.smithi063.stdout:5/669: unlink d8/d19/d30/l71 0 2024-05-09T06:38:18.633 INFO:tasks.workunit.client.0.smithi063.stdout:7/648: mkdir d0/d15/d89/d98/de3 0 2024-05-09T06:38:18.635 INFO:tasks.workunit.client.0.smithi063.stdout:3/702: dwrite d7/da/d2b/d66/f95 [0,4194304] 0 2024-05-09T06:38:18.636 INFO:tasks.workunit.client.1.smithi193.stdout:2/300: creat db/d25/d38/d61/f70 x:0 0 0 2024-05-09T06:38:18.636 INFO:tasks.workunit.client.1.smithi193.stdout:5/340: write d0/f8 [4096118,118271] 0 2024-05-09T06:38:18.638 INFO:tasks.workunit.client.0.smithi063.stdout:6/687: unlink d2/d4/lcd 0 2024-05-09T06:38:18.640 INFO:tasks.workunit.client.1.smithi193.stdout:2/301: stat db/d4b/d56 0 2024-05-09T06:38:18.640 INFO:tasks.workunit.client.1.smithi193.stdout:7/349: getdents d6/d8 0 2024-05-09T06:38:18.642 INFO:tasks.workunit.client.1.smithi193.stdout:9/381: dread dd/d1e/d66/d49/f64 [0,4194304] 0 2024-05-09T06:38:18.644 INFO:tasks.workunit.client.1.smithi193.stdout:0/329: getdents d18/d27 0 2024-05-09T06:38:18.646 INFO:tasks.workunit.client.1.smithi193.stdout:6/347: mknod d1/d25/d2c/d76/c7c 0 2024-05-09T06:38:18.647 INFO:tasks.workunit.client.1.smithi193.stdout:6/348: chown d1/fa 138 1 2024-05-09T06:38:18.649 INFO:tasks.workunit.client.1.smithi193.stdout:8/303: symlink d6/d1c/d53/d56/l59 0 2024-05-09T06:38:18.649 INFO:tasks.workunit.client.1.smithi193.stdout:7/350: dread d6/d8/d29/d2f/f4c [0,4194304] 0 2024-05-09T06:38:18.650 INFO:tasks.workunit.client.0.smithi063.stdout:9/710: mkdir d0/d4/d57/df8 0 2024-05-09T06:38:18.650 INFO:tasks.workunit.client.1.smithi193.stdout:8/304: fdatasync d6/d1c/d27/f4d 0 2024-05-09T06:38:18.650 INFO:tasks.workunit.client.1.smithi193.stdout:5/341: dwrite d0/d13/d2c/d37/f11 [0,4194304] 0 2024-05-09T06:38:18.651 INFO:tasks.workunit.client.0.smithi063.stdout:4/674: dread d7/df/d35/d78/f86 [0,4194304] 0 2024-05-09T06:38:18.652 INFO:tasks.workunit.client.1.smithi193.stdout:8/305: fdatasync d6/d1c/d27/f3b 0 2024-05-09T06:38:18.653 INFO:tasks.workunit.client.0.smithi063.stdout:5/670: mknod d8/d17/d75/dc9/ce0 0 2024-05-09T06:38:18.653 INFO:tasks.workunit.client.1.smithi193.stdout:4/408: truncate f11 1188539 0 2024-05-09T06:38:18.654 INFO:tasks.workunit.client.1.smithi193.stdout:4/409: chown d1a/d6b/d60 80 1 2024-05-09T06:38:18.656 INFO:tasks.workunit.client.0.smithi063.stdout:4/675: dwrite d7/d50/f9d [0,4194304] 0 2024-05-09T06:38:18.657 INFO:tasks.workunit.client.1.smithi193.stdout:6/349: dread d1/d25/d2c/d3c/d3e/f43 [0,4194304] 0 2024-05-09T06:38:18.658 INFO:tasks.workunit.client.0.smithi063.stdout:4/676: readlink d7/da3/d28/d31/d32/d34/db4/ld6 0 2024-05-09T06:38:18.659 INFO:tasks.workunit.client.1.smithi193.stdout:8/306: dread d6/d1c/f1e [0,4194304] 0 2024-05-09T06:38:18.660 INFO:tasks.workunit.client.1.smithi193.stdout:8/307: chown d6/d1c/d1b/f4a 670283 1 2024-05-09T06:38:18.662 INFO:tasks.workunit.client.1.smithi193.stdout:8/308: chown d6/d1c/d18/d2f/f44 48046 1 2024-05-09T06:38:18.664 INFO:tasks.workunit.client.1.smithi193.stdout:8/309: dread - d6/d1c/d27/f4d zero size 2024-05-09T06:38:18.665 INFO:tasks.workunit.client.0.smithi063.stdout:3/703: creat d7/d70/feb x:0 0 0 2024-05-09T06:38:18.667 INFO:tasks.workunit.client.0.smithi063.stdout:6/688: truncate d2/d20/d4a/f94 276590 0 2024-05-09T06:38:18.674 INFO:tasks.workunit.client.0.smithi063.stdout:0/723: dwrite dd/f6b [0,4194304] 0 2024-05-09T06:38:18.674 INFO:tasks.workunit.client.1.smithi193.stdout:1/311: write d4/d14/f2f [2914212,120367] 0 2024-05-09T06:38:18.678 INFO:tasks.workunit.client.1.smithi193.stdout:9/382: dwrite dd/df/d15/f56 [0,4194304] 0 2024-05-09T06:38:18.679 INFO:tasks.workunit.client.0.smithi063.stdout:5/671: creat d8/d17/da2/fe1 x:0 0 0 2024-05-09T06:38:18.681 INFO:tasks.workunit.client.1.smithi193.stdout:9/383: chown dd/df/d15/d16/c2c 576112 1 2024-05-09T06:38:18.685 INFO:tasks.workunit.client.1.smithi193.stdout:4/410: rename d1a/d24/d30 to d1a/d1f/d4b/d7c 0 2024-05-09T06:38:18.687 INFO:tasks.workunit.client.0.smithi063.stdout:7/649: write d0/d15/d49/d4f/d66/d67/d86/f90 [891640,129852] 0 2024-05-09T06:38:18.689 INFO:tasks.workunit.client.1.smithi193.stdout:4/411: write d1a/d6b/f66 [765330,92585] 0 2024-05-09T06:38:18.690 INFO:tasks.workunit.client.0.smithi063.stdout:7/650: write d0/d15/d4a/d50/f2e [8084818,129650] 0 2024-05-09T06:38:18.690 INFO:tasks.workunit.client.0.smithi063.stdout:4/677: creat d7/da3/d28/d31/d71/dbb/fd7 x:0 0 0 2024-05-09T06:38:18.690 INFO:tasks.workunit.client.0.smithi063.stdout:7/651: readlink d0/d15/d49/d54/d9f/lc5 0 2024-05-09T06:38:18.692 INFO:tasks.workunit.client.1.smithi193.stdout:2/302: truncate f6 1384258 0 2024-05-09T06:38:18.694 INFO:tasks.workunit.client.0.smithi063.stdout:6/689: write d2/d20/d4a/f93 [4889101,95556] 0 2024-05-09T06:38:18.696 INFO:tasks.workunit.client.0.smithi063.stdout:8/747: getdents d9/da/d18/d24/d71 0 2024-05-09T06:38:18.697 INFO:tasks.workunit.client.1.smithi193.stdout:0/330: symlink d18/d65/l67 0 2024-05-09T06:38:18.697 INFO:tasks.workunit.client.0.smithi063.stdout:9/711: getdents d0/d4/d26/d39/d48/d80/d84/dcd 0 2024-05-09T06:38:18.703 INFO:tasks.workunit.client.0.smithi063.stdout:5/672: dread d8/d19/d30/d46/f5d [0,4194304] 0 2024-05-09T06:38:18.704 INFO:tasks.workunit.client.0.smithi063.stdout:7/652: rmdir d0/dcb 39 2024-05-09T06:38:18.705 INFO:tasks.workunit.client.1.smithi193.stdout:0/331: dwrite d18/f61 [0,4194304] 0 2024-05-09T06:38:18.705 INFO:tasks.workunit.client.0.smithi063.stdout:7/653: read - d0/d15/fcc zero size 2024-05-09T06:38:18.708 INFO:tasks.workunit.client.0.smithi063.stdout:3/704: link d7/d22/d28/dab/fe0 d7/d22/d28/fec 0 2024-05-09T06:38:18.708 INFO:tasks.workunit.client.0.smithi063.stdout:3/705: rename d7 to d7/da/de4/de8/ded 22 2024-05-09T06:38:18.714 INFO:tasks.workunit.client.1.smithi193.stdout:4/412: unlink d1a/d6b/f72 0 2024-05-09T06:38:18.718 INFO:tasks.workunit.client.1.smithi193.stdout:8/310: mknod d6/d1c/d18/c5a 0 2024-05-09T06:38:18.719 INFO:tasks.workunit.client.1.smithi193.stdout:9/384: dread dd/df/d31/d7e/f62 [0,4194304] 0 2024-05-09T06:38:18.725 INFO:tasks.workunit.client.1.smithi193.stdout:1/312: mkdir d4/d14/d15/d4f/d59 0 2024-05-09T06:38:18.729 INFO:tasks.workunit.client.1.smithi193.stdout:5/342: creat d0/d13/d2c/d37/d5b/f70 x:0 0 0 2024-05-09T06:38:18.730 INFO:tasks.workunit.client.0.smithi063.stdout:7/654: mknod d0/d15/d4a/d68/ce4 0 2024-05-09T06:38:18.731 INFO:tasks.workunit.client.1.smithi193.stdout:6/350: dwrite d1/de/d11/f22 [0,4194304] 0 2024-05-09T06:38:18.733 INFO:tasks.workunit.client.1.smithi193.stdout:5/343: write d0/d13/d2c/d37/d5b/d4e/d5d/f69 [949118,64271] 0 2024-05-09T06:38:18.735 INFO:tasks.workunit.client.0.smithi063.stdout:4/678: link d7/df/d35/f36 d7/df/d35/d5a/d60/fd8 0 2024-05-09T06:38:18.737 INFO:tasks.workunit.client.0.smithi063.stdout:8/748: dwrite d9/da/d45/de4/ff3 [0,4194304] 0 2024-05-09T06:38:18.738 INFO:tasks.workunit.client.1.smithi193.stdout:0/332: creat d18/d4c/d47/d54/f68 x:0 0 0 2024-05-09T06:38:18.738 INFO:tasks.workunit.client.0.smithi063.stdout:1/615: dwrite d9/d2a/d35/d24/d68/f98 [0,4194304] 0 2024-05-09T06:38:18.742 INFO:tasks.workunit.client.0.smithi063.stdout:3/706: write d7/d22/d28/f3c [3971659,16272] 0 2024-05-09T06:38:18.744 INFO:tasks.workunit.client.1.smithi193.stdout:4/413: rename d1a/f23 to d1a/d1f/d4b/d7c/d5b/f7d 0 2024-05-09T06:38:18.745 INFO:tasks.workunit.client.0.smithi063.stdout:9/712: symlink d0/d8/lf9 0 2024-05-09T06:38:18.746 INFO:tasks.workunit.client.0.smithi063.stdout:4/679: dwrite d7/da3/d28/d31/d71/dbb/fd7 [0,4194304] 0 2024-05-09T06:38:18.750 INFO:tasks.workunit.client.0.smithi063.stdout:8/749: read d9/d37/fa9 [1503285,94214] 0 2024-05-09T06:38:18.751 INFO:tasks.workunit.client.0.smithi063.stdout:8/750: readlink d9/da/d45/l5d 0 2024-05-09T06:38:18.755 INFO:tasks.workunit.client.1.smithi193.stdout:2/303: link db/df/d12/f32 db/d4b/f71 0 2024-05-09T06:38:18.755 INFO:tasks.workunit.client.1.smithi193.stdout:9/385: dread dd/d1e/d1f/f22 [0,4194304] 0 2024-05-09T06:38:18.756 INFO:tasks.workunit.client.1.smithi193.stdout:7/351: getdents d6/d8/d1a 0 2024-05-09T06:38:18.759 INFO:tasks.workunit.client.0.smithi063.stdout:1/616: mknod d9/d10/cd4 0 2024-05-09T06:38:18.760 INFO:tasks.workunit.client.0.smithi063.stdout:1/617: write d9/d2a/d3c/d4c/fbb [466414,10178] 0 2024-05-09T06:38:18.762 INFO:tasks.workunit.client.1.smithi193.stdout:6/351: creat d1/d25/d2c/d76/f7d x:0 0 0 2024-05-09T06:38:18.764 INFO:tasks.workunit.client.1.smithi193.stdout:5/344: mknod d0/d13/d2c/d37/d2b/d47/d66/d3a/c71 0 2024-05-09T06:38:18.765 INFO:tasks.workunit.client.0.smithi063.stdout:9/713: fsync d0/d4/d57/fa4 0 2024-05-09T06:38:18.766 INFO:tasks.workunit.client.0.smithi063.stdout:4/680: mkdir d7/df/d35/dd9 0 2024-05-09T06:38:18.767 INFO:tasks.workunit.client.1.smithi193.stdout:8/311: mkdir d6/d5b 0 2024-05-09T06:38:18.769 INFO:tasks.workunit.client.0.smithi063.stdout:5/673: creat d8/d19/fe2 x:0 0 0 2024-05-09T06:38:18.770 INFO:tasks.workunit.client.1.smithi193.stdout:4/414: creat d1a/d24/d73/f7e x:0 0 0 2024-05-09T06:38:18.771 INFO:tasks.workunit.client.0.smithi063.stdout:5/674: write d8/d19/d30/f43 [165469,32253] 0 2024-05-09T06:38:18.771 INFO:tasks.workunit.client.0.smithi063.stdout:8/751: rmdir d9/d41/dec/da1 39 2024-05-09T06:38:18.772 INFO:tasks.workunit.client.0.smithi063.stdout:5/675: chown d8/d53/c63 509604102 1 2024-05-09T06:38:18.775 INFO:tasks.workunit.client.1.smithi193.stdout:2/304: rename db/df/f4a to db/d25/d38/d40/f72 0 2024-05-09T06:38:18.780 INFO:tasks.workunit.client.1.smithi193.stdout:9/386: dread dd/d1e/f27 [0,4194304] 0 2024-05-09T06:38:18.784 INFO:tasks.workunit.client.1.smithi193.stdout:2/305: dread db/d25/d38/d61/f65 [0,4194304] 0 2024-05-09T06:38:18.784 INFO:tasks.workunit.client.0.smithi063.stdout:0/724: sync 2024-05-09T06:38:18.784 INFO:tasks.workunit.client.0.smithi063.stdout:2/711: sync 2024-05-09T06:38:18.784 INFO:tasks.workunit.client.0.smithi063.stdout:4/681: dwrite d7/d3d/fab [0,4194304] 0 2024-05-09T06:38:18.784 INFO:tasks.workunit.client.0.smithi063.stdout:7/655: mknod d0/dcb/ce5 0 2024-05-09T06:38:18.785 INFO:tasks.workunit.client.1.smithi193.stdout:9/387: read fa [840358,86701] 0 2024-05-09T06:38:18.785 INFO:tasks.workunit.client.1.smithi193.stdout:7/352: mkdir d6/dd/d72 0 2024-05-09T06:38:18.786 INFO:tasks.workunit.client.0.smithi063.stdout:6/690: getdents d2/df/d91 0 2024-05-09T06:38:18.786 INFO:tasks.workunit.client.0.smithi063.stdout:2/712: chown dc/dd/l28 514 1 2024-05-09T06:38:18.787 INFO:tasks.workunit.client.0.smithi063.stdout:1/618: dread - d9/d46/f63 zero size 2024-05-09T06:38:18.789 INFO:tasks.workunit.client.1.smithi193.stdout:0/333: mknod d18/d1d/d56/c69 0 2024-05-09T06:38:18.790 INFO:tasks.workunit.client.0.smithi063.stdout:1/619: write d9/d2a/d39/d5d/fc6 [494977,68541] 0 2024-05-09T06:38:18.792 INFO:tasks.workunit.client.1.smithi193.stdout:3/326: sync 2024-05-09T06:38:18.792 INFO:tasks.workunit.client.1.smithi193.stdout:3/327: stat d1/l27 0 2024-05-09T06:38:18.793 INFO:tasks.workunit.client.1.smithi193.stdout:1/313: dwrite d4/d16/d17/d1a/f1e [0,4194304] 0 2024-05-09T06:38:18.797 INFO:tasks.workunit.client.0.smithi063.stdout:8/752: creat d9/da/d45/de4/f10e x:0 0 0 2024-05-09T06:38:18.799 INFO:tasks.workunit.client.0.smithi063.stdout:8/753: truncate d9/da/d18/d24/f109 275138 0 2024-05-09T06:38:18.799 INFO:tasks.workunit.client.1.smithi193.stdout:0/334: read d18/d1d/f33 [359093,111710] 0 2024-05-09T06:38:18.799 INFO:tasks.workunit.client.1.smithi193.stdout:8/312: dread d6/d1c/d1b/f4a [0,4194304] 0 2024-05-09T06:38:18.799 INFO:tasks.workunit.client.0.smithi063.stdout:5/676: readlink d8/d19/d30/d74/d77/l90 0 2024-05-09T06:38:18.800 INFO:tasks.workunit.client.0.smithi063.stdout:8/754: dread - d9/da/d34/f10d zero size 2024-05-09T06:38:18.800 INFO:tasks.workunit.client.1.smithi193.stdout:0/335: read - d18/d27/f41 zero size 2024-05-09T06:38:18.801 INFO:tasks.workunit.client.0.smithi063.stdout:8/755: fsync d9/da/d34/fac 0 2024-05-09T06:38:18.803 INFO:tasks.workunit.client.0.smithi063.stdout:7/656: unlink d0/d15/d4a/lb1 0 2024-05-09T06:38:18.807 INFO:tasks.workunit.client.0.smithi063.stdout:8/756: dwrite d9/d41/fef [0,4194304] 0 2024-05-09T06:38:18.807 INFO:tasks.workunit.client.0.smithi063.stdout:4/682: rename d7/df/d57/c64 to d7/d3d/dca/cda 0 2024-05-09T06:38:18.810 INFO:tasks.workunit.client.1.smithi193.stdout:9/388: fdatasync dd/d1e/d66/d49/f64 0 2024-05-09T06:38:18.811 INFO:tasks.workunit.client.0.smithi063.stdout:6/691: chown d2/d1f/d29/d3f/d5c/f9d 2 1 2024-05-09T06:38:18.813 INFO:tasks.workunit.client.1.smithi193.stdout:7/353: stat d6/dd/d53/c6f 0 2024-05-09T06:38:18.813 INFO:tasks.workunit.client.1.smithi193.stdout:9/389: truncate dd/df/d31/d7e/d71/f84 784076 0 2024-05-09T06:38:18.815 INFO:tasks.workunit.client.1.smithi193.stdout:9/390: fdatasync dd/df/d31/d7e/f35 0 2024-05-09T06:38:18.815 INFO:tasks.workunit.client.0.smithi063.stdout:5/677: mknod d8/d19/d30/d74/d77/ce3 0 2024-05-09T06:38:18.819 INFO:tasks.workunit.client.0.smithi063.stdout:6/692: dwrite d2/d20/f9c [0,4194304] 0 2024-05-09T06:38:18.821 INFO:tasks.workunit.client.1.smithi193.stdout:8/313: dread d6/d1c/d18/f26 [0,4194304] 0 2024-05-09T06:38:18.822 INFO:tasks.workunit.client.1.smithi193.stdout:7/354: dwrite d6/d17/f52 [0,4194304] 0 2024-05-09T06:38:18.827 INFO:tasks.workunit.client.0.smithi063.stdout:2/713: write f6 [557788,101674] 0 2024-05-09T06:38:18.831 INFO:tasks.workunit.client.1.smithi193.stdout:7/355: dwrite d6/d8/d1a/f3c [0,4194304] 0 2024-05-09T06:38:18.834 INFO:tasks.workunit.client.1.smithi193.stdout:4/415: unlink d1a/d6b/l39 0 2024-05-09T06:38:18.837 INFO:tasks.workunit.client.0.smithi063.stdout:3/707: getdents d7/d22 0 2024-05-09T06:38:18.838 INFO:tasks.workunit.client.1.smithi193.stdout:5/345: truncate d0/d13/d2c/d37/d2b/d47/f63 476501 0 2024-05-09T06:38:18.840 INFO:tasks.workunit.client.1.smithi193.stdout:5/346: write d0/d13/d2c/d37/d2b/d47/d66/f23 [4189979,104705] 0 2024-05-09T06:38:18.842 INFO:tasks.workunit.client.0.smithi063.stdout:3/708: dwrite d7/da/d2b/d66/f95 [0,4194304] 0 2024-05-09T06:38:18.842 INFO:tasks.workunit.client.1.smithi193.stdout:2/306: link db/d4b/f5d db/d25/d38/d61/f73 0 2024-05-09T06:38:18.846 INFO:tasks.workunit.client.0.smithi063.stdout:6/693: fdatasync d2/d1f/d29/d3f/d5c/f9d 0 2024-05-09T06:38:18.848 INFO:tasks.workunit.client.0.smithi063.stdout:6/694: write d2/df/d59/faf [571979,121650] 0 2024-05-09T06:38:18.848 INFO:tasks.workunit.client.1.smithi193.stdout:5/347: dwrite d0/f29 [0,4194304] 0 2024-05-09T06:38:18.848 INFO:tasks.workunit.client.0.smithi063.stdout:2/714: rmdir dc/d36/d43/d46/d63/d79/d7d/d8d/d8e 39 2024-05-09T06:38:18.850 INFO:tasks.workunit.client.1.smithi193.stdout:9/391: mkdir dd/d1e/d1f/d48/d55/d86 0 2024-05-09T06:38:18.851 INFO:tasks.workunit.client.1.smithi193.stdout:2/307: dread db/d25/d38/d40/f72 [4194304,4194304] 0 2024-05-09T06:38:18.860 INFO:tasks.workunit.client.1.smithi193.stdout:9/392: dwrite dd/d1e/d66/d49/f70 [0,4194304] 0 2024-05-09T06:38:18.862 INFO:tasks.workunit.client.1.smithi193.stdout:7/356: mknod d6/d8/d29/c73 0 2024-05-09T06:38:18.865 INFO:tasks.workunit.client.1.smithi193.stdout:4/416: fsync d1a/f65 0 2024-05-09T06:38:18.869 INFO:tasks.workunit.client.0.smithi063.stdout:4/683: creat d7/da3/d28/d31/fdb x:0 0 0 2024-05-09T06:38:18.870 INFO:tasks.workunit.client.0.smithi063.stdout:4/684: chown d7/df/l37 6 1 2024-05-09T06:38:18.870 INFO:tasks.workunit.client.1.smithi193.stdout:0/336: creat d18/d4c/d5f/f6a x:0 0 0 2024-05-09T06:38:18.871 INFO:tasks.workunit.client.0.smithi063.stdout:6/695: mkdir d2/df/d91/ddf 0 2024-05-09T06:38:18.873 INFO:tasks.workunit.client.0.smithi063.stdout:2/715: creat dc/dd/d45/d5a/d6c/d8f/dbf/fd4 x:0 0 0 2024-05-09T06:38:18.875 INFO:tasks.workunit.client.0.smithi063.stdout:1/620: rename d9/d2a/d2e/f7d to d9/d2a/d35/d1e/fd5 0 2024-05-09T06:38:18.877 INFO:tasks.workunit.client.0.smithi063.stdout:8/757: link d9/d3b/c49 d9/da/d18/dc7/d8f/c10f 0 2024-05-09T06:38:18.878 INFO:tasks.workunit.client.1.smithi193.stdout:6/352: getdents d1/d25/d2c 0 2024-05-09T06:38:18.879 INFO:tasks.workunit.client.0.smithi063.stdout:5/678: link d8/l92 d8/d19/d30/d74/d77/le4 0 2024-05-09T06:38:18.880 INFO:tasks.workunit.client.0.smithi063.stdout:5/679: readlink d8/d19/d30/d74/d77/l90 0 2024-05-09T06:38:18.880 INFO:tasks.workunit.client.1.smithi193.stdout:5/348: chown d0/d13/d2c/d37/c20 396837238 1 2024-05-09T06:38:18.882 INFO:tasks.workunit.client.0.smithi063.stdout:7/657: link d0/d15/d49/d4f/d66/c7b d0/d15/d49/d54/d9f/ce6 0 2024-05-09T06:38:18.883 INFO:tasks.workunit.client.0.smithi063.stdout:3/709: mkdir d7/da/d21/d30/d44/dee 0 2024-05-09T06:38:18.884 INFO:tasks.workunit.client.0.smithi063.stdout:3/710: dread - d7/d4c/d25/fa0 zero size 2024-05-09T06:38:18.884 INFO:tasks.workunit.client.1.smithi193.stdout:6/353: dread d1/d25/d2c/d3c/d3e/f3f [0,4194304] 0 2024-05-09T06:38:18.885 INFO:tasks.workunit.client.1.smithi193.stdout:3/328: rename d1/d2d/c64 to d1/db/c76 0 2024-05-09T06:38:18.885 INFO:tasks.workunit.client.0.smithi063.stdout:2/716: dread dc/dd/d2d/d60/dd0/d7b/fae [0,4194304] 0 2024-05-09T06:38:18.885 INFO:tasks.workunit.client.1.smithi193.stdout:6/354: dread - d1/de/d11/d2d/d3a/f3b zero size 2024-05-09T06:38:18.887 INFO:tasks.workunit.client.0.smithi063.stdout:1/621: creat d9/d2a/d3c/d4c/d83/d88/dc2/dd0/fd6 x:0 0 0 2024-05-09T06:38:18.888 INFO:tasks.workunit.client.1.smithi193.stdout:2/308: symlink db/d25/d38/d5b/l74 0 2024-05-09T06:38:18.888 INFO:tasks.workunit.client.0.smithi063.stdout:6/696: dread f1 [0,4194304] 0 2024-05-09T06:38:18.889 INFO:tasks.workunit.client.0.smithi063.stdout:1/622: write d9/d2a/d35/d1e/fd1 [577063,114772] 0 2024-05-09T06:38:18.889 INFO:tasks.workunit.client.0.smithi063.stdout:8/758: creat d9/da/d53/dbc/d102/f110 x:0 0 0 2024-05-09T06:38:18.890 INFO:tasks.workunit.client.1.smithi193.stdout:9/393: symlink dd/d1e/d1f/d48/l87 0 2024-05-09T06:38:18.892 INFO:tasks.workunit.client.0.smithi063.stdout:1/623: dread - d9/d2a/d3c/d4c/d83/d88/dc2/dd0/fd6 zero size 2024-05-09T06:38:18.893 INFO:tasks.workunit.client.0.smithi063.stdout:5/680: rename d8/d17/d75/dc9/dd5/la4 to d8/d19/d30/d74/d77/dde/le5 0 2024-05-09T06:38:18.893 INFO:tasks.workunit.client.1.smithi193.stdout:7/357: mkdir d6/d8/d1a/d36/d4d/d74 0 2024-05-09T06:38:18.893 INFO:tasks.workunit.client.1.smithi193.stdout:7/358: readlink d6/l1d 0 2024-05-09T06:38:18.894 INFO:tasks.workunit.client.0.smithi063.stdout:7/658: unlink d0/d15/d49/d4f/d66/d67/lcd 0 2024-05-09T06:38:18.895 INFO:tasks.workunit.client.0.smithi063.stdout:4/685: mkdir d7/da3/ddc 0 2024-05-09T06:38:18.897 INFO:tasks.workunit.client.0.smithi063.stdout:3/711: fdatasync d7/f16 0 2024-05-09T06:38:18.900 INFO:tasks.workunit.client.0.smithi063.stdout:6/697: dread d2/d4/f3c [0,4194304] 0 2024-05-09T06:38:18.900 INFO:tasks.workunit.client.0.smithi063.stdout:8/759: mknod d9/da/d18/dc7/dd3/df1/c111 0 2024-05-09T06:38:18.902 INFO:tasks.workunit.client.0.smithi063.stdout:4/686: dwrite d7/d3d/fab [0,4194304] 0 2024-05-09T06:38:18.903 INFO:tasks.workunit.client.0.smithi063.stdout:5/681: mkdir d8/d19/d30/d46/de6 0 2024-05-09T06:38:18.903 INFO:tasks.workunit.client.0.smithi063.stdout:4/687: write d7/df/f12 [1010263,32513] 0 2024-05-09T06:38:18.907 INFO:tasks.workunit.client.1.smithi193.stdout:2/309: creat db/d25/d38/f75 x:0 0 0 2024-05-09T06:38:18.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/394: write dd/d1e/f27 [6029403,52173] 0 2024-05-09T06:38:18.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/395: read - dd/f82 zero size 2024-05-09T06:38:18.910 INFO:tasks.workunit.client.0.smithi063.stdout:3/712: symlink d7/da/de4/lef 0 2024-05-09T06:38:18.913 INFO:tasks.workunit.client.0.smithi063.stdout:2/717: mkdir dc/dd/d45/d5a/d6c/d8c/d9a/dd5 0 2024-05-09T06:38:18.917 INFO:tasks.workunit.client.0.smithi063.stdout:1/624: mknod d9/d2a/d35/cd7 0 2024-05-09T06:38:18.920 INFO:tasks.workunit.client.1.smithi193.stdout:8/314: getdents d6 0 2024-05-09T06:38:18.925 INFO:tasks.workunit.client.1.smithi193.stdout:1/314: rename d4/d14/c1d to d4/d16/d17/d1a/d3a/d4a/c5a 0 2024-05-09T06:38:18.925 INFO:tasks.workunit.client.1.smithi193.stdout:4/417: truncate d1a/f1b 4407398 0 2024-05-09T06:38:18.925 INFO:tasks.workunit.client.0.smithi063.stdout:4/688: symlink d7/df/d35/da5/ldd 0 2024-05-09T06:38:18.928 INFO:tasks.workunit.client.0.smithi063.stdout:3/713: rename d7/da/d2b/d66/d74/d7e/f8b to d7/d70/d98/d86/ff0 0 2024-05-09T06:38:18.929 INFO:tasks.workunit.client.0.smithi063.stdout:3/714: stat d7/d4c/d25/c59 0 2024-05-09T06:38:18.930 INFO:tasks.workunit.client.0.smithi063.stdout:5/682: write d8/f8e [1087520,57191] 0 2024-05-09T06:38:18.931 INFO:tasks.workunit.client.0.smithi063.stdout:8/760: mknod d9/d41/dec/c112 0 2024-05-09T06:38:18.932 INFO:tasks.workunit.client.0.smithi063.stdout:3/715: write d7/d22/d28/f3c [5042788,105450] 0 2024-05-09T06:38:18.933 INFO:tasks.workunit.client.0.smithi063.stdout:4/689: dwrite d7/da3/d28/d31/d61/f6d [0,4194304] 0 2024-05-09T06:38:18.935 INFO:tasks.workunit.client.0.smithi063.stdout:4/690: stat d7/d50/l52 0 2024-05-09T06:38:18.936 INFO:tasks.workunit.client.0.smithi063.stdout:6/698: link d2/d1f/d29/d3f/d5c/f9e d2/df/d91/fe0 0 2024-05-09T06:38:18.937 INFO:tasks.workunit.client.0.smithi063.stdout:4/691: readlink d7/df/l5c 0 2024-05-09T06:38:18.938 INFO:tasks.workunit.client.0.smithi063.stdout:4/692: dread - d7/d3d/d73/fd4 zero size 2024-05-09T06:38:18.940 INFO:tasks.workunit.client.0.smithi063.stdout:2/718: dwrite dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f86 [0,4194304] 0 2024-05-09T06:38:18.942 INFO:tasks.workunit.client.0.smithi063.stdout:8/761: dwrite d9/da/d34/f59 [4194304,4194304] 0 2024-05-09T06:38:18.943 INFO:tasks.workunit.client.0.smithi063.stdout:3/716: creat d7/d70/d98/dbb/ff1 x:0 0 0 2024-05-09T06:38:18.943 INFO:tasks.workunit.client.0.smithi063.stdout:2/719: truncate dc/dd/fb5 103589 0 2024-05-09T06:38:18.945 INFO:tasks.workunit.client.0.smithi063.stdout:5/683: symlink d8/d19/d30/db1/le7 0 2024-05-09T06:38:18.946 INFO:tasks.workunit.client.0.smithi063.stdout:5/684: fsync d8/d35/f41 0 2024-05-09T06:38:18.946 INFO:tasks.workunit.client.0.smithi063.stdout:4/693: getdents d7/da3/da2 0 2024-05-09T06:38:18.947 INFO:tasks.workunit.client.0.smithi063.stdout:8/762: creat d9/da/d18/dc7/f113 x:0 0 0 2024-05-09T06:38:18.949 INFO:tasks.workunit.client.0.smithi063.stdout:3/717: symlink d7/da/de4/de8/lf2 0 2024-05-09T06:38:18.949 INFO:tasks.workunit.client.0.smithi063.stdout:2/720: dread dc/dd/d2d/d60/dd0/d7b/fae [0,4194304] 0 2024-05-09T06:38:18.950 INFO:tasks.workunit.client.0.smithi063.stdout:5/685: readlink d8/d35/lcc 0 2024-05-09T06:38:18.952 INFO:tasks.workunit.client.0.smithi063.stdout:1/625: link d9/d2a/d35/d1e/f42 d9/d2a/fd8 0 2024-05-09T06:38:18.953 INFO:tasks.workunit.client.0.smithi063.stdout:1/626: chown d9/d2a/d35/d1e/d9d/f82 1557024 1 2024-05-09T06:38:18.956 INFO:tasks.workunit.client.0.smithi063.stdout:5/686: creat d8/d17/d18/d9c/fe8 x:0 0 0 2024-05-09T06:38:18.957 INFO:tasks.workunit.client.0.smithi063.stdout:1/627: write d9/d2a/d35/d24/f53 [2641076,81878] 0 2024-05-09T06:38:18.958 INFO:tasks.workunit.client.0.smithi063.stdout:4/694: link d7/d3d/fab d7/d3d/fde 0 2024-05-09T06:38:18.959 INFO:tasks.workunit.client.0.smithi063.stdout:2/721: mknod dc/d36/d43/d46/d63/d79/d7d/cd6 0 2024-05-09T06:38:18.962 INFO:tasks.workunit.client.0.smithi063.stdout:1/628: creat d9/d2a/d35/d1e/d9d/d3f/d59/fd9 x:0 0 0 2024-05-09T06:38:18.963 INFO:tasks.workunit.client.0.smithi063.stdout:1/629: chown d9/d2a/d3c/d4c/f92 192 1 2024-05-09T06:38:18.963 INFO:tasks.workunit.client.0.smithi063.stdout:4/695: rename d7/ld5 to d7/da3/d28/d31/d32/d34/ldf 0 2024-05-09T06:38:18.964 INFO:tasks.workunit.client.0.smithi063.stdout:2/722: symlink dc/dd/d2d/d60/dd0/d7b/ld7 0 2024-05-09T06:38:18.965 INFO:tasks.workunit.client.0.smithi063.stdout:2/723: stat dc/d36/d43/d46/d63/d79/d7d/d8d/l95 0 2024-05-09T06:38:18.966 INFO:tasks.workunit.client.0.smithi063.stdout:4/696: mkdir d7/d3d/dca/de0 0 2024-05-09T06:38:18.969 INFO:tasks.workunit.client.0.smithi063.stdout:4/697: mkdir d7/da3/d28/d31/d32/d34/db4/dcc/de1 0 2024-05-09T06:38:18.973 INFO:tasks.workunit.client.0.smithi063.stdout:2/724: dwrite dc/d32/f42 [0,4194304] 0 2024-05-09T06:38:18.974 INFO:tasks.workunit.client.0.smithi063.stdout:4/698: rename d7/da6/fd2 to d7/df/d1a/fe2 0 2024-05-09T06:38:18.975 INFO:tasks.workunit.client.0.smithi063.stdout:3/718: dread d7/da/f18 [0,4194304] 0 2024-05-09T06:38:18.975 INFO:tasks.workunit.client.0.smithi063.stdout:4/699: stat d7/da3/l83 0 2024-05-09T06:38:18.982 INFO:tasks.workunit.client.0.smithi063.stdout:5/687: dread d8/d19/f3f [0,4194304] 0 2024-05-09T06:38:18.982 INFO:tasks.workunit.client.0.smithi063.stdout:6/699: dread d2/df/f1d [0,4194304] 0 2024-05-09T06:38:18.983 INFO:tasks.workunit.client.0.smithi063.stdout:4/700: dread d7/da3/d28/d31/d32/d34/db4/fd1 [0,4194304] 0 2024-05-09T06:38:18.986 INFO:tasks.workunit.client.0.smithi063.stdout:4/701: mknod d7/df/d35/d5a/d60/d75/ce3 0 2024-05-09T06:38:18.989 INFO:tasks.workunit.client.0.smithi063.stdout:4/702: fsync d7/d3d/f49 0 2024-05-09T06:38:18.989 INFO:tasks.workunit.client.1.smithi193.stdout:9/396: creat dd/d1e/f88 x:0 0 0 2024-05-09T06:38:18.989 INFO:tasks.workunit.client.1.smithi193.stdout:7/359: getdents d6/d8/d1a/d36/d4d/d74 0 2024-05-09T06:38:18.990 INFO:tasks.workunit.client.1.smithi193.stdout:0/337: rename d18/d27/c35 to d18/d4c/d64/c6b 0 2024-05-09T06:38:18.990 INFO:tasks.workunit.client.0.smithi063.stdout:5/688: getdents d8/d35/db6 0 2024-05-09T06:38:18.991 INFO:tasks.workunit.client.1.smithi193.stdout:1/315: mkdir d4/d16/d17/d1a/d5b 0 2024-05-09T06:38:18.991 INFO:tasks.workunit.client.1.smithi193.stdout:4/418: mknod d1a/c7f 0 2024-05-09T06:38:18.993 INFO:tasks.workunit.client.1.smithi193.stdout:4/419: truncate d1a/d24/f2e 1110287 0 2024-05-09T06:38:18.996 INFO:tasks.workunit.client.1.smithi193.stdout:0/338: read fe [4517322,85954] 0 2024-05-09T06:38:18.996 INFO:tasks.workunit.client.0.smithi063.stdout:5/689: creat d8/dc8/fe9 x:0 0 0 2024-05-09T06:38:18.997 INFO:tasks.workunit.client.1.smithi193.stdout:9/397: dwrite dd/d1e/d1f/d43/d5b/f5d [0,4194304] 0 2024-05-09T06:38:18.999 INFO:tasks.workunit.client.1.smithi193.stdout:3/329: getdents d1 0 2024-05-09T06:38:18.999 INFO:tasks.workunit.client.0.smithi063.stdout:4/703: rmdir d7/da3/d28/d31/d7b/d81 39 2024-05-09T06:38:19.001 INFO:tasks.workunit.client.0.smithi063.stdout:5/690: unlink d8/f8f 0 2024-05-09T06:38:19.002 INFO:tasks.workunit.client.1.smithi193.stdout:1/316: rmdir d4/d14 39 2024-05-09T06:38:19.004 INFO:tasks.workunit.client.0.smithi063.stdout:4/704: unlink d7/df/d35/d5a/f6f 0 2024-05-09T06:38:19.006 INFO:tasks.workunit.client.0.smithi063.stdout:1/630: write d9/d2a/d35/d1e/f70 [1895684,35115] 0 2024-05-09T06:38:19.007 INFO:tasks.workunit.client.0.smithi063.stdout:4/705: dread d7/df/d57/f91 [0,4194304] 0 2024-05-09T06:38:19.008 INFO:tasks.workunit.client.1.smithi193.stdout:1/317: dwrite d4/f57 [0,4194304] 0 2024-05-09T06:38:19.010 INFO:tasks.workunit.client.1.smithi193.stdout:4/420: mkdir d1a/d80 0 2024-05-09T06:38:19.013 INFO:tasks.workunit.client.0.smithi063.stdout:3/719: write d7/d4c/f80 [2396865,9277] 0 2024-05-09T06:38:19.018 INFO:tasks.workunit.client.0.smithi063.stdout:5/691: symlink d8/d19/d30/d74/lea 0 2024-05-09T06:38:19.020 INFO:tasks.workunit.client.0.smithi063.stdout:3/720: rmdir d7/d70 39 2024-05-09T06:38:19.021 INFO:tasks.workunit.client.0.smithi063.stdout:2/725: dwrite dc/d6e/d7f/f97 [0,4194304] 0 2024-05-09T06:38:19.024 INFO:tasks.workunit.client.0.smithi063.stdout:6/700: dwrite d2/d1f/d29/d3f/d5c/d64/f6e [0,4194304] 0 2024-05-09T06:38:19.025 INFO:tasks.workunit.client.0.smithi063.stdout:4/706: creat d7/df/d77/fe4 x:0 0 0 2024-05-09T06:38:19.028 INFO:tasks.workunit.client.1.smithi193.stdout:7/360: write d6/d8/d1a/d36/f3b [2046937,60217] 0 2024-05-09T06:38:19.028 INFO:tasks.workunit.client.0.smithi063.stdout:4/707: dread - d7/d3d/d73/fd4 zero size 2024-05-09T06:38:19.028 INFO:tasks.workunit.client.0.smithi063.stdout:1/631: dwrite d9/d2a/d3c/d4c/d83/d88/fcd [0,4194304] 0 2024-05-09T06:38:19.029 INFO:tasks.workunit.client.1.smithi193.stdout:1/318: creat d4/d3c/d3e/f5c x:0 0 0 2024-05-09T06:38:19.029 INFO:tasks.workunit.client.1.smithi193.stdout:7/361: chown d6/d8/d1a/f3d 3020 1 2024-05-09T06:38:19.030 INFO:tasks.workunit.client.0.smithi063.stdout:4/708: write d7/da3/d28/d31/d32/f8c [4923485,129469] 0 2024-05-09T06:38:19.032 INFO:tasks.workunit.client.0.smithi063.stdout:3/721: dread - d7/d70/feb zero size 2024-05-09T06:38:19.033 INFO:tasks.workunit.client.1.smithi193.stdout:4/421: mknod d1a/d24/c81 0 2024-05-09T06:38:19.034 INFO:tasks.workunit.client.0.smithi063.stdout:2/726: fdatasync dc/f1c 0 2024-05-09T06:38:19.045 INFO:tasks.workunit.client.1.smithi193.stdout:3/330: symlink d1/db/d59/d63/l77 0 2024-05-09T06:38:19.046 INFO:tasks.workunit.client.0.smithi063.stdout:1/632: chown d9/d2a/d35/c2c 40027126 1 2024-05-09T06:38:19.046 INFO:tasks.workunit.client.0.smithi063.stdout:1/633: dread - d9/fbe zero size 2024-05-09T06:38:19.047 INFO:tasks.workunit.client.1.smithi193.stdout:6/355: rename d1/de/d11/d18/d36/d37/d4f/l79 to d1/l7e 0 2024-05-09T06:38:19.047 INFO:tasks.workunit.client.1.smithi193.stdout:2/310: rename db to db/df/d54/d76 22 2024-05-09T06:38:19.049 INFO:tasks.workunit.client.1.smithi193.stdout:6/356: dread - d1/f42 zero size 2024-05-09T06:38:19.049 INFO:tasks.workunit.client.1.smithi193.stdout:8/315: link c2 d6/d1c/d18/d2f/c5c 0 2024-05-09T06:38:19.052 INFO:tasks.workunit.client.1.smithi193.stdout:0/339: symlink d18/d27/l6c 0 2024-05-09T06:38:19.052 INFO:tasks.workunit.client.1.smithi193.stdout:8/316: fsync d6/d1c/d1b/f30 0 2024-05-09T06:38:19.056 INFO:tasks.workunit.client.1.smithi193.stdout:3/331: dwrite d1/d2d/d51/f65 [0,4194304] 0 2024-05-09T06:38:19.062 INFO:tasks.workunit.client.1.smithi193.stdout:7/362: dwrite d6/d8/f35 [0,4194304] 0 2024-05-09T06:38:19.063 INFO:tasks.workunit.client.1.smithi193.stdout:9/398: write dd/f13 [1662985,62680] 0 2024-05-09T06:38:19.064 INFO:tasks.workunit.client.1.smithi193.stdout:7/363: stat d6/d8/d1a/d36/f65 0 2024-05-09T06:38:19.065 INFO:tasks.workunit.client.1.smithi193.stdout:6/357: symlink d1/d25/d2c/l7f 0 2024-05-09T06:38:19.068 INFO:tasks.workunit.client.0.smithi063.stdout:5/692: getdents d8/d19/d30 0 2024-05-09T06:38:19.069 INFO:tasks.workunit.client.1.smithi193.stdout:7/364: dread - d6/dd/d5e/f61 zero size 2024-05-09T06:38:19.069 INFO:tasks.workunit.client.1.smithi193.stdout:3/332: dwrite d1/d2d/d51/f6a [0,4194304] 0 2024-05-09T06:38:19.070 INFO:tasks.workunit.client.1.smithi193.stdout:7/365: write d6/dd/f47 [4723729,21315] 0 2024-05-09T06:38:19.072 INFO:tasks.workunit.client.1.smithi193.stdout:2/311: dread f8 [0,4194304] 0 2024-05-09T06:38:19.072 INFO:tasks.workunit.client.1.smithi193.stdout:0/340: symlink d18/d4c/d64/l6d 0 2024-05-09T06:38:19.073 INFO:tasks.workunit.client.0.smithi063.stdout:6/701: write d2/df/d59/d99/fc2 [586181,44710] 0 2024-05-09T06:38:19.075 INFO:tasks.workunit.client.1.smithi193.stdout:2/312: truncate db/df/d12/f13 2447117 0 2024-05-09T06:38:19.075 INFO:tasks.workunit.client.0.smithi063.stdout:2/727: rename dc/d6e/d7f/fa1 to dc/fd8 0 2024-05-09T06:38:19.076 INFO:tasks.workunit.client.0.smithi063.stdout:1/634: write d9/d46/f6e [349650,74986] 0 2024-05-09T06:38:19.081 INFO:tasks.workunit.client.1.smithi193.stdout:1/319: truncate d4/d16/de/f12 3468881 0 2024-05-09T06:38:19.081 INFO:tasks.workunit.client.1.smithi193.stdout:7/366: dwrite d6/d17/f52 [0,4194304] 0 2024-05-09T06:38:19.082 INFO:tasks.workunit.client.0.smithi063.stdout:4/709: getdents d7/d3d 0 2024-05-09T06:38:19.083 INFO:tasks.workunit.client.1.smithi193.stdout:8/317: truncate d6/d1c/d18/d2f/d40/f2c 686809 0 2024-05-09T06:38:19.083 INFO:tasks.workunit.client.0.smithi063.stdout:4/710: read d7/da3/f65 [1478773,6335] 0 2024-05-09T06:38:19.085 INFO:tasks.workunit.client.1.smithi193.stdout:0/341: dread d18/d1d/f23 [0,4194304] 0 2024-05-09T06:38:19.085 INFO:tasks.workunit.client.0.smithi063.stdout:1/635: dwrite d9/d2a/d35/d1e/d9d/f33 [0,4194304] 0 2024-05-09T06:38:19.085 INFO:tasks.workunit.client.1.smithi193.stdout:9/399: rename dd/df/d31/l3b to dd/d1e/l89 0 2024-05-09T06:38:19.086 INFO:tasks.workunit.client.1.smithi193.stdout:9/400: chown dd/d1e 219 1 2024-05-09T06:38:19.087 INFO:tasks.workunit.client.0.smithi063.stdout:1/636: write d9/d46/f54 [3536737,6181] 0 2024-05-09T06:38:19.087 INFO:tasks.workunit.client.1.smithi193.stdout:9/401: chown dd/d1e/d3c 245561560 1 2024-05-09T06:38:19.088 INFO:tasks.workunit.client.1.smithi193.stdout:0/342: truncate d18/d4c/d47/f58 1794757 0 2024-05-09T06:38:19.089 INFO:tasks.workunit.client.1.smithi193.stdout:6/358: rmdir d1/de/d11/d2d 39 2024-05-09T06:38:19.089 INFO:tasks.workunit.client.1.smithi193.stdout:6/359: read - d1/de/f38 zero size 2024-05-09T06:38:19.096 INFO:tasks.workunit.client.1.smithi193.stdout:3/333: mkdir d1/db/d2f/d78 0 2024-05-09T06:38:19.098 INFO:tasks.workunit.client.0.smithi063.stdout:4/711: mkdir d7/d3d/d73/de5 0 2024-05-09T06:38:19.099 INFO:tasks.workunit.client.0.smithi063.stdout:5/693: dwrite d8/f10 [0,4194304] 0 2024-05-09T06:38:19.100 INFO:tasks.workunit.client.0.smithi063.stdout:6/702: creat d2/df/d91/ddf/fe1 x:0 0 0 2024-05-09T06:38:19.102 INFO:tasks.workunit.client.0.smithi063.stdout:1/637: unlink d9/d46/c8d 0 2024-05-09T06:38:19.103 INFO:tasks.workunit.client.0.smithi063.stdout:5/694: truncate d8/d17/d18/d9c/fac 226700 0 2024-05-09T06:38:19.103 INFO:tasks.workunit.client.0.smithi063.stdout:2/728: dread dc/dd/d2d/d60/fb2 [0,4194304] 0 2024-05-09T06:38:19.108 INFO:tasks.workunit.client.1.smithi193.stdout:9/402: fdatasync dd/df/d31/f4c 0 2024-05-09T06:38:19.108 INFO:tasks.workunit.client.0.smithi063.stdout:4/712: dwrite d7/da3/d28/fd0 [0,4194304] 0 2024-05-09T06:38:19.115 INFO:tasks.workunit.client.1.smithi193.stdout:0/343: symlink d18/d4c/d64/l6e 0 2024-05-09T06:38:19.116 INFO:tasks.workunit.client.0.smithi063.stdout:1/638: fdatasync d9/d2a/d35/d1e/f27 0 2024-05-09T06:38:19.117 INFO:tasks.workunit.client.0.smithi063.stdout:1/639: stat d9/d46/l47 0 2024-05-09T06:38:19.119 INFO:tasks.workunit.client.1.smithi193.stdout:6/360: rename d1/d25/d2c/d76/c7c to d1/d25/c80 0 2024-05-09T06:38:19.121 INFO:tasks.workunit.client.0.smithi063.stdout:9/714: sync 2024-05-09T06:38:19.121 INFO:tasks.workunit.client.0.smithi063.stdout:0/725: sync 2024-05-09T06:38:19.122 INFO:tasks.workunit.client.1.smithi193.stdout:3/334: chown d1/db/c1f 13200 1 2024-05-09T06:38:19.122 INFO:tasks.workunit.client.1.smithi193.stdout:3/335: fdatasync d1/db/d59/f69 0 2024-05-09T06:38:19.123 INFO:tasks.workunit.client.0.smithi063.stdout:2/729: creat dc/dd/d45/d5a/d6c/d8f/dbf/fd9 x:0 0 0 2024-05-09T06:38:19.123 INFO:tasks.workunit.client.1.smithi193.stdout:3/336: write d1/f6d [227992,85668] 0 2024-05-09T06:38:19.125 INFO:tasks.workunit.client.0.smithi063.stdout:5/695: dread d8/d17/f8d [0,4194304] 0 2024-05-09T06:38:19.126 INFO:tasks.workunit.client.1.smithi193.stdout:2/313: mkdir db/df/d54/d67/d77 0 2024-05-09T06:38:19.126 INFO:tasks.workunit.client.1.smithi193.stdout:2/314: fsync db/d25/d38/f75 0 2024-05-09T06:38:19.130 INFO:tasks.workunit.client.1.smithi193.stdout:7/367: creat d6/dd/d5e/d6b/f75 x:0 0 0 2024-05-09T06:38:19.131 INFO:tasks.workunit.client.1.smithi193.stdout:7/368: readlink d6/d8/d1a/l25 0 2024-05-09T06:38:19.131 INFO:tasks.workunit.client.0.smithi063.stdout:5/696: dwrite d8/d53/fa5 [0,4194304] 0 2024-05-09T06:38:19.131 INFO:tasks.workunit.client.0.smithi063.stdout:1/640: dread - d9/d2a/d35/d1e/f8e zero size 2024-05-09T06:38:19.136 INFO:tasks.workunit.client.0.smithi063.stdout:0/726: symlink dd/d1a/d2c/d51/d72/d9e/d23/d31/le4 0 2024-05-09T06:38:19.137 INFO:tasks.workunit.client.1.smithi193.stdout:9/403: mkdir dd/d1e/d1f/d43/d5b/d8a 0 2024-05-09T06:38:19.137 INFO:tasks.workunit.client.0.smithi063.stdout:6/703: dwrite d2/d4/d7c/d42/f61 [0,4194304] 0 2024-05-09T06:38:19.137 INFO:tasks.workunit.client.0.smithi063.stdout:0/727: stat dd/d1a/d2c/d42/fcf 0 2024-05-09T06:38:19.137 INFO:tasks.workunit.client.1.smithi193.stdout:8/318: dwrite d6/f21 [0,4194304] 0 2024-05-09T06:38:19.138 INFO:tasks.workunit.client.1.smithi193.stdout:8/319: chown c2 4 1 2024-05-09T06:38:19.140 INFO:tasks.workunit.client.0.smithi063.stdout:2/730: unlink dc/d36/d43/d46/c96 0 2024-05-09T06:38:19.141 INFO:tasks.workunit.client.1.smithi193.stdout:0/344: mkdir d18/d65/d6f 0 2024-05-09T06:38:19.148 INFO:tasks.workunit.client.1.smithi193.stdout:6/361: rename d1/de/d11/d18/d36 to d1/d25/d2c/d81 0 2024-05-09T06:38:19.148 INFO:tasks.workunit.client.1.smithi193.stdout:3/337: rename d1/db/d59/d63 to d1/db/d59/d63/d79 22 2024-05-09T06:38:19.150 INFO:tasks.workunit.client.0.smithi063.stdout:5/697: dread - d8/d65/f6f zero size 2024-05-09T06:38:19.151 INFO:tasks.workunit.client.1.smithi193.stdout:2/315: symlink db/d25/d38/d61/l78 0 2024-05-09T06:38:19.153 INFO:tasks.workunit.client.1.smithi193.stdout:1/320: write d4/d14/f2e [918364,125609] 0 2024-05-09T06:38:19.155 INFO:tasks.workunit.client.1.smithi193.stdout:1/321: write d4/d16/d17/d1a/d3a/d3f/f41 [634444,34279] 0 2024-05-09T06:38:19.158 INFO:tasks.workunit.client.0.smithi063.stdout:2/731: mkdir dc/d36/d43/dda 0 2024-05-09T06:38:19.159 INFO:tasks.workunit.client.1.smithi193.stdout:9/404: creat dd/d1e/d1f/d43/f8b x:0 0 0 2024-05-09T06:38:19.160 INFO:tasks.workunit.client.0.smithi063.stdout:5/698: fsync f6 0 2024-05-09T06:38:19.160 INFO:tasks.workunit.client.1.smithi193.stdout:9/405: stat dd/d1e/d1f/d61 0 2024-05-09T06:38:19.161 INFO:tasks.workunit.client.0.smithi063.stdout:5/699: chown d8/l57 18 1 2024-05-09T06:38:19.164 INFO:tasks.workunit.client.0.smithi063.stdout:9/715: rename d0/d43/f77 to d0/d8/ffa 0 2024-05-09T06:38:19.166 INFO:tasks.workunit.client.1.smithi193.stdout:0/345: read d18/d4c/f3f [37815,117092] 0 2024-05-09T06:38:19.169 INFO:tasks.workunit.client.0.smithi063.stdout:4/713: write d7/da3/d28/d31/d7b/d81/f8a [1016011,96329] 0 2024-05-09T06:38:19.170 INFO:tasks.workunit.client.0.smithi063.stdout:2/732: fdatasync dc/dd/d2d/d60/dd0/d7b/f9c 0 2024-05-09T06:38:19.175 INFO:tasks.workunit.client.0.smithi063.stdout:7/659: sync 2024-05-09T06:38:19.175 INFO:tasks.workunit.client.0.smithi063.stdout:3/722: sync 2024-05-09T06:38:19.175 INFO:tasks.workunit.client.0.smithi063.stdout:8/763: sync 2024-05-09T06:38:19.175 INFO:tasks.workunit.client.0.smithi063.stdout:6/704: write d2/d20/f44 [1354894,9254] 0 2024-05-09T06:38:19.176 INFO:tasks.workunit.client.0.smithi063.stdout:7/660: fdatasync d0/d15/fd1 0 2024-05-09T06:38:19.181 INFO:tasks.workunit.client.0.smithi063.stdout:7/661: chown d0/d15/d4a/d50/l4d 0 1 2024-05-09T06:38:19.182 INFO:tasks.workunit.client.0.smithi063.stdout:8/764: dread d9/da/d18/d46/f64 [0,4194304] 0 2024-05-09T06:38:19.183 INFO:tasks.workunit.client.0.smithi063.stdout:7/662: write d0/d15/d49/d4f/d66/d67/d6b/fc3 [1105378,16112] 0 2024-05-09T06:38:19.183 INFO:tasks.workunit.client.0.smithi063.stdout:0/728: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d89/f8b [0,4194304] 0 2024-05-09T06:38:19.183 INFO:tasks.workunit.client.0.smithi063.stdout:7/663: chown d0/d15/d49/d54/l78 2475 1 2024-05-09T06:38:19.184 INFO:tasks.workunit.client.0.smithi063.stdout:2/733: mkdir dc/d36/d43/d46/d63/d9e/ddb 0 2024-05-09T06:38:19.184 INFO:tasks.workunit.client.0.smithi063.stdout:8/765: read d9/d37/fa9 [2556785,27973] 0 2024-05-09T06:38:19.184 INFO:tasks.workunit.client.0.smithi063.stdout:1/641: write d9/d2a/d35/d1e/f42 [3616743,39202] 0 2024-05-09T06:38:19.185 INFO:tasks.workunit.client.0.smithi063.stdout:3/723: unlink d7/d22/cc8 0 2024-05-09T06:38:19.186 INFO:tasks.workunit.client.0.smithi063.stdout:1/642: dread - d9/d2a/d39/d5d/d75/fa9 zero size 2024-05-09T06:38:19.187 INFO:tasks.workunit.client.0.smithi063.stdout:3/724: read d7/f16 [49694,112308] 0 2024-05-09T06:38:19.191 INFO:tasks.workunit.client.0.smithi063.stdout:6/705: unlink d2/d1f/d29/d3f/lc3 0 2024-05-09T06:38:19.194 INFO:tasks.workunit.client.0.smithi063.stdout:0/729: creat dd/d60/fe5 x:0 0 0 2024-05-09T06:38:19.194 INFO:tasks.workunit.client.0.smithi063.stdout:7/664: fsync d0/d15/d49/fbf 0 2024-05-09T06:38:19.194 INFO:tasks.workunit.client.0.smithi063.stdout:2/734: rmdir dc/d36/d43 39 2024-05-09T06:38:19.195 INFO:tasks.workunit.client.0.smithi063.stdout:1/643: fdatasync d9/d2a/d39/d5d/f9a 0 2024-05-09T06:38:19.196 INFO:tasks.workunit.client.0.smithi063.stdout:7/665: dread - d0/d15/d49/d54/d9f/d97/fdc zero size 2024-05-09T06:38:19.196 INFO:tasks.workunit.client.0.smithi063.stdout:9/716: rmdir d0/d4/d26/df6 0 2024-05-09T06:38:19.197 INFO:tasks.workunit.client.0.smithi063.stdout:3/725: readlink d7/da/d21/lb2 0 2024-05-09T06:38:19.198 INFO:tasks.workunit.client.0.smithi063.stdout:4/714: creat d7/da3/d28/d31/fe6 x:0 0 0 2024-05-09T06:38:19.201 INFO:tasks.workunit.client.0.smithi063.stdout:7/666: mkdir d0/d15/d89/d98/de7 0 2024-05-09T06:38:19.217 INFO:tasks.workunit.client.1.smithi193.stdout:2/316: mknod db/d25/d38/d5b/c79 0 2024-05-09T06:38:19.218 INFO:tasks.workunit.client.1.smithi193.stdout:1/322: mknod d4/d3c/d3e/c5d 0 2024-05-09T06:38:19.218 INFO:tasks.workunit.client.1.smithi193.stdout:9/406: rename dd/df/d15/d16 to dd/df/d15/d8c 0 2024-05-09T06:38:19.219 INFO:tasks.workunit.client.1.smithi193.stdout:6/362: truncate d1/f4d 773836 0 2024-05-09T06:38:19.220 INFO:tasks.workunit.client.1.smithi193.stdout:0/346: write d18/d4c/d47/f58 [426786,46455] 0 2024-05-09T06:38:19.221 INFO:tasks.workunit.client.1.smithi193.stdout:6/363: truncate d1/de/f10 1722527 0 2024-05-09T06:38:19.221 INFO:tasks.workunit.client.1.smithi193.stdout:7/369: link d6/d8/d1a/l39 d6/d8/l76 0 2024-05-09T06:38:19.222 INFO:tasks.workunit.client.1.smithi193.stdout:2/317: mknod db/d37/c7a 0 2024-05-09T06:38:19.223 INFO:tasks.workunit.client.1.smithi193.stdout:1/323: creat d4/d16/d35/f5e x:0 0 0 2024-05-09T06:38:19.224 INFO:tasks.workunit.client.1.smithi193.stdout:8/320: getdents d6/d1c/d1b/d29 0 2024-05-09T06:38:19.224 INFO:tasks.workunit.client.1.smithi193.stdout:6/364: rmdir d1/d25/d2c/d81 39 2024-05-09T06:38:19.227 INFO:tasks.workunit.client.1.smithi193.stdout:9/407: dread f6 [0,4194304] 0 2024-05-09T06:38:19.228 INFO:tasks.workunit.client.1.smithi193.stdout:7/370: rename d6/dd/d5e/f61 to d6/dd/d5e/f77 0 2024-05-09T06:38:19.229 INFO:tasks.workunit.client.1.smithi193.stdout:2/318: dread db/d25/f27 [0,4194304] 0 2024-05-09T06:38:19.231 INFO:tasks.workunit.client.1.smithi193.stdout:1/324: unlink d4/d16/d17/f54 0 2024-05-09T06:38:19.233 INFO:tasks.workunit.client.1.smithi193.stdout:0/347: write d18/f57 [1440751,88851] 0 2024-05-09T06:38:19.235 INFO:tasks.workunit.client.1.smithi193.stdout:8/321: mkdir d6/d1c/d1b/d5d 0 2024-05-09T06:38:19.236 INFO:tasks.workunit.client.1.smithi193.stdout:7/371: dwrite d6/d8/d1a/d33/f51 [4194304,4194304] 0 2024-05-09T06:38:19.237 INFO:tasks.workunit.client.1.smithi193.stdout:2/319: rename db/d25/d38/d61/l78 to db/df/d54/l7b 0 2024-05-09T06:38:19.239 INFO:tasks.workunit.client.1.smithi193.stdout:1/325: creat d4/d16/f5f x:0 0 0 2024-05-09T06:38:19.240 INFO:tasks.workunit.client.1.smithi193.stdout:9/408: link dd/d1e/d1f/f6f dd/d1e/d66/f8d 0 2024-05-09T06:38:19.241 INFO:tasks.workunit.client.1.smithi193.stdout:0/348: chown f8 110 1 2024-05-09T06:38:19.242 INFO:tasks.workunit.client.1.smithi193.stdout:8/322: mkdir d6/d1c/d18/d2f/d40/d5e 0 2024-05-09T06:38:19.244 INFO:tasks.workunit.client.1.smithi193.stdout:8/323: fsync d6/f21 0 2024-05-09T06:38:19.245 INFO:tasks.workunit.client.1.smithi193.stdout:7/372: dwrite d6/dd/f68 [4194304,4194304] 0 2024-05-09T06:38:19.247 INFO:tasks.workunit.client.1.smithi193.stdout:9/409: fdatasync dd/df/d31/d7e/f32 0 2024-05-09T06:38:19.249 INFO:tasks.workunit.client.1.smithi193.stdout:9/410: write dd/df/d31/d7e/d37/f4d [1650388,17585] 0 2024-05-09T06:38:19.250 INFO:tasks.workunit.client.1.smithi193.stdout:6/365: getdents d1/d25/d2c/d3c 0 2024-05-09T06:38:19.252 INFO:tasks.workunit.client.1.smithi193.stdout:8/324: chown d6/c31 4924 1 2024-05-09T06:38:19.264 INFO:tasks.workunit.client.1.smithi193.stdout:7/373: link d6/dd/d53/f6a d6/d8/d1a/d36/d4d/f78 0 2024-05-09T06:38:19.266 INFO:tasks.workunit.client.1.smithi193.stdout:7/374: write d6/f16 [11134705,33091] 0 2024-05-09T06:38:19.266 INFO:tasks.workunit.client.1.smithi193.stdout:9/411: dwrite dd/df/f6e [0,4194304] 0 2024-05-09T06:38:19.268 INFO:tasks.workunit.client.1.smithi193.stdout:7/375: chown d6/dd/d53/f6a 3 1 2024-05-09T06:38:19.271 INFO:tasks.workunit.client.1.smithi193.stdout:8/325: symlink d6/d1c/d1b/d29/d57/l5f 0 2024-05-09T06:38:19.276 INFO:tasks.workunit.client.1.smithi193.stdout:0/349: dread d18/d27/f53 [0,4194304] 0 2024-05-09T06:38:19.276 INFO:tasks.workunit.client.1.smithi193.stdout:2/320: dwrite db/d25/f33 [0,4194304] 0 2024-05-09T06:38:19.280 INFO:tasks.workunit.client.0.smithi063.stdout:2/735: rename dc/d6e/d7f/dc7 to dc/d36/d43/d46/d49/ddc 0 2024-05-09T06:38:19.280 INFO:tasks.workunit.client.0.smithi063.stdout:1/644: mknod d9/d2a/cda 0 2024-05-09T06:38:19.281 INFO:tasks.workunit.client.0.smithi063.stdout:3/726: symlink d7/da/lf3 0 2024-05-09T06:38:19.281 INFO:tasks.workunit.client.1.smithi193.stdout:9/412: rmdir dd 39 2024-05-09T06:38:19.281 INFO:tasks.workunit.client.0.smithi063.stdout:8/766: write d9/da/d45/f8d [802571,35246] 0 2024-05-09T06:38:19.282 INFO:tasks.workunit.client.0.smithi063.stdout:1/645: chown d9/d76 545453534 1 2024-05-09T06:38:19.283 INFO:tasks.workunit.client.0.smithi063.stdout:0/730: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d46/fdf [0,4194304] 0 2024-05-09T06:38:19.284 INFO:tasks.workunit.client.1.smithi193.stdout:7/376: symlink d6/d8/d1a/l79 0 2024-05-09T06:38:19.286 INFO:tasks.workunit.client.0.smithi063.stdout:0/731: fdatasync dd/d1a/f35 0 2024-05-09T06:38:19.286 INFO:tasks.workunit.client.1.smithi193.stdout:6/366: write d1/de/d11/d2d/f40 [523083,99803] 0 2024-05-09T06:38:19.286 INFO:tasks.workunit.client.1.smithi193.stdout:7/377: write d6/d8/f44 [1516557,3274] 0 2024-05-09T06:38:19.289 INFO:tasks.workunit.client.0.smithi063.stdout:0/732: stat dd/d60/fd4 0 2024-05-09T06:38:19.290 INFO:tasks.workunit.client.0.smithi063.stdout:2/736: fsync dc/f2a 0 2024-05-09T06:38:19.291 INFO:tasks.workunit.client.0.smithi063.stdout:3/727: mknod d7/da/de4/cf4 0 2024-05-09T06:38:19.292 INFO:tasks.workunit.client.1.smithi193.stdout:0/350: rmdir d18/d4c 39 2024-05-09T06:38:19.294 INFO:tasks.workunit.client.0.smithi063.stdout:6/706: dwrite d2/df/f1d [0,4194304] 0 2024-05-09T06:38:19.294 INFO:tasks.workunit.client.0.smithi063.stdout:2/737: chown dc/d36/d43/l94 4378 1 2024-05-09T06:38:19.298 INFO:tasks.workunit.client.0.smithi063.stdout:8/767: dwrite d9/da/f9a [0,4194304] 0 2024-05-09T06:38:19.299 INFO:tasks.workunit.client.0.smithi063.stdout:0/733: mknod dd/d60/d44/ce6 0 2024-05-09T06:38:19.301 INFO:tasks.workunit.client.0.smithi063.stdout:4/715: getdents d7 0 2024-05-09T06:38:19.302 INFO:tasks.workunit.client.0.smithi063.stdout:0/734: write dd/d1a/d2c/d51/d72/fd3 [391480,91442] 0 2024-05-09T06:38:19.302 INFO:tasks.workunit.client.0.smithi063.stdout:4/716: chown d7 6823 1 2024-05-09T06:38:19.302 INFO:tasks.workunit.client.1.smithi193.stdout:1/326: rename d4/d14/d15 to d4/d16/d60 0 2024-05-09T06:38:19.304 INFO:tasks.workunit.client.0.smithi063.stdout:4/717: read - d7/da3/d28/d31/fe6 zero size 2024-05-09T06:38:19.306 INFO:tasks.workunit.client.1.smithi193.stdout:2/321: write db/d25/f27 [1783971,100570] 0 2024-05-09T06:38:19.308 INFO:tasks.workunit.client.0.smithi063.stdout:3/728: creat d7/da/d2b/d66/dc0/ff5 x:0 0 0 2024-05-09T06:38:19.310 INFO:tasks.workunit.client.1.smithi193.stdout:8/326: dwrite d6/d1c/d18/f26 [0,4194304] 0 2024-05-09T06:38:19.311 INFO:tasks.workunit.client.0.smithi063.stdout:1/646: unlink d9/d2a/d39/d5d/l97 0 2024-05-09T06:38:19.312 INFO:tasks.workunit.client.1.smithi193.stdout:0/351: chown d18/d27/l2e 154 1 2024-05-09T06:38:19.312 INFO:tasks.workunit.client.0.smithi063.stdout:4/718: dwrite d7/da3/d28/d31/d71/dbb/fc0 [0,4194304] 0 2024-05-09T06:38:19.312 INFO:tasks.workunit.client.0.smithi063.stdout:7/667: dread d0/d15/d49/f4c [0,4194304] 0 2024-05-09T06:38:19.313 INFO:tasks.workunit.client.0.smithi063.stdout:2/738: creat dc/dd/d45/d5a/fdd x:0 0 0 2024-05-09T06:38:19.314 INFO:tasks.workunit.client.0.smithi063.stdout:8/768: dread - d9/da/d53/fcb zero size 2024-05-09T06:38:19.314 INFO:tasks.workunit.client.1.smithi193.stdout:9/413: creat dd/df/d15/f8e x:0 0 0 2024-05-09T06:38:19.315 INFO:tasks.workunit.client.0.smithi063.stdout:0/735: creat dd/d1a/d2c/d51/d76/d97/fe7 x:0 0 0 2024-05-09T06:38:19.315 INFO:tasks.workunit.client.1.smithi193.stdout:1/327: readlink d4/d16/de/l56 0 2024-05-09T06:38:19.316 INFO:tasks.workunit.client.1.smithi193.stdout:7/378: readlink d6/d8/d1a/l20 0 2024-05-09T06:38:19.318 INFO:tasks.workunit.client.1.smithi193.stdout:6/367: mknod d1/d25/d2c/c82 0 2024-05-09T06:38:19.319 INFO:tasks.workunit.client.1.smithi193.stdout:6/368: fdatasync d1/de/d11/d18/f2a 0 2024-05-09T06:38:19.322 INFO:tasks.workunit.client.0.smithi063.stdout:7/668: creat d0/d15/d4a/d68/fe8 x:0 0 0 2024-05-09T06:38:19.323 INFO:tasks.workunit.client.1.smithi193.stdout:8/327: creat d6/d1c/d53/f60 x:0 0 0 2024-05-09T06:38:19.326 INFO:tasks.workunit.client.0.smithi063.stdout:4/719: creat d7/da3/d28/fe7 x:0 0 0 2024-05-09T06:38:19.327 INFO:tasks.workunit.client.1.smithi193.stdout:9/414: symlink dd/df/d31/d7e/l8f 0 2024-05-09T06:38:19.328 INFO:tasks.workunit.client.1.smithi193.stdout:9/415: chown dd/df/d31/l3d 97112 1 2024-05-09T06:38:19.328 INFO:tasks.workunit.client.1.smithi193.stdout:9/416: readlink dd/df/l34 0 2024-05-09T06:38:19.329 INFO:tasks.workunit.client.1.smithi193.stdout:9/417: dread - dd/d1e/d1f/d48/d55/d41/d68/f78 zero size 2024-05-09T06:38:19.330 INFO:tasks.workunit.client.0.smithi063.stdout:0/736: chown dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/cb0 2151 1 2024-05-09T06:38:19.331 INFO:tasks.workunit.client.1.smithi193.stdout:7/379: creat d6/d8/d29/f7a x:0 0 0 2024-05-09T06:38:19.333 INFO:tasks.workunit.client.0.smithi063.stdout:2/739: fdatasync dc/f1b 0 2024-05-09T06:38:19.336 INFO:tasks.workunit.client.0.smithi063.stdout:7/669: rename d0/d15/f35 to d0/d15/d4a/dc8/fe9 0 2024-05-09T06:38:19.338 INFO:tasks.workunit.client.0.smithi063.stdout:7/670: chown d0/d15/d4a/d50/f3f 236 1 2024-05-09T06:38:19.339 INFO:tasks.workunit.client.1.smithi193.stdout:8/328: rename d6/d1c/l45 to d6/d1c/d1b/d29/l61 0 2024-05-09T06:38:19.339 INFO:tasks.workunit.client.0.smithi063.stdout:6/707: dread d2/d12/f24 [0,4194304] 0 2024-05-09T06:38:19.340 INFO:tasks.workunit.client.1.smithi193.stdout:8/329: fdatasync d6/f21 0 2024-05-09T06:38:19.341 INFO:tasks.workunit.client.0.smithi063.stdout:2/740: dwrite dc/f21 [8388608,4194304] 0 2024-05-09T06:38:19.344 INFO:tasks.workunit.client.1.smithi193.stdout:1/328: dread d4/d16/d60/f58 [0,4194304] 0 2024-05-09T06:38:19.345 INFO:tasks.workunit.client.0.smithi063.stdout:1/647: write d9/d2a/d3c/d4c/f92 [644404,77089] 0 2024-05-09T06:38:19.351 INFO:tasks.workunit.client.1.smithi193.stdout:8/330: dread d6/f21 [0,4194304] 0 2024-05-09T06:38:19.351 INFO:tasks.workunit.client.0.smithi063.stdout:8/769: dread d9/d41/dec/dc5/fd2 [0,4194304] 0 2024-05-09T06:38:19.353 INFO:tasks.workunit.client.1.smithi193.stdout:8/331: fdatasync d6/d1c/d53/f60 0 2024-05-09T06:38:19.353 INFO:tasks.workunit.client.1.smithi193.stdout:2/322: dwrite db/df/d12/f14 [0,4194304] 0 2024-05-09T06:38:19.353 INFO:tasks.workunit.client.0.smithi063.stdout:8/770: truncate d9/d41/dec/db8/fd4 527111 0 2024-05-09T06:38:19.353 INFO:tasks.workunit.client.0.smithi063.stdout:6/708: truncate d2/d1f/d29/d3f/d5c/f70 2997365 0 2024-05-09T06:38:19.354 INFO:tasks.workunit.client.1.smithi193.stdout:8/332: chown d6/d1c/d27/f3b 3888612 1 2024-05-09T06:38:19.357 INFO:tasks.workunit.client.1.smithi193.stdout:6/369: fsync d1/de/d11/d18/f65 0 2024-05-09T06:38:19.358 INFO:tasks.workunit.client.0.smithi063.stdout:1/648: symlink d9/d2a/d35/d1e/d9d/d3f/ldb 0 2024-05-09T06:38:19.360 INFO:tasks.workunit.client.0.smithi063.stdout:0/737: dwrite dd/d1a/d2c/f92 [0,4194304] 0 2024-05-09T06:38:19.361 INFO:tasks.workunit.client.0.smithi063.stdout:8/771: fdatasync d9/da/d18/d46/f50 0 2024-05-09T06:38:19.361 INFO:tasks.workunit.client.0.smithi063.stdout:6/709: read d2/d12/f76 [1061355,111808] 0 2024-05-09T06:38:19.363 INFO:tasks.workunit.client.0.smithi063.stdout:2/741: link f3 dc/dd/d45/d5a/d6c/d8c/d9a/dd5/fde 0 2024-05-09T06:38:19.363 INFO:tasks.workunit.client.0.smithi063.stdout:1/649: truncate d9/d2a/d3c/d4c/dc3/fd2 83633 0 2024-05-09T06:38:19.367 INFO:tasks.workunit.client.1.smithi193.stdout:0/352: mkdir d18/d4c/d47/d5e/d70 0 2024-05-09T06:38:19.369 INFO:tasks.workunit.client.0.smithi063.stdout:8/772: mkdir d9/da/d45/d114 0 2024-05-09T06:38:19.371 INFO:tasks.workunit.client.0.smithi063.stdout:2/742: dwrite dc/d36/d43/d46/fcc [0,4194304] 0 2024-05-09T06:38:19.372 INFO:tasks.workunit.client.0.smithi063.stdout:2/743: chown dc/d32/d33/f61 44672715 1 2024-05-09T06:38:19.374 INFO:tasks.workunit.client.0.smithi063.stdout:6/710: rename d2/d1f/d29/d3b to d2/d1f/d29/de2 0 2024-05-09T06:38:19.375 INFO:tasks.workunit.client.0.smithi063.stdout:6/711: dread - d2/d1f/d6c/fb7 zero size 2024-05-09T06:38:19.377 INFO:tasks.workunit.client.0.smithi063.stdout:2/744: dread dc/dd/d45/d5a/d6c/d8c/d9a/dd5/fde [0,4194304] 0 2024-05-09T06:38:19.378 INFO:tasks.workunit.client.0.smithi063.stdout:1/650: creat d9/d2a/d3c/d4a/fdc x:0 0 0 2024-05-09T06:38:19.379 INFO:tasks.workunit.client.1.smithi193.stdout:7/380: mknod d6/c7b 0 2024-05-09T06:38:19.381 INFO:tasks.workunit.client.0.smithi063.stdout:7/671: getdents d0/d15/d49/d4f/d66 0 2024-05-09T06:38:19.382 INFO:tasks.workunit.client.0.smithi063.stdout:7/672: chown d0/d15/d4a/d68/f6f 0 1 2024-05-09T06:38:19.383 INFO:tasks.workunit.client.0.smithi063.stdout:7/673: write d0/d15/d4a/f7e [496252,47275] 0 2024-05-09T06:38:19.385 INFO:tasks.workunit.client.1.smithi193.stdout:8/333: creat d6/d1c/d1b/d3a/f62 x:0 0 0 2024-05-09T06:38:19.390 INFO:tasks.workunit.client.0.smithi063.stdout:8/773: rename d9/f13 to d9/d3b/f115 0 2024-05-09T06:38:19.390 INFO:tasks.workunit.client.1.smithi193.stdout:8/334: write d6/d1c/f3d [845751,101157] 0 2024-05-09T06:38:19.390 INFO:tasks.workunit.client.1.smithi193.stdout:0/353: mknod d18/d4c/d47/d5e/c71 0 2024-05-09T06:38:19.391 INFO:tasks.workunit.client.1.smithi193.stdout:2/323: dwrite f8 [8388608,4194304] 0 2024-05-09T06:38:19.394 INFO:tasks.workunit.client.1.smithi193.stdout:1/329: rename d4/d16/f25 to d4/d16/d60/d4f/f61 0 2024-05-09T06:38:19.395 INFO:tasks.workunit.client.1.smithi193.stdout:1/330: readlink d4/d14/l49 0 2024-05-09T06:38:19.396 INFO:tasks.workunit.client.0.smithi063.stdout:8/774: dwrite d9/da/d51/dc2/fd9 [0,4194304] 0 2024-05-09T06:38:19.401 INFO:tasks.workunit.client.0.smithi063.stdout:1/651: write d9/d2a/d3c/d4c/dc3/fd2 [965033,53152] 0 2024-05-09T06:38:19.403 INFO:tasks.workunit.client.1.smithi193.stdout:9/418: truncate dd/df/d15/d8c/f2a 1638108 0 2024-05-09T06:38:19.407 INFO:tasks.workunit.client.0.smithi063.stdout:6/712: dread d2/d4/f92 [0,4194304] 0 2024-05-09T06:38:19.407 INFO:tasks.workunit.client.1.smithi193.stdout:8/335: rename d6/d1c/d1b/f2b to d6/d1c/d53/f63 0 2024-05-09T06:38:19.409 INFO:tasks.workunit.client.1.smithi193.stdout:8/336: write d6/d1c/d1b/d29/f4f [5023833,21807] 0 2024-05-09T06:38:19.412 INFO:tasks.workunit.client.0.smithi063.stdout:7/674: dread d0/d15/d4a/d68/f9e [0,4194304] 0 2024-05-09T06:38:19.413 INFO:tasks.workunit.client.0.smithi063.stdout:8/775: truncate d9/da/d18/dc7/f78 247673 0 2024-05-09T06:38:19.414 INFO:tasks.workunit.client.1.smithi193.stdout:9/419: mknod dd/d1e/d66/d49/c90 0 2024-05-09T06:38:19.414 INFO:tasks.workunit.client.0.smithi063.stdout:8/776: write d9/da/d18/dc7/f113 [587116,96998] 0 2024-05-09T06:38:19.417 INFO:tasks.workunit.client.0.smithi063.stdout:2/745: link dc/f25 dc/d36/d43/fdf 0 2024-05-09T06:38:19.417 INFO:tasks.workunit.client.0.smithi063.stdout:7/675: dread d0/d15/d49/d4f/d66/d67/d86/f90 [0,4194304] 0 2024-05-09T06:38:19.417 INFO:tasks.workunit.client.0.smithi063.stdout:2/746: stat dc/d32/d33/l54 0 2024-05-09T06:38:19.417 INFO:tasks.workunit.client.0.smithi063.stdout:7/676: fdatasync d0/ff 0 2024-05-09T06:38:19.418 INFO:tasks.workunit.client.0.smithi063.stdout:7/677: fdatasync d0/d15/f76 0 2024-05-09T06:38:19.419 INFO:tasks.workunit.client.0.smithi063.stdout:7/678: chown d0/d15/d4a/d68/f9e 141120950 1 2024-05-09T06:38:19.421 INFO:tasks.workunit.client.1.smithi193.stdout:9/420: symlink dd/d1e/d1f/d48/d55/l91 0 2024-05-09T06:38:19.422 INFO:tasks.workunit.client.1.smithi193.stdout:9/421: stat dd/df/d15/d8c/c2c 0 2024-05-09T06:38:19.423 INFO:tasks.workunit.client.1.smithi193.stdout:8/337: creat d6/d5b/f64 x:0 0 0 2024-05-09T06:38:19.423 INFO:tasks.workunit.client.1.smithi193.stdout:9/422: write dd/d1e/f88 [553914,56074] 0 2024-05-09T06:38:19.424 INFO:tasks.workunit.client.0.smithi063.stdout:0/738: getdents dd/d1a/d2c/d51/d72/d9e/d23/d46 0 2024-05-09T06:38:19.424 INFO:tasks.workunit.client.1.smithi193.stdout:2/324: write db/d25/f2a [116826,63471] 0 2024-05-09T06:38:19.425 INFO:tasks.workunit.client.1.smithi193.stdout:2/325: readlink db/d25/d38/d40/l52 0 2024-05-09T06:38:19.429 INFO:tasks.workunit.client.1.smithi193.stdout:7/381: creat d6/d8/d1a/d36/d4d/d74/f7c x:0 0 0 2024-05-09T06:38:19.430 INFO:tasks.workunit.client.0.smithi063.stdout:8/777: rmdir d9/da/d53/dbc 39 2024-05-09T06:38:19.432 INFO:tasks.workunit.client.1.smithi193.stdout:8/338: creat d6/d1c/d27/f65 x:0 0 0 2024-05-09T06:38:19.432 INFO:tasks.workunit.client.1.smithi193.stdout:2/326: dwrite db/d25/d38/f75 [0,4194304] 0 2024-05-09T06:38:19.433 INFO:tasks.workunit.client.1.smithi193.stdout:9/423: dread - dd/df/d15/d8c/f50 zero size 2024-05-09T06:38:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:5/700: sync 2024-05-09T06:38:19.435 INFO:tasks.workunit.client.1.smithi193.stdout:1/331: link d4/d16/d60/c23 d4/d16/c62 0 2024-05-09T06:38:19.437 INFO:tasks.workunit.client.0.smithi063.stdout:2/747: mknod dc/dd/d45/d5a/d6a/ce0 0 2024-05-09T06:38:19.445 INFO:tasks.workunit.client.0.smithi063.stdout:0/739: creat dd/d1a/d2c/d51/d72/d9e/d23/d89/fe8 x:0 0 0 2024-05-09T06:38:19.448 INFO:tasks.workunit.client.1.smithi193.stdout:2/327: creat db/df/d54/f7c x:0 0 0 2024-05-09T06:38:19.448 INFO:tasks.workunit.client.1.smithi193.stdout:7/382: dread d6/f30 [0,4194304] 0 2024-05-09T06:38:19.451 INFO:tasks.workunit.client.1.smithi193.stdout:9/424: unlink dd/d1e/d1f/d48/f53 0 2024-05-09T06:38:19.451 INFO:tasks.workunit.client.0.smithi063.stdout:4/720: sync 2024-05-09T06:38:19.451 INFO:tasks.workunit.client.0.smithi063.stdout:9/717: sync 2024-05-09T06:38:19.451 INFO:tasks.workunit.client.0.smithi063.stdout:3/729: sync 2024-05-09T06:38:19.452 INFO:tasks.workunit.client.0.smithi063.stdout:8/778: truncate d9/da/d18/d46/f64 902960 0 2024-05-09T06:38:19.453 INFO:tasks.workunit.client.0.smithi063.stdout:8/779: chown d9/da/d18/dc7/f91 26 1 2024-05-09T06:38:19.453 INFO:tasks.workunit.client.0.smithi063.stdout:3/730: write d7/da/d2b/d66/fdb [2931350,21239] 0 2024-05-09T06:38:19.454 INFO:tasks.workunit.client.0.smithi063.stdout:7/679: write d0/d15/f25 [5207699,2621] 0 2024-05-09T06:38:19.457 INFO:tasks.workunit.client.0.smithi063.stdout:8/780: write d9/da/d45/de4/ff3 [4577343,127404] 0 2024-05-09T06:38:19.458 INFO:tasks.workunit.client.0.smithi063.stdout:3/731: truncate d7/d4c/d25/fa0 678342 0 2024-05-09T06:38:19.460 INFO:tasks.workunit.client.0.smithi063.stdout:3/732: dread - d7/da/d2b/d66/dc0/ff5 zero size 2024-05-09T06:38:19.460 INFO:tasks.workunit.client.0.smithi063.stdout:9/718: dwrite d0/d4/d26/d47/fa8 [0,4194304] 0 2024-05-09T06:38:19.463 INFO:tasks.workunit.client.1.smithi193.stdout:1/332: getdents d4/d16/d60/d4f/d59 0 2024-05-09T06:38:19.466 INFO:tasks.workunit.client.0.smithi063.stdout:4/721: creat d7/d3d/fe8 x:0 0 0 2024-05-09T06:38:19.468 INFO:tasks.workunit.client.1.smithi193.stdout:8/339: rename d6/d1c/d1b/d3a to d6/d1c/d18/d66 0 2024-05-09T06:38:19.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/701: write d8/d53/dca/f67 [703105,100307] 0 2024-05-09T06:38:19.469 INFO:tasks.workunit.client.1.smithi193.stdout:8/340: chown d6/d1c/d18/c28 10908904 1 2024-05-09T06:38:19.470 INFO:tasks.workunit.client.0.smithi063.stdout:5/702: fdatasync d8/d19/d30/d46/f6c 0 2024-05-09T06:38:19.470 INFO:tasks.workunit.client.0.smithi063.stdout:1/652: getdents d9 0 2024-05-09T06:38:19.471 INFO:tasks.workunit.client.1.smithi193.stdout:2/328: mkdir db/df/d12/d60/d7d 0 2024-05-09T06:38:19.472 INFO:tasks.workunit.client.1.smithi193.stdout:7/383: mkdir d6/d17/d7d 0 2024-05-09T06:38:19.475 INFO:tasks.workunit.client.1.smithi193.stdout:8/341: dread d6/fd [0,4194304] 0 2024-05-09T06:38:19.481 INFO:tasks.workunit.client.0.smithi063.stdout:0/740: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fb6 [0,4194304] 0 2024-05-09T06:38:19.482 INFO:tasks.workunit.client.1.smithi193.stdout:9/425: dread dd/df/f1d [0,4194304] 0 2024-05-09T06:38:19.485 INFO:tasks.workunit.client.0.smithi063.stdout:2/748: truncate dc/d36/d43/d46/fcc 950251 0 2024-05-09T06:38:19.486 INFO:tasks.workunit.client.0.smithi063.stdout:6/713: getdents d2/df/d59/da3 0 2024-05-09T06:38:19.487 INFO:tasks.workunit.client.1.smithi193.stdout:8/342: unlink d6/d1c/d27/f3b 0 2024-05-09T06:38:19.489 INFO:tasks.workunit.client.0.smithi063.stdout:0/741: dread dd/d1a/d2c/d42/fcf [0,4194304] 0 2024-05-09T06:38:19.490 INFO:tasks.workunit.client.1.smithi193.stdout:7/384: mkdir d6/dd/d7e 0 2024-05-09T06:38:19.491 INFO:tasks.workunit.client.1.smithi193.stdout:9/426: symlink dd/d1e/d1f/d48/d55/l92 0 2024-05-09T06:38:19.492 INFO:tasks.workunit.client.1.smithi193.stdout:2/329: dread db/df/d12/f17 [0,4194304] 0 2024-05-09T06:38:19.492 INFO:tasks.workunit.client.1.smithi193.stdout:9/427: fdatasync dd/df/d31/f5f 0 2024-05-09T06:38:19.492 INFO:tasks.workunit.client.0.smithi063.stdout:6/714: dwrite f1 [0,4194304] 0 2024-05-09T06:38:19.496 INFO:tasks.workunit.client.1.smithi193.stdout:8/343: dread d6/d1c/d1b/f4a [0,4194304] 0 2024-05-09T06:38:19.496 INFO:tasks.workunit.client.0.smithi063.stdout:5/703: mkdir d8/d17/da2/deb 0 2024-05-09T06:38:19.497 INFO:tasks.workunit.client.1.smithi193.stdout:8/344: dread - d6/d1c/d53/f60 zero size 2024-05-09T06:38:19.497 INFO:tasks.workunit.client.0.smithi063.stdout:5/704: write d8/d17/da2/fe1 [993263,58580] 0 2024-05-09T06:38:19.498 INFO:tasks.workunit.client.0.smithi063.stdout:5/705: stat d8/d53/dca 0 2024-05-09T06:38:19.498 INFO:tasks.workunit.client.1.smithi193.stdout:1/333: rename d4/d16/d17/d1a/d3a/d3f/f41 to d4/d16/f63 0 2024-05-09T06:38:19.499 INFO:tasks.workunit.client.0.smithi063.stdout:5/706: dread - d8/d17/d75/dc9/fd7 zero size 2024-05-09T06:38:19.500 INFO:tasks.workunit.client.0.smithi063.stdout:5/707: truncate d8/d17/f6a 177792 0 2024-05-09T06:38:19.501 INFO:tasks.workunit.client.0.smithi063.stdout:5/708: write d8/d53/fa5 [4826759,60785] 0 2024-05-09T06:38:19.502 INFO:tasks.workunit.client.0.smithi063.stdout:7/680: symlink d0/lea 0 2024-05-09T06:38:19.503 INFO:tasks.workunit.client.0.smithi063.stdout:7/681: fsync d0/d15/d49/d54/d9f/d97/fde 0 2024-05-09T06:38:19.503 INFO:tasks.workunit.client.0.smithi063.stdout:5/709: chown d8/d19/d30/d74/d77 1006 1 2024-05-09T06:38:19.504 INFO:tasks.workunit.client.0.smithi063.stdout:7/682: truncate d0/f32 4401727 0 2024-05-09T06:38:19.505 INFO:tasks.workunit.client.1.smithi193.stdout:9/428: rmdir dd/d1e/d1f/d48/d55/d41/d68 39 2024-05-09T06:38:19.507 INFO:tasks.workunit.client.1.smithi193.stdout:9/429: dread f6 [0,4194304] 0 2024-05-09T06:38:19.508 INFO:tasks.workunit.client.1.smithi193.stdout:7/385: symlink d6/d8/d1a/l7f 0 2024-05-09T06:38:19.509 INFO:tasks.workunit.client.1.smithi193.stdout:7/386: write d6/d8/f35 [1920922,20862] 0 2024-05-09T06:38:19.509 INFO:tasks.workunit.client.1.smithi193.stdout:5/349: sync 2024-05-09T06:38:19.510 INFO:tasks.workunit.client.1.smithi193.stdout:8/345: symlink d6/d1c/d18/d66/l67 0 2024-05-09T06:38:19.513 INFO:tasks.workunit.client.1.smithi193.stdout:2/330: creat db/df/d12/d60/d7d/f7e x:0 0 0 2024-05-09T06:38:19.514 INFO:tasks.workunit.client.1.smithi193.stdout:2/331: dread - db/d25/d38/f6b zero size 2024-05-09T06:38:19.516 INFO:tasks.workunit.client.1.smithi193.stdout:9/430: mkdir dd/d1e/d93 0 2024-05-09T06:38:19.519 INFO:tasks.workunit.client.0.smithi063.stdout:4/722: unlink d7/c11 0 2024-05-09T06:38:19.521 INFO:tasks.workunit.client.0.smithi063.stdout:6/715: symlink d2/d1f/le3 0 2024-05-09T06:38:19.523 INFO:tasks.workunit.client.0.smithi063.stdout:1/653: unlink d9/d76/fce 0 2024-05-09T06:38:19.525 INFO:tasks.workunit.client.0.smithi063.stdout:5/710: dread d8/d19/d30/f43 [0,4194304] 0 2024-05-09T06:38:19.526 INFO:tasks.workunit.client.1.smithi193.stdout:8/346: dread d6/fd [0,4194304] 0 2024-05-09T06:38:19.527 INFO:tasks.workunit.client.1.smithi193.stdout:1/334: write d4/f9 [281820,121742] 0 2024-05-09T06:38:19.528 INFO:tasks.workunit.client.0.smithi063.stdout:0/742: write dd/f1d [100491,86896] 0 2024-05-09T06:38:19.529 INFO:tasks.workunit.client.0.smithi063.stdout:0/743: read - dd/d1a/d50/f7a zero size 2024-05-09T06:38:19.530 INFO:tasks.workunit.client.1.smithi193.stdout:7/387: dwrite d6/d8/d1a/d33/f3e [0,4194304] 0 2024-05-09T06:38:19.530 INFO:tasks.workunit.client.0.smithi063.stdout:0/744: chown dd/d1a/l3d 1528296 1 2024-05-09T06:38:19.533 INFO:tasks.workunit.client.1.smithi193.stdout:9/431: rename dd/l63 to dd/df/d31/d7e/l94 0 2024-05-09T06:38:19.534 INFO:tasks.workunit.client.0.smithi063.stdout:2/749: creat dc/dd/d2d/d60/dd0/d90/dc1/fe1 x:0 0 0 2024-05-09T06:38:19.535 INFO:tasks.workunit.client.1.smithi193.stdout:5/350: mknod d0/d13/d2c/d37/d2b/d6a/c72 0 2024-05-09T06:38:19.538 INFO:tasks.workunit.client.0.smithi063.stdout:3/733: creat d7/da/d2b/d66/ff6 x:0 0 0 2024-05-09T06:38:19.540 INFO:tasks.workunit.client.0.smithi063.stdout:9/719: rmdir d0/d4/d57/df8 0 2024-05-09T06:38:19.541 INFO:tasks.workunit.client.1.smithi193.stdout:2/332: write db/df/f47 [710211,36144] 0 2024-05-09T06:38:19.545 INFO:tasks.workunit.client.1.smithi193.stdout:1/335: symlink d4/d16/de/l64 0 2024-05-09T06:38:19.546 INFO:tasks.workunit.client.0.smithi063.stdout:4/723: mkdir d7/df/d35/d5a/dad/de9 0 2024-05-09T06:38:19.548 INFO:tasks.workunit.client.0.smithi063.stdout:6/716: mknod d2/d1f/ce4 0 2024-05-09T06:38:19.548 INFO:tasks.workunit.client.0.smithi063.stdout:6/717: chown d2/d20/f44 26 1 2024-05-09T06:38:19.549 INFO:tasks.workunit.client.1.smithi193.stdout:7/388: unlink d6/d8/d1a/d33/f51 0 2024-05-09T06:38:19.552 INFO:tasks.workunit.client.1.smithi193.stdout:9/432: read dd/f29 [312617,2935] 0 2024-05-09T06:38:19.554 INFO:tasks.workunit.client.0.smithi063.stdout:7/683: mknod d0/d15/d89/d98/de7/ceb 0 2024-05-09T06:38:19.554 INFO:tasks.workunit.client.0.smithi063.stdout:0/745: rmdir dd/d1a/d2c/d51/d72/d9e/d23/d89/da8 39 2024-05-09T06:38:19.556 INFO:tasks.workunit.client.1.smithi193.stdout:8/347: dwrite d6/d1c/d1b/f4a [0,4194304] 0 2024-05-09T06:38:19.556 INFO:tasks.workunit.client.1.smithi193.stdout:1/336: creat d4/d16/d17/d1a/d3a/d3f/f65 x:0 0 0 2024-05-09T06:38:19.561 INFO:tasks.workunit.client.1.smithi193.stdout:9/433: dwrite dd/df/d31/d7e/d37/f4d [0,4194304] 0 2024-05-09T06:38:19.562 INFO:tasks.workunit.client.0.smithi063.stdout:3/734: mknod d7/d4c/cf7 0 2024-05-09T06:38:19.563 INFO:tasks.workunit.client.0.smithi063.stdout:9/720: mknod d0/d8/da6/cfb 0 2024-05-09T06:38:19.564 INFO:tasks.workunit.client.1.smithi193.stdout:5/351: symlink d0/d13/d2c/l73 0 2024-05-09T06:38:19.566 INFO:tasks.workunit.client.0.smithi063.stdout:4/724: truncate d7/da3/d28/d31/d32/f51 4164248 0 2024-05-09T06:38:19.571 INFO:tasks.workunit.client.1.smithi193.stdout:2/333: unlink db/d4b/d56/c6c 0 2024-05-09T06:38:19.574 INFO:tasks.workunit.client.0.smithi063.stdout:7/684: readlink d0/l57 0 2024-05-09T06:38:19.575 INFO:tasks.workunit.client.0.smithi063.stdout:3/735: dwrite d7/d22/d28/f3c [0,4194304] 0 2024-05-09T06:38:19.577 INFO:tasks.workunit.client.1.smithi193.stdout:8/348: mknod d6/d5b/c68 0 2024-05-09T06:38:19.578 INFO:tasks.workunit.client.0.smithi063.stdout:9/721: dwrite d0/d4/d26/d47/fa8 [0,4194304] 0 2024-05-09T06:38:19.578 INFO:tasks.workunit.client.1.smithi193.stdout:1/337: creat d4/d16/d17/d1a/f66 x:0 0 0 2024-05-09T06:38:19.580 INFO:tasks.workunit.client.1.smithi193.stdout:1/338: write d4/d16/de/f28 [1142517,121598] 0 2024-05-09T06:38:19.580 INFO:tasks.workunit.client.0.smithi063.stdout:3/736: chown d7/da/d2b/d66/dc0/c63 17 1 2024-05-09T06:38:19.582 INFO:tasks.workunit.client.0.smithi063.stdout:4/725: dwrite d7/df/f12 [0,4194304] 0 2024-05-09T06:38:19.584 INFO:tasks.workunit.client.1.smithi193.stdout:5/352: fsync d0/d13/d2c/d37/d5b/d4e/d5d/f61 0 2024-05-09T06:38:19.585 INFO:tasks.workunit.client.1.smithi193.stdout:2/334: chown db/d25/d38/d5b/l5f 258001 1 2024-05-09T06:38:19.586 INFO:tasks.workunit.client.1.smithi193.stdout:5/353: write d0/d13/d2c/d37/d5b/f6 [568840,13618] 0 2024-05-09T06:38:19.587 INFO:tasks.workunit.client.1.smithi193.stdout:2/335: write db/f53 [99340,55086] 0 2024-05-09T06:38:19.590 INFO:tasks.workunit.client.1.smithi193.stdout:8/349: rename d6/d1c/d1b/c24 to d6/d1c/d18/d2f/d40/c69 0 2024-05-09T06:38:19.594 INFO:tasks.workunit.client.0.smithi063.stdout:8/781: sync 2024-05-09T06:38:19.598 INFO:tasks.workunit.client.0.smithi063.stdout:8/782: chown d9/da/d18/d46/lc0 1 1 2024-05-09T06:38:19.601 INFO:tasks.workunit.client.1.smithi193.stdout:2/336: dread - db/d37/f42 zero size 2024-05-09T06:38:19.602 INFO:tasks.workunit.client.1.smithi193.stdout:7/389: write d6/d8/d1a/d36/f5b [1190915,121383] 0 2024-05-09T06:38:19.603 INFO:tasks.workunit.client.1.smithi193.stdout:8/350: write d6/d1c/d53/f63 [2928343,97530] 0 2024-05-09T06:38:19.604 INFO:tasks.workunit.client.1.smithi193.stdout:7/390: fdatasync d6/d8/d1a/d36/f3b 0 2024-05-09T06:38:19.611 INFO:tasks.workunit.client.1.smithi193.stdout:9/434: getdents dd/df/d15 0 2024-05-09T06:38:19.615 INFO:tasks.workunit.client.1.smithi193.stdout:4/422: sync 2024-05-09T06:38:19.615 INFO:tasks.workunit.client.1.smithi193.stdout:3/338: sync 2024-05-09T06:38:19.616 INFO:tasks.workunit.client.1.smithi193.stdout:8/351: unlink d6/d1c/d1b/f30 0 2024-05-09T06:38:19.625 INFO:tasks.workunit.client.1.smithi193.stdout:1/339: truncate d4/f57 92840 0 2024-05-09T06:38:19.625 INFO:tasks.workunit.client.1.smithi193.stdout:7/391: dread d6/f13 [0,4194304] 0 2024-05-09T06:38:19.626 INFO:tasks.workunit.client.1.smithi193.stdout:3/339: creat d1/db/d1b/f7a x:0 0 0 2024-05-09T06:38:19.641 INFO:tasks.workunit.client.1.smithi193.stdout:2/337: getdents db/df/d54 0 2024-05-09T06:38:19.642 INFO:tasks.workunit.client.1.smithi193.stdout:4/423: dwrite f5 [0,4194304] 0 2024-05-09T06:38:19.643 INFO:tasks.workunit.client.1.smithi193.stdout:3/340: truncate d1/d2d/d51/f54 3997265 0 2024-05-09T06:38:19.646 INFO:tasks.workunit.client.1.smithi193.stdout:1/340: dread d4/d16/fb [0,4194304] 0 2024-05-09T06:38:19.651 INFO:tasks.workunit.client.1.smithi193.stdout:2/338: mknod db/d25/c7f 0 2024-05-09T06:38:19.653 INFO:tasks.workunit.client.0.smithi063.stdout:1/654: creat d9/d46/fdd x:0 0 0 2024-05-09T06:38:19.654 INFO:tasks.workunit.client.0.smithi063.stdout:1/655: fsync d9/d2a/d35/d1e/d9d/d3f/d59/fd9 0 2024-05-09T06:38:19.656 INFO:tasks.workunit.client.0.smithi063.stdout:5/711: link d8/d17/d18/f2d d8/d19/d30/d74/fec 0 2024-05-09T06:38:19.659 INFO:tasks.workunit.client.1.smithi193.stdout:3/341: truncate d1/d2d/d51/f3d 43374 0 2024-05-09T06:38:19.661 INFO:tasks.workunit.client.0.smithi063.stdout:2/750: rename dc/dd/f20 to dc/d36/d43/d46/d63/d79/d7d/fe2 0 2024-05-09T06:38:19.662 INFO:tasks.workunit.client.1.smithi193.stdout:1/341: mknod d4/d16/d60/c67 0 2024-05-09T06:38:19.662 INFO:tasks.workunit.client.0.smithi063.stdout:1/656: dwrite d9/d2a/d3c/d4a/fdc [0,4194304] 0 2024-05-09T06:38:19.662 INFO:tasks.workunit.client.1.smithi193.stdout:7/392: truncate d6/d17/f23 1210725 0 2024-05-09T06:38:19.663 INFO:tasks.workunit.client.1.smithi193.stdout:1/342: write d4/d16/d17/d1a/f66 [97622,93225] 0 2024-05-09T06:38:19.664 INFO:tasks.workunit.client.0.smithi063.stdout:2/751: stat dc/dd/d45/d5a/d6c/d8c/d9a/dd5 0 2024-05-09T06:38:19.664 INFO:tasks.workunit.client.1.smithi193.stdout:8/352: getdents d6/d5b 0 2024-05-09T06:38:19.665 INFO:tasks.workunit.client.0.smithi063.stdout:5/712: dwrite d8/d53/f93 [0,4194304] 0 2024-05-09T06:38:19.665 INFO:tasks.workunit.client.1.smithi193.stdout:8/353: stat d6/d1c/d18/d66 0 2024-05-09T06:38:19.666 INFO:tasks.workunit.client.1.smithi193.stdout:8/354: dread - d6/d1c/d18/d2f/f44 zero size 2024-05-09T06:38:19.666 INFO:tasks.workunit.client.0.smithi063.stdout:0/746: mknod dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dcb/ce9 0 2024-05-09T06:38:19.668 INFO:tasks.workunit.client.0.smithi063.stdout:3/737: creat d7/d70/d98/d5b/ff8 x:0 0 0 2024-05-09T06:38:19.670 INFO:tasks.workunit.client.0.smithi063.stdout:1/657: dwrite d9/d2a/d35/d24/f53 [0,4194304] 0 2024-05-09T06:38:19.675 INFO:tasks.workunit.client.1.smithi193.stdout:4/424: dwrite d1a/d1f/f2a [0,4194304] 0 2024-05-09T06:38:19.676 INFO:tasks.workunit.client.0.smithi063.stdout:3/738: dwrite d7/d22/d28/dab/fd4 [0,4194304] 0 2024-05-09T06:38:19.680 INFO:tasks.workunit.client.1.smithi193.stdout:7/393: mknod d6/dd/d5e/c80 0 2024-05-09T06:38:19.688 INFO:tasks.workunit.client.1.smithi193.stdout:8/355: mkdir d6/d1c/d18/d2f/d6a 0 2024-05-09T06:38:19.690 INFO:tasks.workunit.client.0.smithi063.stdout:6/718: rename d2/d20/d4a/f93 to d2/d20/d26/d7f/dcf/fe5 0 2024-05-09T06:38:19.691 INFO:tasks.workunit.client.1.smithi193.stdout:2/339: getdents db/d25/d38/d6a 0 2024-05-09T06:38:19.692 INFO:tasks.workunit.client.0.smithi063.stdout:8/783: dread d9/d3b/f85 [0,4194304] 0 2024-05-09T06:38:19.692 INFO:tasks.workunit.client.0.smithi063.stdout:4/726: write d7/d50/f99 [1093816,87684] 0 2024-05-09T06:38:19.696 INFO:tasks.workunit.client.1.smithi193.stdout:3/342: link d1/db/d59/f52 d1/db/d59/f7b 0 2024-05-09T06:38:19.697 INFO:tasks.workunit.client.0.smithi063.stdout:2/752: creat dc/dd/d45/fe3 x:0 0 0 2024-05-09T06:38:19.699 INFO:tasks.workunit.client.0.smithi063.stdout:5/713: mknod d8/d65/d9e/ced 0 2024-05-09T06:38:19.700 INFO:tasks.workunit.client.0.smithi063.stdout:0/747: creat dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0/fea x:0 0 0 2024-05-09T06:38:19.701 INFO:tasks.workunit.client.1.smithi193.stdout:7/394: read d6/d8/d1a/f3d [368168,23177] 0 2024-05-09T06:38:19.702 INFO:tasks.workunit.client.1.smithi193.stdout:1/343: unlink d4/d16/d35/f5e 0 2024-05-09T06:38:19.703 INFO:tasks.workunit.client.0.smithi063.stdout:1/658: rmdir d9/d2a/d3c/d4c/d83 39 2024-05-09T06:38:19.703 INFO:tasks.workunit.client.1.smithi193.stdout:1/344: write d4/d14/f44 [3766008,79245] 0 2024-05-09T06:38:19.703 INFO:tasks.workunit.client.0.smithi063.stdout:1/659: chown d9/d2a/d35/d24/d68/l84 351 1 2024-05-09T06:38:19.707 INFO:tasks.workunit.client.1.smithi193.stdout:4/425: mkdir d1a/d67/d82 0 2024-05-09T06:38:19.708 INFO:tasks.workunit.client.0.smithi063.stdout:6/719: creat d2/dbf/fe6 x:0 0 0 2024-05-09T06:38:19.709 INFO:tasks.workunit.client.0.smithi063.stdout:6/720: chown d2/d1f/d29/d3f/f58 1 1 2024-05-09T06:38:19.712 INFO:tasks.workunit.client.0.smithi063.stdout:8/784: read - d9/da/d53/fa2 zero size 2024-05-09T06:38:19.713 INFO:tasks.workunit.client.1.smithi193.stdout:4/426: dwrite f5 [0,4194304] 0 2024-05-09T06:38:19.715 INFO:tasks.workunit.client.1.smithi193.stdout:8/356: symlink d6/d1c/d18/d2f/d40/d5e/l6b 0 2024-05-09T06:38:19.716 INFO:tasks.workunit.client.1.smithi193.stdout:8/357: stat d6/d1c/d53/d56/l59 0 2024-05-09T06:38:19.720 INFO:tasks.workunit.client.1.smithi193.stdout:4/427: dwrite f10 [0,4194304] 0 2024-05-09T06:38:19.721 INFO:tasks.workunit.client.1.smithi193.stdout:1/345: creat d4/d16/d17/d1a/d3a/d3f/f68 x:0 0 0 2024-05-09T06:38:19.723 INFO:tasks.workunit.client.1.smithi193.stdout:4/428: stat d1a/d22/l6a 0 2024-05-09T06:38:19.724 INFO:tasks.workunit.client.1.smithi193.stdout:1/346: stat d4/d16/d17/d1a/d3a/d3f 0 2024-05-09T06:38:19.725 INFO:tasks.workunit.client.0.smithi063.stdout:1/660: unlink d9/d2a/d35/d1e/d9d/f99 0 2024-05-09T06:38:19.725 INFO:tasks.workunit.client.1.smithi193.stdout:8/358: dwrite d6/d1c/d18/d66/f62 [0,4194304] 0 2024-05-09T06:38:19.727 INFO:tasks.workunit.client.1.smithi193.stdout:8/359: truncate d6/d1c/d18/d2f/f44 213843 0 2024-05-09T06:38:19.732 INFO:tasks.workunit.client.0.smithi063.stdout:9/722: rename d0/d4/d26/d39/f6c to d0/d3b/da0/dcf/ffc 0 2024-05-09T06:38:19.734 INFO:tasks.workunit.client.0.smithi063.stdout:3/739: write d7/da/d21/d30/d44/d48/f53 [1110258,74166] 0 2024-05-09T06:38:19.734 INFO:tasks.workunit.client.1.smithi193.stdout:7/395: symlink d6/l81 0 2024-05-09T06:38:19.738 INFO:tasks.workunit.client.1.smithi193.stdout:2/340: dwrite db/d4b/f5d [0,4194304] 0 2024-05-09T06:38:19.738 INFO:tasks.workunit.client.1.smithi193.stdout:3/343: rename d1/d2d/d51/f54 to d1/d2d/f7c 0 2024-05-09T06:38:19.743 INFO:tasks.workunit.client.0.smithi063.stdout:7/685: link d0/fc d0/d15/d49/d54/d9f/d97/fec 0 2024-05-09T06:38:19.744 INFO:tasks.workunit.client.0.smithi063.stdout:2/753: write f3 [927503,35843] 0 2024-05-09T06:38:19.745 INFO:tasks.workunit.client.0.smithi063.stdout:7/686: truncate d0/d15/d89/d98/fe0 614879 0 2024-05-09T06:38:19.745 INFO:tasks.workunit.client.1.smithi193.stdout:8/360: creat d6/d1c/d53/d56/f6c x:0 0 0 2024-05-09T06:38:19.748 INFO:tasks.workunit.client.0.smithi063.stdout:7/687: dread d0/d15/d4a/f7e [0,4194304] 0 2024-05-09T06:38:19.750 INFO:tasks.workunit.client.0.smithi063.stdout:9/723: creat d0/d4/d7d/ffd x:0 0 0 2024-05-09T06:38:19.750 INFO:tasks.workunit.client.1.smithi193.stdout:3/344: mkdir d1/d44/d7d 0 2024-05-09T06:38:19.751 INFO:tasks.workunit.client.0.smithi063.stdout:6/721: mkdir d2/d20/de7 0 2024-05-09T06:38:19.751 INFO:tasks.workunit.client.1.smithi193.stdout:1/347: mknod d4/d16/d17/d1a/d5b/c69 0 2024-05-09T06:38:19.752 INFO:tasks.workunit.client.1.smithi193.stdout:1/348: stat d4/ff 0 2024-05-09T06:38:19.753 INFO:tasks.workunit.client.0.smithi063.stdout:6/722: fsync d2/d1f/d29/d3f/d5c/d64/f6e 0 2024-05-09T06:38:19.753 INFO:tasks.workunit.client.0.smithi063.stdout:3/740: unlink d7/da/d2b/d66/fdb 0 2024-05-09T06:38:19.754 INFO:tasks.workunit.client.0.smithi063.stdout:4/727: creat d7/df/d35/d5a/d60/fea x:0 0 0 2024-05-09T06:38:19.755 INFO:tasks.workunit.client.1.smithi193.stdout:7/396: creat d6/dd/d7e/f82 x:0 0 0 2024-05-09T06:38:19.755 INFO:tasks.workunit.client.0.smithi063.stdout:4/728: stat d7/da3/l83 0 2024-05-09T06:38:19.755 INFO:tasks.workunit.client.1.smithi193.stdout:7/397: fdatasync d6/dd/f22 0 2024-05-09T06:38:19.756 INFO:tasks.workunit.client.0.smithi063.stdout:8/785: unlink d9/da/d45/de4/lf2 0 2024-05-09T06:38:19.756 INFO:tasks.workunit.client.0.smithi063.stdout:9/724: dwrite d0/d3b/da0/f9c [4194304,4194304] 0 2024-05-09T06:38:19.757 INFO:tasks.workunit.client.1.smithi193.stdout:7/398: chown d6/d8/d1a/d36/l49 119 1 2024-05-09T06:38:19.761 INFO:tasks.workunit.client.1.smithi193.stdout:1/349: creat d4/d16/d17/d1a/d3a/d3f/f6a x:0 0 0 2024-05-09T06:38:19.763 INFO:tasks.workunit.client.0.smithi063.stdout:0/748: link dd/d1a/d2c/d51/d72/d9e/d23/d31/f4a dd/d1a/d2c/d51/d72/d9e/feb 0 2024-05-09T06:38:19.764 INFO:tasks.workunit.client.1.smithi193.stdout:8/361: creat d6/d1c/d1b/d5d/f6d x:0 0 0 2024-05-09T06:38:19.768 INFO:tasks.workunit.client.0.smithi063.stdout:5/714: truncate d8/f16 4007105 0 2024-05-09T06:38:19.769 INFO:tasks.workunit.client.0.smithi063.stdout:5/715: chown d8/f21 833 1 2024-05-09T06:38:19.770 INFO:tasks.workunit.client.1.smithi193.stdout:4/429: rename d1a/d1f/f33 to d1a/d22/f83 0 2024-05-09T06:38:19.771 INFO:tasks.workunit.client.1.smithi193.stdout:4/430: write d1a/d24/d73/f7e [843830,33101] 0 2024-05-09T06:38:19.777 INFO:tasks.workunit.client.1.smithi193.stdout:7/399: rename d6/c7b to d6/d8/d1a/d36/c83 0 2024-05-09T06:38:19.779 INFO:tasks.workunit.client.1.smithi193.stdout:8/362: dread d6/d1c/d18/f1d [0,4194304] 0 2024-05-09T06:38:19.780 INFO:tasks.workunit.client.1.smithi193.stdout:2/341: getdents db/d37 0 2024-05-09T06:38:19.780 INFO:tasks.workunit.client.1.smithi193.stdout:2/342: chown db/df/l11 29364091 1 2024-05-09T06:38:19.781 INFO:tasks.workunit.client.1.smithi193.stdout:2/343: stat db/d4b/d56 0 2024-05-09T06:38:19.781 INFO:tasks.workunit.client.0.smithi063.stdout:4/729: mkdir d7/d50/deb 0 2024-05-09T06:38:19.782 INFO:tasks.workunit.client.1.smithi193.stdout:2/344: write db/f44 [2404043,128617] 0 2024-05-09T06:38:19.782 INFO:tasks.workunit.client.0.smithi063.stdout:6/723: dwrite d2/d4/f92 [0,4194304] 0 2024-05-09T06:38:19.783 INFO:tasks.workunit.client.1.smithi193.stdout:2/345: chown db/d25/d38/c5c 28108 1 2024-05-09T06:38:19.783 INFO:tasks.workunit.client.1.smithi193.stdout:1/350: mkdir d4/d16/d6b 0 2024-05-09T06:38:19.785 INFO:tasks.workunit.client.0.smithi063.stdout:6/724: fsync d2/df/f14 0 2024-05-09T06:38:19.787 INFO:tasks.workunit.client.1.smithi193.stdout:4/431: creat d1a/d67/f84 x:0 0 0 2024-05-09T06:38:19.794 INFO:tasks.workunit.client.1.smithi193.stdout:1/351: dwrite d4/f32 [4194304,4194304] 0 2024-05-09T06:38:19.794 INFO:tasks.workunit.client.1.smithi193.stdout:2/346: dwrite db/d25/f57 [0,4194304] 0 2024-05-09T06:38:19.796 INFO:tasks.workunit.client.0.smithi063.stdout:1/661: write d9/d10/f1c [2329946,13453] 0 2024-05-09T06:38:19.797 INFO:tasks.workunit.client.0.smithi063.stdout:2/754: symlink dc/d36/d43/d46/d63/d79/d7d/le4 0 2024-05-09T06:38:19.807 INFO:tasks.workunit.client.1.smithi193.stdout:2/347: dwrite db/f45 [0,4194304] 0 2024-05-09T06:38:19.808 INFO:tasks.workunit.client.0.smithi063.stdout:5/716: creat d8/d17/d75/dc9/dd5/fee x:0 0 0 2024-05-09T06:38:19.808 INFO:tasks.workunit.client.0.smithi063.stdout:5/717: chown d8/d17/cb7 861543911 1 2024-05-09T06:38:19.808 INFO:tasks.workunit.client.1.smithi193.stdout:1/352: stat d4/d16/f27 0 2024-05-09T06:38:19.810 INFO:tasks.workunit.client.1.smithi193.stdout:1/353: readlink d4/d16/d17/d1a/l4b 0 2024-05-09T06:38:19.813 INFO:tasks.workunit.client.1.smithi193.stdout:4/432: mknod d1a/d80/c85 0 2024-05-09T06:38:19.814 INFO:tasks.workunit.client.0.smithi063.stdout:8/786: symlink d9/da/d18/d24/l116 0 2024-05-09T06:38:19.816 INFO:tasks.workunit.client.0.smithi063.stdout:6/725: mknod d2/df/d91/ce8 0 2024-05-09T06:38:19.817 INFO:tasks.workunit.client.0.smithi063.stdout:6/726: write d2/fd [2244016,3311] 0 2024-05-09T06:38:19.817 INFO:tasks.workunit.client.1.smithi193.stdout:2/348: dwrite db/df/f50 [0,4194304] 0 2024-05-09T06:38:19.820 INFO:tasks.workunit.client.1.smithi193.stdout:2/349: chown db/df/d54/d63/l6f 328557 1 2024-05-09T06:38:19.821 INFO:tasks.workunit.client.1.smithi193.stdout:2/350: write db/d4b/f5d [2231926,129204] 0 2024-05-09T06:38:19.822 INFO:tasks.workunit.client.0.smithi063.stdout:8/787: dwrite d9/da/d45/f8d [0,4194304] 0 2024-05-09T06:38:19.822 INFO:tasks.workunit.client.1.smithi193.stdout:4/433: creat d1a/d1f/d4b/d7c/f86 x:0 0 0 2024-05-09T06:38:19.824 INFO:tasks.workunit.client.0.smithi063.stdout:2/755: mkdir dc/d32/de5 0 2024-05-09T06:38:19.826 INFO:tasks.workunit.client.0.smithi063.stdout:0/749: mkdir dd/d1a/d2c/d51/d72/d9e/dec 0 2024-05-09T06:38:19.829 INFO:tasks.workunit.client.0.smithi063.stdout:5/718: mknod d8/d17/da2/cef 0 2024-05-09T06:38:19.832 INFO:tasks.workunit.client.1.smithi193.stdout:2/351: rename db/df/d12 to db/d37/d6d/d80 0 2024-05-09T06:38:19.833 INFO:tasks.workunit.client.1.smithi193.stdout:2/352: fsync f8 0 2024-05-09T06:38:19.833 INFO:tasks.workunit.client.0.smithi063.stdout:7/688: link d0/d15/d49/d54/c83 d0/d15/d49/d54/d9f/dd2/ced 0 2024-05-09T06:38:19.834 INFO:tasks.workunit.client.0.smithi063.stdout:7/689: fdatasync d0/d15/d49/d54/fbd 0 2024-05-09T06:38:19.835 INFO:tasks.workunit.client.1.smithi193.stdout:4/434: creat d1a/d1f/d4b/d7c/d5b/f87 x:0 0 0 2024-05-09T06:38:19.837 INFO:tasks.workunit.client.1.smithi193.stdout:4/435: write d1a/d22/f7a [603797,118884] 0 2024-05-09T06:38:19.838 INFO:tasks.workunit.client.1.smithi193.stdout:1/354: creat d4/d14/f6c x:0 0 0 2024-05-09T06:38:19.839 INFO:tasks.workunit.client.1.smithi193.stdout:1/355: chown d4/d14/f1b 2653381 1 2024-05-09T06:38:19.841 INFO:tasks.workunit.client.1.smithi193.stdout:1/356: chown d4/d16/de/l64 125 1 2024-05-09T06:38:19.842 INFO:tasks.workunit.client.0.smithi063.stdout:9/725: write d0/d3b/f6e [630738,52765] 0 2024-05-09T06:38:19.846 INFO:tasks.workunit.client.0.smithi063.stdout:6/727: truncate d2/d1f/d29/f84 3020985 0 2024-05-09T06:38:19.847 INFO:tasks.workunit.client.1.smithi193.stdout:2/353: mknod db/df/d54/c81 0 2024-05-09T06:38:19.847 INFO:tasks.workunit.client.1.smithi193.stdout:8/363: write d6/d1c/d18/f1d [1880824,80600] 0 2024-05-09T06:38:19.847 INFO:tasks.workunit.client.1.smithi193.stdout:4/436: dwrite d1a/d67/f84 [0,4194304] 0 2024-05-09T06:38:19.848 INFO:tasks.workunit.client.0.smithi063.stdout:6/728: stat d2/d20/d4a 0 2024-05-09T06:38:19.848 INFO:tasks.workunit.client.1.smithi193.stdout:8/364: dread - d6/d1c/d27/f65 zero size 2024-05-09T06:38:19.848 INFO:tasks.workunit.client.1.smithi193.stdout:2/354: dread - db/d37/f42 zero size 2024-05-09T06:38:19.849 INFO:tasks.workunit.client.0.smithi063.stdout:9/726: dwrite d0/d4/d26/d39/d3e/d66/fe3 [0,4194304] 0 2024-05-09T06:38:19.850 INFO:tasks.workunit.client.1.smithi193.stdout:4/437: stat d1a/d24/d73/c76 0 2024-05-09T06:38:19.854 INFO:tasks.workunit.client.0.smithi063.stdout:2/756: creat dc/dd/d45/d5a/d6c/d8c/d9a/fe6 x:0 0 0 2024-05-09T06:38:19.858 INFO:tasks.workunit.client.1.smithi193.stdout:1/357: symlink d4/d16/d6b/l6d 0 2024-05-09T06:38:19.862 INFO:tasks.workunit.client.1.smithi193.stdout:4/438: truncate fa 3020355 0 2024-05-09T06:38:19.863 INFO:tasks.workunit.client.1.smithi193.stdout:4/439: truncate d1a/d6b/f66 932552 0 2024-05-09T06:38:19.864 INFO:tasks.workunit.client.1.smithi193.stdout:1/358: mkdir d4/d16/d6b/d6e 0 2024-05-09T06:38:19.867 INFO:tasks.workunit.client.1.smithi193.stdout:8/365: symlink d6/d1c/d1b/d3f/l6e 0 2024-05-09T06:38:19.868 INFO:tasks.workunit.client.1.smithi193.stdout:8/366: chown d6/d1c/f12 469592 1 2024-05-09T06:38:19.871 INFO:tasks.workunit.client.1.smithi193.stdout:2/355: dread f6 [0,4194304] 0 2024-05-09T06:38:19.871 INFO:tasks.workunit.client.1.smithi193.stdout:4/440: symlink d1a/d22/l88 0 2024-05-09T06:38:19.872 INFO:tasks.workunit.client.1.smithi193.stdout:1/359: dwrite d4/f37 [0,4194304] 0 2024-05-09T06:38:19.873 INFO:tasks.workunit.client.1.smithi193.stdout:2/356: write db/df/f69 [89459,1131] 0 2024-05-09T06:38:19.875 INFO:tasks.workunit.client.1.smithi193.stdout:8/367: unlink d6/d1c/d1b/d29/f49 0 2024-05-09T06:38:19.875 INFO:tasks.workunit.client.1.smithi193.stdout:1/360: truncate d4/d16/d17/d1a/f1f 4277316 0 2024-05-09T06:38:19.877 INFO:tasks.workunit.client.1.smithi193.stdout:1/361: chown d4/d16/d17/d1a/d3a 0 1 2024-05-09T06:38:19.882 INFO:tasks.workunit.client.1.smithi193.stdout:8/368: dread d6/d1c/d27/f43 [0,4194304] 0 2024-05-09T06:38:19.882 INFO:tasks.workunit.client.1.smithi193.stdout:2/357: symlink db/df/d54/d67/d77/l82 0 2024-05-09T06:38:19.882 INFO:tasks.workunit.client.0.smithi063.stdout:0/750: creat dd/d1a/d2c/d51/d72/fed x:0 0 0 2024-05-09T06:38:19.883 INFO:tasks.workunit.client.0.smithi063.stdout:3/741: write d7/f17 [1128291,33809] 0 2024-05-09T06:38:19.884 INFO:tasks.workunit.client.0.smithi063.stdout:5/719: creat d8/d19/d30/ff0 x:0 0 0 2024-05-09T06:38:19.885 INFO:tasks.workunit.client.0.smithi063.stdout:6/729: mknod d2/d12/ce9 0 2024-05-09T06:38:19.885 INFO:tasks.workunit.client.0.smithi063.stdout:1/662: write d9/d2a/d35/d1e/d9d/f51 [142997,43405] 0 2024-05-09T06:38:19.886 INFO:tasks.workunit.client.1.smithi193.stdout:8/369: mknod d6/d1c/d18/d2f/c6f 0 2024-05-09T06:38:19.886 INFO:tasks.workunit.client.1.smithi193.stdout:8/370: readlink d6/d1c/d1b/d29/d57/l5f 0 2024-05-09T06:38:19.887 INFO:tasks.workunit.client.1.smithi193.stdout:2/358: symlink db/d37/d6d/d80/l83 0 2024-05-09T06:38:19.887 INFO:tasks.workunit.client.1.smithi193.stdout:8/371: dread - d6/d1c/f34 zero size 2024-05-09T06:38:19.888 INFO:tasks.workunit.client.1.smithi193.stdout:1/362: dread d4/d16/d35/f48 [0,4194304] 0 2024-05-09T06:38:19.891 INFO:tasks.workunit.client.0.smithi063.stdout:3/742: symlink d7/d4c/dcf/lf9 0 2024-05-09T06:38:19.891 INFO:tasks.workunit.client.0.smithi063.stdout:8/788: creat d9/da/d53/f117 x:0 0 0 2024-05-09T06:38:19.894 INFO:tasks.workunit.client.1.smithi193.stdout:2/359: unlink f6 0 2024-05-09T06:38:19.895 INFO:tasks.workunit.client.0.smithi063.stdout:6/730: dread d2/df/d59/faf [0,4194304] 0 2024-05-09T06:38:19.898 INFO:tasks.workunit.client.1.smithi193.stdout:8/372: mknod d6/d1c/d1b/d29/c70 0 2024-05-09T06:38:19.898 INFO:tasks.workunit.client.0.smithi063.stdout:5/720: dwrite d8/dc8/fe9 [0,4194304] 0 2024-05-09T06:38:19.899 INFO:tasks.workunit.client.0.smithi063.stdout:1/663: dread d9/d2a/d35/d1e/d9d/f33 [0,4194304] 0 2024-05-09T06:38:19.900 INFO:tasks.workunit.client.1.smithi193.stdout:2/360: rmdir db/df/d54/d67/d77 39 2024-05-09T06:38:19.901 INFO:tasks.workunit.client.1.smithi193.stdout:4/441: truncate d1a/d1f/d4b/d7c/f49 3018943 0 2024-05-09T06:38:19.904 INFO:tasks.workunit.client.0.smithi063.stdout:2/757: symlink dc/d36/d43/d46/le7 0 2024-05-09T06:38:19.904 INFO:tasks.workunit.client.0.smithi063.stdout:6/731: dread d2/d12/f24 [0,4194304] 0 2024-05-09T06:38:19.904 INFO:tasks.workunit.client.1.smithi193.stdout:1/363: rename d4/f57 to d4/d3c/f6f 0 2024-05-09T06:38:19.905 INFO:tasks.workunit.client.1.smithi193.stdout:8/373: dwrite d6/d1c/d27/f4d [0,4194304] 0 2024-05-09T06:38:19.909 INFO:tasks.workunit.client.0.smithi063.stdout:8/789: mknod d9/da/d53/da3/c118 0 2024-05-09T06:38:19.913 INFO:tasks.workunit.client.1.smithi193.stdout:4/442: write d1a/d1f/d4b/d7c/d5b/f70 [3901897,50021] 0 2024-05-09T06:38:19.914 INFO:tasks.workunit.client.1.smithi193.stdout:4/443: fdatasync d1a/d22/f7a 0 2024-05-09T06:38:19.915 INFO:tasks.workunit.client.1.smithi193.stdout:1/364: fdatasync d4/d16/d35/f48 0 2024-05-09T06:38:19.918 INFO:tasks.workunit.client.1.smithi193.stdout:8/374: creat d6/d1c/d53/f71 x:0 0 0 2024-05-09T06:38:19.920 INFO:tasks.workunit.client.0.smithi063.stdout:1/664: symlink d9/d2a/d3c/lde 0 2024-05-09T06:38:19.920 INFO:tasks.workunit.client.0.smithi063.stdout:4/730: sync 2024-05-09T06:38:19.921 INFO:tasks.workunit.client.0.smithi063.stdout:9/727: dwrite d0/d4/d26/f2a [0,4194304] 0 2024-05-09T06:38:19.923 INFO:tasks.workunit.client.0.smithi063.stdout:1/665: truncate d9/d2a/d3c/fc9 549948 0 2024-05-09T06:38:19.924 INFO:tasks.workunit.client.0.smithi063.stdout:0/751: creat dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/dd6/fee x:0 0 0 2024-05-09T06:38:19.927 INFO:tasks.workunit.client.0.smithi063.stdout:9/728: chown d0/d4/d26/d47/f7a 15664186 1 2024-05-09T06:38:19.927 INFO:tasks.workunit.client.1.smithi193.stdout:2/361: write db/d4b/f71 [2948376,99250] 0 2024-05-09T06:38:19.930 INFO:tasks.workunit.client.0.smithi063.stdout:8/790: mkdir d9/da/d51/dc2/d119 0 2024-05-09T06:38:19.930 INFO:tasks.workunit.client.1.smithi193.stdout:1/365: rmdir d4/d16/d17/d1a/d3a/d4a 39 2024-05-09T06:38:19.931 INFO:tasks.workunit.client.1.smithi193.stdout:1/366: chown d4/d3c/f36 6377 1 2024-05-09T06:38:19.937 INFO:tasks.workunit.client.1.smithi193.stdout:2/362: symlink db/d25/d38/d5b/l84 0 2024-05-09T06:38:19.937 INFO:tasks.workunit.client.1.smithi193.stdout:2/363: dread - db/d37/f42 zero size 2024-05-09T06:38:19.938 INFO:tasks.workunit.client.0.smithi063.stdout:9/729: rmdir d0/d4/d26/d47 39 2024-05-09T06:38:19.941 INFO:tasks.workunit.client.0.smithi063.stdout:8/791: unlink d9/d3b/l86 0 2024-05-09T06:38:19.942 INFO:tasks.workunit.client.0.smithi063.stdout:8/792: chown d9/da/d18/d24/f62 5 1 2024-05-09T06:38:19.945 INFO:tasks.workunit.client.1.smithi193.stdout:4/444: write d1a/d1f/f20 [297827,104648] 0 2024-05-09T06:38:19.947 INFO:tasks.workunit.client.0.smithi063.stdout:9/730: unlink d0/d4/f4d 0 2024-05-09T06:38:19.948 INFO:tasks.workunit.client.1.smithi193.stdout:2/364: rmdir db/d25/d38 39 2024-05-09T06:38:19.949 INFO:tasks.workunit.client.0.smithi063.stdout:1/666: dread d9/d2a/d35/d1e/d9d/d3f/d59/d62/fa5 [0,4194304] 0 2024-05-09T06:38:19.950 INFO:tasks.workunit.client.1.smithi193.stdout:2/365: write db/d37/d6d/d80/f13 [2982955,68830] 0 2024-05-09T06:38:19.950 INFO:tasks.workunit.client.0.smithi063.stdout:2/758: dread dc/f27 [4194304,4194304] 0 2024-05-09T06:38:19.951 INFO:tasks.workunit.client.0.smithi063.stdout:4/731: creat d7/da3/d28/d31/d7b/fec x:0 0 0 2024-05-09T06:38:19.952 INFO:tasks.workunit.client.1.smithi193.stdout:2/366: write db/d25/f33 [2021813,57718] 0 2024-05-09T06:38:19.953 INFO:tasks.workunit.client.1.smithi193.stdout:4/445: dwrite d1a/d1f/f20 [0,4194304] 0 2024-05-09T06:38:19.953 INFO:tasks.workunit.client.1.smithi193.stdout:1/367: rename d4/d3c/f6f to d4/d16/d6b/d6e/f70 0 2024-05-09T06:38:19.954 INFO:tasks.workunit.client.1.smithi193.stdout:1/368: readlink d4/l6 0 2024-05-09T06:38:19.954 INFO:tasks.workunit.client.0.smithi063.stdout:6/732: getdents d2/d1f/d29/d3f/d5c/d64 0 2024-05-09T06:38:19.955 INFO:tasks.workunit.client.1.smithi193.stdout:8/375: link d6/d1c/d18/d2f/d40/l4e d6/d1c/d18/d66/l72 0 2024-05-09T06:38:19.957 INFO:tasks.workunit.client.0.smithi063.stdout:0/752: dread dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fc3 [0,4194304] 0 2024-05-09T06:38:19.960 INFO:tasks.workunit.client.1.smithi193.stdout:2/367: dread - db/d37/d6d/d80/f46 zero size 2024-05-09T06:38:19.961 INFO:tasks.workunit.client.1.smithi193.stdout:2/368: fdatasync db/d4b/f71 0 2024-05-09T06:38:19.962 INFO:tasks.workunit.client.1.smithi193.stdout:2/369: readlink db/l51 0 2024-05-09T06:38:19.963 INFO:tasks.workunit.client.0.smithi063.stdout:1/667: dwrite d9/d2a/d35/f29 [0,4194304] 0 2024-05-09T06:38:19.963 INFO:tasks.workunit.client.0.smithi063.stdout:4/732: write d7/df/d57/f93 [1937264,27464] 0 2024-05-09T06:38:19.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/759: symlink dc/dd/d45/d5a/d6c/d8c/dad/le8 0 2024-05-09T06:38:19.963 INFO:tasks.workunit.client.1.smithi193.stdout:4/446: write d1a/d22/d43/f75 [1070611,1657] 0 2024-05-09T06:38:19.964 INFO:tasks.workunit.client.0.smithi063.stdout:4/733: fdatasync d7/d3d/d73/fb1 0 2024-05-09T06:38:19.965 INFO:tasks.workunit.client.0.smithi063.stdout:2/760: readlink dc/dd/d45/d5a/d6c/d8c/dad/le8 0 2024-05-09T06:38:19.967 INFO:tasks.workunit.client.0.smithi063.stdout:4/734: write d7/d50/f9d [1910350,4444] 0 2024-05-09T06:38:19.971 INFO:tasks.workunit.client.1.smithi193.stdout:6/370: sync 2024-05-09T06:38:19.971 INFO:tasks.workunit.client.1.smithi193.stdout:0/354: sync 2024-05-09T06:38:19.972 INFO:tasks.workunit.client.1.smithi193.stdout:6/371: chown d1/f42 209477 1 2024-05-09T06:38:19.973 INFO:tasks.workunit.client.1.smithi193.stdout:2/370: chown db/d25/d38/d5b/f66 2112 1 2024-05-09T06:38:19.974 INFO:tasks.workunit.client.0.smithi063.stdout:1/668: rmdir d9/d76/d94 39 2024-05-09T06:38:19.975 INFO:tasks.workunit.client.1.smithi193.stdout:4/447: creat d1a/d1f/d4b/d7c/f89 x:0 0 0 2024-05-09T06:38:19.977 INFO:tasks.workunit.client.1.smithi193.stdout:8/376: truncate d6/d1c/d1b/d29/f4f 565517 0 2024-05-09T06:38:19.977 INFO:tasks.workunit.client.0.smithi063.stdout:4/735: truncate d7/df/f95 2870724 0 2024-05-09T06:38:19.978 INFO:tasks.workunit.client.0.smithi063.stdout:8/793: dread d9/d37/f4d [0,4194304] 0 2024-05-09T06:38:19.979 INFO:tasks.workunit.client.1.smithi193.stdout:6/372: dread d1/de/d11/f1f [0,4194304] 0 2024-05-09T06:38:19.981 INFO:tasks.workunit.client.1.smithi193.stdout:6/373: chown d1/d25/d2c/l7f 872 1 2024-05-09T06:38:19.983 INFO:tasks.workunit.client.1.smithi193.stdout:0/355: rmdir d18/d27 39 2024-05-09T06:38:19.984 INFO:tasks.workunit.client.1.smithi193.stdout:8/377: dwrite d6/d1c/d18/d66/f47 [0,4194304] 0 2024-05-09T06:38:19.984 INFO:tasks.workunit.client.1.smithi193.stdout:0/356: chown f17 620392 1 2024-05-09T06:38:19.985 INFO:tasks.workunit.client.1.smithi193.stdout:8/378: write d6/d5b/f64 [1031470,115700] 0 2024-05-09T06:38:19.986 INFO:tasks.workunit.client.1.smithi193.stdout:8/379: dread - d6/d1c/d27/f65 zero size 2024-05-09T06:38:19.987 INFO:tasks.workunit.client.1.smithi193.stdout:8/380: chown d6/d1c/d1b/d29/c70 191554188 1 2024-05-09T06:38:19.987 INFO:tasks.workunit.client.1.smithi193.stdout:2/371: rename db/d25/f33 to db/d37/d6d/d80/d60/d7d/f85 0 2024-05-09T06:38:19.988 INFO:tasks.workunit.client.0.smithi063.stdout:6/733: dwrite d2/df/f5d [0,4194304] 0 2024-05-09T06:38:19.989 INFO:tasks.workunit.client.0.smithi063.stdout:0/753: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fa6 [0,4194304] 0 2024-05-09T06:38:19.990 INFO:tasks.workunit.client.1.smithi193.stdout:4/448: mknod d1a/d1f/d4b/d7c/c8a 0 2024-05-09T06:38:19.994 INFO:tasks.workunit.client.1.smithi193.stdout:1/369: truncate d4/d16/f27 3404702 0 2024-05-09T06:38:19.996 INFO:tasks.workunit.client.0.smithi063.stdout:4/736: creat d7/df/d57/fed x:0 0 0 2024-05-09T06:38:19.997 INFO:tasks.workunit.client.0.smithi063.stdout:4/737: write f5 [7681178,24333] 0 2024-05-09T06:38:19.998 INFO:tasks.workunit.client.1.smithi193.stdout:6/374: symlink d1/de/d11/d2d/d3a/l83 0 2024-05-09T06:38:19.999 INFO:tasks.workunit.client.0.smithi063.stdout:8/794: mkdir d9/da/d53/db2/d11a 0 2024-05-09T06:38:20.001 INFO:tasks.workunit.client.0.smithi063.stdout:8/795: chown d9/da/d51/fe5 0 1 2024-05-09T06:38:20.001 INFO:tasks.workunit.client.0.smithi063.stdout:4/738: chown d7/da3/d28/d31/d32/f8c 6744 1 2024-05-09T06:38:20.004 INFO:tasks.workunit.client.0.smithi063.stdout:6/734: mknod d2/df/cea 0 2024-05-09T06:38:20.005 INFO:tasks.workunit.client.1.smithi193.stdout:2/372: rmdir db/df 39 2024-05-09T06:38:20.007 INFO:tasks.workunit.client.1.smithi193.stdout:2/373: chown db/d37/d6d/d80/c16 20 1 2024-05-09T06:38:20.007 INFO:tasks.workunit.client.1.smithi193.stdout:2/374: fdatasync db/d37/d6d/d80/f13 0 2024-05-09T06:38:20.008 INFO:tasks.workunit.client.0.smithi063.stdout:0/754: mknod dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/cef 0 2024-05-09T06:38:20.010 INFO:tasks.workunit.client.1.smithi193.stdout:1/370: creat d4/d16/d6b/d6e/f71 x:0 0 0 2024-05-09T06:38:20.011 INFO:tasks.workunit.client.1.smithi193.stdout:6/375: dread d1/de/f10 [0,4194304] 0 2024-05-09T06:38:20.015 INFO:tasks.workunit.client.1.smithi193.stdout:0/357: mknod d18/d4c/d47/d55/c72 0 2024-05-09T06:38:20.016 INFO:tasks.workunit.client.0.smithi063.stdout:9/731: link d0/d8/le d0/lfe 0 2024-05-09T06:38:20.017 INFO:tasks.workunit.client.1.smithi193.stdout:4/449: link d1a/d24/d73/f7e d1a/d22/d43/f8b 0 2024-05-09T06:38:20.024 INFO:tasks.workunit.client.0.smithi063.stdout:1/669: rmdir d9/dc8 0 2024-05-09T06:38:20.025 INFO:tasks.workunit.client.1.smithi193.stdout:1/371: mkdir d4/d16/d60/d4f/d72 0 2024-05-09T06:38:20.025 INFO:tasks.workunit.client.1.smithi193.stdout:1/372: stat d4/d14/l49 0 2024-05-09T06:38:20.027 INFO:tasks.workunit.client.0.smithi063.stdout:0/755: unlink dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/cb0 0 2024-05-09T06:38:20.029 INFO:tasks.workunit.client.0.smithi063.stdout:9/732: symlink d0/da5/lff 0 2024-05-09T06:38:20.030 INFO:tasks.workunit.client.1.smithi193.stdout:2/375: dread db/f1e [0,4194304] 0 2024-05-09T06:38:20.032 INFO:tasks.workunit.client.1.smithi193.stdout:0/358: creat d18/d4c/d47/d54/f73 x:0 0 0 2024-05-09T06:38:20.034 INFO:tasks.workunit.client.1.smithi193.stdout:4/450: unlink d1a/d22/l88 0 2024-05-09T06:38:20.038 INFO:tasks.workunit.client.1.smithi193.stdout:4/451: dread d1a/d22/f7a [0,4194304] 0 2024-05-09T06:38:20.038 INFO:tasks.workunit.client.1.smithi193.stdout:1/373: read d4/d16/de/f12 [1053489,31092] 0 2024-05-09T06:38:20.044 INFO:tasks.workunit.client.1.smithi193.stdout:1/374: dread d4/d16/d17/d1a/f1e [0,4194304] 0 2024-05-09T06:38:20.045 INFO:tasks.workunit.client.0.smithi063.stdout:0/756: symlink dd/d1a/d2c/d51/d72/d9e/d23/d46/lf0 0 2024-05-09T06:38:20.046 INFO:tasks.workunit.client.0.smithi063.stdout:6/735: write d2/d20/f21 [3660183,122885] 0 2024-05-09T06:38:20.046 INFO:tasks.workunit.client.1.smithi193.stdout:4/452: dwrite d1a/d1f/d4b/d7c/f89 [0,4194304] 0 2024-05-09T06:38:20.046 INFO:tasks.workunit.client.0.smithi063.stdout:9/733: truncate d0/d4/d57/d60/fc9 335196 0 2024-05-09T06:38:20.047 INFO:tasks.workunit.client.0.smithi063.stdout:8/796: dread d9/da/f2f [0,4194304] 0 2024-05-09T06:38:20.052 INFO:tasks.workunit.client.1.smithi193.stdout:0/359: dread fe [0,4194304] 0 2024-05-09T06:38:20.053 INFO:tasks.workunit.client.1.smithi193.stdout:6/376: write d1/de/f53 [4700355,121051] 0 2024-05-09T06:38:20.053 INFO:tasks.workunit.client.0.smithi063.stdout:0/757: mkdir dd/d1a/d2c/dba/df1 0 2024-05-09T06:38:20.054 INFO:tasks.workunit.client.1.smithi193.stdout:2/376: dwrite db/df/d54/f7c [0,4194304] 0 2024-05-09T06:38:20.055 INFO:tasks.workunit.client.0.smithi063.stdout:6/736: dwrite d2/d20/d26/fca [0,4194304] 0 2024-05-09T06:38:20.056 INFO:tasks.workunit.client.1.smithi193.stdout:4/453: rmdir d1a/d6d 39 2024-05-09T06:38:20.056 INFO:tasks.workunit.client.1.smithi193.stdout:2/377: write db/df/d54/f7c [968363,86783] 0 2024-05-09T06:38:20.057 INFO:tasks.workunit.client.0.smithi063.stdout:8/797: creat d9/da/d51/dc2/f11b x:0 0 0 2024-05-09T06:38:20.059 INFO:tasks.workunit.client.1.smithi193.stdout:2/378: stat db/d25/d38/d5b/f66 0 2024-05-09T06:38:20.065 INFO:tasks.workunit.client.1.smithi193.stdout:0/360: rmdir d18/d65 39 2024-05-09T06:38:20.068 INFO:tasks.workunit.client.1.smithi193.stdout:6/377: fdatasync d1/d25/d4c/f6b 0 2024-05-09T06:38:20.069 INFO:tasks.workunit.client.0.smithi063.stdout:6/737: fdatasync d2/d12/f24 0 2024-05-09T06:38:20.071 INFO:tasks.workunit.client.0.smithi063.stdout:0/758: rename dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fd0 to dd/d1a/d2c/d51/d72/d9e/d23/d31/ff2 0 2024-05-09T06:38:20.071 INFO:tasks.workunit.client.0.smithi063.stdout:6/738: dread - d2/d1f/d29/fb1 zero size 2024-05-09T06:38:20.072 INFO:tasks.workunit.client.1.smithi193.stdout:2/379: creat db/d4b/f86 x:0 0 0 2024-05-09T06:38:20.073 INFO:tasks.workunit.client.0.smithi063.stdout:9/734: link d0/d4/d26/d39/d48/d80/fc1 d0/d4/d57/f100 0 2024-05-09T06:38:20.074 INFO:tasks.workunit.client.0.smithi063.stdout:1/670: mkdir d9/d2a/d3c/d4a/db8/ddf 0 2024-05-09T06:38:20.075 INFO:tasks.workunit.client.1.smithi193.stdout:0/361: mkdir d18/d4c/d64/d74 0 2024-05-09T06:38:20.075 INFO:tasks.workunit.client.1.smithi193.stdout:0/362: chown d18/d4c/d47/f4a 14467 1 2024-05-09T06:38:20.076 INFO:tasks.workunit.client.1.smithi193.stdout:6/378: creat d1/d25/d2c/d3c/d3e/d58/f84 x:0 0 0 2024-05-09T06:38:20.077 INFO:tasks.workunit.client.0.smithi063.stdout:6/739: chown d2/d1f/d29/f84 390 1 2024-05-09T06:38:20.078 INFO:tasks.workunit.client.1.smithi193.stdout:2/380: chown db/d37/d6d/d80/f24 103639 1 2024-05-09T06:38:20.078 INFO:tasks.workunit.client.0.smithi063.stdout:6/740: chown d2/d1f/d29/d3f/fc9 840681 1 2024-05-09T06:38:20.078 INFO:tasks.workunit.client.0.smithi063.stdout:6/741: stat d2/d1f 0 2024-05-09T06:38:20.079 INFO:tasks.workunit.client.0.smithi063.stdout:6/742: chown d2/d1f/d73/d88 865402 1 2024-05-09T06:38:20.079 INFO:tasks.workunit.client.0.smithi063.stdout:0/759: unlink dd/d1a/d2c/d51/d72/d9e/d23/d31/fc7 0 2024-05-09T06:38:20.080 INFO:tasks.workunit.client.0.smithi063.stdout:0/760: readlink dd/d60/d44/d90/da4/ld5 0 2024-05-09T06:38:20.081 INFO:tasks.workunit.client.1.smithi193.stdout:1/375: rename d4/d16/d60/c23 to d4/d16/c73 0 2024-05-09T06:38:20.082 INFO:tasks.workunit.client.1.smithi193.stdout:2/381: dread db/d4b/f5d [0,4194304] 0 2024-05-09T06:38:20.082 INFO:tasks.workunit.client.0.smithi063.stdout:9/735: truncate d0/f65 488686 0 2024-05-09T06:38:20.082 INFO:tasks.workunit.client.0.smithi063.stdout:0/761: dread dd/d60/d44/d90/fce [0,4194304] 0 2024-05-09T06:38:20.083 INFO:tasks.workunit.client.1.smithi193.stdout:0/363: creat d18/d4c/f75 x:0 0 0 2024-05-09T06:38:20.083 INFO:tasks.workunit.client.1.smithi193.stdout:2/382: read db/d25/f2a [1059868,28921] 0 2024-05-09T06:38:20.086 INFO:tasks.workunit.client.1.smithi193.stdout:6/379: unlink d1/de/d11/d18/f2a 0 2024-05-09T06:38:20.087 INFO:tasks.workunit.client.1.smithi193.stdout:6/380: dread - d1/de/d11/d2d/d3a/d47/f7a zero size 2024-05-09T06:38:20.088 INFO:tasks.workunit.client.0.smithi063.stdout:0/762: dwrite dd/d1a/d2c/f92 [0,4194304] 0 2024-05-09T06:38:20.090 INFO:tasks.workunit.client.1.smithi193.stdout:6/381: dread d1/de/f10 [0,4194304] 0 2024-05-09T06:38:20.090 INFO:tasks.workunit.client.0.smithi063.stdout:8/798: write d9/d37/f3a [1212558,42623] 0 2024-05-09T06:38:20.095 INFO:tasks.workunit.client.0.smithi063.stdout:9/736: rmdir d0/d4/d26 39 2024-05-09T06:38:20.102 INFO:tasks.workunit.client.1.smithi193.stdout:2/383: fdatasync db/f39 0 2024-05-09T06:38:20.107 INFO:tasks.workunit.client.1.smithi193.stdout:4/454: getdents d1a/d22 0 2024-05-09T06:38:20.108 INFO:tasks.workunit.client.0.smithi063.stdout:8/799: rmdir d9/da/d51 39 2024-05-09T06:38:20.108 INFO:tasks.workunit.client.0.smithi063.stdout:8/800: chown d9/da/d53/db2/cba 158057414 1 2024-05-09T06:38:20.109 INFO:tasks.workunit.client.1.smithi193.stdout:6/382: mknod d1/d25/d2c/d3c/d3e/d58/c85 0 2024-05-09T06:38:20.112 INFO:tasks.workunit.client.0.smithi063.stdout:7/690: sync 2024-05-09T06:38:20.114 INFO:tasks.workunit.client.0.smithi063.stdout:1/671: rename d9/f11 to d9/d2a/d3c/d4c/d83/d88/dc2/fe0 0 2024-05-09T06:38:20.114 INFO:tasks.workunit.client.1.smithi193.stdout:0/364: write f9 [8521390,52464] 0 2024-05-09T06:38:20.115 INFO:tasks.workunit.client.1.smithi193.stdout:1/376: truncate d4/d16/f27 1350123 0 2024-05-09T06:38:20.116 INFO:tasks.workunit.client.0.smithi063.stdout:0/763: unlink dd/cc8 0 2024-05-09T06:38:20.118 INFO:tasks.workunit.client.0.smithi063.stdout:6/743: creat d2/d1f/d29/de2/d4e/feb x:0 0 0 2024-05-09T06:38:20.118 INFO:tasks.workunit.client.0.smithi063.stdout:8/801: mknod d9/d41/c11c 0 2024-05-09T06:38:20.120 INFO:tasks.workunit.client.1.smithi193.stdout:4/455: creat d1a/d6b/f8c x:0 0 0 2024-05-09T06:38:20.122 INFO:tasks.workunit.client.1.smithi193.stdout:1/377: creat d4/d16/d35/f74 x:0 0 0 2024-05-09T06:38:20.123 INFO:tasks.workunit.client.0.smithi063.stdout:0/764: symlink dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dcb/lf3 0 2024-05-09T06:38:20.123 INFO:tasks.workunit.client.0.smithi063.stdout:0/765: chown dd/d60/dcd 32983716 1 2024-05-09T06:38:20.124 INFO:tasks.workunit.client.1.smithi193.stdout:2/384: mkdir db/df/d87 0 2024-05-09T06:38:20.125 INFO:tasks.workunit.client.1.smithi193.stdout:4/456: rmdir d1a/d1f 39 2024-05-09T06:38:20.130 INFO:tasks.workunit.client.1.smithi193.stdout:0/365: mkdir d18/d4c/d47/d59/d76 0 2024-05-09T06:38:20.130 INFO:tasks.workunit.client.0.smithi063.stdout:6/744: dwrite d2/d1f/d29/fb1 [0,4194304] 0 2024-05-09T06:38:20.131 INFO:tasks.workunit.client.0.smithi063.stdout:7/691: mknod d0/d15/d4a/cee 0 2024-05-09T06:38:20.134 INFO:tasks.workunit.client.1.smithi193.stdout:2/385: chown db/c29 1738955731 1 2024-05-09T06:38:20.136 INFO:tasks.workunit.client.1.smithi193.stdout:4/457: unlink d1a/d24/d73/c76 0 2024-05-09T06:38:20.139 INFO:tasks.workunit.client.1.smithi193.stdout:6/383: creat d1/f86 x:0 0 0 2024-05-09T06:38:20.139 INFO:tasks.workunit.client.0.smithi063.stdout:7/692: dwrite d0/d15/d49/f5b [0,4194304] 0 2024-05-09T06:38:20.141 INFO:tasks.workunit.client.1.smithi193.stdout:4/458: dread d1a/d22/d43/f8b [0,4194304] 0 2024-05-09T06:38:20.145 INFO:tasks.workunit.client.1.smithi193.stdout:2/386: creat db/d25/d38/d5b/f88 x:0 0 0 2024-05-09T06:38:20.147 INFO:tasks.workunit.client.0.smithi063.stdout:6/745: mkdir d2/d20/d95/dec 0 2024-05-09T06:38:20.149 INFO:tasks.workunit.client.0.smithi063.stdout:0/766: write dd/d60/d44/f69 [1204414,75668] 0 2024-05-09T06:38:20.150 INFO:tasks.workunit.client.0.smithi063.stdout:0/767: chown dd/d1a/d2c/d51/d76/d97/fa5 106262 1 2024-05-09T06:38:20.153 INFO:tasks.workunit.client.0.smithi063.stdout:0/768: write dd/d1a/d2c/d51/d72/d9e/d23/d46/fdf [5218102,113189] 0 2024-05-09T06:38:20.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/459: write d1a/d1f/d4b/d7c/f86 [1027148,129592] 0 2024-05-09T06:38:20.157 INFO:tasks.workunit.client.0.smithi063.stdout:6/746: symlink d2/d20/d4a/led 0 2024-05-09T06:38:20.160 INFO:tasks.workunit.client.1.smithi193.stdout:1/378: truncate d4/d16/f47 3592692 0 2024-05-09T06:38:20.162 INFO:tasks.workunit.client.1.smithi193.stdout:4/460: mknod d1a/d24/c8d 0 2024-05-09T06:38:20.164 INFO:tasks.workunit.client.1.smithi193.stdout:2/387: mkdir db/df/d89 0 2024-05-09T06:38:20.164 INFO:tasks.workunit.client.1.smithi193.stdout:2/388: stat db/df/d54/d63 0 2024-05-09T06:38:20.165 INFO:tasks.workunit.client.1.smithi193.stdout:6/384: rename d1/de/d11/f1f to d1/de/d11/f87 0 2024-05-09T06:38:20.167 INFO:tasks.workunit.client.0.smithi063.stdout:9/737: getdents d0/d3b/da0/dcf 0 2024-05-09T06:38:20.168 INFO:tasks.workunit.client.1.smithi193.stdout:4/461: mkdir d1a/d22/d8e 0 2024-05-09T06:38:20.171 INFO:tasks.workunit.client.0.smithi063.stdout:6/747: creat d2/d1f/d29/d3f/d5c/d64/fee x:0 0 0 2024-05-09T06:38:20.172 INFO:tasks.workunit.client.0.smithi063.stdout:3/743: sync 2024-05-09T06:38:20.172 INFO:tasks.workunit.client.0.smithi063.stdout:1/672: truncate d9/d2a/fd8 2103380 0 2024-05-09T06:38:20.173 INFO:tasks.workunit.client.1.smithi193.stdout:2/389: rmdir db/d25 39 2024-05-09T06:38:20.174 INFO:tasks.workunit.client.0.smithi063.stdout:8/802: getdents d9/da/d53/dbc/d102 0 2024-05-09T06:38:20.176 INFO:tasks.workunit.client.0.smithi063.stdout:1/673: dread d9/d2a/d3c/d4c/f56 [0,4194304] 0 2024-05-09T06:38:20.181 INFO:tasks.workunit.client.0.smithi063.stdout:6/748: mkdir d2/d4/d7c/d42/def 0 2024-05-09T06:38:20.185 INFO:tasks.workunit.client.1.smithi193.stdout:6/385: creat d1/d25/d2c/d76/d69/f88 x:0 0 0 2024-05-09T06:38:20.188 INFO:tasks.workunit.client.1.smithi193.stdout:1/379: truncate d4/fc 1073820 0 2024-05-09T06:38:20.188 INFO:tasks.workunit.client.0.smithi063.stdout:8/803: mkdir d9/da/d18/d24/d71/d11d 0 2024-05-09T06:38:20.188 INFO:tasks.workunit.client.1.smithi193.stdout:2/390: dwrite db/d25/d38/d61/f70 [0,4194304] 0 2024-05-09T06:38:20.190 INFO:tasks.workunit.client.0.smithi063.stdout:7/693: getdents d0/d15/d49/d4f/d66/ddd 0 2024-05-09T06:38:20.194 INFO:tasks.workunit.client.0.smithi063.stdout:7/694: write d0/d15/d49/d4f/d66/ddd/fdf [865354,4807] 0 2024-05-09T06:38:20.194 INFO:tasks.workunit.client.0.smithi063.stdout:1/674: creat d9/d2a/d3c/d4c/d83/d88/fe1 x:0 0 0 2024-05-09T06:38:20.195 INFO:tasks.workunit.client.0.smithi063.stdout:9/738: dread - d0/d4/d26/d39/d48/d80/fc3 zero size 2024-05-09T06:38:20.197 INFO:tasks.workunit.client.0.smithi063.stdout:6/749: truncate d2/d20/f7b 2423283 0 2024-05-09T06:38:20.198 INFO:tasks.workunit.client.0.smithi063.stdout:1/675: write d9/d2a/d39/d5d/fc6 [499639,110863] 0 2024-05-09T06:38:20.198 INFO:tasks.workunit.client.0.smithi063.stdout:3/744: dwrite d7/d22/f47 [0,4194304] 0 2024-05-09T06:38:20.199 INFO:tasks.workunit.client.1.smithi193.stdout:2/391: write db/d25/f49 [682069,117297] 0 2024-05-09T06:38:20.203 INFO:tasks.workunit.client.0.smithi063.stdout:8/804: symlink d9/d41/l11e 0 2024-05-09T06:38:20.206 INFO:tasks.workunit.client.0.smithi063.stdout:9/739: write d0/d4/d26/d47/f7a [2822450,75661] 0 2024-05-09T06:38:20.208 INFO:tasks.workunit.client.1.smithi193.stdout:6/386: rmdir d1/de/d11/d20/d78 0 2024-05-09T06:38:20.210 INFO:tasks.workunit.client.0.smithi063.stdout:7/695: symlink d0/d15/d49/d4f/d66/d67/d86/lef 0 2024-05-09T06:38:20.214 INFO:tasks.workunit.client.1.smithi193.stdout:1/380: creat d4/d14/f75 x:0 0 0 2024-05-09T06:38:20.215 INFO:tasks.workunit.client.0.smithi063.stdout:3/745: mkdir d7/da/d2b/d66/d74/d7e/dfa 0 2024-05-09T06:38:20.216 INFO:tasks.workunit.client.1.smithi193.stdout:1/381: dread d4/f9 [0,4194304] 0 2024-05-09T06:38:20.217 INFO:tasks.workunit.client.1.smithi193.stdout:1/382: chown d4/d3c 68140224 1 2024-05-09T06:38:20.217 INFO:tasks.workunit.client.0.smithi063.stdout:1/676: mknod d9/d2a/d3c/d4c/d83/d88/ce2 0 2024-05-09T06:38:20.219 INFO:tasks.workunit.client.0.smithi063.stdout:9/740: rename d0/d4/d26/f33 to d0/d8/dea/f101 0 2024-05-09T06:38:20.220 INFO:tasks.workunit.client.1.smithi193.stdout:4/462: getdents d1a/d22/d5d 0 2024-05-09T06:38:20.220 INFO:tasks.workunit.client.1.smithi193.stdout:4/463: read d1a/d1f/f20 [2206139,123120] 0 2024-05-09T06:38:20.221 INFO:tasks.workunit.client.0.smithi063.stdout:3/746: fdatasync f2 0 2024-05-09T06:38:20.221 INFO:tasks.workunit.client.1.smithi193.stdout:4/464: fdatasync d1a/d6b/f77 0 2024-05-09T06:38:20.222 INFO:tasks.workunit.client.0.smithi063.stdout:3/747: stat d7/da/d21/d30/d44/d48 0 2024-05-09T06:38:20.222 INFO:tasks.workunit.client.1.smithi193.stdout:4/465: stat d1a/f65 0 2024-05-09T06:38:20.222 INFO:tasks.workunit.client.1.smithi193.stdout:5/354: sync 2024-05-09T06:38:20.223 INFO:tasks.workunit.client.0.smithi063.stdout:7/696: creat d0/d15/d49/d4f/d66/d67/d86/ff0 x:0 0 0 2024-05-09T06:38:20.223 INFO:tasks.workunit.client.1.smithi193.stdout:5/355: write d0/d13/d2c/d37/d5b/f6 [90632,35741] 0 2024-05-09T06:38:20.225 INFO:tasks.workunit.client.1.smithi193.stdout:5/356: write d0/d13/d2c/d37/d5b/d4e/f57 [486219,89529] 0 2024-05-09T06:38:20.227 INFO:tasks.workunit.client.0.smithi063.stdout:1/677: readlink l4 0 2024-05-09T06:38:20.227 INFO:tasks.workunit.client.0.smithi063.stdout:6/750: dread d2/d1f/d29/d3f/d5c/f70 [0,4194304] 0 2024-05-09T06:38:20.230 INFO:tasks.workunit.client.1.smithi193.stdout:5/357: read d0/d13/d2c/d37/d2b/d47/d66/f15 [521044,125658] 0 2024-05-09T06:38:20.231 INFO:tasks.workunit.client.0.smithi063.stdout:7/697: dwrite d0/d15/d49/d4f/d66/d67/d86/ff0 [0,4194304] 0 2024-05-09T06:38:20.234 INFO:tasks.workunit.client.1.smithi193.stdout:2/392: write f9 [842449,48411] 0 2024-05-09T06:38:20.240 INFO:tasks.workunit.client.0.smithi063.stdout:8/805: mkdir d9/da/d51/dc2/d119/d11f 0 2024-05-09T06:38:20.240 INFO:tasks.workunit.client.0.smithi063.stdout:8/806: chown d9/d3b/f40 33 1 2024-05-09T06:38:20.241 INFO:tasks.workunit.client.0.smithi063.stdout:8/807: dread - d9/da/d51/fdd zero size 2024-05-09T06:38:20.242 INFO:tasks.workunit.client.0.smithi063.stdout:8/808: dread - d9/da/d51/fdd zero size 2024-05-09T06:38:20.245 INFO:tasks.workunit.client.0.smithi063.stdout:1/678: rename d9/d2a/fa2 to d9/d2a/d3c/d4c/d83/d88/fe3 0 2024-05-09T06:38:20.246 INFO:tasks.workunit.client.0.smithi063.stdout:9/741: creat d0/d4/d57/d60/df5/f102 x:0 0 0 2024-05-09T06:38:20.247 INFO:tasks.workunit.client.0.smithi063.stdout:7/698: mkdir d0/d15/d49/d4f/d66/df1 0 2024-05-09T06:38:20.248 INFO:tasks.workunit.client.0.smithi063.stdout:8/809: mkdir d9/da/d53/dbc/d102/d120 0 2024-05-09T06:38:20.250 INFO:tasks.workunit.client.0.smithi063.stdout:3/748: creat d7/d70/d98/d2d/ffb x:0 0 0 2024-05-09T06:38:20.250 INFO:tasks.workunit.client.0.smithi063.stdout:8/810: rmdir d9/da/d18/dc7/d8f 39 2024-05-09T06:38:20.251 INFO:tasks.workunit.client.0.smithi063.stdout:7/699: chown d0/d15/d4a/c8e 7415 1 2024-05-09T06:38:20.253 INFO:tasks.workunit.client.0.smithi063.stdout:1/679: mknod d9/d76/d94/ce4 0 2024-05-09T06:38:20.255 INFO:tasks.workunit.client.0.smithi063.stdout:9/742: getdents d0/d4/d57 0 2024-05-09T06:38:20.256 INFO:tasks.workunit.client.0.smithi063.stdout:7/700: symlink d0/d15/d49/d54/d9f/d97/lf2 0 2024-05-09T06:38:20.259 INFO:tasks.workunit.client.0.smithi063.stdout:7/701: chown d0/d15/d4a/cb2 2032267 1 2024-05-09T06:38:20.260 INFO:tasks.workunit.client.0.smithi063.stdout:3/749: dwrite d7/d70/d98/d5b/f8f [0,4194304] 0 2024-05-09T06:38:20.260 INFO:tasks.workunit.client.0.smithi063.stdout:9/743: creat d0/d4/d26/d39/d48/d80/d84/dd8/ddf/f103 x:0 0 0 2024-05-09T06:38:20.263 INFO:tasks.workunit.client.0.smithi063.stdout:7/702: mkdir d0/d15/d89/df3 0 2024-05-09T06:38:20.265 INFO:tasks.workunit.client.0.smithi063.stdout:7/703: fdatasync d0/d15/d89/d98/fe0 0 2024-05-09T06:38:20.266 INFO:tasks.workunit.client.0.smithi063.stdout:9/744: getdents d0/d4/d57/dba/dd5 0 2024-05-09T06:38:20.267 INFO:tasks.workunit.client.0.smithi063.stdout:7/704: unlink d0/d15/d49/f5e 0 2024-05-09T06:38:20.269 INFO:tasks.workunit.client.0.smithi063.stdout:7/705: fsync d0/f18 0 2024-05-09T06:38:20.276 INFO:tasks.workunit.client.0.smithi063.stdout:7/706: dwrite d0/d15/d49/fbf [8388608,4194304] 0 2024-05-09T06:38:20.278 INFO:tasks.workunit.client.1.smithi193.stdout:2/393: fdatasync db/d37/d6d/d80/f14 0 2024-05-09T06:38:20.279 INFO:tasks.workunit.client.1.smithi193.stdout:6/387: truncate d1/de/d11/d18/f61 3160700 0 2024-05-09T06:38:20.279 INFO:tasks.workunit.client.1.smithi193.stdout:1/383: dread d4/f26 [0,4194304] 0 2024-05-09T06:38:20.280 INFO:tasks.workunit.client.0.smithi063.stdout:7/707: symlink d0/dcb/lf4 0 2024-05-09T06:38:20.281 INFO:tasks.workunit.client.1.smithi193.stdout:5/358: dwrite d0/d13/d2c/d37/d5b/f21 [0,4194304] 0 2024-05-09T06:38:20.281 INFO:tasks.workunit.client.1.smithi193.stdout:1/384: readlink d4/l6 0 2024-05-09T06:38:20.282 INFO:tasks.workunit.client.1.smithi193.stdout:2/394: mknod db/d4b/c8a 0 2024-05-09T06:38:20.284 INFO:tasks.workunit.client.1.smithi193.stdout:2/395: chown db/df/d54/f7c 5235994 1 2024-05-09T06:38:20.288 INFO:tasks.workunit.client.1.smithi193.stdout:5/359: mkdir d0/d13/d2c/d37/d74 0 2024-05-09T06:38:20.290 INFO:tasks.workunit.client.1.smithi193.stdout:5/360: truncate d0/d13/d2c/f3f 1092711 0 2024-05-09T06:38:20.291 INFO:tasks.workunit.client.1.smithi193.stdout:1/385: mkdir d4/d76 0 2024-05-09T06:38:20.291 INFO:tasks.workunit.client.1.smithi193.stdout:5/361: stat d0/d13/d2c/d37/d5b/l48 0 2024-05-09T06:38:20.295 INFO:tasks.workunit.client.1.smithi193.stdout:2/396: rename db/df/d54/d67 to db/d25/d38/d40/d8b 0 2024-05-09T06:38:20.297 INFO:tasks.workunit.client.1.smithi193.stdout:1/386: readlink d4/d16/ld 0 2024-05-09T06:38:20.298 INFO:tasks.workunit.client.0.smithi063.stdout:8/811: dread d9/d3b/f115 [0,4194304] 0 2024-05-09T06:38:20.298 INFO:tasks.workunit.client.1.smithi193.stdout:5/362: mkdir d0/d13/d2c/d37/d2b/d47/d60/d75 0 2024-05-09T06:38:20.300 INFO:tasks.workunit.client.1.smithi193.stdout:2/397: creat db/df/d54/f8c x:0 0 0 2024-05-09T06:38:20.301 INFO:tasks.workunit.client.1.smithi193.stdout:6/388: link d1/de/d11/d2d/c35 d1/d25/d2c/d81/c89 0 2024-05-09T06:38:20.305 INFO:tasks.workunit.client.1.smithi193.stdout:1/387: rename d4/d16/d17/d1a/d3a/d3f/f6a to d4/d3c/d3e/f77 0 2024-05-09T06:38:20.307 INFO:tasks.workunit.client.0.smithi063.stdout:8/812: link d9/da/d18/d46/fe7 d9/da/d18/dc7/d58/f121 0 2024-05-09T06:38:20.308 INFO:tasks.workunit.client.0.smithi063.stdout:8/813: chown d9/da/d18/fc4 38 1 2024-05-09T06:38:20.309 INFO:tasks.workunit.client.1.smithi193.stdout:6/389: mknod d1/d25/d2c/d3c/d3e/d58/c8a 0 2024-05-09T06:38:20.310 INFO:tasks.workunit.client.1.smithi193.stdout:6/390: dread - d1/d25/d2c/d76/d69/f88 zero size 2024-05-09T06:38:20.310 INFO:tasks.workunit.client.1.smithi193.stdout:2/398: dread db/d25/d38/d61/f73 [0,4194304] 0 2024-05-09T06:38:20.311 INFO:tasks.workunit.client.0.smithi063.stdout:8/814: fdatasync d9/da/d18/f80 0 2024-05-09T06:38:20.314 INFO:tasks.workunit.client.1.smithi193.stdout:5/363: creat d0/d13/d2c/d4d/d67/f76 x:0 0 0 2024-05-09T06:38:20.315 INFO:tasks.workunit.client.1.smithi193.stdout:5/364: write d0/d13/d2c/d4d/d67/f76 [1032336,78908] 0 2024-05-09T06:38:20.316 INFO:tasks.workunit.client.1.smithi193.stdout:6/391: truncate d1/de/d11/d18/f29 1700776 0 2024-05-09T06:38:20.318 INFO:tasks.workunit.client.1.smithi193.stdout:1/388: truncate d4/d16/f47 233659 0 2024-05-09T06:38:20.319 INFO:tasks.workunit.client.0.smithi063.stdout:1/680: truncate d9/d2a/d35/d24/d68/f98 701496 0 2024-05-09T06:38:20.320 INFO:tasks.workunit.client.0.smithi063.stdout:8/815: mknod d9/da/d18/d46/d10c/c122 0 2024-05-09T06:38:20.320 INFO:tasks.workunit.client.0.smithi063.stdout:1/681: read d9/d2a/d35/d1e/d9d/f33 [1770486,114917] 0 2024-05-09T06:38:20.323 INFO:tasks.workunit.client.1.smithi193.stdout:2/399: dread db/f39 [0,4194304] 0 2024-05-09T06:38:20.323 INFO:tasks.workunit.client.1.smithi193.stdout:5/365: symlink d0/d6d/l77 0 2024-05-09T06:38:20.324 INFO:tasks.workunit.client.1.smithi193.stdout:5/366: readlink d0/d13/d2c/d37/d2b/d47/l5f 0 2024-05-09T06:38:20.324 INFO:tasks.workunit.client.0.smithi063.stdout:1/682: creat d9/d2a/d3c/d4a/fe5 x:0 0 0 2024-05-09T06:38:20.326 INFO:tasks.workunit.client.0.smithi063.stdout:9/745: dwrite d0/d4/d26/d39/d3e/ff2 [0,4194304] 0 2024-05-09T06:38:20.330 INFO:tasks.workunit.client.0.smithi063.stdout:7/708: truncate d0/d15/d49/d4f/d66/d67/d6b/fc3 81257 0 2024-05-09T06:38:20.330 INFO:tasks.workunit.client.0.smithi063.stdout:4/739: sync 2024-05-09T06:38:20.330 INFO:tasks.workunit.client.0.smithi063.stdout:5/721: sync 2024-05-09T06:38:20.330 INFO:tasks.workunit.client.0.smithi063.stdout:2/761: sync 2024-05-09T06:38:20.331 INFO:tasks.workunit.client.1.smithi193.stdout:5/367: dwrite d0/f5c [0,4194304] 0 2024-05-09T06:38:20.332 INFO:tasks.workunit.client.0.smithi063.stdout:8/816: creat d9/da/f123 x:0 0 0 2024-05-09T06:38:20.332 INFO:tasks.workunit.client.0.smithi063.stdout:5/722: stat d8/d53/dca/f81 0 2024-05-09T06:38:20.333 INFO:tasks.workunit.client.0.smithi063.stdout:8/817: chown d9/da/d18/d24/d71/feb 118 1 2024-05-09T06:38:20.334 INFO:tasks.workunit.client.0.smithi063.stdout:9/746: stat d0/f65 0 2024-05-09T06:38:20.336 INFO:tasks.workunit.client.1.smithi193.stdout:6/392: creat d1/de/d11/f8b x:0 0 0 2024-05-09T06:38:20.343 INFO:tasks.workunit.client.0.smithi063.stdout:8/818: dwrite d9/da/d53/dbc/d102/f110 [0,4194304] 0 2024-05-09T06:38:20.344 INFO:tasks.workunit.client.0.smithi063.stdout:4/740: creat d7/d3d/fee x:0 0 0 2024-05-09T06:38:20.348 INFO:tasks.workunit.client.1.smithi193.stdout:2/400: rename db/l3e to db/d25/d38/l8d 0 2024-05-09T06:38:20.349 INFO:tasks.workunit.client.1.smithi193.stdout:5/368: unlink d0/d13/d2c/d37/d5b/f2a 0 2024-05-09T06:38:20.351 INFO:tasks.workunit.client.0.smithi063.stdout:5/723: truncate d8/d17/da3/fbf 1167115 0 2024-05-09T06:38:20.351 INFO:tasks.workunit.client.1.smithi193.stdout:6/393: mknod d1/de/d11/d2d/d3a/c8c 0 2024-05-09T06:38:20.358 INFO:tasks.workunit.client.1.smithi193.stdout:1/389: getdents d4/d16/d35 0 2024-05-09T06:38:20.359 INFO:tasks.workunit.client.0.smithi063.stdout:1/683: truncate d9/d2a/d35/d1e/f27 2309796 0 2024-05-09T06:38:20.363 INFO:tasks.workunit.client.0.smithi063.stdout:8/819: mknod d9/da/d18/d46/c124 0 2024-05-09T06:38:20.366 INFO:tasks.workunit.client.0.smithi063.stdout:4/741: stat d7/df/d35/d5a/dad/lb5 0 2024-05-09T06:38:20.366 INFO:tasks.workunit.client.0.smithi063.stdout:1/684: dwrite d9/d2a/d3c/d4a/fdc [4194304,4194304] 0 2024-05-09T06:38:20.367 INFO:tasks.workunit.client.0.smithi063.stdout:4/742: chown d7/d50/fa1 12192354 1 2024-05-09T06:38:20.367 INFO:tasks.workunit.client.0.smithi063.stdout:7/709: creat d0/d15/d89/d98/de3/ff5 x:0 0 0 2024-05-09T06:38:20.369 INFO:tasks.workunit.client.0.smithi063.stdout:4/743: chown d7/df/d1a/fe2 433 1 2024-05-09T06:38:20.369 INFO:tasks.workunit.client.0.smithi063.stdout:7/710: chown d0/d15/l41 7861 1 2024-05-09T06:38:20.391 INFO:tasks.workunit.client.1.smithi193.stdout:2/401: read db/d25/f48 [2098353,104555] 0 2024-05-09T06:38:20.392 INFO:tasks.workunit.client.1.smithi193.stdout:6/394: symlink d1/d25/d2c/d81/d37/l8d 0 2024-05-09T06:38:20.394 INFO:tasks.workunit.client.1.smithi193.stdout:5/369: dread d0/d13/d2c/f4a [0,4194304] 0 2024-05-09T06:38:20.394 INFO:tasks.workunit.client.1.smithi193.stdout:6/395: write d1/d25/d2c/d81/d37/f5b [3669084,65460] 0 2024-05-09T06:38:20.396 INFO:tasks.workunit.client.1.smithi193.stdout:5/370: write d0/d13/d2c/d37/d5b/d4e/d5d/f61 [1375828,96155] 0 2024-05-09T06:38:20.396 INFO:tasks.workunit.client.1.smithi193.stdout:2/402: unlink db/d37/c7a 0 2024-05-09T06:38:20.400 INFO:tasks.workunit.client.1.smithi193.stdout:6/396: symlink d1/d25/d2c/d3c/d3e/l8e 0 2024-05-09T06:38:20.401 INFO:tasks.workunit.client.1.smithi193.stdout:2/403: symlink db/d25/l8e 0 2024-05-09T06:38:20.403 INFO:tasks.workunit.client.1.smithi193.stdout:2/404: read db/d25/f49 [143597,66208] 0 2024-05-09T06:38:20.405 INFO:tasks.workunit.client.1.smithi193.stdout:2/405: write db/d25/d38/f6b [226696,110632] 0 2024-05-09T06:38:20.408 INFO:tasks.workunit.client.1.smithi193.stdout:6/397: dwrite d1/d25/d2c/d81/d37/d4f/f66 [0,4194304] 0 2024-05-09T06:38:20.411 INFO:tasks.workunit.client.1.smithi193.stdout:6/398: rmdir d1/d25/d2c/d81/d37 39 2024-05-09T06:38:20.412 INFO:tasks.workunit.client.0.smithi063.stdout:1/685: mknod d9/d2a/d3c/d4c/d83/ce6 0 2024-05-09T06:38:20.412 INFO:tasks.workunit.client.0.smithi063.stdout:7/711: dread - d0/d15/fd5 zero size 2024-05-09T06:38:20.414 INFO:tasks.workunit.client.0.smithi063.stdout:2/762: dwrite fa [0,4194304] 0 2024-05-09T06:38:20.418 INFO:tasks.workunit.client.1.smithi193.stdout:6/399: symlink d1/de/d11/d2d/d3a/l8f 0 2024-05-09T06:38:20.418 INFO:tasks.workunit.client.0.smithi063.stdout:7/712: write d0/d15/d89/fa9 [398231,94540] 0 2024-05-09T06:38:20.418 INFO:tasks.workunit.client.0.smithi063.stdout:4/744: unlink d7/da3/l83 0 2024-05-09T06:38:20.419 INFO:tasks.workunit.client.0.smithi063.stdout:2/763: read - dc/dd/d2d/d60/dd0/d90/dc1/fe1 zero size 2024-05-09T06:38:20.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/747: creat d0/f104 x:0 0 0 2024-05-09T06:38:20.419 INFO:tasks.workunit.client.1.smithi193.stdout:6/400: chown d1/d25/d2c/d76/f57 1360532459 1 2024-05-09T06:38:20.421 INFO:tasks.workunit.client.0.smithi063.stdout:2/764: read - dc/dd/d2d/d60/fac zero size 2024-05-09T06:38:20.421 INFO:tasks.workunit.client.1.smithi193.stdout:2/406: getdents db/d25/d38/d40 0 2024-05-09T06:38:20.422 INFO:tasks.workunit.client.1.smithi193.stdout:6/401: write d1/f6 [7167934,51146] 0 2024-05-09T06:38:20.423 INFO:tasks.workunit.client.0.smithi063.stdout:2/765: truncate dc/d32/d33/d4e/f88 1602755 0 2024-05-09T06:38:20.424 INFO:tasks.workunit.client.1.smithi193.stdout:2/407: truncate db/f1e 4616047 0 2024-05-09T06:38:20.425 INFO:tasks.workunit.client.1.smithi193.stdout:1/390: dwrite d4/d16/d6b/d6e/f70 [0,4194304] 0 2024-05-09T06:38:20.426 INFO:tasks.workunit.client.0.smithi063.stdout:1/686: fsync d9/d2a/d35/d1e/d9d/f82 0 2024-05-09T06:38:20.428 INFO:tasks.workunit.client.1.smithi193.stdout:6/402: write d1/fd [703208,37598] 0 2024-05-09T06:38:20.429 INFO:tasks.workunit.client.0.smithi063.stdout:9/748: dread d0/d8/dea/f101 [0,4194304] 0 2024-05-09T06:38:20.430 INFO:tasks.workunit.client.0.smithi063.stdout:4/745: creat d7/da3/d28/d31/d71/d98/fef x:0 0 0 2024-05-09T06:38:20.432 INFO:tasks.workunit.client.0.smithi063.stdout:2/766: symlink dc/dd/d45/d5a/d6a/le9 0 2024-05-09T06:38:20.434 INFO:tasks.workunit.client.0.smithi063.stdout:1/687: truncate d9/d46/fdd 748560 0 2024-05-09T06:38:20.437 INFO:tasks.workunit.client.1.smithi193.stdout:1/391: dwrite d4/d14/f2e [0,4194304] 0 2024-05-09T06:38:20.444 INFO:tasks.workunit.client.1.smithi193.stdout:1/392: creat d4/d16/d6b/d6e/f78 x:0 0 0 2024-05-09T06:38:20.446 INFO:tasks.workunit.client.0.smithi063.stdout:5/724: link d8/d53/fa5 d8/d19/d30/d74/ff1 0 2024-05-09T06:38:20.447 INFO:tasks.workunit.client.1.smithi193.stdout:6/403: mknod d1/d25/d2c/d81/d37/c90 0 2024-05-09T06:38:20.447 INFO:tasks.workunit.client.0.smithi063.stdout:5/725: write d8/d17/d18/fb3 [1506026,80250] 0 2024-05-09T06:38:20.448 INFO:tasks.workunit.client.0.smithi063.stdout:7/713: creat d0/d15/d49/ff6 x:0 0 0 2024-05-09T06:38:20.450 INFO:tasks.workunit.client.0.smithi063.stdout:2/767: dread dc/dd/d2d/fa0 [4194304,4194304] 0 2024-05-09T06:38:20.451 INFO:tasks.workunit.client.1.smithi193.stdout:2/408: getdents db/d37 0 2024-05-09T06:38:20.453 INFO:tasks.workunit.client.1.smithi193.stdout:1/393: dread d4/f13 [0,4194304] 0 2024-05-09T06:38:20.454 INFO:tasks.workunit.client.0.smithi063.stdout:5/726: read d8/f4c [4016283,749] 0 2024-05-09T06:38:20.454 INFO:tasks.workunit.client.0.smithi063.stdout:2/768: mknod dc/dd/d45/d5a/d6c/d8f/cea 0 2024-05-09T06:38:20.455 INFO:tasks.workunit.client.1.smithi193.stdout:6/404: dwrite d1/d25/d2c/d3c/d3e/f43 [0,4194304] 0 2024-05-09T06:38:20.455 INFO:tasks.workunit.client.0.smithi063.stdout:2/769: chown dc/dd/d2d 110404337 1 2024-05-09T06:38:20.456 INFO:tasks.workunit.client.1.smithi193.stdout:2/409: mkdir db/d25/d38/d40/d8f 0 2024-05-09T06:38:20.456 INFO:tasks.workunit.client.0.smithi063.stdout:2/770: readlink dc/dd/d2d/d60/dd0/d90/l78 0 2024-05-09T06:38:20.458 INFO:tasks.workunit.client.0.smithi063.stdout:7/714: link d0/d15/f4e d0/d15/d49/d54/ff7 0 2024-05-09T06:38:20.458 INFO:tasks.workunit.client.0.smithi063.stdout:5/727: readlink d8/d19/d30/d74/d77/dde/le5 0 2024-05-09T06:38:20.459 INFO:tasks.workunit.client.1.smithi193.stdout:1/394: mknod d4/d14/c79 0 2024-05-09T06:38:20.460 INFO:tasks.workunit.client.1.smithi193.stdout:6/405: rmdir d1/d25/d2c/d81 39 2024-05-09T06:38:20.461 INFO:tasks.workunit.client.1.smithi193.stdout:2/410: rmdir db/d25/d38 39 2024-05-09T06:38:20.464 INFO:tasks.workunit.client.0.smithi063.stdout:5/728: creat d8/d17/d75/dc9/dd5/d96/ff2 x:0 0 0 2024-05-09T06:38:20.465 INFO:tasks.workunit.client.0.smithi063.stdout:5/729: dread - d8/d19/fe2 zero size 2024-05-09T06:38:20.466 INFO:tasks.workunit.client.0.smithi063.stdout:0/769: sync 2024-05-09T06:38:20.466 INFO:tasks.workunit.client.0.smithi063.stdout:7/715: truncate d0/d15/d4a/f7e 268048 0 2024-05-09T06:38:20.467 INFO:tasks.workunit.client.0.smithi063.stdout:0/770: chown dd/d1a/d2c/ca3 234251241 1 2024-05-09T06:38:20.467 INFO:tasks.workunit.client.1.smithi193.stdout:2/411: write db/d25/d38/f75 [4037626,59599] 0 2024-05-09T06:38:20.468 INFO:tasks.workunit.client.1.smithi193.stdout:1/395: creat d4/d16/d60/d4f/d72/f7a x:0 0 0 2024-05-09T06:38:20.470 INFO:tasks.workunit.client.1.smithi193.stdout:2/412: rmdir db/d25/d38 39 2024-05-09T06:38:20.472 INFO:tasks.workunit.client.1.smithi193.stdout:2/413: write db/d37/d6d/d80/f14 [570552,14431] 0 2024-05-09T06:38:20.473 INFO:tasks.workunit.client.0.smithi063.stdout:2/771: getdents dc/dd/d45/d5a/d6c 0 2024-05-09T06:38:20.474 INFO:tasks.workunit.client.0.smithi063.stdout:7/716: dwrite d0/d15/f76 [0,4194304] 0 2024-05-09T06:38:20.474 INFO:tasks.workunit.client.1.smithi193.stdout:1/396: mknod d4/d16/d17/d1a/c7b 0 2024-05-09T06:38:20.475 INFO:tasks.workunit.client.0.smithi063.stdout:2/772: write dc/dd/fb5 [813619,89598] 0 2024-05-09T06:38:20.475 INFO:tasks.workunit.client.0.smithi063.stdout:7/717: chown d0/l57 140372438 1 2024-05-09T06:38:20.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/414: dread db/d37/d6d/d80/f24 [0,4194304] 0 2024-05-09T06:38:20.477 INFO:tasks.workunit.client.0.smithi063.stdout:2/773: write f6 [1278584,62561] 0 2024-05-09T06:38:20.479 INFO:tasks.workunit.client.0.smithi063.stdout:5/730: rename d8/d19/d30/d74/d77/le4 to d8/d19/d30/d74/d77/dde/lf3 0 2024-05-09T06:38:20.479 INFO:tasks.workunit.client.0.smithi063.stdout:1/688: write d9/d10/f22 [26814,16009] 0 2024-05-09T06:38:20.486 INFO:tasks.workunit.client.0.smithi063.stdout:7/718: chown d0/d15/cd0 1939796 1 2024-05-09T06:38:20.487 INFO:tasks.workunit.client.1.smithi193.stdout:2/415: creat db/d4b/f90 x:0 0 0 2024-05-09T06:38:20.487 INFO:tasks.workunit.client.0.smithi063.stdout:7/719: write d0/d15/d49/ff6 [31754,113685] 0 2024-05-09T06:38:20.490 INFO:tasks.workunit.client.1.smithi193.stdout:1/397: fdatasync d4/d16/f47 0 2024-05-09T06:38:20.493 INFO:tasks.workunit.client.0.smithi063.stdout:0/771: rename dd/d1a/d2c/dba to dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4 0 2024-05-09T06:38:20.493 INFO:tasks.workunit.client.1.smithi193.stdout:2/416: fdatasync db/f28 0 2024-05-09T06:38:20.499 INFO:tasks.workunit.client.0.smithi063.stdout:2/774: unlink dc/dd/l51 0 2024-05-09T06:38:20.501 INFO:tasks.workunit.client.0.smithi063.stdout:7/720: creat d0/d15/d89/d98/de3/ff8 x:0 0 0 2024-05-09T06:38:20.501 INFO:tasks.workunit.client.0.smithi063.stdout:1/689: dwrite d9/d2a/d35/d24/faa [0,4194304] 0 2024-05-09T06:38:20.504 INFO:tasks.workunit.client.1.smithi193.stdout:2/417: symlink db/l91 0 2024-05-09T06:38:20.505 INFO:tasks.workunit.client.0.smithi063.stdout:7/721: truncate d0/d15/d49/d54/d9f/d97/d9b/d9d/fa8 926822 0 2024-05-09T06:38:20.505 INFO:tasks.workunit.client.1.smithi193.stdout:2/418: readlink db/d37/d6d/d80/l3b 0 2024-05-09T06:38:20.505 INFO:tasks.workunit.client.0.smithi063.stdout:5/731: mkdir d8/d17/d18/df4 0 2024-05-09T06:38:20.508 INFO:tasks.workunit.client.0.smithi063.stdout:2/775: creat dc/d36/d43/d46/d63/d9e/feb x:0 0 0 2024-05-09T06:38:20.509 INFO:tasks.workunit.client.0.smithi063.stdout:2/776: readlink dc/dd/d45/d5a/d6c/l9b 0 2024-05-09T06:38:20.510 INFO:tasks.workunit.client.1.smithi193.stdout:1/398: dwrite d4/d16/d17/d1a/f1e [0,4194304] 0 2024-05-09T06:38:20.510 INFO:tasks.workunit.client.0.smithi063.stdout:5/732: creat d8/d65/d9e/ff5 x:0 0 0 2024-05-09T06:38:20.514 INFO:tasks.workunit.client.0.smithi063.stdout:0/772: creat dd/d1a/d2c/d51/d72/d9e/d23/d89/ff5 x:0 0 0 2024-05-09T06:38:20.516 INFO:tasks.workunit.client.0.smithi063.stdout:2/777: unlink dc/dd/d45/f89 0 2024-05-09T06:38:20.517 INFO:tasks.workunit.client.1.smithi193.stdout:1/399: rmdir d4/d3c 39 2024-05-09T06:38:20.518 INFO:tasks.workunit.client.0.smithi063.stdout:5/733: mknod d8/dc8/dda/cf6 0 2024-05-09T06:38:20.520 INFO:tasks.workunit.client.0.smithi063.stdout:7/722: mkdir d0/d15/d49/d54/d9f/dd2/df9 0 2024-05-09T06:38:20.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/773: creat dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/ff6 x:0 0 0 2024-05-09T06:38:20.524 INFO:tasks.workunit.client.1.smithi193.stdout:1/400: mkdir d4/d3c/d3e/d7c 0 2024-05-09T06:38:20.524 INFO:tasks.workunit.client.1.smithi193.stdout:1/401: chown d4/d3c/d3e/c40 5173 1 2024-05-09T06:38:20.525 INFO:tasks.workunit.client.0.smithi063.stdout:5/734: mknod d8/d17/da3/cf7 0 2024-05-09T06:38:20.525 INFO:tasks.workunit.client.1.smithi193.stdout:2/419: link db/d37/d6d/d80/c4d db/d25/d38/c92 0 2024-05-09T06:38:20.525 INFO:tasks.workunit.client.1.smithi193.stdout:1/402: write d4/d3c/d3e/f5c [976335,15283] 0 2024-05-09T06:38:20.527 INFO:tasks.workunit.client.0.smithi063.stdout:7/723: unlink d0/d15/d4a/d50/l4b 0 2024-05-09T06:38:20.527 INFO:tasks.workunit.client.1.smithi193.stdout:1/403: read d4/d16/d60/f58 [123809,73986] 0 2024-05-09T06:38:20.528 INFO:tasks.workunit.client.1.smithi193.stdout:1/404: chown d4/d16/d17/d1a/c7b 1703 1 2024-05-09T06:38:20.528 INFO:tasks.workunit.client.1.smithi193.stdout:1/405: stat d4/d16 0 2024-05-09T06:38:20.530 INFO:tasks.workunit.client.1.smithi193.stdout:2/420: creat db/d25/d38/d40/d8f/f93 x:0 0 0 2024-05-09T06:38:20.533 INFO:tasks.workunit.client.0.smithi063.stdout:5/735: creat d8/d19/d30/db1/ff8 x:0 0 0 2024-05-09T06:38:20.534 INFO:tasks.workunit.client.0.smithi063.stdout:2/778: dread f2 [4194304,4194304] 0 2024-05-09T06:38:20.534 INFO:tasks.workunit.client.0.smithi063.stdout:7/724: dwrite d0/f81 [0,4194304] 0 2024-05-09T06:38:20.537 INFO:tasks.workunit.client.1.smithi193.stdout:2/421: dwrite db/d4b/f90 [0,4194304] 0 2024-05-09T06:38:20.545 INFO:tasks.workunit.client.1.smithi193.stdout:2/422: dwrite db/d25/d38/f75 [4194304,4194304] 0 2024-05-09T06:38:20.547 INFO:tasks.workunit.client.1.smithi193.stdout:2/423: chown db/d37/d6d/d80/l2c 27481325 1 2024-05-09T06:38:20.548 INFO:tasks.workunit.client.1.smithi193.stdout:2/424: read db/f45 [1444624,129472] 0 2024-05-09T06:38:20.555 INFO:tasks.workunit.client.1.smithi193.stdout:1/406: dwrite d4/d16/fb [0,4194304] 0 2024-05-09T06:38:20.558 INFO:tasks.workunit.client.1.smithi193.stdout:1/407: truncate d4/d16/d17/d1a/d3a/d3f/f68 148732 0 2024-05-09T06:38:20.559 INFO:tasks.workunit.client.1.smithi193.stdout:1/408: chown d4/d3c/c53 825 1 2024-05-09T06:38:20.560 INFO:tasks.workunit.client.1.smithi193.stdout:1/409: symlink d4/d76/l7d 0 2024-05-09T06:38:20.562 INFO:tasks.workunit.client.1.smithi193.stdout:1/410: mkdir d4/d16/d17/d1a/d7e 0 2024-05-09T06:38:20.563 INFO:tasks.workunit.client.1.smithi193.stdout:1/411: dread - d4/d16/d6b/d6e/f78 zero size 2024-05-09T06:38:20.565 INFO:tasks.workunit.client.1.smithi193.stdout:1/412: mkdir d4/d16/d60/d4f/d72/d7f 0 2024-05-09T06:38:20.565 INFO:tasks.workunit.client.1.smithi193.stdout:1/413: write d4/d16/d17/d1a/f66 [1151961,1452] 0 2024-05-09T06:38:20.568 INFO:tasks.workunit.client.1.smithi193.stdout:1/414: creat d4/d76/f80 x:0 0 0 2024-05-09T06:38:20.569 INFO:tasks.workunit.client.1.smithi193.stdout:1/415: symlink d4/d16/d6b/d6e/l81 0 2024-05-09T06:38:20.571 INFO:tasks.workunit.client.1.smithi193.stdout:1/416: symlink d4/d16/d6b/l82 0 2024-05-09T06:38:20.580 INFO:tasks.workunit.client.1.smithi193.stdout:1/417: truncate d4/d16/d6b/d6e/f70 4192816 0 2024-05-09T06:38:20.581 INFO:tasks.workunit.client.1.smithi193.stdout:1/418: fsync d4/d16/d17/d1a/d3a/d3f/f45 0 2024-05-09T06:38:20.582 INFO:tasks.workunit.client.1.smithi193.stdout:1/419: write d4/d16/d17/d1a/d3a/d3f/f68 [96353,29708] 0 2024-05-09T06:38:20.585 INFO:tasks.workunit.client.1.smithi193.stdout:1/420: truncate d4/fc 251454 0 2024-05-09T06:38:20.586 INFO:tasks.workunit.client.1.smithi193.stdout:1/421: fdatasync d4/d16/d17/f29 0 2024-05-09T06:38:20.586 INFO:tasks.workunit.client.0.smithi063.stdout:2/779: fsync dc/d36/d43/d46/fcc 0 2024-05-09T06:38:20.589 INFO:tasks.workunit.client.0.smithi063.stdout:2/780: chown dc/d32/c6f 1235 1 2024-05-09T06:38:20.589 INFO:tasks.workunit.client.0.smithi063.stdout:5/736: dwrite d8/d17/d18/fc3 [0,4194304] 0 2024-05-09T06:38:20.596 INFO:tasks.workunit.client.0.smithi063.stdout:5/737: dread - d8/d17/d75/dc9/dd5/fee zero size 2024-05-09T06:38:20.596 INFO:tasks.workunit.client.0.smithi063.stdout:0/774: dwrite dd/d1a/d2c/fa1 [0,4194304] 0 2024-05-09T06:38:20.600 INFO:tasks.workunit.client.0.smithi063.stdout:0/775: fsync dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0/fc5 0 2024-05-09T06:38:20.600 INFO:tasks.workunit.client.0.smithi063.stdout:7/725: dwrite d0/d15/d49/d54/d9f/d97/fec [0,4194304] 0 2024-05-09T06:38:20.609 INFO:tasks.workunit.client.1.smithi193.stdout:1/422: write d4/d16/d6b/d6e/f70 [2799488,77095] 0 2024-05-09T06:38:20.613 INFO:tasks.workunit.client.0.smithi063.stdout:8/820: sync 2024-05-09T06:38:20.613 INFO:tasks.workunit.client.0.smithi063.stdout:6/751: sync 2024-05-09T06:38:20.613 INFO:tasks.workunit.client.0.smithi063.stdout:3/750: sync 2024-05-09T06:38:20.622 INFO:tasks.workunit.client.0.smithi063.stdout:3/751: mkdir d7/d22/dfc 0 2024-05-09T06:38:20.624 INFO:tasks.workunit.client.0.smithi063.stdout:2/781: mkdir dc/d36/d43/d46/d49/dbb/dec 0 2024-05-09T06:38:20.625 INFO:tasks.workunit.client.0.smithi063.stdout:5/738: unlink d8/d19/d30/f43 0 2024-05-09T06:38:20.625 INFO:tasks.workunit.client.0.smithi063.stdout:0/776: mknod dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4/de1/cf7 0 2024-05-09T06:38:20.627 INFO:tasks.workunit.client.0.smithi063.stdout:5/739: chown d8/d19/d30/d74 111832868 1 2024-05-09T06:38:20.629 INFO:tasks.workunit.client.0.smithi063.stdout:2/782: mkdir dc/d36/d43/d46/ded 0 2024-05-09T06:38:20.631 INFO:tasks.workunit.client.0.smithi063.stdout:0/777: fdatasync dd/d1a/d2c/d51/d72/d9e/d23/d33/f3e 0 2024-05-09T06:38:20.633 INFO:tasks.workunit.client.0.smithi063.stdout:8/821: getdents d9/da/d18/dc7/dd3 0 2024-05-09T06:38:20.634 INFO:tasks.workunit.client.0.smithi063.stdout:7/726: dread d0/d15/d4a/d50/f23 [0,4194304] 0 2024-05-09T06:38:20.635 INFO:tasks.workunit.client.0.smithi063.stdout:2/783: symlink dc/d36/d43/d46/d49/ddc/lee 0 2024-05-09T06:38:20.637 INFO:tasks.workunit.client.0.smithi063.stdout:8/822: truncate d9/da/d18/f63 1266006 0 2024-05-09T06:38:20.638 INFO:tasks.workunit.client.0.smithi063.stdout:0/778: fdatasync dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/f7b 0 2024-05-09T06:38:20.642 INFO:tasks.workunit.client.0.smithi063.stdout:2/784: rename dc/d32/d33/d4e/f88 to dc/dd/d2d/d60/dd0/d90/fef 0 2024-05-09T06:38:20.642 INFO:tasks.workunit.client.0.smithi063.stdout:6/752: dread d2/d20/f21 [0,4194304] 0 2024-05-09T06:38:20.643 INFO:tasks.workunit.client.0.smithi063.stdout:8/823: creat d9/d41/dec/db8/f125 x:0 0 0 2024-05-09T06:38:20.645 INFO:tasks.workunit.client.0.smithi063.stdout:3/752: dread d7/da/d2b/d66/dc0/f6e [0,4194304] 0 2024-05-09T06:38:20.646 INFO:tasks.workunit.client.0.smithi063.stdout:2/785: rmdir dc/d36/d43/d46/d63/d79/d7d/d8d 39 2024-05-09T06:38:20.649 INFO:tasks.workunit.client.0.smithi063.stdout:6/753: creat d2/d4/d7c/d7e/ff0 x:0 0 0 2024-05-09T06:38:20.650 INFO:tasks.workunit.client.1.smithi193.stdout:1/423: write d4/d3c/f36 [307389,119140] 0 2024-05-09T06:38:20.651 INFO:tasks.workunit.client.0.smithi063.stdout:8/824: creat d9/da/d53/dbc/d102/d120/f126 x:0 0 0 2024-05-09T06:38:20.652 INFO:tasks.workunit.client.0.smithi063.stdout:8/825: chown d9/da/d53/da3/lc3 477 1 2024-05-09T06:38:20.653 INFO:tasks.workunit.client.0.smithi063.stdout:6/754: rename d2/d20/d26/la2 to d2/d1f/d29/lf1 0 2024-05-09T06:38:20.653 INFO:tasks.workunit.client.0.smithi063.stdout:6/755: chown d2/d20/d4a/l77 268038439 1 2024-05-09T06:38:20.654 INFO:tasks.workunit.client.0.smithi063.stdout:8/826: rmdir d9/da/d45 39 2024-05-09T06:38:20.655 INFO:tasks.workunit.client.0.smithi063.stdout:8/827: fsync d9/da/f123 0 2024-05-09T06:38:20.659 INFO:tasks.workunit.client.1.smithi193.stdout:1/424: dwrite d4/d16/f42 [0,4194304] 0 2024-05-09T06:38:20.661 INFO:tasks.workunit.client.0.smithi063.stdout:8/828: dread d9/f77 [0,4194304] 0 2024-05-09T06:38:20.662 INFO:tasks.workunit.client.0.smithi063.stdout:8/829: readlink d9/da/d18/d24/d71/l74 0 2024-05-09T06:38:20.662 INFO:tasks.workunit.client.0.smithi063.stdout:9/749: sync 2024-05-09T06:38:20.663 INFO:tasks.workunit.client.1.smithi193.stdout:1/425: symlink d4/d14/l83 0 2024-05-09T06:38:20.664 INFO:tasks.workunit.client.0.smithi063.stdout:9/750: write d0/d4/d7d/ffd [212786,71142] 0 2024-05-09T06:38:20.669 INFO:tasks.workunit.client.1.smithi193.stdout:9/435: sync 2024-05-09T06:38:20.670 INFO:tasks.workunit.client.1.smithi193.stdout:9/436: chown dd/df/d31/d7e/f35 150714089 1 2024-05-09T06:38:20.670 INFO:tasks.workunit.client.1.smithi193.stdout:1/426: unlink d4/d16/d17/d1a/d3a/d3f/f45 0 2024-05-09T06:38:20.672 INFO:tasks.workunit.client.0.smithi063.stdout:9/751: mkdir d0/d4/d7d/d105 0 2024-05-09T06:38:20.672 INFO:tasks.workunit.client.0.smithi063.stdout:5/740: write d8/d65/f7a [437588,121847] 0 2024-05-09T06:38:20.674 INFO:tasks.workunit.client.0.smithi063.stdout:9/752: chown d0/d4/d7d/fb3 86 1 2024-05-09T06:38:20.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:20 smithi063 ceph-mon[27171]: pgmap v13: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 61 MiB/s rd, 193 MiB/s wr, 370 op/s 2024-05-09T06:38:20.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:20 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:20.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:20 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:20.746 INFO:tasks.workunit.client.0.smithi063.stdout:1/690: sync 2024-05-09T06:38:20.746 INFO:tasks.workunit.client.0.smithi063.stdout:4/746: sync 2024-05-09T06:38:20.747 INFO:tasks.workunit.client.0.smithi063.stdout:1/691: rename d9 to d9/d10/de7 22 2024-05-09T06:38:20.757 INFO:tasks.workunit.client.1.smithi193.stdout:9/437: symlink dd/df/d31/d7e/d71/l95 0 2024-05-09T06:38:20.759 INFO:tasks.workunit.client.1.smithi193.stdout:1/427: mknod d4/d16/c84 0 2024-05-09T06:38:20.761 INFO:tasks.workunit.client.1.smithi193.stdout:9/438: rmdir dd/df/d31 39 2024-05-09T06:38:20.761 INFO:tasks.workunit.client.0.smithi063.stdout:9/753: creat d0/d8/da6/db0/db5/f106 x:0 0 0 2024-05-09T06:38:20.763 INFO:tasks.workunit.client.1.smithi193.stdout:1/428: write d4/d16/d17/d1a/f1f [2161605,61527] 0 2024-05-09T06:38:20.763 INFO:tasks.workunit.client.1.smithi193.stdout:1/429: dread - d4/d14/f6c zero size 2024-05-09T06:38:20.764 INFO:tasks.workunit.client.0.smithi063.stdout:9/754: symlink d0/d4/d7d/l107 0 2024-05-09T06:38:20.764 INFO:tasks.workunit.client.1.smithi193.stdout:1/430: chown d4/d16/d17/d1a/l4b 91194705 1 2024-05-09T06:38:20.765 INFO:tasks.workunit.client.1.smithi193.stdout:1/431: chown d4/d3c/d3e/f5c 7361 1 2024-05-09T06:38:20.766 INFO:tasks.workunit.client.1.smithi193.stdout:9/439: creat dd/df/d15/f96 x:0 0 0 2024-05-09T06:38:20.767 INFO:tasks.workunit.client.0.smithi063.stdout:0/779: dwrite dd/d1a/f65 [0,4194304] 0 2024-05-09T06:38:20.767 INFO:tasks.workunit.client.0.smithi063.stdout:6/756: write d2/d20/d26/d7f/dcf/fe5 [1774401,42871] 0 2024-05-09T06:38:20.767 INFO:tasks.workunit.client.0.smithi063.stdout:1/692: truncate d9/d2a/d2e/f50 659233 0 2024-05-09T06:38:20.768 INFO:tasks.workunit.client.1.smithi193.stdout:1/432: mkdir d4/d16/d6b/d85 0 2024-05-09T06:38:20.774 INFO:tasks.workunit.client.0.smithi063.stdout:9/755: rename d0/d4/d26/d47/f2b to d0/d8/d2d/f108 0 2024-05-09T06:38:20.775 INFO:tasks.workunit.client.1.smithi193.stdout:9/440: symlink dd/l97 0 2024-05-09T06:38:20.776 INFO:tasks.workunit.client.0.smithi063.stdout:8/830: write d9/f36 [2250904,72770] 0 2024-05-09T06:38:20.776 INFO:tasks.workunit.client.0.smithi063.stdout:6/757: mknod d2/d89/dd3/cf2 0 2024-05-09T06:38:20.778 INFO:tasks.workunit.client.1.smithi193.stdout:1/433: creat d4/d16/d6b/d85/f86 x:0 0 0 2024-05-09T06:38:20.778 INFO:tasks.workunit.client.0.smithi063.stdout:8/831: readlink d9/da/d53/dbc/l100 0 2024-05-09T06:38:20.778 INFO:tasks.workunit.client.0.smithi063.stdout:6/758: dread - d2/d1f/d29/d3f/d5c/d64/fee zero size 2024-05-09T06:38:20.779 INFO:tasks.workunit.client.1.smithi193.stdout:1/434: write d4/d3c/f36 [173795,89296] 0 2024-05-09T06:38:20.780 INFO:tasks.workunit.client.1.smithi193.stdout:9/441: fdatasync dd/df/d31/d7e/d37/f7c 0 2024-05-09T06:38:20.780 INFO:tasks.workunit.client.0.smithi063.stdout:6/759: chown d2/d4/d7c/d7e/ff0 9 1 2024-05-09T06:38:20.781 INFO:tasks.workunit.client.0.smithi063.stdout:8/832: truncate d9/d41/dec/f105 450844 0 2024-05-09T06:38:20.781 INFO:tasks.workunit.client.1.smithi193.stdout:9/442: chown dd/df/d15/d8c/d7a 6220 1 2024-05-09T06:38:20.781 INFO:tasks.workunit.client.1.smithi193.stdout:1/435: write d4/d16/d17/d1a/d3a/d3f/f55 [1042957,89934] 0 2024-05-09T06:38:20.782 INFO:tasks.workunit.client.1.smithi193.stdout:9/443: dread - dd/df/d15/f8e zero size 2024-05-09T06:38:20.783 INFO:tasks.workunit.client.0.smithi063.stdout:9/756: unlink d0/d4/d57/d60/d76/f8f 0 2024-05-09T06:38:20.786 INFO:tasks.workunit.client.1.smithi193.stdout:9/444: mkdir dd/df/d15/d98 0 2024-05-09T06:38:20.787 INFO:tasks.workunit.client.0.smithi063.stdout:6/760: fsync d2/d4/d7c/fcc 0 2024-05-09T06:38:20.788 INFO:tasks.workunit.client.0.smithi063.stdout:8/833: mkdir d9/da/d51/dc2/d127 0 2024-05-09T06:38:20.790 INFO:tasks.workunit.client.0.smithi063.stdout:1/693: rename d9/d2a/d35/d1e/d9d/d3f/d59/fd9 to d9/d2a/d35/d1e/d9d/fe8 0 2024-05-09T06:38:20.792 INFO:tasks.workunit.client.1.smithi193.stdout:9/445: symlink dd/d1e/d1f/d43/d5b/l99 0 2024-05-09T06:38:20.794 INFO:tasks.workunit.client.1.smithi193.stdout:1/436: dread d4/d16/f10 [0,4194304] 0 2024-05-09T06:38:20.797 INFO:tasks.workunit.client.0.smithi063.stdout:8/834: dwrite d9/da/d34/fac [0,4194304] 0 2024-05-09T06:38:20.800 INFO:tasks.workunit.client.1.smithi193.stdout:1/437: unlink d4/d76/l7d 0 2024-05-09T06:38:20.800 INFO:tasks.workunit.client.0.smithi063.stdout:8/835: readlink d9/d41/l8c 0 2024-05-09T06:38:20.801 INFO:tasks.workunit.client.0.smithi063.stdout:7/727: sync 2024-05-09T06:38:20.801 INFO:tasks.workunit.client.0.smithi063.stdout:2/786: sync 2024-05-09T06:38:20.801 INFO:tasks.workunit.client.0.smithi063.stdout:1/694: mkdir d9/d2a/d3c/d4c/d83/de9 0 2024-05-09T06:38:20.801 INFO:tasks.workunit.client.0.smithi063.stdout:3/753: sync 2024-05-09T06:38:20.805 INFO:tasks.workunit.client.0.smithi063.stdout:0/780: dread dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/f6e [0,4194304] 0 2024-05-09T06:38:20.807 INFO:tasks.workunit.client.1.smithi193.stdout:1/438: creat d4/f87 x:0 0 0 2024-05-09T06:38:20.808 INFO:tasks.workunit.client.0.smithi063.stdout:2/787: stat dc/d36/d43/d46/d63/d79/d7d 0 2024-05-09T06:38:20.808 INFO:tasks.workunit.client.0.smithi063.stdout:3/754: chown d7/d70 9 1 2024-05-09T06:38:20.808 INFO:tasks.workunit.client.0.smithi063.stdout:8/836: write d9/da/d45/de4/ff3 [484874,28157] 0 2024-05-09T06:38:20.811 INFO:tasks.workunit.client.0.smithi063.stdout:7/728: symlink d0/d15/d49/d4f/lfa 0 2024-05-09T06:38:20.811 INFO:tasks.workunit.client.0.smithi063.stdout:5/741: dread d8/d19/f1a [0,4194304] 0 2024-05-09T06:38:20.817 INFO:tasks.workunit.client.0.smithi063.stdout:3/755: mkdir d7/da/de4/de8/dfd 0 2024-05-09T06:38:20.818 INFO:tasks.workunit.client.0.smithi063.stdout:8/837: chown d9/da/l3d 228067 1 2024-05-09T06:38:20.819 INFO:tasks.workunit.client.1.smithi193.stdout:9/446: dread dd/d1e/d66/d49/f64 [0,4194304] 0 2024-05-09T06:38:20.820 INFO:tasks.workunit.client.0.smithi063.stdout:8/838: write d9/da/d53/f117 [38779,91401] 0 2024-05-09T06:38:20.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/439: symlink d4/d14/l88 0 2024-05-09T06:38:20.822 INFO:tasks.workunit.client.1.smithi193.stdout:9/447: chown dd/d1e/d3c 16874 1 2024-05-09T06:38:20.823 INFO:tasks.workunit.client.1.smithi193.stdout:9/448: chown dd/d1e/d1f/d48/d55/d41 72435 1 2024-05-09T06:38:20.824 INFO:tasks.workunit.client.0.smithi063.stdout:1/695: getdents d9/d2a 0 2024-05-09T06:38:20.824 INFO:tasks.workunit.client.0.smithi063.stdout:5/742: chown d8/d53/fab 752005 1 2024-05-09T06:38:20.825 INFO:tasks.workunit.client.1.smithi193.stdout:1/440: mkdir d4/d16/d6b/d89 0 2024-05-09T06:38:20.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:20 smithi193 ceph-mon[35141]: pgmap v13: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 61 MiB/s rd, 193 MiB/s wr, 370 op/s 2024-05-09T06:38:20.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:20 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:20.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:20 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:20.827 INFO:tasks.workunit.client.1.smithi193.stdout:9/449: truncate dd/d1e/f28 1752783 0 2024-05-09T06:38:20.833 INFO:tasks.workunit.client.0.smithi063.stdout:2/788: creat dc/d36/d43/d46/ded/ff0 x:0 0 0 2024-05-09T06:38:20.835 INFO:tasks.workunit.client.0.smithi063.stdout:3/756: creat d7/d4c/ffe x:0 0 0 2024-05-09T06:38:20.835 INFO:tasks.workunit.client.1.smithi193.stdout:9/450: dwrite dd/d1e/d66/f80 [4194304,4194304] 0 2024-05-09T06:38:20.836 INFO:tasks.workunit.client.0.smithi063.stdout:6/761: write d2/d12/f34 [1955680,8683] 0 2024-05-09T06:38:20.838 INFO:tasks.workunit.client.0.smithi063.stdout:8/839: creat d9/da/d18/dc7/d58/f128 x:0 0 0 2024-05-09T06:38:20.839 INFO:tasks.workunit.client.0.smithi063.stdout:8/840: dread - d9/da/d18/d24/d71/fa6 zero size 2024-05-09T06:38:20.842 INFO:tasks.workunit.client.1.smithi193.stdout:9/451: rename dd/d1e/d66/d49/c90 to dd/df/d31/c9a 0 2024-05-09T06:38:20.843 INFO:tasks.workunit.client.0.smithi063.stdout:1/696: creat d9/d2a/d35/d1e/d9d/d3f/d59/d62/d7e/fea x:0 0 0 2024-05-09T06:38:20.845 INFO:tasks.workunit.client.0.smithi063.stdout:2/789: chown dc/dd/d45/d5a/c5b 520785925 1 2024-05-09T06:38:20.846 INFO:tasks.workunit.client.0.smithi063.stdout:2/790: truncate dc/d36/d43/d46/ded/ff0 845104 0 2024-05-09T06:38:20.849 INFO:tasks.workunit.client.0.smithi063.stdout:3/757: unlink d7/d70/d98/d2d/f93 0 2024-05-09T06:38:20.850 INFO:tasks.workunit.client.0.smithi063.stdout:0/781: write dd/d1a/f25 [730370,71079] 0 2024-05-09T06:38:20.850 INFO:tasks.workunit.client.0.smithi063.stdout:6/762: creat d2/df/d91/ddf/ff3 x:0 0 0 2024-05-09T06:38:20.853 INFO:tasks.workunit.client.0.smithi063.stdout:0/782: truncate dd/d1a/d2c/d51/d72/d9e/d23/d31/ff2 944794 0 2024-05-09T06:38:20.854 INFO:tasks.workunit.client.1.smithi193.stdout:1/441: write d4/d16/d60/f18 [1892055,125520] 0 2024-05-09T06:38:20.860 INFO:tasks.workunit.client.1.smithi193.stdout:9/452: dread dd/d1e/d66/f80 [0,4194304] 0 2024-05-09T06:38:20.864 INFO:tasks.workunit.client.1.smithi193.stdout:1/442: symlink d4/d16/l8a 0 2024-05-09T06:38:20.865 INFO:tasks.workunit.client.1.smithi193.stdout:1/443: chown d4/d3c/d3e/c5d 1 1 2024-05-09T06:38:20.865 INFO:tasks.workunit.client.0.smithi063.stdout:6/763: dread d2/d1f/f4c [0,4194304] 0 2024-05-09T06:38:20.865 INFO:tasks.workunit.client.0.smithi063.stdout:6/764: fdatasync d2/f38 0 2024-05-09T06:38:20.868 INFO:tasks.workunit.client.0.smithi063.stdout:0/783: chown dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/f4e 185122 1 2024-05-09T06:38:20.868 INFO:tasks.workunit.client.0.smithi063.stdout:5/743: write d8/d17/d18/f82 [152605,35554] 0 2024-05-09T06:38:20.870 INFO:tasks.workunit.client.1.smithi193.stdout:1/444: rename d4/d16/d6b/d6e/f70 to d4/d16/d60/d4f/d72/d7f/f8b 0 2024-05-09T06:38:20.871 INFO:tasks.workunit.client.0.smithi063.stdout:7/729: getdents d0/d15/d49/d54/d9f/d97 0 2024-05-09T06:38:20.871 INFO:tasks.workunit.client.1.smithi193.stdout:9/453: write dd/d1e/d66/d49/f51 [1661346,52146] 0 2024-05-09T06:38:20.872 INFO:tasks.workunit.client.1.smithi193.stdout:3/345: sync 2024-05-09T06:38:20.873 INFO:tasks.workunit.client.0.smithi063.stdout:1/697: symlink d9/d2a/d35/d24/leb 0 2024-05-09T06:38:20.874 INFO:tasks.workunit.client.0.smithi063.stdout:1/698: fsync d9/d10/f3e 0 2024-05-09T06:38:20.874 INFO:tasks.workunit.client.1.smithi193.stdout:3/346: truncate d1/d44/f4a 596485 0 2024-05-09T06:38:20.875 INFO:tasks.workunit.client.0.smithi063.stdout:2/791: creat dc/dd/d45/d5a/d6c/d8c/d9a/dd5/ff1 x:0 0 0 2024-05-09T06:38:20.875 INFO:tasks.workunit.client.0.smithi063.stdout:5/744: dwrite d8/d17/f3d [4194304,4194304] 0 2024-05-09T06:38:20.877 INFO:tasks.workunit.client.0.smithi063.stdout:8/841: creat d9/da/d53/f129 x:0 0 0 2024-05-09T06:38:20.878 INFO:tasks.workunit.client.1.smithi193.stdout:9/454: dwrite dd/d1e/f27 [0,4194304] 0 2024-05-09T06:38:20.882 INFO:tasks.workunit.client.0.smithi063.stdout:0/784: dread - dd/d1a/d2c/d51/d72/d9e/d23/d31/fb9 zero size 2024-05-09T06:38:20.883 INFO:tasks.workunit.client.0.smithi063.stdout:0/785: readlink dd/l2a 0 2024-05-09T06:38:20.884 INFO:tasks.workunit.client.1.smithi193.stdout:3/347: dwrite d1/db/d1b/f7a [0,4194304] 0 2024-05-09T06:38:20.885 INFO:tasks.workunit.client.1.smithi193.stdout:1/445: link d4/d16/de/f12 d4/f8c 0 2024-05-09T06:38:20.887 INFO:tasks.workunit.client.0.smithi063.stdout:7/730: mkdir d0/d15/d89/d98/dfb 0 2024-05-09T06:38:20.889 INFO:tasks.workunit.client.1.smithi193.stdout:9/455: dwrite dd/df/d31/d7e/f35 [0,4194304] 0 2024-05-09T06:38:20.896 INFO:tasks.workunit.client.1.smithi193.stdout:3/348: write d1/d44/f47 [4799118,6603] 0 2024-05-09T06:38:20.897 INFO:tasks.workunit.client.0.smithi063.stdout:4/747: sync 2024-05-09T06:38:20.898 INFO:tasks.workunit.client.1.smithi193.stdout:1/446: dwrite d4/f37 [0,4194304] 0 2024-05-09T06:38:20.899 INFO:tasks.workunit.client.0.smithi063.stdout:4/748: fdatasync d7/da3/d28/d31/d71/dbb/fc0 0 2024-05-09T06:38:20.904 INFO:tasks.workunit.client.1.smithi193.stdout:3/349: creat d1/d2d/d51/f7e x:0 0 0 2024-05-09T06:38:20.905 INFO:tasks.workunit.client.1.smithi193.stdout:1/447: symlink d4/d16/d60/d4f/d72/l8d 0 2024-05-09T06:38:20.906 INFO:tasks.workunit.client.1.smithi193.stdout:1/448: dread - d4/d14/f75 zero size 2024-05-09T06:38:20.907 INFO:tasks.workunit.client.1.smithi193.stdout:1/449: fsync d4/d16/d6b/d85/f86 0 2024-05-09T06:38:20.908 INFO:tasks.workunit.client.1.smithi193.stdout:1/450: readlink d4/d16/d17/l3d 0 2024-05-09T06:38:20.908 INFO:tasks.workunit.client.1.smithi193.stdout:9/456: mknod dd/c9b 0 2024-05-09T06:38:20.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/457: chown dd/df/d31/d7e/d37 149360415 1 2024-05-09T06:38:20.909 INFO:tasks.workunit.client.0.smithi063.stdout:6/765: dwrite d2/df/f50 [0,4194304] 0 2024-05-09T06:38:20.912 INFO:tasks.workunit.client.0.smithi063.stdout:6/766: readlink d2/d1f/d29/d3f/l16 0 2024-05-09T06:38:20.912 INFO:tasks.workunit.client.0.smithi063.stdout:1/699: mknod d9/d2a/d35/d24/cec 0 2024-05-09T06:38:20.914 INFO:tasks.workunit.client.1.smithi193.stdout:9/458: rmdir dd/df/d31/d7e/d37 39 2024-05-09T06:38:20.917 INFO:tasks.workunit.client.1.smithi193.stdout:1/451: getdents d4/d16/d6b/d89 0 2024-05-09T06:38:20.918 INFO:tasks.workunit.client.0.smithi063.stdout:2/792: creat dc/d32/de5/ff2 x:0 0 0 2024-05-09T06:38:20.919 INFO:tasks.workunit.client.0.smithi063.stdout:2/793: readlink dc/d36/d43/d46/d63/d79/d7d/l91 0 2024-05-09T06:38:20.919 INFO:tasks.workunit.client.0.smithi063.stdout:8/842: creat d9/da/d53/db2/d11a/f12a x:0 0 0 2024-05-09T06:38:20.919 INFO:tasks.workunit.client.1.smithi193.stdout:3/350: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:20.921 INFO:tasks.workunit.client.1.smithi193.stdout:9/459: creat dd/d1e/d1f/d43/d5b/f9c x:0 0 0 2024-05-09T06:38:20.923 INFO:tasks.workunit.client.0.smithi063.stdout:6/767: symlink d2/d4/d7c/d7e/lf4 0 2024-05-09T06:38:20.923 INFO:tasks.workunit.client.0.smithi063.stdout:3/758: sync 2024-05-09T06:38:20.923 INFO:tasks.workunit.client.0.smithi063.stdout:9/757: sync 2024-05-09T06:38:20.924 INFO:tasks.workunit.client.0.smithi063.stdout:9/758: fdatasync d0/d4/f6a 0 2024-05-09T06:38:20.925 INFO:tasks.workunit.client.0.smithi063.stdout:1/700: readlink d9/d2a/d35/d1e/d9d/d3f/d59/d62/d7e/lc1 0 2024-05-09T06:38:20.927 INFO:tasks.workunit.client.1.smithi193.stdout:3/351: symlink d1/db/d59/d63/l7f 0 2024-05-09T06:38:20.928 INFO:tasks.workunit.client.0.smithi063.stdout:2/794: dwrite dc/d6e/d7f/f97 [0,4194304] 0 2024-05-09T06:38:20.928 INFO:tasks.workunit.client.0.smithi063.stdout:1/701: fdatasync d9/d2a/d39/d5d/d75/fcf 0 2024-05-09T06:38:20.930 INFO:tasks.workunit.client.1.smithi193.stdout:1/452: mknod d4/d16/d6b/d89/c8e 0 2024-05-09T06:38:20.930 INFO:tasks.workunit.client.1.smithi193.stdout:1/453: chown d4/c7 1035739506 1 2024-05-09T06:38:20.932 INFO:tasks.workunit.client.0.smithi063.stdout:4/749: symlink d7/da3/d28/d31/dbf/lf0 0 2024-05-09T06:38:20.937 INFO:tasks.workunit.client.1.smithi193.stdout:1/454: mknod d4/d16/d6b/d6e/c8f 0 2024-05-09T06:38:20.940 INFO:tasks.workunit.client.1.smithi193.stdout:8/381: sync 2024-05-09T06:38:20.940 INFO:tasks.workunit.client.1.smithi193.stdout:7/400: sync 2024-05-09T06:38:20.940 INFO:tasks.workunit.client.1.smithi193.stdout:0/366: sync 2024-05-09T06:38:20.941 INFO:tasks.workunit.client.1.smithi193.stdout:9/460: mknod dd/d1e/d1f/d43/d5b/d8a/c9d 0 2024-05-09T06:38:20.942 INFO:tasks.workunit.client.0.smithi063.stdout:0/786: dwrite dd/fb2 [0,4194304] 0 2024-05-09T06:38:20.942 INFO:tasks.workunit.client.1.smithi193.stdout:2/425: sync 2024-05-09T06:38:20.942 INFO:tasks.workunit.client.1.smithi193.stdout:5/371: sync 2024-05-09T06:38:20.942 INFO:tasks.workunit.client.1.smithi193.stdout:6/406: sync 2024-05-09T06:38:20.942 INFO:tasks.workunit.client.1.smithi193.stdout:4/466: sync 2024-05-09T06:38:20.945 INFO:tasks.workunit.client.0.smithi063.stdout:3/759: mknod d7/d70/d98/d5b/cff 0 2024-05-09T06:38:20.946 INFO:tasks.workunit.client.0.smithi063.stdout:7/731: dwrite d0/d15/d49/d4f/d66/d67/d6b/f6e [0,4194304] 0 2024-05-09T06:38:20.946 INFO:tasks.workunit.client.1.smithi193.stdout:8/382: truncate d6/d1c/d18/d66/f62 4238520 0 2024-05-09T06:38:20.946 INFO:tasks.workunit.client.1.smithi193.stdout:1/455: mknod d4/d16/de/c90 0 2024-05-09T06:38:20.947 INFO:tasks.workunit.client.0.smithi063.stdout:6/768: mkdir d2/d20/d26/d7f/dcf/df5 0 2024-05-09T06:38:20.950 INFO:tasks.workunit.client.1.smithi193.stdout:7/401: write d6/f2e [4778450,118475] 0 2024-05-09T06:38:20.951 INFO:tasks.workunit.client.0.smithi063.stdout:2/795: mknod dc/d6e/cf3 0 2024-05-09T06:38:20.953 INFO:tasks.workunit.client.0.smithi063.stdout:2/796: chown dc/d32/d33/f61 33996019 1 2024-05-09T06:38:20.954 INFO:tasks.workunit.client.0.smithi063.stdout:1/702: mknod d9/d2a/d3c/d4c/d83/d88/dc2/ced 0 2024-05-09T06:38:20.961 INFO:tasks.workunit.client.1.smithi193.stdout:9/461: creat dd/df/d15/f9e x:0 0 0 2024-05-09T06:38:20.963 INFO:tasks.workunit.client.1.smithi193.stdout:9/462: truncate dd/df/d15/f96 55391 0 2024-05-09T06:38:20.963 INFO:tasks.workunit.client.0.smithi063.stdout:1/703: dwrite d9/d2a/d35/f44 [0,4194304] 0 2024-05-09T06:38:20.965 INFO:tasks.workunit.client.1.smithi193.stdout:3/352: link d1/d44/l61 d1/d2d/d51/l80 0 2024-05-09T06:38:20.965 INFO:tasks.workunit.client.0.smithi063.stdout:8/843: rename d9/da/d18/dc7/d8f/f90 to d9/da/d53/db2/f12b 0 2024-05-09T06:38:20.966 INFO:tasks.workunit.client.0.smithi063.stdout:2/797: dread dc/dd/f64 [0,4194304] 0 2024-05-09T06:38:20.967 INFO:tasks.workunit.client.1.smithi193.stdout:4/467: creat d1a/d1f/d4b/f8f x:0 0 0 2024-05-09T06:38:20.967 INFO:tasks.workunit.client.1.smithi193.stdout:4/468: stat d1a/d1f/l2b 0 2024-05-09T06:38:20.969 INFO:tasks.workunit.client.1.smithi193.stdout:1/456: readlink d4/d16/d35/l46 0 2024-05-09T06:38:20.969 INFO:tasks.workunit.client.0.smithi063.stdout:9/759: mkdir d0/d109 0 2024-05-09T06:38:20.970 INFO:tasks.workunit.client.0.smithi063.stdout:8/844: write d9/da/d53/dbc/d102/f110 [2114389,100581] 0 2024-05-09T06:38:20.971 INFO:tasks.workunit.client.0.smithi063.stdout:9/760: chown d0/f98 5210766 1 2024-05-09T06:38:20.971 INFO:tasks.workunit.client.0.smithi063.stdout:0/787: dread dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fb6 [0,4194304] 0 2024-05-09T06:38:20.976 INFO:tasks.workunit.client.0.smithi063.stdout:3/760: unlink d7/da/d21/d30/d44/d48/fe7 0 2024-05-09T06:38:20.981 INFO:tasks.workunit.client.1.smithi193.stdout:5/372: write d0/f7 [3897715,18594] 0 2024-05-09T06:38:20.982 INFO:tasks.workunit.client.1.smithi193.stdout:0/367: creat d18/d4c/d47/d59/d76/f77 x:0 0 0 2024-05-09T06:38:20.982 INFO:tasks.workunit.client.1.smithi193.stdout:0/368: dread - d18/f60 zero size 2024-05-09T06:38:20.984 INFO:tasks.workunit.client.1.smithi193.stdout:7/402: fsync d6/d8/d1a/f3c 0 2024-05-09T06:38:20.990 INFO:tasks.workunit.client.0.smithi063.stdout:5/745: getdents d8/dc8/dda 0 2024-05-09T06:38:20.990 INFO:tasks.workunit.client.0.smithi063.stdout:4/750: symlink d7/df/d35/d5a/dad/de9/lf1 0 2024-05-09T06:38:20.992 INFO:tasks.workunit.client.0.smithi063.stdout:1/704: mknod d9/d2a/d3c/cee 0 2024-05-09T06:38:20.993 INFO:tasks.workunit.client.1.smithi193.stdout:8/383: dwrite d6/d1c/f12 [0,4194304] 0 2024-05-09T06:38:20.993 INFO:tasks.workunit.client.0.smithi063.stdout:7/732: dread d0/d15/d49/d4f/d66/d67/d86/f90 [0,4194304] 0 2024-05-09T06:38:20.994 INFO:tasks.workunit.client.1.smithi193.stdout:3/353: rmdir d1/d2d/d51 39 2024-05-09T06:38:20.999 INFO:tasks.workunit.client.1.smithi193.stdout:1/457: fsync d4/d3c/d3e/f77 0 2024-05-09T06:38:21.004 INFO:tasks.workunit.client.1.smithi193.stdout:7/403: mknod d6/d8/d1a/d36/d4d/d60/c84 0 2024-05-09T06:38:21.005 INFO:tasks.workunit.client.1.smithi193.stdout:6/407: creat d1/d25/d2c/f91 x:0 0 0 2024-05-09T06:38:21.006 INFO:tasks.workunit.client.1.smithi193.stdout:6/408: dread - d1/de/d11/d18/f6d zero size 2024-05-09T06:38:21.007 INFO:tasks.workunit.client.0.smithi063.stdout:3/761: creat d7/da/d21/f100 x:0 0 0 2024-05-09T06:38:21.009 INFO:tasks.workunit.client.0.smithi063.stdout:3/762: stat d7/da/d2b/d66/d74/d7e/f45 0 2024-05-09T06:38:21.010 INFO:tasks.workunit.client.1.smithi193.stdout:9/463: unlink dd/d1e/c24 0 2024-05-09T06:38:21.011 INFO:tasks.workunit.client.0.smithi063.stdout:6/769: creat d2/d20/d95/dec/ff6 x:0 0 0 2024-05-09T06:38:21.013 INFO:tasks.workunit.client.1.smithi193.stdout:2/426: creat db/df/f94 x:0 0 0 2024-05-09T06:38:21.014 INFO:tasks.workunit.client.0.smithi063.stdout:1/705: stat d9/l19 0 2024-05-09T06:38:21.014 INFO:tasks.workunit.client.1.smithi193.stdout:2/427: stat db/d25/d38/d40/l52 0 2024-05-09T06:38:21.018 INFO:tasks.workunit.client.0.smithi063.stdout:0/788: write dd/d1a/d2c/d51/d72/d9e/d23/d89/fc4 [4855280,8302] 0 2024-05-09T06:38:21.020 INFO:tasks.workunit.client.0.smithi063.stdout:0/789: chown dd/d1a/d2c/d51/d76/d97/ce2 173701093 1 2024-05-09T06:38:21.020 INFO:tasks.workunit.client.1.smithi193.stdout:3/354: creat d1/db/d59/f81 x:0 0 0 2024-05-09T06:38:21.020 INFO:tasks.workunit.client.0.smithi063.stdout:7/733: fsync d0/d15/d49/d4f/d66/d67/d86/fca 0 2024-05-09T06:38:21.021 INFO:tasks.workunit.client.0.smithi063.stdout:0/790: readlink dd/d1a/d2c/d51/d72/d9e/d23/l45 0 2024-05-09T06:38:21.022 INFO:tasks.workunit.client.0.smithi063.stdout:0/791: chown dd/fb2 4963 1 2024-05-09T06:38:21.023 INFO:tasks.workunit.client.1.smithi193.stdout:6/409: dread d1/f2 [4194304,4194304] 0 2024-05-09T06:38:21.025 INFO:tasks.workunit.client.1.smithi193.stdout:1/458: creat d4/d16/d60/f91 x:0 0 0 2024-05-09T06:38:21.027 INFO:tasks.workunit.client.0.smithi063.stdout:8/845: symlink d9/l12c 0 2024-05-09T06:38:21.027 INFO:tasks.workunit.client.1.smithi193.stdout:1/459: chown d4/d3c/f36 110 1 2024-05-09T06:38:21.027 INFO:tasks.workunit.client.1.smithi193.stdout:1/460: readlink d4/d16/de/l56 0 2024-05-09T06:38:21.029 INFO:tasks.workunit.client.1.smithi193.stdout:5/373: link d0/d13/d2c/d37/d5b/d4e/f6b d0/d13/d2c/d37/d5b/d4e/f78 0 2024-05-09T06:38:21.029 INFO:tasks.workunit.client.0.smithi063.stdout:9/761: mknod d0/d4/d7d/d105/c10a 0 2024-05-09T06:38:21.033 INFO:tasks.workunit.client.1.smithi193.stdout:7/404: mknod d6/d8/d29/d2f/c85 0 2024-05-09T06:38:21.034 INFO:tasks.workunit.client.0.smithi063.stdout:3/763: symlink d7/d22/d9c/l101 0 2024-05-09T06:38:21.035 INFO:tasks.workunit.client.0.smithi063.stdout:3/764: write d7/da/d21/d30/d44/d48/f53 [2217672,74364] 0 2024-05-09T06:38:21.035 INFO:tasks.workunit.client.1.smithi193.stdout:9/464: truncate dd/df/f1d 1135216 0 2024-05-09T06:38:21.035 INFO:tasks.workunit.client.1.smithi193.stdout:9/465: dread - dd/d1e/d1f/d43/d5b/f9c zero size 2024-05-09T06:38:21.038 INFO:tasks.workunit.client.0.smithi063.stdout:6/770: dwrite d2/d20/d26/fde [0,4194304] 0 2024-05-09T06:38:21.041 INFO:tasks.workunit.client.1.smithi193.stdout:3/355: rmdir d1/db/d2f 39 2024-05-09T06:38:21.043 INFO:tasks.workunit.client.1.smithi193.stdout:4/469: link d1a/d22/d5d/l69 d1a/d1f/l90 0 2024-05-09T06:38:21.045 INFO:tasks.workunit.client.0.smithi063.stdout:0/792: creat dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/ff8 x:0 0 0 2024-05-09T06:38:21.047 INFO:tasks.workunit.client.0.smithi063.stdout:9/762: creat d0/d4/d57/d60/df5/f10b x:0 0 0 2024-05-09T06:38:21.050 INFO:tasks.workunit.client.0.smithi063.stdout:1/706: read d9/d2a/d35/d24/d68/f98 [574311,119962] 0 2024-05-09T06:38:21.052 INFO:tasks.workunit.client.1.smithi193.stdout:3/356: dwrite d1/f1d [0,4194304] 0 2024-05-09T06:38:21.058 INFO:tasks.workunit.client.0.smithi063.stdout:7/734: creat d0/d15/d49/d4f/d66/df1/ffc x:0 0 0 2024-05-09T06:38:21.058 INFO:tasks.workunit.client.0.smithi063.stdout:2/798: link dc/d36/d43/d46/fcc dc/d36/ff4 0 2024-05-09T06:38:21.059 INFO:tasks.workunit.client.0.smithi063.stdout:0/793: truncate dd/fbe 401394 0 2024-05-09T06:38:21.059 INFO:tasks.workunit.client.1.smithi193.stdout:6/410: dwrite d1/d25/d2c/d81/d37/f5b [0,4194304] 0 2024-05-09T06:38:21.065 INFO:tasks.workunit.client.0.smithi063.stdout:1/707: creat d9/d2a/d3c/d4c/d83/d88/dc2/fef x:0 0 0 2024-05-09T06:38:21.066 INFO:tasks.workunit.client.0.smithi063.stdout:1/708: fdatasync d9/d10/f22 0 2024-05-09T06:38:21.066 INFO:tasks.workunit.client.0.smithi063.stdout:0/794: dwrite dd/d1a/d2c/f92 [0,4194304] 0 2024-05-09T06:38:21.066 INFO:tasks.workunit.client.1.smithi193.stdout:0/369: rmdir d18/d65/d6f 0 2024-05-09T06:38:21.069 INFO:tasks.workunit.client.0.smithi063.stdout:5/746: link d8/d19/d30/d74/d77/c79 d8/d19/d30/d46/cf9 0 2024-05-09T06:38:21.071 INFO:tasks.workunit.client.0.smithi063.stdout:1/709: dread d9/d2a/d35/f29 [0,4194304] 0 2024-05-09T06:38:21.074 INFO:tasks.workunit.client.1.smithi193.stdout:2/428: creat db/d25/f95 x:0 0 0 2024-05-09T06:38:21.076 INFO:tasks.workunit.client.1.smithi193.stdout:2/429: readlink db/d25/l3a 0 2024-05-09T06:38:21.078 INFO:tasks.workunit.client.1.smithi193.stdout:8/384: getdents d6/d1c/d53/d56 0 2024-05-09T06:38:21.084 INFO:tasks.workunit.client.1.smithi193.stdout:2/430: dwrite db/d25/d38/d40/f72 [8388608,4194304] 0 2024-05-09T06:38:21.084 INFO:tasks.workunit.client.0.smithi063.stdout:9/763: link d0/d4/d26/d47/fa8 d0/d4/d26/d39/d48/d80/d84/f10c 0 2024-05-09T06:38:21.086 INFO:tasks.workunit.client.1.smithi193.stdout:6/411: unlink d1/d25/d2c/d3c/f54 0 2024-05-09T06:38:21.087 INFO:tasks.workunit.client.0.smithi063.stdout:9/764: dread - d0/d4/d26/d39/d48/d80/d84/dd8/ddf/f103 zero size 2024-05-09T06:38:21.091 INFO:tasks.workunit.client.0.smithi063.stdout:3/765: write d7/da/d2b/d66/d74/d7e/fca [668944,4439] 0 2024-05-09T06:38:21.093 INFO:tasks.workunit.client.0.smithi063.stdout:0/795: fsync dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fc3 0 2024-05-09T06:38:21.095 INFO:tasks.workunit.client.1.smithi193.stdout:7/405: link d6/dd/l4a d6/d17/d7d/l86 0 2024-05-09T06:38:21.095 INFO:tasks.workunit.client.0.smithi063.stdout:0/796: truncate dd/d60/d44/d90/faa 4633452 0 2024-05-09T06:38:21.095 INFO:tasks.workunit.client.0.smithi063.stdout:6/771: write d2/d1f/d29/d3f/d5c/f9d [356580,38796] 0 2024-05-09T06:38:21.096 INFO:tasks.workunit.client.0.smithi063.stdout:5/747: mkdir d8/d17/dfa 0 2024-05-09T06:38:21.099 INFO:tasks.workunit.client.0.smithi063.stdout:7/735: getdents d0/d15/d89/df3 0 2024-05-09T06:38:21.100 INFO:tasks.workunit.client.0.smithi063.stdout:7/736: write d0/d15/f48 [567868,38357] 0 2024-05-09T06:38:21.102 INFO:tasks.workunit.client.1.smithi193.stdout:8/385: dread d6/d1c/d18/d2f/f39 [0,4194304] 0 2024-05-09T06:38:21.104 INFO:tasks.workunit.client.0.smithi063.stdout:8/846: getdents d9/d37 0 2024-05-09T06:38:21.107 INFO:tasks.workunit.client.0.smithi063.stdout:9/765: mknod d0/d3b/d63/c10d 0 2024-05-09T06:38:21.111 INFO:tasks.workunit.client.1.smithi193.stdout:1/461: getdents d4/d16 0 2024-05-09T06:38:21.113 INFO:tasks.workunit.client.1.smithi193.stdout:4/470: dwrite d1a/f1b [0,4194304] 0 2024-05-09T06:38:21.115 INFO:tasks.workunit.client.0.smithi063.stdout:5/748: fsync d8/d53/dca/f8c 0 2024-05-09T06:38:21.118 INFO:tasks.workunit.client.0.smithi063.stdout:0/797: creat dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/ff9 x:0 0 0 2024-05-09T06:38:21.118 INFO:tasks.workunit.client.1.smithi193.stdout:0/370: write d18/d1d/f1f [928172,52253] 0 2024-05-09T06:38:21.119 INFO:tasks.workunit.client.0.smithi063.stdout:0/798: chown dd/d1a/l3d 11156 1 2024-05-09T06:38:21.120 INFO:tasks.workunit.client.1.smithi193.stdout:7/406: creat d6/d8/d1a/d33/f87 x:0 0 0 2024-05-09T06:38:21.122 INFO:tasks.workunit.client.1.smithi193.stdout:3/357: mknod d1/db/d2f/d78/c82 0 2024-05-09T06:38:21.123 INFO:tasks.workunit.client.1.smithi193.stdout:3/358: chown d1/db/f6e 81666469 1 2024-05-09T06:38:21.124 INFO:tasks.workunit.client.1.smithi193.stdout:9/466: getdents dd/d1e/d66 0 2024-05-09T06:38:21.124 INFO:tasks.workunit.client.1.smithi193.stdout:9/467: stat dd/df/d15/f9e 0 2024-05-09T06:38:21.126 INFO:tasks.workunit.client.1.smithi193.stdout:9/468: truncate dd/df/d15/f8e 848653 0 2024-05-09T06:38:21.127 INFO:tasks.workunit.client.0.smithi063.stdout:7/737: creat d0/d15/d49/d4f/d66/d67/ffd x:0 0 0 2024-05-09T06:38:21.128 INFO:tasks.workunit.client.1.smithi193.stdout:9/469: read dd/df/d31/d7e/f35 [2477781,37367] 0 2024-05-09T06:38:21.131 INFO:tasks.workunit.client.0.smithi063.stdout:9/766: fsync d0/d4/d26/d39/d3e/fc6 0 2024-05-09T06:38:21.132 INFO:tasks.workunit.client.1.smithi193.stdout:1/462: unlink d4/d14/f6c 0 2024-05-09T06:38:21.133 INFO:tasks.workunit.client.0.smithi063.stdout:7/738: dwrite d0/d15/d49/f65 [0,4194304] 0 2024-05-09T06:38:21.137 INFO:tasks.workunit.client.1.smithi193.stdout:7/407: creat d6/dd/d5e/d6b/f88 x:0 0 0 2024-05-09T06:38:21.142 INFO:tasks.workunit.client.0.smithi063.stdout:2/799: getdents dc/dd/d45/d5a/d6c/d8f/dbf 0 2024-05-09T06:38:21.144 INFO:tasks.workunit.client.0.smithi063.stdout:7/739: dwrite d0/d15/d49/d54/f60 [0,4194304] 0 2024-05-09T06:38:21.145 INFO:tasks.workunit.client.1.smithi193.stdout:9/470: unlink dd/d1e/d66/d49/f7f 0 2024-05-09T06:38:21.145 INFO:tasks.workunit.client.1.smithi193.stdout:0/371: write d18/d4c/f38 [534140,93568] 0 2024-05-09T06:38:21.151 INFO:tasks.workunit.client.0.smithi063.stdout:4/751: sync 2024-05-09T06:38:21.151 INFO:tasks.workunit.client.0.smithi063.stdout:1/710: sync 2024-05-09T06:38:21.151 INFO:tasks.workunit.client.1.smithi193.stdout:6/412: link d1/de/d11/d2d/d3a/c8c d1/d25/d2c/d3c/d3e/d58/c92 0 2024-05-09T06:38:21.152 INFO:tasks.workunit.client.1.smithi193.stdout:6/413: chown d1/de/d11/d18/l33 0 1 2024-05-09T06:38:21.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/471: creat d1a/d6b/d60/f91 x:0 0 0 2024-05-09T06:38:21.155 INFO:tasks.workunit.client.0.smithi063.stdout:6/772: link d2/df/d59/d99/d9f/fba d2/df/d91/dd8/dd9/ff7 0 2024-05-09T06:38:21.158 INFO:tasks.workunit.client.0.smithi063.stdout:0/799: symlink dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/lfa 0 2024-05-09T06:38:21.158 INFO:tasks.workunit.client.0.smithi063.stdout:5/749: truncate d8/d53/dca/f8c 697572 0 2024-05-09T06:38:21.160 INFO:tasks.workunit.client.0.smithi063.stdout:8/847: rename d9/d3b/c75 to d9/da/c12d 0 2024-05-09T06:38:21.161 INFO:tasks.workunit.client.1.smithi193.stdout:1/463: dwrite d4/d16/f10 [0,4194304] 0 2024-05-09T06:38:21.162 INFO:tasks.workunit.client.1.smithi193.stdout:1/464: write d4/d16/d17/f29 [1424687,72208] 0 2024-05-09T06:38:21.165 INFO:tasks.workunit.client.0.smithi063.stdout:3/766: getdents d7/da/d21/d30 0 2024-05-09T06:38:21.166 INFO:tasks.workunit.client.1.smithi193.stdout:8/386: link d6/d1c/d18/d2f/c5c d6/d1c/d1b/d29/c73 0 2024-05-09T06:38:21.171 INFO:tasks.workunit.client.0.smithi063.stdout:9/767: mknod d0/d4/d7d/d94/c10e 0 2024-05-09T06:38:21.173 INFO:tasks.workunit.client.1.smithi193.stdout:2/431: getdents db 0 2024-05-09T06:38:21.174 INFO:tasks.workunit.client.1.smithi193.stdout:2/432: write db/f44 [1404170,71997] 0 2024-05-09T06:38:21.176 INFO:tasks.workunit.client.1.smithi193.stdout:9/471: rename dd/d1e/d1f/f21 to dd/d1e/d1f/d43/d5b/f9f 0 2024-05-09T06:38:21.176 INFO:tasks.workunit.client.0.smithi063.stdout:2/800: dwrite dc/dd/f2f [4194304,4194304] 0 2024-05-09T06:38:21.177 INFO:tasks.workunit.client.1.smithi193.stdout:9/472: chown dd/d1e/d1f/d48/l87 1 1 2024-05-09T06:38:21.177 INFO:tasks.workunit.client.0.smithi063.stdout:1/711: creat d9/d2a/d3c/d4a/ff0 x:0 0 0 2024-05-09T06:38:21.179 INFO:tasks.workunit.client.0.smithi063.stdout:6/773: creat d2/df/d59/d99/ff8 x:0 0 0 2024-05-09T06:38:21.180 INFO:tasks.workunit.client.1.smithi193.stdout:4/472: write d1a/d22/f7a [205588,107011] 0 2024-05-09T06:38:21.182 INFO:tasks.workunit.client.0.smithi063.stdout:8/848: symlink d9/da/d45/l12e 0 2024-05-09T06:38:21.185 INFO:tasks.workunit.client.0.smithi063.stdout:4/752: dread d7/df/d77/f8e [0,4194304] 0 2024-05-09T06:38:21.185 INFO:tasks.workunit.client.0.smithi063.stdout:3/767: creat d7/da/d2b/d66/d74/d7e/d97/f102 x:0 0 0 2024-05-09T06:38:21.187 INFO:tasks.workunit.client.1.smithi193.stdout:3/359: link d1/d2d/d51/f7e d1/d44/d7d/f83 0 2024-05-09T06:38:21.189 INFO:tasks.workunit.client.0.smithi063.stdout:2/801: dwrite dc/d36/d43/d46/d63/d9e/feb [0,4194304] 0 2024-05-09T06:38:21.189 INFO:tasks.workunit.client.0.smithi063.stdout:0/800: dread dd/d1a/d2c/d51/d72/d9e/f64 [0,4194304] 0 2024-05-09T06:38:21.191 INFO:tasks.workunit.client.0.smithi063.stdout:7/740: link d0/d15/d4a/d50/f37 d0/d15/d49/d54/d9f/d97/d9b/d9d/ffe 0 2024-05-09T06:38:21.191 INFO:tasks.workunit.client.0.smithi063.stdout:7/741: stat d0/d15/cd0 0 2024-05-09T06:38:21.192 INFO:tasks.workunit.client.1.smithi193.stdout:1/465: rename d4/d16/d17/d1a to d4/d16/d17/d1a/d3a/d4a/d92 22 2024-05-09T06:38:21.192 INFO:tasks.workunit.client.0.smithi063.stdout:7/742: fsync d0/d15/fcc 0 2024-05-09T06:38:21.193 INFO:tasks.workunit.client.1.smithi193.stdout:7/408: truncate d6/d8/f35 3261905 0 2024-05-09T06:38:21.194 INFO:tasks.workunit.client.1.smithi193.stdout:1/466: dread d4/d14/f1b [0,4194304] 0 2024-05-09T06:38:21.195 INFO:tasks.workunit.client.1.smithi193.stdout:0/372: chown d18/d27/f41 0 1 2024-05-09T06:38:21.196 INFO:tasks.workunit.client.1.smithi193.stdout:0/373: truncate d18/f57 1874837 0 2024-05-09T06:38:21.196 INFO:tasks.workunit.client.0.smithi063.stdout:7/743: write d0/d15/d49/d4f/d66/d67/ffd [603235,124152] 0 2024-05-09T06:38:21.198 INFO:tasks.workunit.client.0.smithi063.stdout:5/750: write d8/d35/f64 [2889009,14535] 0 2024-05-09T06:38:21.199 INFO:tasks.workunit.client.0.smithi063.stdout:5/751: readlink d8/d19/d84/lcb 0 2024-05-09T06:38:21.199 INFO:tasks.workunit.client.0.smithi063.stdout:8/849: dread - d9/d41/dec/da1/ffd zero size 2024-05-09T06:38:21.203 INFO:tasks.workunit.client.1.smithi193.stdout:4/473: symlink d1a/d1f/l92 0 2024-05-09T06:38:21.205 INFO:tasks.workunit.client.0.smithi063.stdout:3/768: rename d7/d70/d98/d5b/fbe to d7/da/d2b/d66/d74/d7e/f103 0 2024-05-09T06:38:21.207 INFO:tasks.workunit.client.1.smithi193.stdout:3/360: mkdir d1/db/d2f/d84 0 2024-05-09T06:38:21.210 INFO:tasks.workunit.client.0.smithi063.stdout:0/801: creat dd/d1a/d50/ffb x:0 0 0 2024-05-09T06:38:21.213 INFO:tasks.workunit.client.0.smithi063.stdout:1/712: creat d9/d2a/d3c/d4a/db8/ddf/ff1 x:0 0 0 2024-05-09T06:38:21.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/744: creat d0/d15/d49/d4f/d66/d67/d86/fff x:0 0 0 2024-05-09T06:38:21.215 INFO:tasks.workunit.client.0.smithi063.stdout:2/802: dread dc/dd/f2f [0,4194304] 0 2024-05-09T06:38:21.216 INFO:tasks.workunit.client.0.smithi063.stdout:6/774: mknod d2/d4/d7c/d42/def/cf9 0 2024-05-09T06:38:21.217 INFO:tasks.workunit.client.1.smithi193.stdout:0/374: mknod d18/d1d/c78 0 2024-05-09T06:38:21.217 INFO:tasks.workunit.client.1.smithi193.stdout:0/375: stat d18/d1d/c78 0 2024-05-09T06:38:21.217 INFO:tasks.workunit.client.0.smithi063.stdout:8/850: rmdir d9 39 2024-05-09T06:38:21.218 INFO:tasks.workunit.client.0.smithi063.stdout:5/752: creat d8/d17/d75/dc9/dd5/d96/ffb x:0 0 0 2024-05-09T06:38:21.218 INFO:tasks.workunit.client.1.smithi193.stdout:9/473: rmdir dd/df/d31/d7e/d37 39 2024-05-09T06:38:21.218 INFO:tasks.workunit.client.1.smithi193.stdout:9/474: chown dd/df/c75 39 1 2024-05-09T06:38:21.219 INFO:tasks.workunit.client.0.smithi063.stdout:4/753: mkdir d7/da3/d28/d31/d7b/df2 0 2024-05-09T06:38:21.219 INFO:tasks.workunit.client.1.smithi193.stdout:9/475: fdatasync dd/d1e/f88 0 2024-05-09T06:38:21.221 INFO:tasks.workunit.client.0.smithi063.stdout:4/754: read d7/df/f2b [1210062,90875] 0 2024-05-09T06:38:21.221 INFO:tasks.workunit.client.0.smithi063.stdout:3/769: creat d7/d70/f104 x:0 0 0 2024-05-09T06:38:21.223 INFO:tasks.workunit.client.1.smithi193.stdout:3/361: symlink d1/d44/l85 0 2024-05-09T06:38:21.226 INFO:tasks.workunit.client.1.smithi193.stdout:3/362: dread d1/f6d [0,4194304] 0 2024-05-09T06:38:21.227 INFO:tasks.workunit.client.0.smithi063.stdout:1/713: stat d9/d2a/d35/d24/f53 0 2024-05-09T06:38:21.231 INFO:tasks.workunit.client.1.smithi193.stdout:7/409: rename d6/d17/f23 to d6/dd/f89 0 2024-05-09T06:38:21.232 INFO:tasks.workunit.client.0.smithi063.stdout:6/775: creat d2/d20/d26/d7f/dcf/ffa x:0 0 0 2024-05-09T06:38:21.233 INFO:tasks.workunit.client.0.smithi063.stdout:0/802: write dd/d1a/d2c/d51/d72/d9e/d23/d31/f4a [2160582,39987] 0 2024-05-09T06:38:21.234 INFO:tasks.workunit.client.1.smithi193.stdout:5/374: sync 2024-05-09T06:38:21.236 INFO:tasks.workunit.client.1.smithi193.stdout:2/433: creat db/d25/f96 x:0 0 0 2024-05-09T06:38:21.238 INFO:tasks.workunit.client.1.smithi193.stdout:0/376: unlink d18/d4c/f46 0 2024-05-09T06:38:21.238 INFO:tasks.workunit.client.1.smithi193.stdout:0/377: chown d18/d4c/d47/d5e/d70 105981 1 2024-05-09T06:38:21.239 INFO:tasks.workunit.client.0.smithi063.stdout:3/770: truncate d7/d70/d98/d2d/f5d 4609661 0 2024-05-09T06:38:21.241 INFO:tasks.workunit.client.1.smithi193.stdout:1/467: truncate d4/f37 22244 0 2024-05-09T06:38:21.242 INFO:tasks.workunit.client.1.smithi193.stdout:6/414: getdents d1/d25/d2c/d3c/d3e/d58 0 2024-05-09T06:38:21.245 INFO:tasks.workunit.client.1.smithi193.stdout:9/476: creat dd/df/d31/d7e/d71/fa0 x:0 0 0 2024-05-09T06:38:21.246 INFO:tasks.workunit.client.0.smithi063.stdout:1/714: symlink d9/d2a/d35/d1e/d9d/d3f/d59/d62/d7e/lf2 0 2024-05-09T06:38:21.248 INFO:tasks.workunit.client.1.smithi193.stdout:4/474: link d1a/d6b/f3c d1a/d22/d5d/f93 0 2024-05-09T06:38:21.248 INFO:tasks.workunit.client.1.smithi193.stdout:1/468: dwrite d4/d16/d17/f29 [0,4194304] 0 2024-05-09T06:38:21.249 INFO:tasks.workunit.client.0.smithi063.stdout:4/755: dread d7/df/f95 [0,4194304] 0 2024-05-09T06:38:21.249 INFO:tasks.workunit.client.0.smithi063.stdout:4/756: chown d7/df/d1a 397575 1 2024-05-09T06:38:21.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/757: write d7/da3/d28/fd0 [842852,69107] 0 2024-05-09T06:38:21.251 INFO:tasks.workunit.client.0.smithi063.stdout:4/758: stat d7/df/d35/d5a/dad/de9/lf1 0 2024-05-09T06:38:21.252 INFO:tasks.workunit.client.0.smithi063.stdout:2/803: symlink dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/lf5 0 2024-05-09T06:38:21.252 INFO:tasks.workunit.client.0.smithi063.stdout:4/759: chown d7/da3 414 1 2024-05-09T06:38:21.256 INFO:tasks.workunit.client.0.smithi063.stdout:7/745: truncate d0/fc 1317759 0 2024-05-09T06:38:21.257 INFO:tasks.workunit.client.0.smithi063.stdout:0/803: mkdir dd/d60/dcd/dfc 0 2024-05-09T06:38:21.257 INFO:tasks.workunit.client.1.smithi193.stdout:3/363: rmdir d1/d2d 39 2024-05-09T06:38:21.258 INFO:tasks.workunit.client.0.smithi063.stdout:4/760: write d7/da3/d28/d31/fdb [104678,49868] 0 2024-05-09T06:38:21.259 INFO:tasks.workunit.client.0.smithi063.stdout:4/761: truncate d7/d50/f99 1308316 0 2024-05-09T06:38:21.259 INFO:tasks.workunit.client.0.smithi063.stdout:6/776: unlink d2/d1f/d29/d3f/d5c/d64/f6e 0 2024-05-09T06:38:21.261 INFO:tasks.workunit.client.1.smithi193.stdout:8/387: getdents d6/d1c 0 2024-05-09T06:38:21.262 INFO:tasks.workunit.client.0.smithi063.stdout:5/753: creat d8/d17/dfa/ffc x:0 0 0 2024-05-09T06:38:21.264 INFO:tasks.workunit.client.1.smithi193.stdout:7/410: rename d6/d8/c2c to d6/d17/d7d/c8a 0 2024-05-09T06:38:21.266 INFO:tasks.workunit.client.0.smithi063.stdout:6/777: dwrite d2/d1f/d6c/fb7 [0,4194304] 0 2024-05-09T06:38:21.266 INFO:tasks.workunit.client.1.smithi193.stdout:5/375: creat d0/d13/d2c/d37/d2b/d47/f79 x:0 0 0 2024-05-09T06:38:21.267 INFO:tasks.workunit.client.1.smithi193.stdout:8/388: dwrite d6/d1c/f12 [0,4194304] 0 2024-05-09T06:38:21.268 INFO:tasks.workunit.client.0.smithi063.stdout:6/778: write d2/d4/d7c/d7e/f87 [84216,77874] 0 2024-05-09T06:38:21.268 INFO:tasks.workunit.client.1.smithi193.stdout:5/376: readlink d0/d13/d2c/d4d/l50 0 2024-05-09T06:38:21.269 INFO:tasks.workunit.client.0.smithi063.stdout:1/715: fdatasync d9/d2a/d35/d1e/d9d/d3f/d59/f79 0 2024-05-09T06:38:21.271 INFO:tasks.workunit.client.1.smithi193.stdout:6/415: creat d1/d25/d2c/d3c/f93 x:0 0 0 2024-05-09T06:38:21.273 INFO:tasks.workunit.client.1.smithi193.stdout:6/416: chown d1/d25/d2c/d3c/d3e/d58/c72 139351183 1 2024-05-09T06:38:21.273 INFO:tasks.workunit.client.1.smithi193.stdout:4/475: read - d1a/d6d/f7b zero size 2024-05-09T06:38:21.277 INFO:tasks.workunit.client.1.smithi193.stdout:8/389: dwrite d6/d1c/d1b/f4a [0,4194304] 0 2024-05-09T06:38:21.278 INFO:tasks.workunit.client.0.smithi063.stdout:2/804: dread - dc/f99 zero size 2024-05-09T06:38:21.278 INFO:tasks.workunit.client.1.smithi193.stdout:1/469: creat d4/d16/d60/d4f/d72/d7f/f93 x:0 0 0 2024-05-09T06:38:21.281 INFO:tasks.workunit.client.0.smithi063.stdout:4/762: fdatasync d7/da3/d28/d31/d32/d34/f59 0 2024-05-09T06:38:21.286 INFO:tasks.workunit.client.0.smithi063.stdout:3/771: symlink d7/l105 0 2024-05-09T06:38:21.286 INFO:tasks.workunit.client.0.smithi063.stdout:3/772: dread - d7/d4c/d25/fd1 zero size 2024-05-09T06:38:21.287 INFO:tasks.workunit.client.1.smithi193.stdout:6/417: dwrite d1/d25/d2c/d3c/d3e/d58/f62 [0,4194304] 0 2024-05-09T06:38:21.287 INFO:tasks.workunit.client.0.smithi063.stdout:6/779: creat d2/d20/d26/ffb x:0 0 0 2024-05-09T06:38:21.288 INFO:tasks.workunit.client.0.smithi063.stdout:6/780: fsync d2/d20/d26/d7f/dcf/fda 0 2024-05-09T06:38:21.290 INFO:tasks.workunit.client.0.smithi063.stdout:1/716: chown d9/d2a/d35/d24/fac 9523 1 2024-05-09T06:38:21.291 INFO:tasks.workunit.client.0.smithi063.stdout:9/768: sync 2024-05-09T06:38:21.292 INFO:tasks.workunit.client.0.smithi063.stdout:9/769: read d0/d8/dea/f101 [1138890,30625] 0 2024-05-09T06:38:21.294 INFO:tasks.workunit.client.1.smithi193.stdout:5/377: creat d0/d13/d2c/d37/d5b/d4e/f7a x:0 0 0 2024-05-09T06:38:21.294 INFO:tasks.workunit.client.0.smithi063.stdout:7/746: symlink d0/d15/d49/d54/d9f/dd2/df9/l100 0 2024-05-09T06:38:21.294 INFO:tasks.workunit.client.1.smithi193.stdout:9/477: creat dd/df/d15/d98/fa1 x:0 0 0 2024-05-09T06:38:21.296 INFO:tasks.workunit.client.0.smithi063.stdout:2/805: mknod dc/d6e/d7f/cf6 0 2024-05-09T06:38:21.302 INFO:tasks.workunit.client.1.smithi193.stdout:3/364: truncate d1/db/d59/f69 571670 0 2024-05-09T06:38:21.304 INFO:tasks.workunit.client.1.smithi193.stdout:4/476: dread d1a/d22/d43/f75 [0,4194304] 0 2024-05-09T06:38:21.305 INFO:tasks.workunit.client.1.smithi193.stdout:1/470: creat d4/d16/d6b/d85/f94 x:0 0 0 2024-05-09T06:38:21.305 INFO:tasks.workunit.client.1.smithi193.stdout:4/477: write d1a/d6b/d60/f91 [602556,95344] 0 2024-05-09T06:38:21.306 INFO:tasks.workunit.client.1.smithi193.stdout:4/478: readlink d1a/l68 0 2024-05-09T06:38:21.308 INFO:tasks.workunit.client.1.smithi193.stdout:8/390: unlink d6/d1c/d1b/f4a 0 2024-05-09T06:38:21.309 INFO:tasks.workunit.client.1.smithi193.stdout:8/391: truncate d6/d1c/f34 384428 0 2024-05-09T06:38:21.312 INFO:tasks.workunit.client.1.smithi193.stdout:7/411: getdents d6/dd/d72 0 2024-05-09T06:38:21.313 INFO:tasks.workunit.client.0.smithi063.stdout:6/781: dread d2/d20/d26/f97 [0,4194304] 0 2024-05-09T06:38:21.314 INFO:tasks.workunit.client.1.smithi193.stdout:7/412: dread - d6/d8/d29/f7a zero size 2024-05-09T06:38:21.316 INFO:tasks.workunit.client.1.smithi193.stdout:0/378: rename d18/d1d/f2d to d18/d65/f79 0 2024-05-09T06:38:21.317 INFO:tasks.workunit.client.0.smithi063.stdout:5/754: write f6 [251278,100266] 0 2024-05-09T06:38:21.317 INFO:tasks.workunit.client.1.smithi193.stdout:7/413: write d6/d8/d1a/d36/d4d/f64 [533104,123921] 0 2024-05-09T06:38:21.317 INFO:tasks.workunit.client.1.smithi193.stdout:6/418: dread d1/de/d11/f22 [0,4194304] 0 2024-05-09T06:38:21.318 INFO:tasks.workunit.client.0.smithi063.stdout:3/773: creat d7/da/de4/de8/dfd/f106 x:0 0 0 2024-05-09T06:38:21.318 INFO:tasks.workunit.client.1.smithi193.stdout:8/392: dwrite d6/d1c/d18/f1d [0,4194304] 0 2024-05-09T06:38:21.320 INFO:tasks.workunit.client.1.smithi193.stdout:7/414: dread - d6/dd/f40 zero size 2024-05-09T06:38:21.322 INFO:tasks.workunit.client.0.smithi063.stdout:1/717: rename d9/d2a/d35/d24/fa6 to d9/d2a/d3c/d4c/d83/d88/dc2/dd0/ff3 0 2024-05-09T06:38:21.327 INFO:tasks.workunit.client.1.smithi193.stdout:0/379: dwrite f9 [0,4194304] 0 2024-05-09T06:38:21.327 INFO:tasks.workunit.client.0.smithi063.stdout:6/782: dread d2/d20/d26/d7f/dcf/fda [0,4194304] 0 2024-05-09T06:38:21.329 INFO:tasks.workunit.client.0.smithi063.stdout:6/783: dread - d2/df/d91/dd8/dd9/ff7 zero size 2024-05-09T06:38:21.333 INFO:tasks.workunit.client.0.smithi063.stdout:9/770: dread d0/d4/d57/d60/d76/fe1 [0,4194304] 0 2024-05-09T06:38:21.335 INFO:tasks.workunit.client.1.smithi193.stdout:4/479: read fa [352450,106821] 0 2024-05-09T06:38:21.336 INFO:tasks.workunit.client.0.smithi063.stdout:3/774: rename d7/da/d2b/d66/d74/d7e/f99 to d7/d22/dfc/f107 0 2024-05-09T06:38:21.337 INFO:tasks.workunit.client.1.smithi193.stdout:4/480: write d1a/d1f/d4b/f54 [635597,108218] 0 2024-05-09T06:38:21.338 INFO:tasks.workunit.client.0.smithi063.stdout:3/775: chown d7/d22/lb5 83500502 1 2024-05-09T06:38:21.338 INFO:tasks.workunit.client.1.smithi193.stdout:4/481: write d1a/f1d [2466666,127163] 0 2024-05-09T06:38:21.340 INFO:tasks.workunit.client.0.smithi063.stdout:1/718: mkdir d9/d2a/d2e/df4 0 2024-05-09T06:38:21.345 INFO:tasks.workunit.client.0.smithi063.stdout:2/806: rmdir dc/d36/d43/d46/d63/d9e/ddb 0 2024-05-09T06:38:21.346 INFO:tasks.workunit.client.0.smithi063.stdout:2/807: readlink dc/d36/d43/lb8 0 2024-05-09T06:38:21.347 INFO:tasks.workunit.client.1.smithi193.stdout:0/380: dread f8 [0,4194304] 0 2024-05-09T06:38:21.348 INFO:tasks.workunit.client.0.smithi063.stdout:5/755: rename d8/d19/d84/ld8 to d8/d19/d84/lfd 0 2024-05-09T06:38:21.355 INFO:tasks.workunit.client.1.smithi193.stdout:0/381: dread f8 [0,4194304] 0 2024-05-09T06:38:21.355 INFO:tasks.workunit.client.1.smithi193.stdout:4/482: dread d1a/d6b/f3c [0,4194304] 0 2024-05-09T06:38:21.357 INFO:tasks.workunit.client.1.smithi193.stdout:4/483: dread - d1a/d1f/d4b/d7c/d5b/f87 zero size 2024-05-09T06:38:21.358 INFO:tasks.workunit.client.1.smithi193.stdout:4/484: write d1a/d6b/f77 [1030960,117899] 0 2024-05-09T06:38:21.359 INFO:tasks.workunit.client.0.smithi063.stdout:6/784: rename d2/d20/de7 to d2/d4/d7c/dfc 0 2024-05-09T06:38:21.360 INFO:tasks.workunit.client.1.smithi193.stdout:9/478: symlink dd/df/d31/d7e/la2 0 2024-05-09T06:38:21.362 INFO:tasks.workunit.client.1.smithi193.stdout:7/415: dread d6/dd/f89 [0,4194304] 0 2024-05-09T06:38:21.362 INFO:tasks.workunit.client.1.smithi193.stdout:9/479: read dd/df/d31/d7e/f62 [3199748,111922] 0 2024-05-09T06:38:21.364 INFO:tasks.workunit.client.1.smithi193.stdout:3/365: readlink d1/d2d/d51/l30 0 2024-05-09T06:38:21.365 INFO:tasks.workunit.client.0.smithi063.stdout:2/808: mknod dc/dd/d45/d5a/d6c/d8c/d9a/dd5/cf7 0 2024-05-09T06:38:21.366 INFO:tasks.workunit.client.0.smithi063.stdout:2/809: chown dc/dd 242 1 2024-05-09T06:38:21.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/756: unlink d8/d53/fab 0 2024-05-09T06:38:21.368 INFO:tasks.workunit.client.0.smithi063.stdout:6/785: mknod d2/d20/d26/cfd 0 2024-05-09T06:38:21.371 INFO:tasks.workunit.client.0.smithi063.stdout:9/771: dwrite d0/d4/d7d/fb3 [0,4194304] 0 2024-05-09T06:38:21.372 INFO:tasks.workunit.client.0.smithi063.stdout:5/757: chown d8/ca6 302 1 2024-05-09T06:38:21.373 INFO:tasks.workunit.client.0.smithi063.stdout:9/772: chown d0/d4/d26/d39/d48/ld0 24126 1 2024-05-09T06:38:21.373 INFO:tasks.workunit.client.0.smithi063.stdout:2/810: creat dc/dd/d45/d5a/d6c/d8f/ff8 x:0 0 0 2024-05-09T06:38:21.374 INFO:tasks.workunit.client.0.smithi063.stdout:3/776: link d7/d4c/lb6 d7/da/d2b/d66/dc0/da4/l108 0 2024-05-09T06:38:21.376 INFO:tasks.workunit.client.0.smithi063.stdout:1/719: creat d9/d2a/d39/ff5 x:0 0 0 2024-05-09T06:38:21.380 INFO:tasks.workunit.client.0.smithi063.stdout:5/758: mkdir d8/d17/dfe 0 2024-05-09T06:38:21.386 INFO:tasks.workunit.client.0.smithi063.stdout:3/777: creat d7/d22/d28/f109 x:0 0 0 2024-05-09T06:38:21.387 INFO:tasks.workunit.client.0.smithi063.stdout:1/720: symlink d9/d2a/d39/d5d/lf6 0 2024-05-09T06:38:21.388 INFO:tasks.workunit.client.0.smithi063.stdout:6/786: mkdir d2/df/dd2/dfe 0 2024-05-09T06:38:21.389 INFO:tasks.workunit.client.0.smithi063.stdout:1/721: read d9/d2a/d3c/d4c/dc3/fd2 [728348,68260] 0 2024-05-09T06:38:21.394 INFO:tasks.workunit.client.0.smithi063.stdout:8/851: sync 2024-05-09T06:38:21.394 INFO:tasks.workunit.client.0.smithi063.stdout:3/778: dread d7/da/d2b/f43 [0,4194304] 0 2024-05-09T06:38:21.395 INFO:tasks.workunit.client.0.smithi063.stdout:6/787: creat d2/d4/d7c/dfc/fff x:0 0 0 2024-05-09T06:38:21.395 INFO:tasks.workunit.client.0.smithi063.stdout:3/779: chown d7/da/de4/de8/dfd 1713941 1 2024-05-09T06:38:21.397 INFO:tasks.workunit.client.0.smithi063.stdout:1/722: rename d9/d2a/d3c/d4c/d83/d88/dc2/dd0/l61 to d9/d2a/d35/d24/d68/lf7 0 2024-05-09T06:38:21.397 INFO:tasks.workunit.client.0.smithi063.stdout:6/788: dread - d2/df/d59/d99/ff8 zero size 2024-05-09T06:38:21.398 INFO:tasks.workunit.client.0.smithi063.stdout:2/811: dwrite dc/f1b [4194304,4194304] 0 2024-05-09T06:38:21.399 INFO:tasks.workunit.client.0.smithi063.stdout:3/780: truncate d7/d22/d28/fec 763355 0 2024-05-09T06:38:21.402 INFO:tasks.workunit.client.1.smithi193.stdout:6/419: mkdir d1/de/d11/d18/d94 0 2024-05-09T06:38:21.403 INFO:tasks.workunit.client.0.smithi063.stdout:9/773: truncate d0/d4/d26/d47/f5d 3068912 0 2024-05-09T06:38:21.403 INFO:tasks.workunit.client.0.smithi063.stdout:1/723: mkdir d9/d2a/d3c/d4c/d83/d88/dc2/df8 0 2024-05-09T06:38:21.404 INFO:tasks.workunit.client.1.smithi193.stdout:0/382: symlink d18/d4c/d47/d55/l7a 0 2024-05-09T06:38:21.405 INFO:tasks.workunit.client.1.smithi193.stdout:5/378: creat d0/d13/d2c/d37/d5b/f7b x:0 0 0 2024-05-09T06:38:21.408 INFO:tasks.workunit.client.1.smithi193.stdout:4/485: write d1a/d22/f31 [886134,42557] 0 2024-05-09T06:38:21.411 INFO:tasks.workunit.client.1.smithi193.stdout:0/383: dwrite d18/d4c/d47/d59/d76/f77 [0,4194304] 0 2024-05-09T06:38:21.413 INFO:tasks.workunit.client.1.smithi193.stdout:1/471: rmdir d4/d16/d17/d1a/d7e 0 2024-05-09T06:38:21.416 INFO:tasks.workunit.client.0.smithi063.stdout:1/724: truncate d9/d2a/d3c/d4a/f7b 1191293 0 2024-05-09T06:38:21.417 INFO:tasks.workunit.client.1.smithi193.stdout:3/366: dread d1/db/d2f/f33 [0,4194304] 0 2024-05-09T06:38:21.418 INFO:tasks.workunit.client.1.smithi193.stdout:8/393: link d6/d1c/d53/f60 d6/d1c/d18/f74 0 2024-05-09T06:38:21.419 INFO:tasks.workunit.client.0.smithi063.stdout:8/852: mkdir d9/d3b/df8/d12f 0 2024-05-09T06:38:21.420 INFO:tasks.workunit.client.1.smithi193.stdout:6/420: creat d1/de/d11/d2d/f95 x:0 0 0 2024-05-09T06:38:21.424 INFO:tasks.workunit.client.0.smithi063.stdout:9/774: unlink d0/d43/f5f 0 2024-05-09T06:38:21.429 INFO:tasks.workunit.client.0.smithi063.stdout:3/781: write d7/d4c/d25/fac [119786,56346] 0 2024-05-09T06:38:21.430 INFO:tasks.workunit.client.0.smithi063.stdout:8/853: creat d9/da/d18/d46/d10c/f130 x:0 0 0 2024-05-09T06:38:21.431 INFO:tasks.workunit.client.0.smithi063.stdout:2/812: truncate dc/f21 3495579 0 2024-05-09T06:38:21.438 INFO:tasks.workunit.client.1.smithi193.stdout:0/384: dread d18/d1d/f33 [0,4194304] 0 2024-05-09T06:38:21.438 INFO:tasks.workunit.client.1.smithi193.stdout:4/486: rename d1a/d1f/d4b to d1a/d24/d73/d94 0 2024-05-09T06:38:21.439 INFO:tasks.workunit.client.0.smithi063.stdout:6/789: getdents d2/d4/d7c 0 2024-05-09T06:38:21.440 INFO:tasks.workunit.client.1.smithi193.stdout:1/472: creat d4/d16/d17/d1a/d3a/d3f/f95 x:0 0 0 2024-05-09T06:38:21.442 INFO:tasks.workunit.client.0.smithi063.stdout:3/782: mkdir d7/d22/d10a 0 2024-05-09T06:38:21.443 INFO:tasks.workunit.client.1.smithi193.stdout:3/367: rmdir d1/db 39 2024-05-09T06:38:21.445 INFO:tasks.workunit.client.1.smithi193.stdout:9/480: truncate dd/d1e/d66/d49/f70 1971420 0 2024-05-09T06:38:21.447 INFO:tasks.workunit.client.1.smithi193.stdout:9/481: fsync dd/d1e/d66/d49/f65 0 2024-05-09T06:38:21.447 INFO:tasks.workunit.client.0.smithi063.stdout:2/813: mknod dc/d36/d43/d46/d49/ddc/cf9 0 2024-05-09T06:38:21.452 INFO:tasks.workunit.client.1.smithi193.stdout:5/379: mkdir d0/d7c 0 2024-05-09T06:38:21.453 INFO:tasks.workunit.client.0.smithi063.stdout:6/790: stat d2/d1f/d29/de2/d4e/c78 0 2024-05-09T06:38:21.454 INFO:tasks.workunit.client.1.smithi193.stdout:7/416: creat d6/f8b x:0 0 0 2024-05-09T06:38:21.454 INFO:tasks.workunit.client.0.smithi063.stdout:6/791: chown d2/d1f/d29/d3f/l16 33 1 2024-05-09T06:38:21.455 INFO:tasks.workunit.client.0.smithi063.stdout:6/792: readlink d2/d12/lc8 0 2024-05-09T06:38:21.456 INFO:tasks.workunit.client.0.smithi063.stdout:9/775: dwrite d0/d4/f37 [0,4194304] 0 2024-05-09T06:38:21.457 INFO:tasks.workunit.client.1.smithi193.stdout:2/434: sync 2024-05-09T06:38:21.457 INFO:tasks.workunit.client.0.smithi063.stdout:7/747: sync 2024-05-09T06:38:21.457 INFO:tasks.workunit.client.0.smithi063.stdout:0/804: sync 2024-05-09T06:38:21.457 INFO:tasks.workunit.client.0.smithi063.stdout:4/763: sync 2024-05-09T06:38:21.459 INFO:tasks.workunit.client.1.smithi193.stdout:4/487: creat d1a/d6b/d60/f95 x:0 0 0 2024-05-09T06:38:21.460 INFO:tasks.workunit.client.0.smithi063.stdout:4/764: dread - d7/da3/d28/d31/d71/d98/fef zero size 2024-05-09T06:38:21.460 INFO:tasks.workunit.client.1.smithi193.stdout:4/488: fdatasync d1a/d6b/f77 0 2024-05-09T06:38:21.462 INFO:tasks.workunit.client.1.smithi193.stdout:4/489: dread - d1a/d6d/f7b zero size 2024-05-09T06:38:21.463 INFO:tasks.workunit.client.1.smithi193.stdout:4/490: chown d1a/l68 50 1 2024-05-09T06:38:21.464 INFO:tasks.workunit.client.0.smithi063.stdout:3/783: rename d7/d22/d9c to d7/d4c/d25/d10b 0 2024-05-09T06:38:21.468 INFO:tasks.workunit.client.1.smithi193.stdout:1/473: mkdir d4/d16/d60/d4f/d72/d96 0 2024-05-09T06:38:21.468 INFO:tasks.workunit.client.0.smithi063.stdout:8/854: dwrite d9/da/d18/d24/f62 [0,4194304] 0 2024-05-09T06:38:21.468 INFO:tasks.workunit.client.0.smithi063.stdout:3/784: dread d7/da/d2b/f43 [0,4194304] 0 2024-05-09T06:38:21.469 INFO:tasks.workunit.client.0.smithi063.stdout:9/776: dwrite d0/d4/d26/d39/d3e/fce [0,4194304] 0 2024-05-09T06:38:21.470 INFO:tasks.workunit.client.0.smithi063.stdout:3/785: dread - d7/da/d2b/d66/d74/d7e/d97/f102 zero size 2024-05-09T06:38:21.473 INFO:tasks.workunit.client.1.smithi193.stdout:9/482: dread fa [0,4194304] 0 2024-05-09T06:38:21.474 INFO:tasks.workunit.client.1.smithi193.stdout:9/483: stat fa 0 2024-05-09T06:38:21.476 INFO:tasks.workunit.client.1.smithi193.stdout:6/421: dwrite d1/d25/d2c/d3c/d3e/f3f [0,4194304] 0 2024-05-09T06:38:21.477 INFO:tasks.workunit.client.0.smithi063.stdout:1/725: getdents d9/d10 0 2024-05-09T06:38:21.481 INFO:tasks.workunit.client.1.smithi193.stdout:5/380: dread d0/f9 [0,4194304] 0 2024-05-09T06:38:21.481 INFO:tasks.workunit.client.0.smithi063.stdout:3/786: dread d7/d70/d98/d5b/f9b [0,4194304] 0 2024-05-09T06:38:21.482 INFO:tasks.workunit.client.0.smithi063.stdout:3/787: write d7/da/d21/f4d [3901207,123373] 0 2024-05-09T06:38:21.483 INFO:tasks.workunit.client.1.smithi193.stdout:6/422: dread d1/d25/d2c/d3c/d3e/f43 [0,4194304] 0 2024-05-09T06:38:21.486 INFO:tasks.workunit.client.1.smithi193.stdout:6/423: truncate d1/d25/d2c/d76/d69/f88 877359 0 2024-05-09T06:38:21.487 INFO:tasks.workunit.client.1.smithi193.stdout:6/424: readlink d1/de/d11/d18/l39 0 2024-05-09T06:38:21.487 INFO:tasks.workunit.client.1.smithi193.stdout:0/385: symlink d18/d4c/d64/d66/l7b 0 2024-05-09T06:38:21.489 INFO:tasks.workunit.client.1.smithi193.stdout:6/425: readlink d1/d25/d2c/d81/d37/l45 0 2024-05-09T06:38:21.490 INFO:tasks.workunit.client.1.smithi193.stdout:6/426: dread - d1/d25/d2c/f75 zero size 2024-05-09T06:38:21.492 INFO:tasks.workunit.client.1.smithi193.stdout:1/474: symlink d4/d16/d17/d1a/l97 0 2024-05-09T06:38:21.492 INFO:tasks.workunit.client.0.smithi063.stdout:0/805: dread dd/fbe [0,4194304] 0 2024-05-09T06:38:21.493 INFO:tasks.workunit.client.0.smithi063.stdout:8/855: mknod d9/d3b/c131 0 2024-05-09T06:38:21.495 INFO:tasks.workunit.client.1.smithi193.stdout:3/368: mkdir d1/db/d2f/d78/d86 0 2024-05-09T06:38:21.496 INFO:tasks.workunit.client.0.smithi063.stdout:2/814: getdents dc/d36/d43/d46/d49/dbb/dec 0 2024-05-09T06:38:21.496 INFO:tasks.workunit.client.1.smithi193.stdout:8/394: creat d6/d1c/d1b/f75 x:0 0 0 2024-05-09T06:38:21.497 INFO:tasks.workunit.client.0.smithi063.stdout:2/815: fdatasync dc/dd/f23 0 2024-05-09T06:38:21.498 INFO:tasks.workunit.client.0.smithi063.stdout:2/816: write dc/dd/d45/fe3 [285058,74512] 0 2024-05-09T06:38:21.498 INFO:tasks.workunit.client.1.smithi193.stdout:8/395: write d6/d1c/f34 [620078,83055] 0 2024-05-09T06:38:21.499 INFO:tasks.workunit.client.0.smithi063.stdout:1/726: symlink d9/d2a/d3c/d4a/db8/ddf/lf9 0 2024-05-09T06:38:21.500 INFO:tasks.workunit.client.1.smithi193.stdout:1/475: dwrite d4/d14/f75 [0,4194304] 0 2024-05-09T06:38:21.501 INFO:tasks.workunit.client.1.smithi193.stdout:8/396: truncate d6/d1c/d53/f60 57951 0 2024-05-09T06:38:21.503 INFO:tasks.workunit.client.1.smithi193.stdout:8/397: read - d6/d1c/d53/d56/f6c zero size 2024-05-09T06:38:21.503 INFO:tasks.workunit.client.1.smithi193.stdout:7/417: creat d6/dd/d72/f8c x:0 0 0 2024-05-09T06:38:21.503 INFO:tasks.workunit.client.0.smithi063.stdout:3/788: chown d7/d70/d98/d2d/d36/lea 8045790 1 2024-05-09T06:38:21.504 INFO:tasks.workunit.client.1.smithi193.stdout:5/381: creat d0/d13/d2c/d37/d2b/d47/d6f/f7d x:0 0 0 2024-05-09T06:38:21.507 INFO:tasks.workunit.client.1.smithi193.stdout:0/386: truncate d18/d1d/f1e 718928 0 2024-05-09T06:38:21.508 INFO:tasks.workunit.client.0.smithi063.stdout:8/856: rmdir d9/da/d53/db2 39 2024-05-09T06:38:21.511 INFO:tasks.workunit.client.1.smithi193.stdout:6/427: fdatasync d1/de/d11/f68 0 2024-05-09T06:38:21.512 INFO:tasks.workunit.client.1.smithi193.stdout:2/435: write db/f21 [4225535,14124] 0 2024-05-09T06:38:21.514 INFO:tasks.workunit.client.0.smithi063.stdout:6/793: dwrite d2/d1f/d29/d3f/d5c/f70 [0,4194304] 0 2024-05-09T06:38:21.514 INFO:tasks.workunit.client.1.smithi193.stdout:2/436: chown db/d37/d6d/d80/c16 1947769371 1 2024-05-09T06:38:21.515 INFO:tasks.workunit.client.1.smithi193.stdout:2/437: truncate db/df/d54/f8c 224973 0 2024-05-09T06:38:21.516 INFO:tasks.workunit.client.0.smithi063.stdout:2/817: rename dc/d36/c59 to dc/d6e/cfa 0 2024-05-09T06:38:21.519 INFO:tasks.workunit.client.0.smithi063.stdout:1/727: mknod d9/d2a/d3c/cfa 0 2024-05-09T06:38:21.522 INFO:tasks.workunit.client.0.smithi063.stdout:3/789: rmdir d7/d70/d98/d2d 39 2024-05-09T06:38:21.523 INFO:tasks.workunit.client.0.smithi063.stdout:5/759: sync 2024-05-09T06:38:21.524 INFO:tasks.workunit.client.0.smithi063.stdout:5/760: dread - d8/d35/f7b zero size 2024-05-09T06:38:21.524 INFO:tasks.workunit.client.1.smithi193.stdout:9/484: creat dd/d1e/d93/fa3 x:0 0 0 2024-05-09T06:38:21.525 INFO:tasks.workunit.client.0.smithi063.stdout:7/748: link d0/d15/d49/d54/cbc d0/d15/d4a/d68/c101 0 2024-05-09T06:38:21.528 INFO:tasks.workunit.client.1.smithi193.stdout:1/476: mkdir d4/d16/d35/d98 0 2024-05-09T06:38:21.528 INFO:tasks.workunit.client.1.smithi193.stdout:1/477: fdatasync d4/d3c/f36 0 2024-05-09T06:38:21.529 INFO:tasks.workunit.client.1.smithi193.stdout:1/478: fsync d4/d14/f51 0 2024-05-09T06:38:21.534 INFO:tasks.workunit.client.1.smithi193.stdout:1/479: dread d4/d16/d17/f29 [4194304,4194304] 0 2024-05-09T06:38:21.535 INFO:tasks.workunit.client.0.smithi063.stdout:2/818: dread - dc/dd/d45/d5a/d6c/d8c/dad/faf zero size 2024-05-09T06:38:21.536 INFO:tasks.workunit.client.0.smithi063.stdout:2/819: fdatasync dc/dd/d2d/d60/dd0/d90/dc1/fe1 0 2024-05-09T06:38:21.539 INFO:tasks.workunit.client.0.smithi063.stdout:1/728: mkdir d9/d2a/d3c/d4c/d83/d88/dfb 0 2024-05-09T06:38:21.539 INFO:tasks.workunit.client.0.smithi063.stdout:3/790: dread - d7/d4c/fc4 zero size 2024-05-09T06:38:21.540 INFO:tasks.workunit.client.1.smithi193.stdout:3/369: dwrite d1/db/d59/f37 [0,4194304] 0 2024-05-09T06:38:21.540 INFO:tasks.workunit.client.0.smithi063.stdout:3/791: stat d7/d70/d98/d5b/f8f 0 2024-05-09T06:38:21.541 INFO:tasks.workunit.client.0.smithi063.stdout:5/761: stat d8/cd 0 2024-05-09T06:38:21.542 INFO:tasks.workunit.client.0.smithi063.stdout:9/777: sync 2024-05-09T06:38:21.543 INFO:tasks.workunit.client.1.smithi193.stdout:9/485: mkdir dd/df/d15/d98/da4 0 2024-05-09T06:38:21.545 INFO:tasks.workunit.client.0.smithi063.stdout:7/749: symlink d0/d15/d49/d4f/d66/l102 0 2024-05-09T06:38:21.546 INFO:tasks.workunit.client.1.smithi193.stdout:0/387: write d18/d4c/f4b [1166778,97936] 0 2024-05-09T06:38:21.548 INFO:tasks.workunit.client.0.smithi063.stdout:4/765: rename d7/df/d35/d5a/ccb to d7/df/d35/d5a/cf3 0 2024-05-09T06:38:21.551 INFO:tasks.workunit.client.0.smithi063.stdout:0/806: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/f93 [0,4194304] 0 2024-05-09T06:38:21.552 INFO:tasks.workunit.client.1.smithi193.stdout:7/418: link d6/dd/d7e/f82 d6/d8/d29/d2f/f8d 0 2024-05-09T06:38:21.553 INFO:tasks.workunit.client.0.smithi063.stdout:6/794: mknod d2/d1f/d29/d3f/d5c/c100 0 2024-05-09T06:38:21.554 INFO:tasks.workunit.client.0.smithi063.stdout:6/795: chown d2/df/l37 215879 1 2024-05-09T06:38:21.556 INFO:tasks.workunit.client.1.smithi193.stdout:2/438: dread db/d25/f2a [0,4194304] 0 2024-05-09T06:38:21.556 INFO:tasks.workunit.client.0.smithi063.stdout:6/796: write d2/d20/d26/ffb [410715,82440] 0 2024-05-09T06:38:21.557 INFO:tasks.workunit.client.0.smithi063.stdout:1/729: unlink d9/d2a/d35/d1e/d9d/c7c 0 2024-05-09T06:38:21.557 INFO:tasks.workunit.client.1.smithi193.stdout:2/439: dread - db/d25/d38/d40/d8f/f93 zero size 2024-05-09T06:38:21.558 INFO:tasks.workunit.client.1.smithi193.stdout:2/440: fdatasync db/df/f94 0 2024-05-09T06:38:21.559 INFO:tasks.workunit.client.0.smithi063.stdout:3/792: rename d7/da/de4/de8/dfd to d7/da/d21/d30/d44/dee/d10c 0 2024-05-09T06:38:21.560 INFO:tasks.workunit.client.1.smithi193.stdout:3/370: rename d1/db/c1a to d1/db/d2f/d78/d86/c87 0 2024-05-09T06:38:21.562 INFO:tasks.workunit.client.0.smithi063.stdout:9/778: mkdir d0/d4/d26/d39/d48/d80/d84/dc2/d10f 0 2024-05-09T06:38:21.562 INFO:tasks.workunit.client.1.smithi193.stdout:9/486: fdatasync dd/df/d31/d7e/f62 0 2024-05-09T06:38:21.563 INFO:tasks.workunit.client.0.smithi063.stdout:0/807: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/fdd [0,4194304] 0 2024-05-09T06:38:21.563 INFO:tasks.workunit.client.1.smithi193.stdout:9/487: chown dd/d1e/d1f/d48/l6c 71 1 2024-05-09T06:38:21.564 INFO:tasks.workunit.client.0.smithi063.stdout:7/750: symlink d0/d15/d49/d4f/d66/ddd/l103 0 2024-05-09T06:38:21.567 INFO:tasks.workunit.client.0.smithi063.stdout:2/820: dread dc/d36/d43/d46/fcc [0,4194304] 0 2024-05-09T06:38:21.567 INFO:tasks.workunit.client.1.smithi193.stdout:8/398: write d6/d1c/d18/d2f/d40/f2c [1244929,45973] 0 2024-05-09T06:38:21.573 INFO:tasks.workunit.client.1.smithi193.stdout:5/382: write d0/d13/d2c/d37/d5b/f32 [342788,28076] 0 2024-05-09T06:38:21.575 INFO:tasks.workunit.client.1.smithi193.stdout:5/383: readlink d0/d13/d2c/d37/d5b/l48 0 2024-05-09T06:38:21.575 INFO:tasks.workunit.client.0.smithi063.stdout:9/779: dwrite d0/d4/d7d/ffd [0,4194304] 0 2024-05-09T06:38:21.576 INFO:tasks.workunit.client.1.smithi193.stdout:2/441: symlink db/df/d54/l97 0 2024-05-09T06:38:21.577 INFO:tasks.workunit.client.0.smithi063.stdout:7/751: dwrite d0/d15/d89/fa9 [0,4194304] 0 2024-05-09T06:38:21.578 INFO:tasks.workunit.client.1.smithi193.stdout:2/442: write db/df/d54/f7c [4348810,70828] 0 2024-05-09T06:38:21.582 INFO:tasks.workunit.client.0.smithi063.stdout:2/821: dread dc/d36/d43/d46/d63/d9e/feb [0,4194304] 0 2024-05-09T06:38:21.583 INFO:tasks.workunit.client.0.smithi063.stdout:9/780: write d0/d4/d57/f9b [731974,1489] 0 2024-05-09T06:38:21.584 INFO:tasks.workunit.client.1.smithi193.stdout:9/488: fsync dd/df/d15/d8c/f50 0 2024-05-09T06:38:21.585 INFO:tasks.workunit.client.1.smithi193.stdout:3/371: dread d1/db/d59/f4e [0,4194304] 0 2024-05-09T06:38:21.585 INFO:tasks.workunit.client.1.smithi193.stdout:8/399: mknod d6/d1c/d18/d2f/d40/d5e/c76 0 2024-05-09T06:38:21.587 INFO:tasks.workunit.client.1.smithi193.stdout:3/372: dread - d1/db/d59/f81 zero size 2024-05-09T06:38:21.587 INFO:tasks.workunit.client.1.smithi193.stdout:9/489: chown dd/df/d15/f9e 57503112 1 2024-05-09T06:38:21.588 INFO:tasks.workunit.client.1.smithi193.stdout:0/388: dread d18/d27/f34 [0,4194304] 0 2024-05-09T06:38:21.590 INFO:tasks.workunit.client.0.smithi063.stdout:3/793: mkdir d7/d70/d98/dbb/d10d 0 2024-05-09T06:38:21.590 INFO:tasks.workunit.client.0.smithi063.stdout:5/762: truncate d8/d19/d30/d46/f6c 562811 0 2024-05-09T06:38:21.592 INFO:tasks.workunit.client.0.smithi063.stdout:1/730: dwrite d9/d2a/d35/d1e/fd5 [0,4194304] 0 2024-05-09T06:38:21.594 INFO:tasks.workunit.client.0.smithi063.stdout:1/731: dread - d9/d2a/d3c/d4a/db8/ddf/ff1 zero size 2024-05-09T06:38:21.596 INFO:tasks.workunit.client.0.smithi063.stdout:0/808: mkdir dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/dfd 0 2024-05-09T06:38:21.596 INFO:tasks.workunit.client.1.smithi193.stdout:2/443: rmdir db/d37 39 2024-05-09T06:38:21.597 INFO:tasks.workunit.client.0.smithi063.stdout:8/857: getdents d9/da/d18/dc7/d58 0 2024-05-09T06:38:21.600 INFO:tasks.workunit.client.0.smithi063.stdout:4/766: mknod d7/da3/d28/d31/d7b/cf4 0 2024-05-09T06:38:21.600 INFO:tasks.workunit.client.0.smithi063.stdout:1/732: truncate d9/d2a/d3c/d4c/d83/d88/dc2/fef 465977 0 2024-05-09T06:38:21.601 INFO:tasks.workunit.client.1.smithi193.stdout:1/480: getdents d4/d16/d35 0 2024-05-09T06:38:21.602 INFO:tasks.workunit.client.1.smithi193.stdout:1/481: readlink d4/d16/d60/d4f/d72/l8d 0 2024-05-09T06:38:21.603 INFO:tasks.workunit.client.1.smithi193.stdout:3/373: symlink d1/d44/l88 0 2024-05-09T06:38:21.604 INFO:tasks.workunit.client.0.smithi063.stdout:2/822: mknod dc/d36/d43/d46/ded/cfb 0 2024-05-09T06:38:21.606 INFO:tasks.workunit.client.1.smithi193.stdout:0/389: creat d18/d4c/d5f/f7c x:0 0 0 2024-05-09T06:38:21.608 INFO:tasks.workunit.client.1.smithi193.stdout:5/384: mkdir d0/d13/d2c/d7e 0 2024-05-09T06:38:21.611 INFO:tasks.workunit.client.1.smithi193.stdout:1/482: dwrite d4/d16/de/f28 [0,4194304] 0 2024-05-09T06:38:21.614 INFO:tasks.workunit.client.1.smithi193.stdout:5/385: chown d0/d13/d2c/d37/d2b/d47/c6c 5871622 1 2024-05-09T06:38:21.615 INFO:tasks.workunit.client.1.smithi193.stdout:9/490: getdents dd/df/d15/d8c/d7a 0 2024-05-09T06:38:21.619 INFO:tasks.workunit.client.1.smithi193.stdout:0/390: dwrite d18/d4c/d47/d54/f68 [0,4194304] 0 2024-05-09T06:38:21.620 INFO:tasks.workunit.client.1.smithi193.stdout:2/444: dread - db/d37/d6d/d80/d60/d7d/f7e zero size 2024-05-09T06:38:21.623 INFO:tasks.workunit.client.1.smithi193.stdout:8/400: rename d6/d1c/d1b/d29/c70 to d6/d1c/d1b/c77 0 2024-05-09T06:38:21.625 INFO:tasks.workunit.client.1.smithi193.stdout:8/401: write d6/d1c/d1b/f75 [1002348,20696] 0 2024-05-09T06:38:21.635 INFO:tasks.workunit.client.1.smithi193.stdout:3/374: rename d1/c13 to d1/db/d59/d63/c89 0 2024-05-09T06:38:21.637 INFO:tasks.workunit.client.1.smithi193.stdout:8/402: creat d6/d1c/d18/d66/f78 x:0 0 0 2024-05-09T06:38:21.642 INFO:tasks.workunit.client.1.smithi193.stdout:9/491: symlink dd/d1e/d1f/d48/d55/d86/la5 0 2024-05-09T06:38:21.642 INFO:tasks.workunit.client.1.smithi193.stdout:0/391: mkdir d18/d27/d7d 0 2024-05-09T06:38:21.643 INFO:tasks.workunit.client.1.smithi193.stdout:9/492: dread dd/f73 [0,4194304] 0 2024-05-09T06:38:21.644 INFO:tasks.workunit.client.1.smithi193.stdout:2/445: mknod db/c98 0 2024-05-09T06:38:21.646 INFO:tasks.workunit.client.1.smithi193.stdout:1/483: rename d4/d16/d60/d4f/d72/d7f/f8b to d4/d16/d6b/d89/f99 0 2024-05-09T06:38:21.648 INFO:tasks.workunit.client.1.smithi193.stdout:8/403: mknod d6/d1c/d27/c79 0 2024-05-09T06:38:21.655 INFO:tasks.workunit.client.1.smithi193.stdout:5/386: dwrite d0/d13/d2c/d37/d2b/d47/d66/d3a/f52 [0,4194304] 0 2024-05-09T06:38:21.657 INFO:tasks.workunit.client.1.smithi193.stdout:3/375: rename d1/db/d59/l5c to d1/d2d/d51/l8a 0 2024-05-09T06:38:21.662 INFO:tasks.workunit.client.1.smithi193.stdout:9/493: dread dd/d1e/d66/d49/f70 [0,4194304] 0 2024-05-09T06:38:21.663 INFO:tasks.workunit.client.1.smithi193.stdout:9/494: stat dd/d1e/d1f/d43/d5b/f9c 0 2024-05-09T06:38:21.664 INFO:tasks.workunit.client.1.smithi193.stdout:2/446: symlink db/d4b/d56/l99 0 2024-05-09T06:38:21.665 INFO:tasks.workunit.client.1.smithi193.stdout:9/495: truncate dd/d1e/d93/fa3 601911 0 2024-05-09T06:38:21.668 INFO:tasks.workunit.client.1.smithi193.stdout:3/376: unlink d1/db/d2f/l75 0 2024-05-09T06:38:21.669 INFO:tasks.workunit.client.1.smithi193.stdout:3/377: chown d1/db/d2f/f33 0 1 2024-05-09T06:38:21.675 INFO:tasks.workunit.client.1.smithi193.stdout:9/496: creat dd/d1e/d1f/d43/fa6 x:0 0 0 2024-05-09T06:38:21.676 INFO:tasks.workunit.client.1.smithi193.stdout:1/484: dread d4/d16/d60/d4f/f61 [0,4194304] 0 2024-05-09T06:38:21.676 INFO:tasks.workunit.client.0.smithi063.stdout:3/794: mknod d7/da/d21/d30/d44/d48/c10e 0 2024-05-09T06:38:21.677 INFO:tasks.workunit.client.1.smithi193.stdout:8/404: link d6/d1c/d18/d66/l46 d6/d1c/d18/d2f/d40/d5e/l7a 0 2024-05-09T06:38:21.678 INFO:tasks.workunit.client.1.smithi193.stdout:8/405: chown d6/c31 109 1 2024-05-09T06:38:21.682 INFO:tasks.workunit.client.1.smithi193.stdout:8/406: dread d6/d1c/d18/d2f/f44 [0,4194304] 0 2024-05-09T06:38:21.682 INFO:tasks.workunit.client.1.smithi193.stdout:9/497: dwrite dd/df/d15/f56 [0,4194304] 0 2024-05-09T06:38:21.684 INFO:tasks.workunit.client.0.smithi063.stdout:8/858: unlink d9/da/d18/d46/c124 0 2024-05-09T06:38:21.687 INFO:tasks.workunit.client.0.smithi063.stdout:6/797: creat d2/d1f/d29/d3f/d5c/f101 x:0 0 0 2024-05-09T06:38:21.688 INFO:tasks.workunit.client.1.smithi193.stdout:3/378: dread d1/d2d/d51/f6a [0,4194304] 0 2024-05-09T06:38:21.688 INFO:tasks.workunit.client.1.smithi193.stdout:1/485: truncate d4/d14/f39 119819 0 2024-05-09T06:38:21.688 INFO:tasks.workunit.client.0.smithi063.stdout:6/798: dread - d2/d1f/d29/d3f/f82 zero size 2024-05-09T06:38:21.689 INFO:tasks.workunit.client.0.smithi063.stdout:4/767: rmdir d7/da3/d28/d31/d71 39 2024-05-09T06:38:21.690 INFO:tasks.workunit.client.1.smithi193.stdout:8/407: dread d6/d1c/d18/d2f/d40/f2c [0,4194304] 0 2024-05-09T06:38:21.695 INFO:tasks.workunit.client.1.smithi193.stdout:2/447: symlink db/d25/d38/d40/d8b/d77/l9a 0 2024-05-09T06:38:21.696 INFO:tasks.workunit.client.0.smithi063.stdout:7/752: truncate d0/f10 3517697 0 2024-05-09T06:38:21.698 INFO:tasks.workunit.client.1.smithi193.stdout:0/392: write d18/f28 [1765213,82381] 0 2024-05-09T06:38:21.701 INFO:tasks.workunit.client.0.smithi063.stdout:9/781: rename d0/d4/da to d0/d4/d7d/d105/d110 0 2024-05-09T06:38:21.702 INFO:tasks.workunit.client.1.smithi193.stdout:3/379: creat d1/db/d2f/d78/f8b x:0 0 0 2024-05-09T06:38:21.704 INFO:tasks.workunit.client.1.smithi193.stdout:8/408: mknod d6/d1c/d1b/d29/c7b 0 2024-05-09T06:38:21.704 INFO:tasks.workunit.client.1.smithi193.stdout:8/409: stat d6/d1c/d1b/d29/c51 0 2024-05-09T06:38:21.705 INFO:tasks.workunit.client.1.smithi193.stdout:8/410: stat d6/d1c/f12 0 2024-05-09T06:38:21.708 INFO:tasks.workunit.client.1.smithi193.stdout:2/448: write db/f21 [4509684,66615] 0 2024-05-09T06:38:21.709 INFO:tasks.workunit.client.0.smithi063.stdout:0/809: creat dd/d1a/d2c/d51/d72/d9e/d23/dbb/ffe x:0 0 0 2024-05-09T06:38:21.709 INFO:tasks.workunit.client.1.smithi193.stdout:2/449: readlink db/d25/d38/d5b/l5f 0 2024-05-09T06:38:21.710 INFO:tasks.workunit.client.0.smithi063.stdout:0/810: chown dd/d1a/d2c/f92 10086762 1 2024-05-09T06:38:21.710 INFO:tasks.workunit.client.1.smithi193.stdout:2/450: readlink db/d25/d38/d5b/l74 0 2024-05-09T06:38:21.711 INFO:tasks.workunit.client.0.smithi063.stdout:0/811: stat dd/d1a/d2c/d51/d72/d9e/d23/d46/c98 0 2024-05-09T06:38:21.712 INFO:tasks.workunit.client.0.smithi063.stdout:8/859: unlink d9/da/d34/f87 0 2024-05-09T06:38:21.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/799: mknod d2/d20/d26/c102 0 2024-05-09T06:38:21.714 INFO:tasks.workunit.client.1.smithi193.stdout:2/451: dwrite db/d4b/f71 [0,4194304] 0 2024-05-09T06:38:21.715 INFO:tasks.workunit.client.1.smithi193.stdout:0/393: creat d18/d4c/d47/d5e/f7e x:0 0 0 2024-05-09T06:38:21.721 INFO:tasks.workunit.client.0.smithi063.stdout:6/800: dwrite d2/d1f/d6c/f7d [0,4194304] 0 2024-05-09T06:38:21.724 INFO:tasks.workunit.client.0.smithi063.stdout:5/763: dread d8/d65/f80 [0,4194304] 0 2024-05-09T06:38:21.728 INFO:tasks.workunit.client.1.smithi193.stdout:2/452: unlink db/d37/d6d/d80/l83 0 2024-05-09T06:38:21.729 INFO:tasks.workunit.client.0.smithi063.stdout:4/768: dwrite d7/df/d77/f8e [0,4194304] 0 2024-05-09T06:38:21.730 INFO:tasks.workunit.client.0.smithi063.stdout:9/782: creat d0/d43/f111 x:0 0 0 2024-05-09T06:38:21.731 INFO:tasks.workunit.client.0.smithi063.stdout:3/795: mknod d7/d70/d98/c10f 0 2024-05-09T06:38:21.732 INFO:tasks.workunit.client.0.smithi063.stdout:9/783: readlink d0/d4/d26/d39/d48/ld0 0 2024-05-09T06:38:21.736 INFO:tasks.workunit.client.0.smithi063.stdout:0/812: mknod dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4/de1/cff 0 2024-05-09T06:38:21.742 INFO:tasks.workunit.client.1.smithi193.stdout:0/394: symlink d18/d27/d7d/l7f 0 2024-05-09T06:38:21.744 INFO:tasks.workunit.client.1.smithi193.stdout:3/380: creat d1/d2d/f8c x:0 0 0 2024-05-09T06:38:21.747 INFO:tasks.workunit.client.1.smithi193.stdout:9/498: truncate dd/df/d31/d7e/d37/f4d 641027 0 2024-05-09T06:38:21.747 INFO:tasks.workunit.client.1.smithi193.stdout:8/411: dread d6/d1c/f14 [0,4194304] 0 2024-05-09T06:38:21.748 INFO:tasks.workunit.client.0.smithi063.stdout:3/796: dread d7/da/d2b/d66/d74/f92 [0,4194304] 0 2024-05-09T06:38:21.749 INFO:tasks.workunit.client.1.smithi193.stdout:0/395: creat d18/d4c/d47/d5e/f80 x:0 0 0 2024-05-09T06:38:21.750 INFO:tasks.workunit.client.0.smithi063.stdout:8/860: write d9/d37/f67 [391703,36002] 0 2024-05-09T06:38:21.751 INFO:tasks.workunit.client.1.smithi193.stdout:3/381: dread - d1/db/f3f zero size 2024-05-09T06:38:21.751 INFO:tasks.workunit.client.0.smithi063.stdout:1/733: dwrite d9/d2a/d35/d1e/f42 [0,4194304] 0 2024-05-09T06:38:21.753 INFO:tasks.workunit.client.1.smithi193.stdout:2/453: rmdir db/d25/d38/d6a 0 2024-05-09T06:38:21.755 INFO:tasks.workunit.client.0.smithi063.stdout:0/813: mkdir dd/d1a/d2c/d51/d100 0 2024-05-09T06:38:21.756 INFO:tasks.workunit.client.0.smithi063.stdout:0/814: fsync dd/f6b 0 2024-05-09T06:38:21.758 INFO:tasks.workunit.client.0.smithi063.stdout:2/823: getdents dc/d6e 0 2024-05-09T06:38:21.758 INFO:tasks.workunit.client.0.smithi063.stdout:1/734: dread d9/d2a/d35/d1e/f42 [0,4194304] 0 2024-05-09T06:38:21.758 INFO:tasks.workunit.client.0.smithi063.stdout:1/735: chown d9/d2a/d3c/d4c 374613 1 2024-05-09T06:38:21.759 INFO:tasks.workunit.client.0.smithi063.stdout:1/736: chown d9/d2a/d3c/d4a/db8/ddf 25444 1 2024-05-09T06:38:21.764 INFO:tasks.workunit.client.1.smithi193.stdout:3/382: rename d1/db/f34 to d1/db/d2f/d84/f8d 0 2024-05-09T06:38:21.764 INFO:tasks.workunit.client.1.smithi193.stdout:8/412: dwrite d6/d1c/d27/f43 [0,4194304] 0 2024-05-09T06:38:21.767 INFO:tasks.workunit.client.1.smithi193.stdout:8/413: write d6/d5b/f64 [31872,124847] 0 2024-05-09T06:38:21.769 INFO:tasks.workunit.client.1.smithi193.stdout:9/499: dread dd/d1e/d1f/d43/d5b/f5d [0,4194304] 0 2024-05-09T06:38:21.772 INFO:tasks.workunit.client.0.smithi063.stdout:9/784: write d0/d4/d57/f64 [787862,27651] 0 2024-05-09T06:38:21.774 INFO:tasks.workunit.client.0.smithi063.stdout:7/753: rename d0/f81 to d0/d15/d49/d54/f104 0 2024-05-09T06:38:21.775 INFO:tasks.workunit.client.0.smithi063.stdout:7/754: readlink d0/dcb/lf4 0 2024-05-09T06:38:21.780 INFO:tasks.workunit.client.0.smithi063.stdout:7/755: dwrite d0/d15/d4a/d50/f1a [0,4194304] 0 2024-05-09T06:38:21.782 INFO:tasks.workunit.client.1.smithi193.stdout:3/383: mknod d1/db/d2f/d78/d86/c8e 0 2024-05-09T06:38:21.783 INFO:tasks.workunit.client.0.smithi063.stdout:0/815: dread - dd/d60/d44/d90/fb1 zero size 2024-05-09T06:38:21.786 INFO:tasks.workunit.client.1.smithi193.stdout:2/454: mkdir db/d9b 0 2024-05-09T06:38:21.788 INFO:tasks.workunit.client.0.smithi063.stdout:1/737: mknod d9/d2a/d3c/d4a/db8/ddf/cfc 0 2024-05-09T06:38:21.794 INFO:tasks.workunit.client.1.smithi193.stdout:3/384: symlink d1/db/d2f/d78/d86/l8f 0 2024-05-09T06:38:21.797 INFO:tasks.workunit.client.0.smithi063.stdout:5/764: rename d8/d17/dfa to d8/d19/d30/d74/d77/dff 0 2024-05-09T06:38:21.799 INFO:tasks.workunit.client.1.smithi193.stdout:2/455: creat db/d25/d38/d40/f9c x:0 0 0 2024-05-09T06:38:21.800 INFO:tasks.workunit.client.1.smithi193.stdout:9/500: mknod dd/ca7 0 2024-05-09T06:38:21.802 INFO:tasks.workunit.client.1.smithi193.stdout:0/396: getdents d18/d4c 0 2024-05-09T06:38:21.804 INFO:tasks.workunit.client.0.smithi063.stdout:7/756: fsync d0/d15/f58 0 2024-05-09T06:38:21.804 INFO:tasks.workunit.client.0.smithi063.stdout:7/757: chown d0/d15 3922 1 2024-05-09T06:38:21.805 INFO:tasks.workunit.client.0.smithi063.stdout:0/816: creat dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/f101 x:0 0 0 2024-05-09T06:38:21.806 INFO:tasks.workunit.client.1.smithi193.stdout:2/456: creat db/d25/d38/f9d x:0 0 0 2024-05-09T06:38:21.807 INFO:tasks.workunit.client.0.smithi063.stdout:6/801: getdents d2/d4/d7c/dfc 0 2024-05-09T06:38:21.809 INFO:tasks.workunit.client.1.smithi193.stdout:9/501: dwrite dd/df/d15/f96 [0,4194304] 0 2024-05-09T06:38:21.809 INFO:tasks.workunit.client.1.smithi193.stdout:0/397: mknod d18/d4c/d5f/c81 0 2024-05-09T06:38:21.810 INFO:tasks.workunit.client.1.smithi193.stdout:8/414: getdents d6/d1c/d18/d2f/d40 0 2024-05-09T06:38:21.812 INFO:tasks.workunit.client.0.smithi063.stdout:4/769: rename d7/da3/d8b/fcd to d7/da3/d28/d31/d32/d34/ff5 0 2024-05-09T06:38:21.813 INFO:tasks.workunit.client.0.smithi063.stdout:4/770: readlink d7/da3/l5d 0 2024-05-09T06:38:21.814 INFO:tasks.workunit.client.0.smithi063.stdout:9/785: mknod d0/d4/d26/d39/d48/d80/d84/dc2/d10f/c112 0 2024-05-09T06:38:21.816 INFO:tasks.workunit.client.1.smithi193.stdout:2/457: write f8 [5726548,130984] 0 2024-05-09T06:38:21.817 INFO:tasks.workunit.client.1.smithi193.stdout:3/385: creat d1/db/f90 x:0 0 0 2024-05-09T06:38:21.817 INFO:tasks.workunit.client.1.smithi193.stdout:2/458: write f8 [5588006,98110] 0 2024-05-09T06:38:21.821 INFO:tasks.workunit.client.0.smithi063.stdout:1/738: symlink d9/d2a/d2e/df4/lfd 0 2024-05-09T06:38:21.822 INFO:tasks.workunit.client.0.smithi063.stdout:1/739: write d9/d2a/d3c/d4c/d83/d88/fcd [3347991,128828] 0 2024-05-09T06:38:21.825 INFO:tasks.workunit.client.1.smithi193.stdout:2/459: dwrite db/d25/d38/d61/f70 [0,4194304] 0 2024-05-09T06:38:21.827 INFO:tasks.workunit.client.0.smithi063.stdout:4/771: symlink d7/df/d57/lf6 0 2024-05-09T06:38:21.829 INFO:tasks.workunit.client.0.smithi063.stdout:9/786: read d0/d8/f69 [852934,114650] 0 2024-05-09T06:38:21.832 INFO:tasks.workunit.client.0.smithi063.stdout:1/740: rmdir d9/d2a/d35/d1e/d9d/d3f/d59/d62 39 2024-05-09T06:38:21.833 INFO:tasks.workunit.client.0.smithi063.stdout:1/741: write d9/d2a/d39/ff5 [275328,14659] 0 2024-05-09T06:38:21.833 INFO:tasks.workunit.client.1.smithi193.stdout:0/398: dread d18/d4c/f45 [0,4194304] 0 2024-05-09T06:38:21.834 INFO:tasks.workunit.client.0.smithi063.stdout:5/765: rename d8/d19/d30/f61 to d8/dc8/f100 0 2024-05-09T06:38:21.835 INFO:tasks.workunit.client.1.smithi193.stdout:0/399: write d18/d4c/d47/d59/d76/f77 [4328253,28302] 0 2024-05-09T06:38:21.841 INFO:tasks.workunit.client.1.smithi193.stdout:3/386: getdents d1/d2d/d51 0 2024-05-09T06:38:21.849 INFO:tasks.workunit.client.0.smithi063.stdout:9/787: dwrite d0/d4/d57/fe8 [0,4194304] 0 2024-05-09T06:38:21.852 INFO:tasks.workunit.client.0.smithi063.stdout:5/766: mknod d8/d35/dd6/c101 0 2024-05-09T06:38:21.852 INFO:tasks.workunit.client.0.smithi063.stdout:8/861: write d9/da/d18/f23 [2137265,109121] 0 2024-05-09T06:38:21.854 INFO:tasks.workunit.client.0.smithi063.stdout:1/742: rename d9/d76 to d9/d46/dfe 0 2024-05-09T06:38:21.855 INFO:tasks.workunit.client.1.smithi193.stdout:0/400: dread d18/f4e [0,4194304] 0 2024-05-09T06:38:21.860 INFO:tasks.workunit.client.0.smithi063.stdout:8/862: symlink d9/d41/l132 0 2024-05-09T06:38:21.860 INFO:tasks.workunit.client.0.smithi063.stdout:9/788: dwrite d0/d4/d7d/d105/d110/f21 [4194304,4194304] 0 2024-05-09T06:38:21.863 INFO:tasks.workunit.client.1.smithi193.stdout:0/401: mkdir d18/d65/d82 0 2024-05-09T06:38:21.863 INFO:tasks.workunit.client.1.smithi193.stdout:0/402: chown d18/d4c 28633 1 2024-05-09T06:38:21.863 INFO:tasks.workunit.client.0.smithi063.stdout:9/789: write d0/d4/d26/d39/d3e/fc6 [1150831,53449] 0 2024-05-09T06:38:21.865 INFO:tasks.workunit.client.1.smithi193.stdout:0/403: truncate d18/d4c/d47/d54/f68 5078251 0 2024-05-09T06:38:21.867 INFO:tasks.workunit.client.0.smithi063.stdout:3/797: write d7/d22/dfc/f107 [3813842,55198] 0 2024-05-09T06:38:21.868 INFO:tasks.workunit.client.0.smithi063.stdout:2/824: dwrite dc/dd/f64 [0,4194304] 0 2024-05-09T06:38:21.870 INFO:tasks.workunit.client.0.smithi063.stdout:8/863: rename d9/da/d34/f4b to d9/d37/f133 0 2024-05-09T06:38:21.871 INFO:tasks.workunit.client.0.smithi063.stdout:2/825: truncate dc/dd/d45/fe3 956596 0 2024-05-09T06:38:21.873 INFO:tasks.workunit.client.1.smithi193.stdout:0/404: rename d18/d1d/f23 to d18/d4c/d47/d5e/d70/f83 0 2024-05-09T06:38:21.875 INFO:tasks.workunit.client.0.smithi063.stdout:5/767: creat d8/d19/d30/f102 x:0 0 0 2024-05-09T06:38:21.878 INFO:tasks.workunit.client.0.smithi063.stdout:9/790: mkdir d0/d8/da6/db0/d113 0 2024-05-09T06:38:21.879 INFO:tasks.workunit.client.1.smithi193.stdout:9/502: truncate dd/df/d31/d7e/d37/f4d 114813 0 2024-05-09T06:38:21.880 INFO:tasks.workunit.client.1.smithi193.stdout:0/405: dwrite d18/d4c/d5f/f6a [0,4194304] 0 2024-05-09T06:38:21.888 INFO:tasks.workunit.client.0.smithi063.stdout:2/826: rmdir dc/d36/d43/d46/d63/d79 39 2024-05-09T06:38:21.888 INFO:tasks.workunit.client.1.smithi193.stdout:9/503: rmdir dd/d1e/d1f/d48/d55/d41 39 2024-05-09T06:38:21.889 INFO:tasks.workunit.client.1.smithi193.stdout:9/504: write dd/df/d31/d7e/d37/f83 [549363,26730] 0 2024-05-09T06:38:21.890 INFO:tasks.workunit.client.0.smithi063.stdout:9/791: truncate d0/d4/d26/f34 5178433 0 2024-05-09T06:38:21.890 INFO:tasks.workunit.client.1.smithi193.stdout:9/505: fdatasync dd/df/d15/f96 0 2024-05-09T06:38:21.891 INFO:tasks.workunit.client.0.smithi063.stdout:9/792: chown d0/d4/d26/d39/d48/d80 3 1 2024-05-09T06:38:21.892 INFO:tasks.workunit.client.0.smithi063.stdout:3/798: mknod d7/da/d2b/d66/d74/d7e/dfa/c110 0 2024-05-09T06:38:21.892 INFO:tasks.workunit.client.1.smithi193.stdout:2/460: rmdir db/d25/d38 39 2024-05-09T06:38:21.892 INFO:tasks.workunit.client.1.smithi193.stdout:9/506: dread - dd/df/d31/d7e/d37/f7c zero size 2024-05-09T06:38:21.893 INFO:tasks.workunit.client.1.smithi193.stdout:9/507: write dd/df/d31/d7e/d71/fa0 [756139,123350] 0 2024-05-09T06:38:21.895 INFO:tasks.workunit.client.1.smithi193.stdout:9/508: truncate dd/df/d31/d7e/d37/f7c 53106 0 2024-05-09T06:38:21.902 INFO:tasks.workunit.client.1.smithi193.stdout:9/509: getdents dd/df/d15/d98/da4 0 2024-05-09T06:38:21.906 INFO:tasks.workunit.client.0.smithi063.stdout:7/758: dwrite d0/d15/d49/f5c [0,4194304] 0 2024-05-09T06:38:21.906 INFO:tasks.workunit.client.1.smithi193.stdout:2/461: link db/d25/d38/d40/d8b/d77/l9a db/d4b/l9e 0 2024-05-09T06:38:21.908 INFO:tasks.workunit.client.1.smithi193.stdout:8/415: fsync d6/d1c/d18/f74 0 2024-05-09T06:38:21.910 INFO:tasks.workunit.client.0.smithi063.stdout:0/817: dwrite dd/d60/d44/d90/fb1 [0,4194304] 0 2024-05-09T06:38:21.911 INFO:tasks.workunit.client.0.smithi063.stdout:4/772: dread d7/df/d35/f36 [0,4194304] 0 2024-05-09T06:38:21.913 INFO:tasks.workunit.client.0.smithi063.stdout:6/802: dwrite d2/d20/d26/f97 [0,4194304] 0 2024-05-09T06:38:21.915 INFO:tasks.workunit.client.0.smithi063.stdout:8/864: rename d9/l6a to d9/da/d53/l134 0 2024-05-09T06:38:21.916 INFO:tasks.workunit.client.1.smithi193.stdout:9/510: creat dd/d1e/d1f/d48/d55/fa8 x:0 0 0 2024-05-09T06:38:21.917 INFO:tasks.workunit.client.1.smithi193.stdout:9/511: dread - dd/df/d15/f9e zero size 2024-05-09T06:38:21.918 INFO:tasks.workunit.client.1.smithi193.stdout:9/512: read dd/d1e/f27 [2188667,2886] 0 2024-05-09T06:38:21.921 INFO:tasks.workunit.client.1.smithi193.stdout:3/387: write d1/d2d/d51/f7e [222850,106004] 0 2024-05-09T06:38:21.923 INFO:tasks.workunit.client.1.smithi193.stdout:8/416: write d6/d1c/d18/d2f/f39 [2399882,123353] 0 2024-05-09T06:38:21.924 INFO:tasks.workunit.client.1.smithi193.stdout:8/417: stat d6/d1c/d18/d66/l67 0 2024-05-09T06:38:21.925 INFO:tasks.workunit.client.0.smithi063.stdout:5/768: link d8/d17/c70 d8/d19/d30/d74/c103 0 2024-05-09T06:38:21.932 INFO:tasks.workunit.client.0.smithi063.stdout:7/759: rmdir d0/d15/d49/d54/d9f/d97/d9b 39 2024-05-09T06:38:21.933 INFO:tasks.workunit.client.0.smithi063.stdout:0/818: dread dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0/fc5 [0,4194304] 0 2024-05-09T06:38:21.934 INFO:tasks.workunit.client.1.smithi193.stdout:9/513: rmdir dd/df/d31/d7e/d37 39 2024-05-09T06:38:21.934 INFO:tasks.workunit.client.0.smithi063.stdout:0/819: readlink dd/d1a/d50/l71 0 2024-05-09T06:38:21.935 INFO:tasks.workunit.client.0.smithi063.stdout:0/820: dread - dd/d1a/d2c/d51/d72/fca zero size 2024-05-09T06:38:21.936 INFO:tasks.workunit.client.0.smithi063.stdout:0/821: readlink dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/lfa 0 2024-05-09T06:38:21.938 INFO:tasks.workunit.client.0.smithi063.stdout:7/760: dread d0/d15/d49/d4f/d66/ddd/fdf [0,4194304] 0 2024-05-09T06:38:21.939 INFO:tasks.workunit.client.1.smithi193.stdout:8/418: creat d6/d1c/d1b/d29/f7c x:0 0 0 2024-05-09T06:38:21.942 INFO:tasks.workunit.client.0.smithi063.stdout:0/822: dwrite dd/d60/fd4 [0,4194304] 0 2024-05-09T06:38:21.945 INFO:tasks.workunit.client.0.smithi063.stdout:6/803: rmdir d2/df/d91/ddf 39 2024-05-09T06:38:21.945 INFO:tasks.workunit.client.0.smithi063.stdout:6/804: chown d2/d89/dd3 989470854 1 2024-05-09T06:38:21.948 INFO:tasks.workunit.client.0.smithi063.stdout:8/865: creat d9/d41/dec/db8/f135 x:0 0 0 2024-05-09T06:38:21.949 INFO:tasks.workunit.client.0.smithi063.stdout:2/827: write dc/dd/f13 [2426176,42790] 0 2024-05-09T06:38:21.950 INFO:tasks.workunit.client.0.smithi063.stdout:8/866: write d9/da/d34/f10d [935901,83369] 0 2024-05-09T06:38:21.952 INFO:tasks.workunit.client.1.smithi193.stdout:8/419: creat d6/d1c/d27/f7d x:0 0 0 2024-05-09T06:38:21.953 INFO:tasks.workunit.client.0.smithi063.stdout:9/793: link d0/d4/d26/d39/d91/fa7 d0/d4/d26/d39/d91/f114 0 2024-05-09T06:38:21.956 INFO:tasks.workunit.client.0.smithi063.stdout:0/823: dread dd/f6b [0,4194304] 0 2024-05-09T06:38:21.958 INFO:tasks.workunit.client.0.smithi063.stdout:1/743: write d9/d2a/d3c/d4c/f78 [474462,62240] 0 2024-05-09T06:38:21.958 INFO:tasks.workunit.client.0.smithi063.stdout:3/799: dwrite d7/d4c/f69 [0,4194304] 0 2024-05-09T06:38:21.959 INFO:tasks.workunit.client.1.smithi193.stdout:9/514: mknod dd/d1e/d1f/d48/d55/d41/ca9 0 2024-05-09T06:38:21.960 INFO:tasks.workunit.client.0.smithi063.stdout:3/800: chown d7/da/d2b/d66/dc0/da4 23109 1 2024-05-09T06:38:21.960 INFO:tasks.workunit.client.1.smithi193.stdout:8/420: dwrite d6/d1c/d18/f26 [0,4194304] 0 2024-05-09T06:38:21.962 INFO:tasks.workunit.client.1.smithi193.stdout:8/421: write d6/f23 [1834821,87053] 0 2024-05-09T06:38:21.964 INFO:tasks.workunit.client.1.smithi193.stdout:8/422: write d6/d1c/f34 [676185,105601] 0 2024-05-09T06:38:21.967 INFO:tasks.workunit.client.1.smithi193.stdout:2/462: dwrite db/f28 [0,4194304] 0 2024-05-09T06:38:21.971 INFO:tasks.workunit.client.1.smithi193.stdout:2/463: dread - db/d37/d6d/d80/f46 zero size 2024-05-09T06:38:21.973 INFO:tasks.workunit.client.0.smithi063.stdout:6/805: mknod d2/d4/d7c/d7e/c103 0 2024-05-09T06:38:21.976 INFO:tasks.workunit.client.0.smithi063.stdout:2/828: stat dc/d36/d43/d46/d63/d79/d7d/d8d/l95 0 2024-05-09T06:38:21.978 INFO:tasks.workunit.client.1.smithi193.stdout:3/388: dwrite d1/f24 [0,4194304] 0 2024-05-09T06:38:21.979 INFO:tasks.workunit.client.1.smithi193.stdout:8/423: dwrite d6/d1c/d53/f63 [0,4194304] 0 2024-05-09T06:38:21.981 INFO:tasks.workunit.client.0.smithi063.stdout:8/867: unlink d9/d41/dec/da1/ffd 0 2024-05-09T06:38:21.982 INFO:tasks.workunit.client.0.smithi063.stdout:8/868: write d9/da/d18/d24/f62 [5212189,87762] 0 2024-05-09T06:38:21.983 INFO:tasks.workunit.client.1.smithi193.stdout:9/515: fdatasync dd/df/d15/d8c/f2a 0 2024-05-09T06:38:21.986 INFO:tasks.workunit.client.0.smithi063.stdout:9/794: mknod d0/d4/d26/d39/d91/c115 0 2024-05-09T06:38:21.987 INFO:tasks.workunit.client.0.smithi063.stdout:4/773: write d7/da3/d28/d31/d32/f51 [476584,35984] 0 2024-05-09T06:38:21.989 INFO:tasks.workunit.client.1.smithi193.stdout:8/424: dwrite d6/f23 [0,4194304] 0 2024-05-09T06:38:21.993 INFO:tasks.workunit.client.0.smithi063.stdout:9/795: dread d0/d4/f37 [0,4194304] 0 2024-05-09T06:38:21.999 INFO:tasks.workunit.client.1.smithi193.stdout:8/425: dwrite d6/d1c/d18/f74 [0,4194304] 0 2024-05-09T06:38:21.999 INFO:tasks.workunit.client.0.smithi063.stdout:7/761: readlink d0/d15/d49/d54/d9f/d97/lf2 0 2024-05-09T06:38:22.005 INFO:tasks.workunit.client.0.smithi063.stdout:2/829: truncate dc/f99 222393 0 2024-05-09T06:38:22.008 INFO:tasks.workunit.client.1.smithi193.stdout:8/426: mkdir d6/d1c/d18/d2f/d40/d55/d7e 0 2024-05-09T06:38:22.009 INFO:tasks.workunit.client.0.smithi063.stdout:0/824: write dd/d60/fa0 [4801618,56764] 0 2024-05-09T06:38:22.010 INFO:tasks.workunit.client.0.smithi063.stdout:8/869: creat d9/d41/dec/dc5/f136 x:0 0 0 2024-05-09T06:38:22.010 INFO:tasks.workunit.client.1.smithi193.stdout:8/427: truncate d6/d1c/d18/d2f/f39 4675826 0 2024-05-09T06:38:22.010 INFO:tasks.workunit.client.1.smithi193.stdout:2/464: mkdir db/df/d87/d9f 0 2024-05-09T06:38:22.013 INFO:tasks.workunit.client.0.smithi063.stdout:4/774: rmdir d7/da3/d28/d31/d32 39 2024-05-09T06:38:22.014 INFO:tasks.workunit.client.0.smithi063.stdout:9/796: rename d0/d8/d2d/c4c to d0/d4/d57/d60/df5/c116 0 2024-05-09T06:38:22.017 INFO:tasks.workunit.client.1.smithi193.stdout:8/428: symlink d6/d5b/l7f 0 2024-05-09T06:38:22.017 INFO:tasks.workunit.client.1.smithi193.stdout:9/516: truncate dd/df/f6e 842849 0 2024-05-09T06:38:22.018 INFO:tasks.workunit.client.0.smithi063.stdout:1/744: symlink d9/d2a/d35/d1e/lff 0 2024-05-09T06:38:22.021 INFO:tasks.workunit.client.0.smithi063.stdout:3/801: link d7/d22/d28/fec d7/da/d2b/d66/dc0/f111 0 2024-05-09T06:38:22.022 INFO:tasks.workunit.client.1.smithi193.stdout:8/429: symlink d6/d1c/d27/l80 0 2024-05-09T06:38:22.023 INFO:tasks.workunit.client.0.smithi063.stdout:7/762: dread - d0/d15/d49/d4f/d66/d67/d6b/f85 zero size 2024-05-09T06:38:22.030 INFO:tasks.workunit.client.1.smithi193.stdout:2/465: write db/f45 [4807609,12865] 0 2024-05-09T06:38:22.030 INFO:tasks.workunit.client.1.smithi193.stdout:9/517: dwrite dd/df/d31/d7e/d37/f83 [0,4194304] 0 2024-05-09T06:38:22.031 INFO:tasks.workunit.client.0.smithi063.stdout:5/769: getdents d8/d53 0 2024-05-09T06:38:22.038 INFO:tasks.workunit.client.0.smithi063.stdout:6/806: truncate d2/d1f/d29/fb1 2937277 0 2024-05-09T06:38:22.039 INFO:tasks.workunit.client.0.smithi063.stdout:6/807: chown d2/d4/c13 25 1 2024-05-09T06:38:22.046 INFO:tasks.workunit.client.0.smithi063.stdout:4/775: rmdir d7 39 2024-05-09T06:38:22.047 INFO:tasks.workunit.client.0.smithi063.stdout:0/825: write dd/d1a/d2c/d51/d72/d9e/d23/d31/fb9 [921875,31349] 0 2024-05-09T06:38:22.047 INFO:tasks.workunit.client.1.smithi193.stdout:8/430: link d6/d1c/f14 d6/d1c/d53/f81 0 2024-05-09T06:38:22.048 INFO:tasks.workunit.client.1.smithi193.stdout:8/431: fsync d6/d1c/d27/f65 0 2024-05-09T06:38:22.049 INFO:tasks.workunit.client.0.smithi063.stdout:0/826: truncate dd/d1a/d2c/d51/d72/fdb 438314 0 2024-05-09T06:38:22.049 INFO:tasks.workunit.client.1.smithi193.stdout:8/432: chown d6/d1c/d18/f26 21770 1 2024-05-09T06:38:22.054 INFO:tasks.workunit.client.0.smithi063.stdout:0/827: write dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/ff8 [735511,42075] 0 2024-05-09T06:38:22.054 INFO:tasks.workunit.client.1.smithi193.stdout:2/466: write db/f1e [1852209,111844] 0 2024-05-09T06:38:22.055 INFO:tasks.workunit.client.1.smithi193.stdout:2/467: chown f8 42905702 1 2024-05-09T06:38:22.058 INFO:tasks.workunit.client.0.smithi063.stdout:7/763: creat d0/d15/d49/d4f/d66/d67/d6b/f105 x:0 0 0 2024-05-09T06:38:22.058 INFO:tasks.workunit.client.1.smithi193.stdout:9/518: dwrite dd/df/d31/d7e/f60 [0,4194304] 0 2024-05-09T06:38:22.059 INFO:tasks.workunit.client.1.smithi193.stdout:2/468: write db/f44 [158131,39285] 0 2024-05-09T06:38:22.061 INFO:tasks.workunit.client.0.smithi063.stdout:6/808: dread d2/df/f14 [0,4194304] 0 2024-05-09T06:38:22.062 INFO:tasks.workunit.client.0.smithi063.stdout:9/797: dwrite d0/d4/d57/dba/dd5/fdb [0,4194304] 0 2024-05-09T06:38:22.063 INFO:tasks.workunit.client.0.smithi063.stdout:7/764: write d0/d15/d49/d4f/d66/d67/d6b/f105 [96505,50978] 0 2024-05-09T06:38:22.070 INFO:tasks.workunit.client.0.smithi063.stdout:5/770: fdatasync d8/d17/f1c 0 2024-05-09T06:38:22.070 INFO:tasks.workunit.client.0.smithi063.stdout:9/798: chown d0/c2 1323009969 1 2024-05-09T06:38:22.071 INFO:tasks.workunit.client.0.smithi063.stdout:6/809: mknod d2/d1f/d29/de2/c104 0 2024-05-09T06:38:22.071 INFO:tasks.workunit.client.0.smithi063.stdout:8/870: link d9/d37/f5a d9/da/d51/dc2/d127/f137 0 2024-05-09T06:38:22.073 INFO:tasks.workunit.client.0.smithi063.stdout:0/828: mknod dd/d1a/d2c/d51/d100/c102 0 2024-05-09T06:38:22.075 INFO:tasks.workunit.client.0.smithi063.stdout:1/745: link d9/d2a/d3c/d4c/d83/d88/dc2/ced d9/d2a/d3c/d4c/d83/d88/dfb/c100 0 2024-05-09T06:38:22.075 INFO:tasks.workunit.client.0.smithi063.stdout:4/776: readlink d7/da3/d28/d31/d32/d34/db4/ld6 0 2024-05-09T06:38:22.077 INFO:tasks.workunit.client.1.smithi193.stdout:8/433: write d6/f20 [2029028,31979] 0 2024-05-09T06:38:22.080 INFO:tasks.workunit.client.0.smithi063.stdout:9/799: dread - d0/d4/d26/d39/d48/d80/d84/faa zero size 2024-05-09T06:38:22.081 INFO:tasks.workunit.client.0.smithi063.stdout:6/810: dwrite d2/f38 [0,4194304] 0 2024-05-09T06:38:22.082 INFO:tasks.workunit.client.0.smithi063.stdout:6/811: stat d2/d20/d4a/l77 0 2024-05-09T06:38:22.083 INFO:tasks.workunit.client.0.smithi063.stdout:3/802: link d7/da/d21/lc3 d7/d70/d98/d5b/l112 0 2024-05-09T06:38:22.083 INFO:tasks.workunit.client.1.smithi193.stdout:8/434: dwrite d6/f20 [0,4194304] 0 2024-05-09T06:38:22.085 INFO:tasks.workunit.client.0.smithi063.stdout:4/777: dread - d7/da3/d28/d31/d71/d98/fef zero size 2024-05-09T06:38:22.086 INFO:tasks.workunit.client.0.smithi063.stdout:1/746: symlink d9/d2a/d3c/d4c/d83/l101 0 2024-05-09T06:38:22.088 INFO:tasks.workunit.client.1.smithi193.stdout:8/435: symlink d6/d1c/d18/d2f/d40/d5e/l82 0 2024-05-09T06:38:22.090 INFO:tasks.workunit.client.0.smithi063.stdout:6/812: creat d2/d1f/d6c/f105 x:0 0 0 2024-05-09T06:38:22.090 INFO:tasks.workunit.client.1.smithi193.stdout:8/436: write d6/d1c/d18/d2f/f39 [3532111,125248] 0 2024-05-09T06:38:22.094 INFO:tasks.workunit.client.0.smithi063.stdout:3/803: symlink d7/da/d2b/d66/d74/d7e/dfa/l113 0 2024-05-09T06:38:22.095 INFO:tasks.workunit.client.0.smithi063.stdout:3/804: chown d7/da/d2b/la5 63 1 2024-05-09T06:38:22.095 INFO:tasks.workunit.client.0.smithi063.stdout:3/805: chown d7/l105 25153 1 2024-05-09T06:38:22.100 INFO:tasks.workunit.client.0.smithi063.stdout:8/871: creat d9/da/d18/dc7/f138 x:0 0 0 2024-05-09T06:38:22.101 INFO:tasks.workunit.client.0.smithi063.stdout:9/800: symlink d0/d8/l117 0 2024-05-09T06:38:22.102 INFO:tasks.workunit.client.0.smithi063.stdout:6/813: mknod d2/d1f/d29/de2/c106 0 2024-05-09T06:38:22.103 INFO:tasks.workunit.client.0.smithi063.stdout:5/771: link d8/d35/l4d d8/d19/d30/db1/l104 0 2024-05-09T06:38:22.104 INFO:tasks.workunit.client.0.smithi063.stdout:0/829: truncate dd/d60/fd4 3977487 0 2024-05-09T06:38:22.105 INFO:tasks.workunit.client.0.smithi063.stdout:3/806: mkdir d7/da/d2b/d66/d74/d7e/d114 0 2024-05-09T06:38:22.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/872: rename d9/da/d18/d46 to d9/da/d45/d114/d139 0 2024-05-09T06:38:22.111 INFO:tasks.workunit.client.1.smithi193.stdout:8/437: truncate d6/d1c/d27/f43 2116408 0 2024-05-09T06:38:22.111 INFO:tasks.workunit.client.0.smithi063.stdout:7/765: dread d0/d15/d4a/d50/f3c [0,4194304] 0 2024-05-09T06:38:22.111 INFO:tasks.workunit.client.0.smithi063.stdout:8/873: chown d9/da/d45/cf0 932326535 1 2024-05-09T06:38:22.112 INFO:tasks.workunit.client.0.smithi063.stdout:5/772: write d8/dc8/f100 [4936470,110465] 0 2024-05-09T06:38:22.112 INFO:tasks.workunit.client.0.smithi063.stdout:7/766: readlink d0/d15/d49/d4f/d66/ddd/le2 0 2024-05-09T06:38:22.114 INFO:tasks.workunit.client.0.smithi063.stdout:6/814: creat d2/df/d59/d99/d9f/f107 x:0 0 0 2024-05-09T06:38:22.115 INFO:tasks.workunit.client.1.smithi193.stdout:8/438: rename d6/d1c/d27/l80 to d6/d1c/d1b/d29/d57/l83 0 2024-05-09T06:38:22.116 INFO:tasks.workunit.client.0.smithi063.stdout:1/747: link d9/d2a/d39/d5d/f9a d9/d2a/d39/f102 0 2024-05-09T06:38:22.117 INFO:tasks.workunit.client.0.smithi063.stdout:5/773: truncate d8/d17/da2/fe1 2000720 0 2024-05-09T06:38:22.117 INFO:tasks.workunit.client.0.smithi063.stdout:4/778: dread d7/da3/d28/d31/d32/f8c [0,4194304] 0 2024-05-09T06:38:22.118 INFO:tasks.workunit.client.0.smithi063.stdout:6/815: chown d2/d4/d96/cd4 2 1 2024-05-09T06:38:22.119 INFO:tasks.workunit.client.0.smithi063.stdout:4/779: chown d7/da3/d28/d31/d32/d34 2102 1 2024-05-09T06:38:22.121 INFO:tasks.workunit.client.0.smithi063.stdout:7/767: mkdir d0/d15/d49/d54/d9f/d106 0 2024-05-09T06:38:22.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/816: unlink d2/d89/dd3/cf2 0 2024-05-09T06:38:22.127 INFO:tasks.workunit.client.0.smithi063.stdout:0/830: dread dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fa6 [0,4194304] 0 2024-05-09T06:38:22.128 INFO:tasks.workunit.client.0.smithi063.stdout:3/807: dread d7/da/d2b/d66/dc0/fa9 [0,4194304] 0 2024-05-09T06:38:22.131 INFO:tasks.workunit.client.0.smithi063.stdout:8/874: write d9/da/d53/fdf [89769,103227] 0 2024-05-09T06:38:22.132 INFO:tasks.workunit.client.0.smithi063.stdout:8/875: readlink d9/d41/dec/lc6 0 2024-05-09T06:38:22.133 INFO:tasks.workunit.client.0.smithi063.stdout:4/780: mkdir d7/da3/d28/d31/d7b/df7 0 2024-05-09T06:38:22.134 INFO:tasks.workunit.client.0.smithi063.stdout:7/768: link d0/d15/d49/f5c d0/d15/d49/d4f/d66/f107 0 2024-05-09T06:38:22.134 INFO:tasks.workunit.client.0.smithi063.stdout:4/781: truncate d7/da3/d28/d31/fdb 949768 0 2024-05-09T06:38:22.136 INFO:tasks.workunit.client.0.smithi063.stdout:1/748: creat d9/d2a/d35/d1e/d9d/f103 x:0 0 0 2024-05-09T06:38:22.137 INFO:tasks.workunit.client.0.smithi063.stdout:5/774: truncate d8/d17/fc1 3086560 0 2024-05-09T06:38:22.139 INFO:tasks.workunit.client.0.smithi063.stdout:0/831: mknod dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/c103 0 2024-05-09T06:38:22.142 INFO:tasks.workunit.client.0.smithi063.stdout:1/749: creat d9/d2a/d35/d24/d68/f104 x:0 0 0 2024-05-09T06:38:22.142 INFO:tasks.workunit.client.0.smithi063.stdout:7/769: dwrite d0/d15/fd5 [0,4194304] 0 2024-05-09T06:38:22.145 INFO:tasks.workunit.client.0.smithi063.stdout:3/808: mknod d7/d70/d98/d2d/d36/db7/c115 0 2024-05-09T06:38:22.145 INFO:tasks.workunit.client.0.smithi063.stdout:0/832: read dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/fc3 [2346678,63899] 0 2024-05-09T06:38:22.147 INFO:tasks.workunit.client.0.smithi063.stdout:3/809: fsync d7/da/d21/d30/d44/d48/f53 0 2024-05-09T06:38:22.148 INFO:tasks.workunit.client.0.smithi063.stdout:5/775: mkdir d8/d19/d105 0 2024-05-09T06:38:22.151 INFO:tasks.workunit.client.0.smithi063.stdout:4/782: dread d7/da3/d28/d31/d71/dbb/fd7 [0,4194304] 0 2024-05-09T06:38:22.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/491: sync 2024-05-09T06:38:22.155 INFO:tasks.workunit.client.0.smithi063.stdout:0/833: mkdir dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0/d104 0 2024-05-09T06:38:22.155 INFO:tasks.workunit.client.0.smithi063.stdout:3/810: dwrite d7/d70/d98/d2d/ffb [0,4194304] 0 2024-05-09T06:38:22.156 INFO:tasks.workunit.client.0.smithi063.stdout:4/783: truncate d7/df/d35/d5a/dad/fc3 959243 0 2024-05-09T06:38:22.156 INFO:tasks.workunit.client.0.smithi063.stdout:6/817: truncate d2/d1f/d6c/fb7 3237624 0 2024-05-09T06:38:22.157 INFO:tasks.workunit.client.0.smithi063.stdout:0/834: readlink dd/d1a/d50/l71 0 2024-05-09T06:38:22.158 INFO:tasks.workunit.client.0.smithi063.stdout:3/811: chown d7/da/d21/d30/cdc 1856 1 2024-05-09T06:38:22.159 INFO:tasks.workunit.client.0.smithi063.stdout:4/784: write d7/df/d35/d5a/d60/f6e [852398,130106] 0 2024-05-09T06:38:22.161 INFO:tasks.workunit.client.0.smithi063.stdout:6/818: truncate d2/d4/d7c/fdb 320137 0 2024-05-09T06:38:22.164 INFO:tasks.workunit.client.0.smithi063.stdout:4/785: readlink d7/da3/l5d 0 2024-05-09T06:38:22.165 INFO:tasks.workunit.client.0.smithi063.stdout:8/876: dread d9/da/d51/f88 [0,4194304] 0 2024-05-09T06:38:22.167 INFO:tasks.workunit.client.0.smithi063.stdout:3/812: mkdir d7/d70/d98/d2d/d36/d116 0 2024-05-09T06:38:22.169 INFO:tasks.workunit.client.1.smithi193.stdout:4/492: link d1a/d22/c29 d1a/d24/d73/c96 0 2024-05-09T06:38:22.170 INFO:tasks.workunit.client.1.smithi193.stdout:4/493: dread - d1a/d6b/d60/f95 zero size 2024-05-09T06:38:22.171 INFO:tasks.workunit.client.0.smithi063.stdout:7/770: getdents d0/d15/d49/d54 0 2024-05-09T06:38:22.176 INFO:tasks.workunit.client.0.smithi063.stdout:1/750: dwrite d9/d2a/d35/d1e/d9d/d3f/d59/d62/d7e/fa4 [0,4194304] 0 2024-05-09T06:38:22.177 INFO:tasks.workunit.client.0.smithi063.stdout:4/786: unlink d7/d50/fc9 0 2024-05-09T06:38:22.178 INFO:tasks.workunit.client.0.smithi063.stdout:5/776: dread d8/d17/f3d [0,4194304] 0 2024-05-09T06:38:22.178 INFO:tasks.workunit.client.0.smithi063.stdout:4/787: readlink d7/d50/l52 0 2024-05-09T06:38:22.179 INFO:tasks.workunit.client.0.smithi063.stdout:3/813: mkdir d7/da/d2b/d66/dc0/da4/d117 0 2024-05-09T06:38:22.180 INFO:tasks.workunit.client.0.smithi063.stdout:0/835: write dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/f7b [1313247,50041] 0 2024-05-09T06:38:22.180 INFO:tasks.workunit.client.1.smithi193.stdout:4/494: link d1a/c7f d1a/d6d/c97 0 2024-05-09T06:38:22.181 INFO:tasks.workunit.client.0.smithi063.stdout:7/771: creat d0/d15/f108 x:0 0 0 2024-05-09T06:38:22.181 INFO:tasks.workunit.client.0.smithi063.stdout:3/814: chown d7/d70/f104 13 1 2024-05-09T06:38:22.182 INFO:tasks.workunit.client.1.smithi193.stdout:4/495: write d1a/d24/d73/d94/d7c/f86 [51283,57595] 0 2024-05-09T06:38:22.183 INFO:tasks.workunit.client.0.smithi063.stdout:7/772: write d0/d15/d49/d4f/d66/d67/ffd [232116,15629] 0 2024-05-09T06:38:22.183 INFO:tasks.workunit.client.1.smithi193.stdout:4/496: fdatasync d1a/d24/d73/d94/f8f 0 2024-05-09T06:38:22.184 INFO:tasks.workunit.client.0.smithi063.stdout:6/819: rename d2/d20/f7b to d2/d4/d7c/f108 0 2024-05-09T06:38:22.190 INFO:tasks.workunit.client.0.smithi063.stdout:3/815: dwrite d7/da/d21/d30/d44/d48/f53 [0,4194304] 0 2024-05-09T06:38:22.190 INFO:tasks.workunit.client.0.smithi063.stdout:4/788: creat d7/df/d35/d5a/dad/ff8 x:0 0 0 2024-05-09T06:38:22.193 INFO:tasks.workunit.client.0.smithi063.stdout:8/877: rename d9/da/d45/l12e to d9/da/d45/d114/d139/l13a 0 2024-05-09T06:38:22.194 INFO:tasks.workunit.client.0.smithi063.stdout:6/820: rmdir d2/d1f/d73/d88 39 2024-05-09T06:38:22.201 INFO:tasks.workunit.client.0.smithi063.stdout:4/789: creat d7/df/d57/ff9 x:0 0 0 2024-05-09T06:38:22.203 INFO:tasks.workunit.client.0.smithi063.stdout:4/790: write d7/da3/d28/d31/fdb [1877319,15866] 0 2024-05-09T06:38:22.204 INFO:tasks.workunit.client.0.smithi063.stdout:1/751: creat d9/d2a/d39/f105 x:0 0 0 2024-05-09T06:38:22.204 INFO:tasks.workunit.client.0.smithi063.stdout:3/816: dwrite d7/da/d2b/d66/dc0/fa9 [0,4194304] 0 2024-05-09T06:38:22.206 INFO:tasks.workunit.client.0.smithi063.stdout:8/878: readlink d9/da/d18/dc7/le3 0 2024-05-09T06:38:22.206 INFO:tasks.workunit.client.0.smithi063.stdout:2/830: sync 2024-05-09T06:38:22.207 INFO:tasks.workunit.client.1.smithi193.stdout:4/497: truncate f5 2846725 0 2024-05-09T06:38:22.207 INFO:tasks.workunit.client.1.smithi193.stdout:4/498: readlink l14 0 2024-05-09T06:38:22.207 INFO:tasks.workunit.client.1.smithi193.stdout:4/499: readlink d1a/d6d/l71 0 2024-05-09T06:38:22.209 INFO:tasks.workunit.client.0.smithi063.stdout:1/752: mknod d9/d2a/d3c/d4c/d83/d88/c106 0 2024-05-09T06:38:22.210 INFO:tasks.workunit.client.0.smithi063.stdout:3/817: mknod d7/d22/d28/dab/c118 0 2024-05-09T06:38:22.210 INFO:tasks.workunit.client.0.smithi063.stdout:7/773: write d0/d15/f58 [1655973,41132] 0 2024-05-09T06:38:22.213 INFO:tasks.workunit.client.0.smithi063.stdout:5/777: write d8/f27 [2832810,38574] 0 2024-05-09T06:38:22.214 INFO:tasks.workunit.client.0.smithi063.stdout:6/821: creat d2/df/d91/f109 x:0 0 0 2024-05-09T06:38:22.217 INFO:tasks.workunit.client.0.smithi063.stdout:8/879: mkdir d9/da/d18/d24/d71/d13b 0 2024-05-09T06:38:22.218 INFO:tasks.workunit.client.0.smithi063.stdout:6/822: write d2/d12/f34 [3987764,50729] 0 2024-05-09T06:38:22.218 INFO:tasks.workunit.client.1.smithi193.stdout:4/500: getdents d1a 0 2024-05-09T06:38:22.220 INFO:tasks.workunit.client.1.smithi193.stdout:4/501: write d1a/d22/f31 [461422,2979] 0 2024-05-09T06:38:22.220 INFO:tasks.workunit.client.0.smithi063.stdout:7/774: dwrite d0/d15/f76 [0,4194304] 0 2024-05-09T06:38:22.220 INFO:tasks.workunit.client.1.smithi193.stdout:4/502: chown d1a/f64 3070 1 2024-05-09T06:38:22.222 INFO:tasks.workunit.client.1.smithi193.stdout:4/503: chown d1a/d24/f41 623417138 1 2024-05-09T06:38:22.222 INFO:tasks.workunit.client.1.smithi193.stdout:4/504: chown f7 16099897 1 2024-05-09T06:38:22.222 INFO:tasks.workunit.client.0.smithi063.stdout:7/775: chown d0/d15/d49/d54/fc1 2451 1 2024-05-09T06:38:22.222 INFO:tasks.workunit.client.0.smithi063.stdout:7/776: chown d0/d15/d49/d54/fc1 1572 1 2024-05-09T06:38:22.225 INFO:tasks.workunit.client.0.smithi063.stdout:7/777: read d0/d15/d49/d4f/d66/d67/d6b/f105 [38488,7671] 0 2024-05-09T06:38:22.225 INFO:tasks.workunit.client.1.smithi193.stdout:4/505: mkdir d1a/d98 0 2024-05-09T06:38:22.226 INFO:tasks.workunit.client.0.smithi063.stdout:3/818: getdents d7/da/d2b/d66 0 2024-05-09T06:38:22.231 INFO:tasks.workunit.client.0.smithi063.stdout:7/778: creat d0/d15/f109 x:0 0 0 2024-05-09T06:38:22.232 INFO:tasks.workunit.client.0.smithi063.stdout:3/819: read - d7/d70/d98/f9f zero size 2024-05-09T06:38:22.234 INFO:tasks.workunit.client.0.smithi063.stdout:3/820: fdatasync d7/d4c/fc4 0 2024-05-09T06:38:22.235 INFO:tasks.workunit.client.0.smithi063.stdout:5/778: truncate d8/d17/da2/fbb 471493 0 2024-05-09T06:38:22.237 INFO:tasks.workunit.client.0.smithi063.stdout:7/779: truncate d0/d15/d4a/d50/f3c 1305744 0 2024-05-09T06:38:22.238 INFO:tasks.workunit.client.0.smithi063.stdout:8/880: getdents d9/da/d53/db2 0 2024-05-09T06:38:22.240 INFO:tasks.workunit.client.0.smithi063.stdout:5/779: symlink d8/d35/l106 0 2024-05-09T06:38:22.241 INFO:tasks.workunit.client.0.smithi063.stdout:4/791: dread d7/df/f21 [0,4194304] 0 2024-05-09T06:38:22.241 INFO:tasks.workunit.client.0.smithi063.stdout:8/881: write d9/da/d45/fa4 [3165674,10760] 0 2024-05-09T06:38:22.242 INFO:tasks.workunit.client.0.smithi063.stdout:7/780: mkdir d0/d15/d4a/d68/d10a 0 2024-05-09T06:38:22.245 INFO:tasks.workunit.client.0.smithi063.stdout:4/792: dread - d7/da3/d28/d31/d7b/fbe zero size 2024-05-09T06:38:22.251 INFO:tasks.workunit.client.0.smithi063.stdout:7/781: truncate d0/d15/d4a/d68/f7d 814121 0 2024-05-09T06:38:22.252 INFO:tasks.workunit.client.0.smithi063.stdout:1/753: dread d9/fe [0,4194304] 0 2024-05-09T06:38:22.253 INFO:tasks.workunit.client.0.smithi063.stdout:3/821: write d7/da/d2b/d66/fd2 [22739,53906] 0 2024-05-09T06:38:22.256 INFO:tasks.workunit.client.1.smithi193.stdout:4/506: link d1a/d1f/l92 d1a/d24/d73/d94/d7c/d5b/l99 0 2024-05-09T06:38:22.257 INFO:tasks.workunit.client.0.smithi063.stdout:4/793: dwrite d7/da3/d28/d31/d32/fce [0,4194304] 0 2024-05-09T06:38:22.258 INFO:tasks.workunit.client.0.smithi063.stdout:7/782: read d0/d15/d49/f5f [721756,102700] 0 2024-05-09T06:38:22.262 INFO:tasks.workunit.client.1.smithi193.stdout:4/507: dread d1a/f64 [0,4194304] 0 2024-05-09T06:38:22.264 INFO:tasks.workunit.client.1.smithi193.stdout:4/508: mkdir d1a/d22/d43/d9a 0 2024-05-09T06:38:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:5/780: dread d8/d35/f41 [0,4194304] 0 2024-05-09T06:38:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:8/882: dread d9/da/d45/d114/d139/f97 [0,4194304] 0 2024-05-09T06:38:22.268 INFO:tasks.workunit.client.1.smithi193.stdout:4/509: truncate d1a/d6b/f77 1425579 0 2024-05-09T06:38:22.268 INFO:tasks.workunit.client.0.smithi063.stdout:1/754: creat d9/d2a/d3c/d4c/d83/d88/dfb/f107 x:0 0 0 2024-05-09T06:38:22.268 INFO:tasks.workunit.client.0.smithi063.stdout:3/822: dwrite d7/da/d2b/d66/dc0/fa9 [0,4194304] 0 2024-05-09T06:38:22.269 INFO:tasks.workunit.client.1.smithi193.stdout:4/510: write d1a/d6b/f77 [296595,65874] 0 2024-05-09T06:38:22.270 INFO:tasks.workunit.client.0.smithi063.stdout:3/823: chown d7/d4c/d25/c59 854952 1 2024-05-09T06:38:22.271 INFO:tasks.workunit.client.0.smithi063.stdout:8/883: creat d9/da/d53/dbc/d102/d120/f13c x:0 0 0 2024-05-09T06:38:22.274 INFO:tasks.workunit.client.0.smithi063.stdout:4/794: creat d7/da3/ffa x:0 0 0 2024-05-09T06:38:22.274 INFO:tasks.workunit.client.1.smithi193.stdout:4/511: rename d1a/d6b/f44 to d1a/d80/f9b 0 2024-05-09T06:38:22.275 INFO:tasks.workunit.client.0.smithi063.stdout:8/884: write d9/da/d45/de4/f10e [652605,115830] 0 2024-05-09T06:38:22.277 INFO:tasks.workunit.client.0.smithi063.stdout:4/795: write d7/df/d35/da5/fb9 [5443,113722] 0 2024-05-09T06:38:22.277 INFO:tasks.workunit.client.0.smithi063.stdout:4/796: readlink d7/d3d/d73/l82 0 2024-05-09T06:38:22.277 INFO:tasks.workunit.client.0.smithi063.stdout:4/797: chown d7/d3d/fb7 64560 1 2024-05-09T06:38:22.278 INFO:tasks.workunit.client.0.smithi063.stdout:7/783: link d0/c6a d0/d15/d49/d54/d9f/d106/c10b 0 2024-05-09T06:38:22.278 INFO:tasks.workunit.client.0.smithi063.stdout:1/755: fdatasync d9/d2a/d35/d1e/f27 0 2024-05-09T06:38:22.283 INFO:tasks.workunit.client.0.smithi063.stdout:8/885: fsync d9/da/d53/faa 0 2024-05-09T06:38:22.284 INFO:tasks.workunit.client.0.smithi063.stdout:8/886: readlink d9/da/d18/d24/l116 0 2024-05-09T06:38:22.285 INFO:tasks.workunit.client.1.smithi193.stdout:4/512: link d1a/d1f/l90 d1a/d22/d8e/l9c 0 2024-05-09T06:38:22.289 INFO:tasks.workunit.client.0.smithi063.stdout:8/887: dwrite d9/d3b/ffb [0,4194304] 0 2024-05-09T06:38:22.291 INFO:tasks.workunit.client.0.smithi063.stdout:7/784: rename d0/d15/d89/d98 to d0/d15/d89/d10c 0 2024-05-09T06:38:22.294 INFO:tasks.workunit.client.0.smithi063.stdout:5/781: link d8/c11 d8/d65/c107 0 2024-05-09T06:38:22.295 INFO:tasks.workunit.client.0.smithi063.stdout:5/782: chown d8/d53/dca/f81 420 1 2024-05-09T06:38:22.296 INFO:tasks.workunit.client.0.smithi063.stdout:3/824: link d7/d22/d28/dab/fe0 d7/d70/d98/d86/f119 0 2024-05-09T06:38:22.300 INFO:tasks.workunit.client.0.smithi063.stdout:4/798: symlink d7/da3/d28/d31/d32/d34/db4/dcc/de1/lfb 0 2024-05-09T06:38:22.301 INFO:tasks.workunit.client.0.smithi063.stdout:4/799: write d7/da3/ffa [877870,53301] 0 2024-05-09T06:38:22.302 INFO:tasks.workunit.client.1.smithi193.stdout:4/513: link d1a/d6d/c97 d1a/d6b/c9d 0 2024-05-09T06:38:22.303 INFO:tasks.workunit.client.0.smithi063.stdout:1/756: mknod d9/d2a/d3c/d4c/d83/de9/c108 0 2024-05-09T06:38:22.305 INFO:tasks.workunit.client.0.smithi063.stdout:5/783: unlink d8/d65/f6f 0 2024-05-09T06:38:22.306 INFO:tasks.workunit.client.1.smithi193.stdout:4/514: mknod d1a/d24/d73/d94/d7c/d5b/c9e 0 2024-05-09T06:38:22.307 INFO:tasks.workunit.client.1.smithi193.stdout:4/515: chown d1a/d22/d5d 36964584 1 2024-05-09T06:38:22.307 INFO:tasks.workunit.client.0.smithi063.stdout:3/825: symlink d7/d70/d98/d86/l11a 0 2024-05-09T06:38:22.309 INFO:tasks.workunit.client.0.smithi063.stdout:4/800: write d7/d3d/fb7 [410971,40356] 0 2024-05-09T06:38:22.310 INFO:tasks.workunit.client.1.smithi193.stdout:4/516: write d1a/f1d [1708241,94599] 0 2024-05-09T06:38:22.311 INFO:tasks.workunit.client.0.smithi063.stdout:5/784: mknod d8/d17/d75/dc9/c108 0 2024-05-09T06:38:22.313 INFO:tasks.workunit.client.0.smithi063.stdout:3/826: symlink d7/da/de4/l11b 0 2024-05-09T06:38:22.314 INFO:tasks.workunit.client.0.smithi063.stdout:3/827: chown d7/da/fe 57912513 1 2024-05-09T06:38:22.315 INFO:tasks.workunit.client.1.smithi193.stdout:4/517: chown d1a/f65 7 1 2024-05-09T06:38:22.316 INFO:tasks.workunit.client.0.smithi063.stdout:8/888: write d9/da/d18/f1b [1084138,2088] 0 2024-05-09T06:38:22.317 INFO:tasks.workunit.client.0.smithi063.stdout:8/889: chown d9/da/d53/db2/fb5 298471467 1 2024-05-09T06:38:22.317 INFO:tasks.workunit.client.0.smithi063.stdout:1/757: symlink d9/d2a/d39/l109 0 2024-05-09T06:38:22.319 INFO:tasks.workunit.client.0.smithi063.stdout:7/785: mknod d0/d15/d49/d54/d9f/d97/d9b/c10d 0 2024-05-09T06:38:22.321 INFO:tasks.workunit.client.0.smithi063.stdout:3/828: symlink d7/da/d2b/d66/dc0/da4/l11c 0 2024-05-09T06:38:22.321 INFO:tasks.workunit.client.0.smithi063.stdout:7/786: write d0/d15/d49/d4f/d66/d67/d86/fff [1005005,42647] 0 2024-05-09T06:38:22.321 INFO:tasks.workunit.client.0.smithi063.stdout:4/801: creat d7/df/d35/dd9/ffc x:0 0 0 2024-05-09T06:38:22.322 INFO:tasks.workunit.client.0.smithi063.stdout:3/829: readlink d7/d70/d98/d5b/l71 0 2024-05-09T06:38:22.324 INFO:tasks.workunit.client.0.smithi063.stdout:3/830: dread - d7/da/d21/d30/f9e zero size 2024-05-09T06:38:22.324 INFO:tasks.workunit.client.1.smithi193.stdout:4/518: getdents d1a/d24/d73/d94/d7c 0 2024-05-09T06:38:22.325 INFO:tasks.workunit.client.0.smithi063.stdout:1/758: unlink d9/f90 0 2024-05-09T06:38:22.332 INFO:tasks.workunit.client.0.smithi063.stdout:5/785: write d8/d17/da3/fbf [1628129,11675] 0 2024-05-09T06:38:22.333 INFO:tasks.workunit.client.0.smithi063.stdout:3/831: mkdir d7/da/d21/d30/d44/d48/d11d 0 2024-05-09T06:38:22.335 INFO:tasks.workunit.client.0.smithi063.stdout:7/787: truncate d0/d15/d49/d4f/d66/d67/d6b/fc3 763996 0 2024-05-09T06:38:22.337 INFO:tasks.workunit.client.0.smithi063.stdout:4/802: link d7/da3/d28/d31/d32/d34/db4/fbd d7/da3/d8b/ffd 0 2024-05-09T06:38:22.338 INFO:tasks.workunit.client.1.smithi193.stdout:4/519: dwrite f17 [0,4194304] 0 2024-05-09T06:38:22.339 INFO:tasks.workunit.client.0.smithi063.stdout:1/759: mkdir d9/d2a/d3c/d4c/d83/de9/d10a 0 2024-05-09T06:38:22.339 INFO:tasks.workunit.client.0.smithi063.stdout:8/890: dwrite d9/da/d45/d114/d139/f64 [0,4194304] 0 2024-05-09T06:38:22.341 INFO:tasks.workunit.client.0.smithi063.stdout:7/788: unlink d0/d15/f72 0 2024-05-09T06:38:22.341 INFO:tasks.workunit.client.0.smithi063.stdout:3/832: dread d7/d4c/f69 [0,4194304] 0 2024-05-09T06:38:22.343 INFO:tasks.workunit.client.1.smithi193.stdout:4/520: unlink d1a/d22/d5d/f5f 0 2024-05-09T06:38:22.347 INFO:tasks.workunit.client.0.smithi063.stdout:4/803: mknod d7/da3/da2/cfe 0 2024-05-09T06:38:22.348 INFO:tasks.workunit.client.0.smithi063.stdout:4/804: chown d7/df/d1a/f20 14 1 2024-05-09T06:38:22.351 INFO:tasks.workunit.client.0.smithi063.stdout:8/891: symlink d9/d3b/l13d 0 2024-05-09T06:38:22.357 INFO:tasks.workunit.client.1.smithi193.stdout:4/521: write d1a/d24/f62 [1037205,38155] 0 2024-05-09T06:38:22.358 INFO:tasks.workunit.client.0.smithi063.stdout:1/760: rename d9/d2a/d3c/d4c/d83/d88/dc2/dd0/c9f to d9/d2a/d3c/d4c/d83/de9/c10b 0 2024-05-09T06:38:22.359 INFO:tasks.workunit.client.0.smithi063.stdout:5/786: creat d8/d19/f109 x:0 0 0 2024-05-09T06:38:22.360 INFO:tasks.workunit.client.0.smithi063.stdout:5/787: readlink d8/d65/dbd/lc4 0 2024-05-09T06:38:22.360 INFO:tasks.workunit.client.0.smithi063.stdout:4/805: rmdir d7/df/d35/d5a 39 2024-05-09T06:38:22.361 INFO:tasks.workunit.client.0.smithi063.stdout:8/892: rmdir d9/da/d53/da3/de9 39 2024-05-09T06:38:22.362 INFO:tasks.workunit.client.0.smithi063.stdout:8/893: read d9/da/d45/d114/d139/f97 [3148209,2694] 0 2024-05-09T06:38:22.363 INFO:tasks.workunit.client.1.smithi193.stdout:4/522: fsync f5 0 2024-05-09T06:38:22.400 INFO:tasks.workunit.client.0.smithi063.stdout:9/801: sync 2024-05-09T06:38:22.490 INFO:tasks.workunit.client.0.smithi063.stdout:3/833: write d7/da/d2b/d66/d74/f92 [1884394,2951] 0 2024-05-09T06:38:22.491 INFO:tasks.workunit.client.0.smithi063.stdout:7/789: mknod d0/d15/d89/d10c/dfb/c10e 0 2024-05-09T06:38:22.492 INFO:tasks.workunit.client.0.smithi063.stdout:3/834: write d7/da/d21/d30/d44/fd6 [431300,6131] 0 2024-05-09T06:38:22.496 INFO:tasks.workunit.client.0.smithi063.stdout:5/788: mknod d8/d17/d75/dc9/dd5/d96/c10a 0 2024-05-09T06:38:22.499 INFO:tasks.workunit.client.0.smithi063.stdout:4/806: mkdir d7/da3/d28/d31/d71/d98/dff 0 2024-05-09T06:38:22.500 INFO:tasks.workunit.client.0.smithi063.stdout:8/894: mknod d9/d3b/c13e 0 2024-05-09T06:38:22.501 INFO:tasks.workunit.client.0.smithi063.stdout:8/895: write d9/da/d51/f10b [2085757,121712] 0 2024-05-09T06:38:22.502 INFO:tasks.workunit.client.0.smithi063.stdout:8/896: chown d9/d41/c11c 25881589 1 2024-05-09T06:38:22.505 INFO:tasks.workunit.client.0.smithi063.stdout:1/761: write d9/d2a/d3c/d4c/d83/d88/fab [203827,10238] 0 2024-05-09T06:38:22.506 INFO:tasks.workunit.client.0.smithi063.stdout:3/835: creat d7/d22/dfc/f11e x:0 0 0 2024-05-09T06:38:22.509 INFO:tasks.workunit.client.0.smithi063.stdout:5/789: rename d8/dc8 to d8/d17/d10b 0 2024-05-09T06:38:22.510 INFO:tasks.workunit.client.0.smithi063.stdout:4/807: mknod d7/da3/da2/c100 0 2024-05-09T06:38:22.513 INFO:tasks.workunit.client.0.smithi063.stdout:0/836: sync 2024-05-09T06:38:22.516 INFO:tasks.workunit.client.0.smithi063.stdout:1/762: rmdir d9/d2a/d3c/d4c/d83/d88 39 2024-05-09T06:38:22.517 INFO:tasks.workunit.client.0.smithi063.stdout:1/763: readlink d9/d2a/d39/d5d/lf6 0 2024-05-09T06:38:22.518 INFO:tasks.workunit.client.0.smithi063.stdout:4/808: mknod d7/d50/c101 0 2024-05-09T06:38:22.521 INFO:tasks.workunit.client.0.smithi063.stdout:8/897: mkdir d9/d37/d106/d13f 0 2024-05-09T06:38:22.521 INFO:tasks.workunit.client.1.smithi193.stdout:4/523: dwrite d1a/d6b/f4a [0,4194304] 0 2024-05-09T06:38:22.522 INFO:tasks.workunit.client.0.smithi063.stdout:4/809: dread d7/fb [0,4194304] 0 2024-05-09T06:38:22.525 INFO:tasks.workunit.client.0.smithi063.stdout:0/837: rename dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/ff9 to dd/d1a/d2c/d51/d72/d9e/d23/f105 0 2024-05-09T06:38:22.525 INFO:tasks.workunit.client.0.smithi063.stdout:9/802: truncate d0/d8/dea/f101 1043038 0 2024-05-09T06:38:22.526 INFO:tasks.workunit.client.0.smithi063.stdout:0/838: stat dd/d1a/d2c/d51/d100/c102 0 2024-05-09T06:38:22.528 INFO:tasks.workunit.client.0.smithi063.stdout:1/764: unlink d9/d2a/d3c/d4a/db8/ddf/ff1 0 2024-05-09T06:38:22.529 INFO:tasks.workunit.client.1.smithi193.stdout:4/524: mknod d1a/d24/d73/d94/c9f 0 2024-05-09T06:38:22.530 INFO:tasks.workunit.client.0.smithi063.stdout:3/836: truncate d7/da/d21/f2c 2798933 0 2024-05-09T06:38:22.533 INFO:tasks.workunit.client.0.smithi063.stdout:7/790: getdents d0/d15/d4a/d50 0 2024-05-09T06:38:22.536 INFO:tasks.workunit.client.0.smithi063.stdout:7/791: chown d0/d15/d4a/d68/d10a 9 1 2024-05-09T06:38:22.536 INFO:tasks.workunit.client.0.smithi063.stdout:0/839: dread dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/f93 [0,4194304] 0 2024-05-09T06:38:22.537 INFO:tasks.workunit.client.0.smithi063.stdout:9/803: dwrite d0/d8/dea/ff3 [0,4194304] 0 2024-05-09T06:38:22.538 INFO:tasks.workunit.client.0.smithi063.stdout:8/898: dread d9/da/d53/dbc/d102/f110 [0,4194304] 0 2024-05-09T06:38:22.539 INFO:tasks.workunit.client.0.smithi063.stdout:4/810: creat d7/da3/d28/d31/d71/d98/dff/f102 x:0 0 0 2024-05-09T06:38:22.541 INFO:tasks.workunit.client.0.smithi063.stdout:3/837: mknod d7/da/d2b/d66/dc0/c11f 0 2024-05-09T06:38:22.543 INFO:tasks.workunit.client.0.smithi063.stdout:5/790: mknod d8/d17/d18/c10c 0 2024-05-09T06:38:22.546 INFO:tasks.workunit.client.0.smithi063.stdout:1/765: creat d9/d2a/d3c/d4c/d83/de9/d10a/f10c x:0 0 0 2024-05-09T06:38:22.548 INFO:tasks.workunit.client.0.smithi063.stdout:0/840: mknod dd/d1a/d2c/d51/d72/c106 0 2024-05-09T06:38:22.549 INFO:tasks.workunit.client.0.smithi063.stdout:0/841: dread - dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/f101 zero size 2024-05-09T06:38:22.550 INFO:tasks.workunit.client.0.smithi063.stdout:4/811: creat d7/df/d35/dd9/f103 x:0 0 0 2024-05-09T06:38:22.551 INFO:tasks.workunit.client.0.smithi063.stdout:7/792: truncate d0/d15/d49/d54/d9f/d97/fde 571386 0 2024-05-09T06:38:22.551 INFO:tasks.workunit.client.0.smithi063.stdout:2/831: sync 2024-05-09T06:38:22.551 INFO:tasks.workunit.client.0.smithi063.stdout:6/823: sync 2024-05-09T06:38:22.552 INFO:tasks.workunit.client.0.smithi063.stdout:0/842: readlink dd/d1a/d2c/d51/d72/d9e/d23/l45 0 2024-05-09T06:38:22.552 INFO:tasks.workunit.client.0.smithi063.stdout:8/899: rename d9/da/d34/f10d to d9/da/d18/dc7/d8f/f140 0 2024-05-09T06:38:22.552 INFO:tasks.workunit.client.1.smithi193.stdout:4/525: dwrite d1a/d1f/f2c [0,4194304] 0 2024-05-09T06:38:22.552 INFO:tasks.workunit.client.0.smithi063.stdout:4/812: write d7/da6/fd3 [907748,103704] 0 2024-05-09T06:38:22.553 INFO:tasks.workunit.client.0.smithi063.stdout:3/838: mknod d7/d22/c120 0 2024-05-09T06:38:22.556 INFO:tasks.workunit.client.0.smithi063.stdout:0/843: write dd/d1a/d2c/d51/d72/d9e/feb [1782802,52587] 0 2024-05-09T06:38:22.558 INFO:tasks.workunit.client.0.smithi063.stdout:0/844: chown dd/d1a/f35 19708021 1 2024-05-09T06:38:22.560 INFO:tasks.workunit.client.0.smithi063.stdout:6/824: creat d2/d12/f10a x:0 0 0 2024-05-09T06:38:22.561 INFO:tasks.workunit.client.0.smithi063.stdout:7/793: dwrite d0/d15/d49/d54/f60 [4194304,4194304] 0 2024-05-09T06:38:22.563 INFO:tasks.workunit.client.0.smithi063.stdout:6/825: truncate d2/d20/d26/fc4 653472 0 2024-05-09T06:38:22.566 INFO:tasks.workunit.client.0.smithi063.stdout:4/813: truncate d7/da3/d28/f7d 1106417 0 2024-05-09T06:38:22.567 INFO:tasks.workunit.client.1.smithi193.stdout:7/419: sync 2024-05-09T06:38:22.567 INFO:tasks.workunit.client.1.smithi193.stdout:6/428: sync 2024-05-09T06:38:22.570 INFO:tasks.workunit.client.0.smithi063.stdout:6/826: dwrite d2/d20/d26/fc4 [0,4194304] 0 2024-05-09T06:38:22.571 INFO:tasks.workunit.client.0.smithi063.stdout:6/827: chown d2/d1f/d29/de2/d4e/lbc 81134558 1 2024-05-09T06:38:22.572 INFO:tasks.workunit.client.1.smithi193.stdout:4/526: write d1a/d24/f40 [4344516,92666] 0 2024-05-09T06:38:22.575 INFO:tasks.workunit.client.0.smithi063.stdout:9/804: rename d0/d4/d26/c3d to d0/d4/d26/d39/d48/d80/d84/dd8/c118 0 2024-05-09T06:38:22.576 INFO:tasks.workunit.client.1.smithi193.stdout:7/420: dwrite d6/dd/f40 [0,4194304] 0 2024-05-09T06:38:22.577 INFO:tasks.workunit.client.1.smithi193.stdout:7/421: write d6/dd/f22 [3684532,112019] 0 2024-05-09T06:38:22.578 INFO:tasks.workunit.client.0.smithi063.stdout:0/845: dread dd/d1a/d2c/d51/d72/d9e/f64 [0,4194304] 0 2024-05-09T06:38:22.579 INFO:tasks.workunit.client.0.smithi063.stdout:7/794: truncate d0/f32 509468 0 2024-05-09T06:38:22.584 INFO:tasks.workunit.client.0.smithi063.stdout:5/791: dwrite d8/d17/d18/f8a [0,4194304] 0 2024-05-09T06:38:22.586 INFO:tasks.workunit.client.1.smithi193.stdout:7/422: creat d6/dd/d70/f8e x:0 0 0 2024-05-09T06:38:22.587 INFO:tasks.workunit.client.0.smithi063.stdout:6/828: truncate d2/d1f/d29/d3f/f82 531036 0 2024-05-09T06:38:22.590 INFO:tasks.workunit.client.0.smithi063.stdout:7/795: mkdir d0/d15/d4a/d50/d10f 0 2024-05-09T06:38:22.596 INFO:tasks.workunit.client.0.smithi063.stdout:1/766: getdents d9/d2a/d3c/d4a/db8/ddf 0 2024-05-09T06:38:22.597 INFO:tasks.workunit.client.0.smithi063.stdout:3/839: link d7/da/d21/d30/d44/cc9 d7/da/d2b/d66/dc0/da4/c121 0 2024-05-09T06:38:22.598 INFO:tasks.workunit.client.0.smithi063.stdout:8/900: getdents d9/da/d18/dc7/d8f 0 2024-05-09T06:38:22.598 INFO:tasks.workunit.client.0.smithi063.stdout:6/829: rename d2/d1f/d29/d3f/d5c/f90 to d2/d1f/d29/d3f/d5c/d64/f10b 0 2024-05-09T06:38:22.602 INFO:tasks.workunit.client.0.smithi063.stdout:4/814: write d7/d3d/f49 [1096293,14593] 0 2024-05-09T06:38:22.603 INFO:tasks.workunit.client.0.smithi063.stdout:5/792: dread d8/d65/f7a [0,4194304] 0 2024-05-09T06:38:22.605 INFO:tasks.workunit.client.0.smithi063.stdout:5/793: read d8/d17/d75/dc9/dd5/fbe [359070,4036] 0 2024-05-09T06:38:22.609 INFO:tasks.workunit.client.0.smithi063.stdout:9/805: dwrite d0/d4/d26/d39/f53 [4194304,4194304] 0 2024-05-09T06:38:22.624 INFO:tasks.workunit.client.1.smithi193.stdout:7/423: chown d6/f2e 15853091 1 2024-05-09T06:38:22.628 INFO:tasks.workunit.client.1.smithi193.stdout:7/424: rmdir d6/d8/d29/d2f 39 2024-05-09T06:38:22.629 INFO:tasks.workunit.client.1.smithi193.stdout:4/527: dwrite d1a/d24/d73/f7e [0,4194304] 0 2024-05-09T06:38:22.630 INFO:tasks.workunit.client.1.smithi193.stdout:7/425: dread - d6/d8/d1a/d36/f65 zero size 2024-05-09T06:38:22.633 INFO:tasks.workunit.client.1.smithi193.stdout:4/528: symlink d1a/d22/d43/la0 0 2024-05-09T06:38:22.637 INFO:tasks.workunit.client.1.smithi193.stdout:4/529: creat d1a/d6d/fa1 x:0 0 0 2024-05-09T06:38:22.639 INFO:tasks.workunit.client.1.smithi193.stdout:4/530: read f10 [1184713,105072] 0 2024-05-09T06:38:22.640 INFO:tasks.workunit.client.1.smithi193.stdout:4/531: readlink d1a/d24/d73/d94/d7c/l5a 0 2024-05-09T06:38:22.642 INFO:tasks.workunit.client.1.smithi193.stdout:7/426: creat d6/d8/d1a/f8f x:0 0 0 2024-05-09T06:38:22.644 INFO:tasks.workunit.client.1.smithi193.stdout:4/532: unlink d1a/d6b/f4a 0 2024-05-09T06:38:22.645 INFO:tasks.workunit.client.0.smithi063.stdout:7/796: fsync d0/d15/d4a/d50/f37 0 2024-05-09T06:38:22.649 INFO:tasks.workunit.client.1.smithi193.stdout:4/533: dwrite d1a/d24/f62 [0,4194304] 0 2024-05-09T06:38:22.650 INFO:tasks.workunit.client.0.smithi063.stdout:1/767: symlink d9/d2a/d3c/d4c/d83/de9/d10a/l10d 0 2024-05-09T06:38:22.651 INFO:tasks.workunit.client.0.smithi063.stdout:7/797: dwrite d0/d15/f76 [0,4194304] 0 2024-05-09T06:38:22.653 INFO:tasks.workunit.client.0.smithi063.stdout:6/830: symlink d2/d4/d7c/dfc/l10c 0 2024-05-09T06:38:22.657 INFO:tasks.workunit.client.0.smithi063.stdout:4/815: unlink d7/d3d/d73/fb1 0 2024-05-09T06:38:22.658 INFO:tasks.workunit.client.0.smithi063.stdout:4/816: stat d7/da3/da2/fb8 0 2024-05-09T06:38:22.666 INFO:tasks.workunit.client.0.smithi063.stdout:0/846: link dd/d1a/d2c/d51/d76/d97/cae dd/d60/dcd/de0/c107 0 2024-05-09T06:38:22.670 INFO:tasks.workunit.client.0.smithi063.stdout:5/794: dwrite d8/d19/d30/db1/ff8 [0,4194304] 0 2024-05-09T06:38:22.670 INFO:tasks.workunit.client.0.smithi063.stdout:0/847: write dd/d1a/d2c/d51/d72/d9e/d23/d89/fc4 [4794506,47204] 0 2024-05-09T06:38:22.671 INFO:tasks.workunit.client.0.smithi063.stdout:3/840: fdatasync d7/da/d2b/d66/d74/d7e/f103 0 2024-05-09T06:38:22.671 INFO:tasks.workunit.client.0.smithi063.stdout:2/832: sync 2024-05-09T06:38:22.671 INFO:tasks.workunit.client.0.smithi063.stdout:6/831: mknod d2/d4/d96/c10d 0 2024-05-09T06:38:22.672 INFO:tasks.workunit.client.0.smithi063.stdout:3/841: dread - d7/da/d21/d30/d44/dee/d10c/f106 zero size 2024-05-09T06:38:22.674 INFO:tasks.workunit.client.0.smithi063.stdout:4/817: symlink d7/da3/d28/d31/d32/d34/db4/dcc/l104 0 2024-05-09T06:38:22.677 INFO:tasks.workunit.client.0.smithi063.stdout:1/768: mknod d9/d2a/d3c/dbc/c10e 0 2024-05-09T06:38:22.680 INFO:tasks.workunit.client.0.smithi063.stdout:5/795: write d8/d35/f64 [501247,24718] 0 2024-05-09T06:38:22.687 INFO:tasks.workunit.client.0.smithi063.stdout:2/833: fdatasync dc/dd/d45/f53 0 2024-05-09T06:38:22.688 INFO:tasks.workunit.client.0.smithi063.stdout:8/901: getdents d9/da/d18/dc7/dd3/df1 0 2024-05-09T06:38:22.690 INFO:tasks.workunit.client.0.smithi063.stdout:3/842: dwrite d7/f1b [8388608,4194304] 0 2024-05-09T06:38:22.692 INFO:tasks.workunit.client.1.smithi193.stdout:5/387: sync 2024-05-09T06:38:22.692 INFO:tasks.workunit.client.1.smithi193.stdout:8/439: sync 2024-05-09T06:38:22.692 INFO:tasks.workunit.client.1.smithi193.stdout:2/469: sync 2024-05-09T06:38:22.692 INFO:tasks.workunit.client.1.smithi193.stdout:1/486: sync 2024-05-09T06:38:22.693 INFO:tasks.workunit.client.1.smithi193.stdout:0/406: sync 2024-05-09T06:38:22.693 INFO:tasks.workunit.client.1.smithi193.stdout:3/389: sync 2024-05-09T06:38:22.693 INFO:tasks.workunit.client.1.smithi193.stdout:9/519: sync 2024-05-09T06:38:22.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/843: readlink d7/da/d21/d30/d44/d48/ld9 0 2024-05-09T06:38:22.694 INFO:tasks.workunit.client.0.smithi063.stdout:6/832: dread d2/d20/d26/d7f/dcf/fe5 [0,4194304] 0 2024-05-09T06:38:22.694 INFO:tasks.workunit.client.0.smithi063.stdout:5/796: rename d8/d17/d75/dc9/dd5/d96/db0 to d8/d19/d30/d74/d77/d10d 0 2024-05-09T06:38:22.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/844: write d7/d70/f104 [744194,93495] 0 2024-05-09T06:38:22.701 INFO:tasks.workunit.client.0.smithi063.stdout:3/845: dread d7/f16 [0,4194304] 0 2024-05-09T06:38:22.701 INFO:tasks.workunit.client.0.smithi063.stdout:4/818: creat d7/df/d35/d5a/d60/f105 x:0 0 0 2024-05-09T06:38:22.701 INFO:tasks.workunit.client.0.smithi063.stdout:8/902: dread d9/d41/dec/dc5/fd2 [0,4194304] 0 2024-05-09T06:38:22.703 INFO:tasks.workunit.client.0.smithi063.stdout:4/819: stat d7/df/d35/d5a/dad/de9 0 2024-05-09T06:38:22.703 INFO:tasks.workunit.client.0.smithi063.stdout:8/903: write d9/da/d53/db2/fe0 [673485,91015] 0 2024-05-09T06:38:22.703 INFO:tasks.workunit.client.0.smithi063.stdout:7/798: getdents d0/d15/d49/d4f/d66/df1 0 2024-05-09T06:38:22.704 INFO:tasks.workunit.client.1.smithi193.stdout:3/390: dwrite d1/d2d/d51/f65 [0,4194304] 0 2024-05-09T06:38:22.705 INFO:tasks.workunit.client.1.smithi193.stdout:1/487: rmdir d4/d16/d6b/d89 39 2024-05-09T06:38:22.705 INFO:tasks.workunit.client.0.smithi063.stdout:8/904: chown d9/da/d51/fe5 757 1 2024-05-09T06:38:22.708 INFO:tasks.workunit.client.1.smithi193.stdout:8/440: dwrite d6/d1c/d53/f63 [0,4194304] 0 2024-05-09T06:38:22.709 INFO:tasks.workunit.client.1.smithi193.stdout:7/427: write d6/d8/d1a/d36/f57 [855865,25860] 0 2024-05-09T06:38:22.711 INFO:tasks.workunit.client.0.smithi063.stdout:6/833: dread d2/d20/d26/d7f/dcf/fda [0,4194304] 0 2024-05-09T06:38:22.712 INFO:tasks.workunit.client.1.smithi193.stdout:0/407: rename d18/d4c/d64/d66/l7b to d18/d4c/d47/d59/d76/l84 0 2024-05-09T06:38:22.715 INFO:tasks.workunit.client.1.smithi193.stdout:3/391: mknod d1/db/d59/d63/c91 0 2024-05-09T06:38:22.716 INFO:tasks.workunit.client.1.smithi193.stdout:4/534: truncate d1a/f1d 1017176 0 2024-05-09T06:38:22.717 INFO:tasks.workunit.client.1.smithi193.stdout:9/520: dwrite dd/d1e/d1f/d48/d55/d41/d68/f78 [0,4194304] 0 2024-05-09T06:38:22.719 INFO:tasks.workunit.client.0.smithi063.stdout:4/820: dwrite d7/da3/f3a [0,4194304] 0 2024-05-09T06:38:22.721 INFO:tasks.workunit.client.1.smithi193.stdout:8/441: mkdir d6/d1c/d1b/d5d/d84 0 2024-05-09T06:38:22.721 INFO:tasks.workunit.client.0.smithi063.stdout:8/905: dwrite d9/d3b/ffb [0,4194304] 0 2024-05-09T06:38:22.722 INFO:tasks.workunit.client.0.smithi063.stdout:9/806: write d0/d4/d57/d60/f7c [446576,57393] 0 2024-05-09T06:38:22.722 INFO:tasks.workunit.client.1.smithi193.stdout:8/442: write d6/d1c/f34 [83612,118952] 0 2024-05-09T06:38:22.725 INFO:tasks.workunit.client.0.smithi063.stdout:2/834: dread dc/d36/f80 [0,4194304] 0 2024-05-09T06:38:22.725 INFO:tasks.workunit.client.0.smithi063.stdout:8/906: dread - d9/da/d53/f129 zero size 2024-05-09T06:38:22.725 INFO:tasks.workunit.client.0.smithi063.stdout:3/846: mknod d7/d22/c122 0 2024-05-09T06:38:22.726 INFO:tasks.workunit.client.1.smithi193.stdout:5/388: rename d0/d13/d2c/d37/d2b/d47/d66/l43 to d0/d13/d2c/d37/d2b/d47/d66/d3a/l7f 0 2024-05-09T06:38:22.726 INFO:tasks.workunit.client.1.smithi193.stdout:6/429: sync 2024-05-09T06:38:22.728 INFO:tasks.workunit.client.1.smithi193.stdout:6/430: write d1/d25/d2c/d3c/d3e/f3f [2780922,23408] 0 2024-05-09T06:38:22.730 INFO:tasks.workunit.client.1.smithi193.stdout:6/431: write d1/d25/d2c/d81/d37/d4f/f66 [1785367,67360] 0 2024-05-09T06:38:22.731 INFO:tasks.workunit.client.1.smithi193.stdout:0/408: creat d18/d4c/d64/f85 x:0 0 0 2024-05-09T06:38:22.736 INFO:tasks.workunit.client.0.smithi063.stdout:3/847: dwrite d7/d22/d28/f109 [0,4194304] 0 2024-05-09T06:38:22.737 INFO:tasks.workunit.client.1.smithi193.stdout:4/535: creat d1a/d22/d43/fa2 x:0 0 0 2024-05-09T06:38:22.742 INFO:tasks.workunit.client.0.smithi063.stdout:0/848: truncate dd/d1a/d2c/fa1 354594 0 2024-05-09T06:38:22.743 INFO:tasks.workunit.client.1.smithi193.stdout:4/536: read d1a/d22/d43/f4d [94716,7130] 0 2024-05-09T06:38:22.744 INFO:tasks.workunit.client.0.smithi063.stdout:0/849: write dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/f7b [3959942,14995] 0 2024-05-09T06:38:22.744 INFO:tasks.workunit.client.0.smithi063.stdout:0/850: readlink dd/l2a 0 2024-05-09T06:38:22.745 INFO:tasks.workunit.client.1.smithi193.stdout:4/537: stat d1a/d80/f9b 0 2024-05-09T06:38:22.747 INFO:tasks.workunit.client.1.smithi193.stdout:1/488: dwrite d4/d16/d6b/d89/f99 [0,4194304] 0 2024-05-09T06:38:22.748 INFO:tasks.workunit.client.0.smithi063.stdout:4/821: creat d7/da3/d28/d31/d32/d34/db4/dcc/f106 x:0 0 0 2024-05-09T06:38:22.753 INFO:tasks.workunit.client.1.smithi193.stdout:8/443: symlink d6/d1c/d1b/d5d/l85 0 2024-05-09T06:38:22.756 INFO:tasks.workunit.client.0.smithi063.stdout:8/907: mkdir d9/da/d45/d114/d139/d141 0 2024-05-09T06:38:22.757 INFO:tasks.workunit.client.1.smithi193.stdout:2/470: rename db/d25/d38/f6b to db/d4b/d56/fa0 0 2024-05-09T06:38:22.758 INFO:tasks.workunit.client.0.smithi063.stdout:8/908: truncate d9/d41/dec/db8/f125 18614 0 2024-05-09T06:38:22.758 INFO:tasks.workunit.client.0.smithi063.stdout:8/909: chown f4 620520486 1 2024-05-09T06:38:22.759 INFO:tasks.workunit.client.0.smithi063.stdout:2/835: symlink dc/d36/d43/d46/d49/lfc 0 2024-05-09T06:38:22.761 INFO:tasks.workunit.client.0.smithi063.stdout:8/910: read d9/da/d51/f10b [2411533,70492] 0 2024-05-09T06:38:22.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/409: symlink d18/d4c/d47/d59/d76/l86 0 2024-05-09T06:38:22.768 INFO:tasks.workunit.client.0.smithi063.stdout:6/834: mkdir d2/df/d91/dd8/dd9/d10e 0 2024-05-09T06:38:22.769 INFO:tasks.workunit.client.0.smithi063.stdout:1/769: sync 2024-05-09T06:38:22.770 INFO:tasks.workunit.client.1.smithi193.stdout:3/392: symlink d1/d2d/l92 0 2024-05-09T06:38:22.774 INFO:tasks.workunit.client.0.smithi063.stdout:6/835: dwrite d2/d4/d7c/d7e/f87 [0,4194304] 0 2024-05-09T06:38:22.775 INFO:tasks.workunit.client.1.smithi193.stdout:0/410: dwrite d18/f60 [0,4194304] 0 2024-05-09T06:38:22.775 INFO:tasks.workunit.client.1.smithi193.stdout:9/521: write dd/d1e/d1f/f6f [208549,107264] 0 2024-05-09T06:38:22.778 INFO:tasks.workunit.client.1.smithi193.stdout:0/411: chown d18/d4c/d47/f58 1425117 1 2024-05-09T06:38:22.779 INFO:tasks.workunit.client.1.smithi193.stdout:9/522: fdatasync dd/d1e/d66/d49/f51 0 2024-05-09T06:38:22.780 INFO:tasks.workunit.client.0.smithi063.stdout:0/851: unlink dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/f101 0 2024-05-09T06:38:22.782 INFO:tasks.workunit.client.1.smithi193.stdout:3/393: dwrite d1/db/d2f/d78/f8b [0,4194304] 0 2024-05-09T06:38:22.782 INFO:tasks.workunit.client.1.smithi193.stdout:1/489: rmdir d4/d76 39 2024-05-09T06:38:22.788 INFO:tasks.workunit.client.0.smithi063.stdout:4/822: unlink d7/df/d35/d5a/dad/laf 0 2024-05-09T06:38:22.789 INFO:tasks.workunit.client.1.smithi193.stdout:7/428: write d6/d8/d29/d2f/f8d [942165,26531] 0 2024-05-09T06:38:22.789 INFO:tasks.workunit.client.0.smithi063.stdout:4/823: dread - d7/df/d35/d5a/d60/f105 zero size 2024-05-09T06:38:22.791 INFO:tasks.workunit.client.0.smithi063.stdout:9/807: creat d0/d4/d7d/d105/d110/df7/f119 x:0 0 0 2024-05-09T06:38:22.792 INFO:tasks.workunit.client.0.smithi063.stdout:9/808: readlink d0/d8/d2d/l40 0 2024-05-09T06:38:22.792 INFO:tasks.workunit.client.1.smithi193.stdout:8/444: mknod d6/d1c/d18/d66/c86 0 2024-05-09T06:38:22.797 INFO:tasks.workunit.client.0.smithi063.stdout:2/836: rename dc/f1c to dc/d36/d43/d46/d49/ffd 0 2024-05-09T06:38:22.798 INFO:tasks.workunit.client.0.smithi063.stdout:2/837: dread - dc/dd/d45/d5a/d6c/d8f/ff8 zero size 2024-05-09T06:38:22.800 INFO:tasks.workunit.client.0.smithi063.stdout:2/838: dread dc/d36/d43/d46/ded/ff0 [0,4194304] 0 2024-05-09T06:38:22.800 INFO:tasks.workunit.client.0.smithi063.stdout:7/799: mkdir d0/d15/d49/d4f/d66/d67/d86/d110 0 2024-05-09T06:38:22.802 INFO:tasks.workunit.client.1.smithi193.stdout:6/432: rename d1/d25/d2c/d3c/f3d to d1/d25/f96 0 2024-05-09T06:38:22.806 INFO:tasks.workunit.client.0.smithi063.stdout:7/800: dwrite d0/ff [0,4194304] 0 2024-05-09T06:38:22.806 INFO:tasks.workunit.client.0.smithi063.stdout:8/911: mknod d9/da/d53/db2/d11a/c142 0 2024-05-09T06:38:22.807 INFO:tasks.workunit.client.1.smithi193.stdout:5/389: creat d0/d13/d2c/d37/d74/f80 x:0 0 0 2024-05-09T06:38:22.808 INFO:tasks.workunit.client.0.smithi063.stdout:8/912: stat d9/da/d45/de4/f107 0 2024-05-09T06:38:22.810 INFO:tasks.workunit.client.0.smithi063.stdout:3/848: truncate d7/d70/d98/d86/ff0 1096440 0 2024-05-09T06:38:22.813 INFO:tasks.workunit.client.0.smithi063.stdout:1/770: fsync d9/d46/f54 0 2024-05-09T06:38:22.814 INFO:tasks.workunit.client.0.smithi063.stdout:1/771: write d9/d2a/d35/d1e/d9d/f82 [2009383,78718] 0 2024-05-09T06:38:22.815 INFO:tasks.workunit.client.0.smithi063.stdout:5/797: getdents d8/d65 0 2024-05-09T06:38:22.816 INFO:tasks.workunit.client.0.smithi063.stdout:5/798: chown d8/d19/d30/d74/cb9 3829 1 2024-05-09T06:38:22.819 INFO:tasks.workunit.client.1.smithi193.stdout:0/412: rmdir d18/d4c/d64 39 2024-05-09T06:38:22.821 INFO:tasks.workunit.client.0.smithi063.stdout:0/852: creat dd/d60/d44/d90/da4/f108 x:0 0 0 2024-05-09T06:38:22.824 INFO:tasks.workunit.client.0.smithi063.stdout:9/809: write d0/d4/d57/d60/d76/fa9 [3182953,71220] 0 2024-05-09T06:38:22.824 INFO:tasks.workunit.client.1.smithi193.stdout:7/429: mknod d6/d8/d1a/d36/d4d/d74/c90 0 2024-05-09T06:38:22.826 INFO:tasks.workunit.client.1.smithi193.stdout:7/430: write d6/d8/d29/d2f/f8d [1732716,49530] 0 2024-05-09T06:38:22.829 INFO:tasks.workunit.client.0.smithi063.stdout:7/801: fsync d0/d15/d4a/fb4 0 2024-05-09T06:38:22.832 INFO:tasks.workunit.client.0.smithi063.stdout:8/913: creat d9/da/d18/d24/d71/f143 x:0 0 0 2024-05-09T06:38:22.833 INFO:tasks.workunit.client.0.smithi063.stdout:3/849: dread d7/d22/d28/dab/fd4 [0,4194304] 0 2024-05-09T06:38:22.834 INFO:tasks.workunit.client.0.smithi063.stdout:7/802: dread d0/d15/d49/d4f/d66/d67/ffd [0,4194304] 0 2024-05-09T06:38:22.834 INFO:tasks.workunit.client.1.smithi193.stdout:4/538: rename d1a/d24/d73/f7e to d1a/d6d/fa3 0 2024-05-09T06:38:22.834 INFO:tasks.workunit.client.1.smithi193.stdout:2/471: dwrite db/d4b/f5d [0,4194304] 0 2024-05-09T06:38:22.835 INFO:tasks.workunit.client.1.smithi193.stdout:6/433: chown d1/de/d11/d2d/c35 6170 1 2024-05-09T06:38:22.837 INFO:tasks.workunit.client.1.smithi193.stdout:6/434: stat d1/de/d11/d18/f6d 0 2024-05-09T06:38:22.838 INFO:tasks.workunit.client.1.smithi193.stdout:5/390: creat d0/d6d/f81 x:0 0 0 2024-05-09T06:38:22.839 INFO:tasks.workunit.client.1.smithi193.stdout:6/435: truncate d1/d25/f96 442661 0 2024-05-09T06:38:22.840 INFO:tasks.workunit.client.0.smithi063.stdout:1/772: unlink d9/d2a/d3c/d4c/f56 0 2024-05-09T06:38:22.841 INFO:tasks.workunit.client.1.smithi193.stdout:0/413: truncate d18/f4f 3332163 0 2024-05-09T06:38:22.841 INFO:tasks.workunit.client.0.smithi063.stdout:1/773: dread - d9/d2a/d3c/d4c/d83/de9/d10a/f10c zero size 2024-05-09T06:38:22.846 INFO:tasks.workunit.client.0.smithi063.stdout:7/803: dread d0/d15/f48 [0,4194304] 0 2024-05-09T06:38:22.847 INFO:tasks.workunit.client.1.smithi193.stdout:1/490: mknod d4/d16/d60/d4f/d59/c9a 0 2024-05-09T06:38:22.847 INFO:tasks.workunit.client.0.smithi063.stdout:7/804: fdatasync d0/d15/f109 0 2024-05-09T06:38:22.848 INFO:tasks.workunit.client.0.smithi063.stdout:6/836: rename d2/df/c48 to d2/d9a/c10f 0 2024-05-09T06:38:22.849 INFO:tasks.workunit.client.0.smithi063.stdout:6/837: write d2/df/d59/d99/ff8 [644800,121198] 0 2024-05-09T06:38:22.853 INFO:tasks.workunit.client.1.smithi193.stdout:8/445: mkdir d6/d1c/d1b/d5d/d84/d87 0 2024-05-09T06:38:22.854 INFO:tasks.workunit.client.1.smithi193.stdout:8/446: write d6/d1c/d53/d56/f6c [274691,22847] 0 2024-05-09T06:38:22.855 INFO:tasks.workunit.client.1.smithi193.stdout:7/431: symlink d6/dd/d5e/d6b/l91 0 2024-05-09T06:38:22.860 INFO:tasks.workunit.client.1.smithi193.stdout:4/539: mknod d1a/d1f/ca4 0 2024-05-09T06:38:22.861 INFO:tasks.workunit.client.1.smithi193.stdout:8/447: dwrite d6/d1c/d18/f26 [4194304,4194304] 0 2024-05-09T06:38:22.863 INFO:tasks.workunit.client.1.smithi193.stdout:2/472: unlink db/df/d54/l68 0 2024-05-09T06:38:22.864 INFO:tasks.workunit.client.1.smithi193.stdout:2/473: write db/f1e [1538108,31289] 0 2024-05-09T06:38:22.865 INFO:tasks.workunit.client.1.smithi193.stdout:5/391: mknod d0/d13/d2c/d37/c82 0 2024-05-09T06:38:22.869 INFO:tasks.workunit.client.1.smithi193.stdout:6/436: read d1/f23 [10932542,62550] 0 2024-05-09T06:38:22.870 INFO:tasks.workunit.client.0.smithi063.stdout:5/799: write d8/d19/f1a [5117740,107215] 0 2024-05-09T06:38:22.874 INFO:tasks.workunit.client.0.smithi063.stdout:3/850: mknod d7/d70/d98/dbb/c123 0 2024-05-09T06:38:22.874 INFO:tasks.workunit.client.0.smithi063.stdout:8/914: dread d9/da/f4e [0,4194304] 0 2024-05-09T06:38:22.883 INFO:tasks.workunit.client.1.smithi193.stdout:0/414: readlink d18/d4c/d64/l6e 0 2024-05-09T06:38:22.883 INFO:tasks.workunit.client.1.smithi193.stdout:9/523: creat dd/d1e/d1f/faa x:0 0 0 2024-05-09T06:38:22.884 INFO:tasks.workunit.client.1.smithi193.stdout:7/432: symlink d6/dd/d70/l92 0 2024-05-09T06:38:22.885 INFO:tasks.workunit.client.0.smithi063.stdout:9/810: dwrite d0/d4/d26/d39/d48/d80/fad [0,4194304] 0 2024-05-09T06:38:22.886 INFO:tasks.workunit.client.1.smithi193.stdout:5/392: dwrite d0/f5c [4194304,4194304] 0 2024-05-09T06:38:22.889 INFO:tasks.workunit.client.1.smithi193.stdout:8/448: dread d6/d1c/d27/f43 [0,4194304] 0 2024-05-09T06:38:22.889 INFO:tasks.workunit.client.0.smithi063.stdout:4/824: creat d7/da3/d28/d31/f107 x:0 0 0 2024-05-09T06:38:22.890 INFO:tasks.workunit.client.1.smithi193.stdout:2/474: mknod db/d25/d38/d40/ca1 0 2024-05-09T06:38:22.892 INFO:tasks.workunit.client.1.smithi193.stdout:2/475: readlink db/d37/d6d/d80/l2c 0 2024-05-09T06:38:22.896 INFO:tasks.workunit.client.1.smithi193.stdout:1/491: write d4/d16/f27 [2319789,107166] 0 2024-05-09T06:38:22.896 INFO:tasks.workunit.client.1.smithi193.stdout:6/437: dread - d1/d25/d2c/d3c/f49 zero size 2024-05-09T06:38:22.897 INFO:tasks.workunit.client.1.smithi193.stdout:1/492: chown d4/d3c/d3e/f5c 587 1 2024-05-09T06:38:22.898 INFO:tasks.workunit.client.1.smithi193.stdout:3/394: link d1/db/d1b/c49 d1/d2d/c93 0 2024-05-09T06:38:22.901 INFO:tasks.workunit.client.1.smithi193.stdout:0/415: creat d18/d4c/d47/d59/d76/f87 x:0 0 0 2024-05-09T06:38:22.903 INFO:tasks.workunit.client.0.smithi063.stdout:3/851: rmdir d7/d70 39 2024-05-09T06:38:22.905 INFO:tasks.workunit.client.1.smithi193.stdout:5/393: write d0/f9 [8405694,84450] 0 2024-05-09T06:38:22.905 INFO:tasks.workunit.client.0.smithi063.stdout:8/915: dread d9/da/f9f [0,4194304] 0 2024-05-09T06:38:22.906 INFO:tasks.workunit.client.0.smithi063.stdout:8/916: readlink d9/d3b/l2e 0 2024-05-09T06:38:22.906 INFO:tasks.workunit.client.1.smithi193.stdout:6/438: dwrite d1/d25/d2c/d3c/d3e/d58/f84 [0,4194304] 0 2024-05-09T06:38:22.906 INFO:tasks.workunit.client.1.smithi193.stdout:9/524: mkdir dd/d1e/d1f/d43/dab 0 2024-05-09T06:38:22.908 INFO:tasks.workunit.client.1.smithi193.stdout:8/449: mknod d6/c88 0 2024-05-09T06:38:22.910 INFO:tasks.workunit.client.0.smithi063.stdout:3/852: dread f2 [4194304,4194304] 0 2024-05-09T06:38:22.912 INFO:tasks.workunit.client.1.smithi193.stdout:2/476: symlink db/d37/la2 0 2024-05-09T06:38:22.913 INFO:tasks.workunit.client.1.smithi193.stdout:2/477: fdatasync db/df/f10 0 2024-05-09T06:38:22.915 INFO:tasks.workunit.client.1.smithi193.stdout:8/450: dread d6/d1c/d1b/f75 [0,4194304] 0 2024-05-09T06:38:22.916 INFO:tasks.workunit.client.1.smithi193.stdout:1/493: fdatasync d4/d16/d17/d1a/f1e 0 2024-05-09T06:38:22.920 INFO:tasks.workunit.client.0.smithi063.stdout:2/839: sync 2024-05-09T06:38:22.920 INFO:tasks.workunit.client.0.smithi063.stdout:9/811: creat d0/d8/da6/db0/db5/f11a x:0 0 0 2024-05-09T06:38:22.920 INFO:tasks.workunit.client.0.smithi063.stdout:9/812: readlink d0/d3b/d63/ldd 0 2024-05-09T06:38:22.921 INFO:tasks.workunit.client.1.smithi193.stdout:0/416: creat d18/d4c/d5f/f88 x:0 0 0 2024-05-09T06:38:22.921 INFO:tasks.workunit.client.0.smithi063.stdout:2/840: chown dc/d36/d43/d46/d49/lfc 121036718 1 2024-05-09T06:38:22.922 INFO:tasks.workunit.client.1.smithi193.stdout:0/417: write d18/d4c/f38 [265773,62996] 0 2024-05-09T06:38:22.924 INFO:tasks.workunit.client.1.smithi193.stdout:0/418: write d18/d4c/d5f/f7c [72061,15257] 0 2024-05-09T06:38:22.928 INFO:tasks.workunit.client.0.smithi063.stdout:5/800: rename d8/d19/d30/d46/f94 to d8/d53/f10e 0 2024-05-09T06:38:22.928 INFO:tasks.workunit.client.0.smithi063.stdout:0/853: dwrite dd/d1a/d2c/fa1 [0,4194304] 0 2024-05-09T06:38:22.928 INFO:tasks.workunit.client.0.smithi063.stdout:5/801: dread - d8/d35/f7b zero size 2024-05-09T06:38:22.929 INFO:tasks.workunit.client.0.smithi063.stdout:8/917: stat d9/da/d45/la5 0 2024-05-09T06:38:22.933 INFO:tasks.workunit.client.1.smithi193.stdout:6/439: mknod d1/d25/d2c/d81/d37/d4f/c97 0 2024-05-09T06:38:22.934 INFO:tasks.workunit.client.1.smithi193.stdout:3/395: dwrite d1/db/d59/f7b [0,4194304] 0 2024-05-09T06:38:22.938 INFO:tasks.workunit.client.1.smithi193.stdout:6/440: readlink d1/d25/d2c/d3c/d3e/d58/l77 0 2024-05-09T06:38:22.940 INFO:tasks.workunit.client.1.smithi193.stdout:2/478: rename db/df/f47 to db/d25/d38/d5b/fa3 0 2024-05-09T06:38:22.941 INFO:tasks.workunit.client.0.smithi063.stdout:6/838: link d2/d1f/f4c d2/d20/d26/d7f/f110 0 2024-05-09T06:38:22.941 INFO:tasks.workunit.client.1.smithi193.stdout:2/479: write db/f28 [686155,66501] 0 2024-05-09T06:38:22.943 INFO:tasks.workunit.client.1.smithi193.stdout:8/451: mkdir d6/d1c/d1b/d29/d89 0 2024-05-09T06:38:22.944 INFO:tasks.workunit.client.0.smithi063.stdout:6/839: dread d2/f38 [0,4194304] 0 2024-05-09T06:38:22.945 INFO:tasks.workunit.client.1.smithi193.stdout:8/452: chown d6/d1c/d18/d2f/d6a 860686 1 2024-05-09T06:38:22.946 INFO:tasks.workunit.client.0.smithi063.stdout:2/841: creat dc/dd/d2d/d60/dd0/ffe x:0 0 0 2024-05-09T06:38:22.950 INFO:tasks.workunit.client.0.smithi063.stdout:0/854: creat dd/d1a/d2c/d51/d72/d9e/d23/f109 x:0 0 0 2024-05-09T06:38:22.953 INFO:tasks.workunit.client.0.smithi063.stdout:2/842: dwrite dc/dd/d45/d5a/d6c/d8c/d9a/dd5/ff1 [0,4194304] 0 2024-05-09T06:38:22.959 INFO:tasks.workunit.client.0.smithi063.stdout:3/853: unlink d7/da/d2b/d66/dc0/da4/l108 0 2024-05-09T06:38:22.963 INFO:tasks.workunit.client.0.smithi063.stdout:7/805: getdents d0/d15/d4a/dc8 0 2024-05-09T06:38:22.964 INFO:tasks.workunit.client.0.smithi063.stdout:4/825: truncate d7/df/d77/f8e 1203909 0 2024-05-09T06:38:22.965 INFO:tasks.workunit.client.1.smithi193.stdout:1/494: write d4/d16/f47 [1071517,105323] 0 2024-05-09T06:38:22.967 INFO:tasks.workunit.client.1.smithi193.stdout:1/495: write d4/d16/d6b/d85/f86 [52233,8370] 0 2024-05-09T06:38:22.968 INFO:tasks.workunit.client.0.smithi063.stdout:9/813: truncate d0/d3b/da0/fde 608843 0 2024-05-09T06:38:22.971 INFO:tasks.workunit.client.1.smithi193.stdout:2/480: rename db/df/c15 to db/d37/d6d/d80/d60/d7d/ca4 0 2024-05-09T06:38:22.973 INFO:tasks.workunit.client.0.smithi063.stdout:6/840: creat d2/df/d59/d99/d9f/f111 x:0 0 0 2024-05-09T06:38:22.978 INFO:tasks.workunit.client.0.smithi063.stdout:5/802: fsync d8/d19/d30/d46/f6c 0 2024-05-09T06:38:22.979 INFO:tasks.workunit.client.1.smithi193.stdout:0/419: dwrite d18/d4c/f2b [0,4194304] 0 2024-05-09T06:38:22.981 INFO:tasks.workunit.client.1.smithi193.stdout:9/525: dwrite dd/d1e/f28 [0,4194304] 0 2024-05-09T06:38:22.984 INFO:tasks.workunit.client.1.smithi193.stdout:7/433: getdents d6/d8/d1a/d33 0 2024-05-09T06:38:22.984 INFO:tasks.workunit.client.0.smithi063.stdout:2/843: read - dc/d32/d33/fc3 zero size 2024-05-09T06:38:22.985 INFO:tasks.workunit.client.0.smithi063.stdout:8/918: truncate d9/da/d18/dc7/f78 722996 0 2024-05-09T06:38:22.986 INFO:tasks.workunit.client.1.smithi193.stdout:7/434: read - d6/d8/d29/f7a zero size 2024-05-09T06:38:22.989 INFO:tasks.workunit.client.0.smithi063.stdout:1/774: getdents d9/d2a/d3c/d4c/d83/d88/dc2/dd0 0 2024-05-09T06:38:22.991 INFO:tasks.workunit.client.1.smithi193.stdout:5/394: creat d0/d13/d2c/d37/d2b/d47/d66/f83 x:0 0 0 2024-05-09T06:38:22.993 INFO:tasks.workunit.client.0.smithi063.stdout:3/854: creat d7/da/d21/f124 x:0 0 0 2024-05-09T06:38:22.994 INFO:tasks.workunit.client.1.smithi193.stdout:1/496: truncate d4/d16/d35/f48 305803 0 2024-05-09T06:38:22.997 INFO:tasks.workunit.client.1.smithi193.stdout:2/481: mknod db/d37/d6d/d80/ca5 0 2024-05-09T06:38:22.998 INFO:tasks.workunit.client.0.smithi063.stdout:4/826: fdatasync d7/da3/da2/fc8 0 2024-05-09T06:38:23.002 INFO:tasks.workunit.client.1.smithi193.stdout:8/453: creat d6/d1c/d18/d2f/d40/d55/d7e/f8a x:0 0 0 2024-05-09T06:38:23.002 INFO:tasks.workunit.client.1.smithi193.stdout:1/497: dwrite d4/d3c/d3e/f5c [0,4194304] 0 2024-05-09T06:38:23.002 INFO:tasks.workunit.client.0.smithi063.stdout:7/806: dread d0/d15/d49/d54/d9f/d97/fae [0,4194304] 0 2024-05-09T06:38:23.026 INFO:tasks.workunit.client.1.smithi193.stdout:1/498: dread d4/f32 [4194304,4194304] 0 2024-05-09T06:38:23.028 INFO:tasks.workunit.client.1.smithi193.stdout:1/499: write d4/d16/d60/d4f/d72/d7f/f93 [898156,109955] 0 2024-05-09T06:38:23.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:22 smithi063 ceph-mon[27171]: pgmap v14: 65 pgs: 65 active+clean; 2.5 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 100 MiB/s rd, 294 MiB/s wr, 584 op/s 2024-05-09T06:38:23.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:22 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:23.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:22 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:23.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:22 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:23.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:22 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:23.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:22 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:23.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:22 smithi063 ceph-mon[27171]: Adjusting osd_memory_target on smithi063 to 1216M 2024-05-09T06:38:23.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:22 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:23.114 INFO:tasks.workunit.client.0.smithi063.stdout:6/841: mknod d2/df/d59/d99/d9f/c112 0 2024-05-09T06:38:23.120 INFO:tasks.workunit.client.1.smithi193.stdout:9/526: creat dd/df/d31/d7e/d71/fac x:0 0 0 2024-05-09T06:38:23.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:22 smithi193 ceph-mon[35141]: pgmap v14: 65 pgs: 65 active+clean; 2.5 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 100 MiB/s rd, 294 MiB/s wr, 584 op/s 2024-05-09T06:38:23.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:22 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:23.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:22 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:23.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:22 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:23.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:22 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:23.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:22 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:23.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:22 smithi193 ceph-mon[35141]: Adjusting osd_memory_target on smithi063 to 1216M 2024-05-09T06:38:23.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:22 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:23.129 INFO:tasks.workunit.client.1.smithi193.stdout:7/435: dwrite d6/dd/f6c [0,4194304] 0 2024-05-09T06:38:23.131 INFO:tasks.workunit.client.0.smithi063.stdout:8/919: rmdir d9/da/d45 39 2024-05-09T06:38:23.133 INFO:tasks.workunit.client.0.smithi063.stdout:9/814: dwrite d0/d8/d2d/faf [0,4194304] 0 2024-05-09T06:38:23.134 INFO:tasks.workunit.client.1.smithi193.stdout:2/482: write db/d37/d6d/d80/d60/d7d/f85 [1962428,47361] 0 2024-05-09T06:38:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:9/815: chown d0/d4/f37 701699 1 2024-05-09T06:38:23.141 INFO:tasks.workunit.client.0.smithi063.stdout:1/775: fdatasync d9/d2a/d35/d1e/f27 0 2024-05-09T06:38:23.143 INFO:tasks.workunit.client.1.smithi193.stdout:8/454: creat d6/d1c/d1b/d29/d57/f8b x:0 0 0 2024-05-09T06:38:23.152 INFO:tasks.workunit.client.1.smithi193.stdout:9/527: unlink dd/d1e/c3e 0 2024-05-09T06:38:23.159 INFO:tasks.workunit.client.1.smithi193.stdout:5/395: mknod d0/d13/d2c/d37/d5b/c84 0 2024-05-09T06:38:23.161 INFO:tasks.workunit.client.0.smithi063.stdout:4/827: dwrite d7/da3/d28/d31/d32/f8c [4194304,4194304] 0 2024-05-09T06:38:23.164 INFO:tasks.workunit.client.1.smithi193.stdout:2/483: readlink db/d25/d38/l8d 0 2024-05-09T06:38:23.167 INFO:tasks.workunit.client.0.smithi063.stdout:7/807: fsync d0/d15/d4a/d50/f3f 0 2024-05-09T06:38:23.171 INFO:tasks.workunit.client.1.smithi193.stdout:8/455: creat d6/d1c/d53/d56/f8c x:0 0 0 2024-05-09T06:38:23.172 INFO:tasks.workunit.client.1.smithi193.stdout:8/456: dread - d6/d1c/d18/d66/f78 zero size 2024-05-09T06:38:23.173 INFO:tasks.workunit.client.1.smithi193.stdout:8/457: fsync d6/d1c/d1b/d5d/f6d 0 2024-05-09T06:38:23.174 INFO:tasks.workunit.client.0.smithi063.stdout:7/808: dwrite d0/d15/d49/d54/d9f/d97/d9b/d9d/fa8 [0,4194304] 0 2024-05-09T06:38:23.177 INFO:tasks.workunit.client.1.smithi193.stdout:5/396: truncate d0/d13/f3d 2789751 0 2024-05-09T06:38:23.182 INFO:tasks.workunit.client.1.smithi193.stdout:2/484: creat db/d37/d6d/d80/fa6 x:0 0 0 2024-05-09T06:38:23.186 INFO:tasks.workunit.client.1.smithi193.stdout:5/397: dwrite d0/d13/d2c/d37/d5b/f70 [0,4194304] 0 2024-05-09T06:38:23.188 INFO:tasks.workunit.client.0.smithi063.stdout:2/844: symlink dc/d36/d43/d46/d49/dbb/dec/lff 0 2024-05-09T06:38:23.190 INFO:tasks.workunit.client.0.smithi063.stdout:2/845: dread - dc/dd/d45/d5a/d6c/d8f/dbf/fd9 zero size 2024-05-09T06:38:23.192 INFO:tasks.workunit.client.1.smithi193.stdout:8/458: creat d6/d5b/f8d x:0 0 0 2024-05-09T06:38:23.194 INFO:tasks.workunit.client.1.smithi193.stdout:8/459: write d6/d1c/f12 [2085526,111908] 0 2024-05-09T06:38:23.195 INFO:tasks.workunit.client.1.smithi193.stdout:5/398: dwrite d0/d13/d2c/d37/d2b/d47/d66/f83 [0,4194304] 0 2024-05-09T06:38:23.196 INFO:tasks.workunit.client.0.smithi063.stdout:9/816: creat d0/d4/d26/d39/d91/f11b x:0 0 0 2024-05-09T06:38:23.201 INFO:tasks.workunit.client.1.smithi193.stdout:2/485: creat db/d25/d38/d40/d8b/d77/fa7 x:0 0 0 2024-05-09T06:38:23.209 INFO:tasks.workunit.client.1.smithi193.stdout:2/486: dwrite db/d25/d38/d61/f73 [0,4194304] 0 2024-05-09T06:38:23.214 INFO:tasks.workunit.client.1.smithi193.stdout:9/528: getdents dd 0 2024-05-09T06:38:23.219 INFO:tasks.workunit.client.1.smithi193.stdout:2/487: mkdir db/d37/d6d/d80/d60/d7d/da8 0 2024-05-09T06:38:23.221 INFO:tasks.workunit.client.0.smithi063.stdout:7/809: mknod d0/d15/d49/d54/c111 0 2024-05-09T06:38:23.222 INFO:tasks.workunit.client.1.smithi193.stdout:5/399: getdents d0/d13/d2c/d37/d2b/d47/d60/d75 0 2024-05-09T06:38:23.224 INFO:tasks.workunit.client.1.smithi193.stdout:9/529: mkdir dd/d1e/d1f/d43/d5b/d8a/dad 0 2024-05-09T06:38:23.224 INFO:tasks.workunit.client.1.smithi193.stdout:5/400: write d0/d13/d2c/d37/d5b/d4e/f57 [1261154,55947] 0 2024-05-09T06:38:23.227 INFO:tasks.workunit.client.1.smithi193.stdout:2/488: dwrite db/d4b/f5d [0,4194304] 0 2024-05-09T06:38:23.228 INFO:tasks.workunit.client.1.smithi193.stdout:5/401: write d0/d13/d2c/d37/f68 [567943,56387] 0 2024-05-09T06:38:23.232 INFO:tasks.workunit.client.0.smithi063.stdout:8/920: symlink d9/da/d51/dc2/d127/l144 0 2024-05-09T06:38:23.233 INFO:tasks.workunit.client.1.smithi193.stdout:9/530: chown dd/df/f6e 10 1 2024-05-09T06:38:23.233 INFO:tasks.workunit.client.1.smithi193.stdout:5/402: dread d0/d13/d2c/d37/d5b/f32 [0,4194304] 0 2024-05-09T06:38:23.235 INFO:tasks.workunit.client.0.smithi063.stdout:9/817: creat d0/d4/d26/d39/d48/d80/f11c x:0 0 0 2024-05-09T06:38:23.237 INFO:tasks.workunit.client.0.smithi063.stdout:9/818: read - d0/d8/da6/db0/db5/f106 zero size 2024-05-09T06:38:23.238 INFO:tasks.workunit.client.0.smithi063.stdout:8/921: dwrite d9/da/d53/db2/d11a/f12a [0,4194304] 0 2024-05-09T06:38:23.240 INFO:tasks.workunit.client.0.smithi063.stdout:3/855: rename d7/fa8 to d7/d70/d98/d2d/d36/f125 0 2024-05-09T06:38:23.240 INFO:tasks.workunit.client.0.smithi063.stdout:8/922: chown d9/d3b/f68 2095222916 1 2024-05-09T06:38:23.248 INFO:tasks.workunit.client.0.smithi063.stdout:7/810: symlink d0/d15/d89/d10c/de7/l112 0 2024-05-09T06:38:23.256 INFO:tasks.workunit.client.0.smithi063.stdout:2/846: symlink dc/d36/d43/d46/d63/d79/da5/l100 0 2024-05-09T06:38:23.262 INFO:tasks.workunit.client.0.smithi063.stdout:9/819: mknod d0/d4/d26/d39/d91/c11d 0 2024-05-09T06:38:23.265 INFO:tasks.workunit.client.0.smithi063.stdout:3/856: creat d7/da/d21/d30/d44/dee/d10c/f126 x:0 0 0 2024-05-09T06:38:23.268 INFO:tasks.workunit.client.0.smithi063.stdout:4/828: creat d7/da3/d28/d31/d7b/f108 x:0 0 0 2024-05-09T06:38:23.268 INFO:tasks.workunit.client.0.smithi063.stdout:8/923: dread d9/d3b/f68 [0,4194304] 0 2024-05-09T06:38:23.268 INFO:tasks.workunit.client.0.smithi063.stdout:4/829: dread - d7/da3/d8b/ffd zero size 2024-05-09T06:38:23.270 INFO:tasks.workunit.client.0.smithi063.stdout:4/830: write d7/df/d35/d5a/d60/f105 [146666,121760] 0 2024-05-09T06:38:23.270 INFO:tasks.workunit.client.0.smithi063.stdout:4/831: chown d7/df/d1a/f3b 0 1 2024-05-09T06:38:23.271 INFO:tasks.workunit.client.1.smithi193.stdout:2/489: getdents db/d25 0 2024-05-09T06:38:23.272 INFO:tasks.workunit.client.0.smithi063.stdout:6/842: getdents d2/d12 0 2024-05-09T06:38:23.272 INFO:tasks.workunit.client.1.smithi193.stdout:2/490: fdatasync db/d37/d6d/d80/d60/d7d/f7e 0 2024-05-09T06:38:23.273 INFO:tasks.workunit.client.0.smithi063.stdout:6/843: fsync d2/df/f50 0 2024-05-09T06:38:23.273 INFO:tasks.workunit.client.1.smithi193.stdout:2/491: write db/f44 [161958,40186] 0 2024-05-09T06:38:23.273 INFO:tasks.workunit.client.0.smithi063.stdout:2/847: rmdir dc/d36/d43/d46/d63/d79 39 2024-05-09T06:38:23.274 INFO:tasks.workunit.client.0.smithi063.stdout:1/776: mknod d9/d2a/d39/d5d/c10f 0 2024-05-09T06:38:23.278 INFO:tasks.workunit.client.0.smithi063.stdout:2/848: dwrite dc/dd/fb5 [0,4194304] 0 2024-05-09T06:38:23.279 INFO:tasks.workunit.client.1.smithi193.stdout:2/492: dwrite db/d37/d6d/d80/f32 [0,4194304] 0 2024-05-09T06:38:23.280 INFO:tasks.workunit.client.0.smithi063.stdout:3/857: rename d7/d70/d98/d2d/f65 to d7/da/d2b/d66/dc0/da4/f127 0 2024-05-09T06:38:23.291 INFO:tasks.workunit.client.0.smithi063.stdout:4/832: mkdir d7/df/d35/d78/d109 0 2024-05-09T06:38:23.298 INFO:tasks.workunit.client.0.smithi063.stdout:2/849: fdatasync dc/f1f 0 2024-05-09T06:38:23.301 INFO:tasks.workunit.client.0.smithi063.stdout:8/924: creat d9/d3b/df8/d12f/f145 x:0 0 0 2024-05-09T06:38:23.302 INFO:tasks.workunit.client.0.smithi063.stdout:7/811: rmdir d0/d15/d89/df3 0 2024-05-09T06:38:23.305 INFO:tasks.workunit.client.0.smithi063.stdout:4/833: creat d7/da3/d28/d31/d32/d34/f10a x:0 0 0 2024-05-09T06:38:23.305 INFO:tasks.workunit.client.0.smithi063.stdout:4/834: stat d7/d3d/fe8 0 2024-05-09T06:38:23.311 INFO:tasks.workunit.client.0.smithi063.stdout:8/925: stat d9/d37/c7b 0 2024-05-09T06:38:23.313 INFO:tasks.workunit.client.0.smithi063.stdout:4/835: dread - d7/d3d/fa7 zero size 2024-05-09T06:38:23.314 INFO:tasks.workunit.client.0.smithi063.stdout:8/926: creat d9/da/d34/f146 x:0 0 0 2024-05-09T06:38:23.316 INFO:tasks.workunit.client.0.smithi063.stdout:7/812: truncate d0/d15/d89/d10c/fe0 61752 0 2024-05-09T06:38:23.318 INFO:tasks.workunit.client.0.smithi063.stdout:3/858: getdents d7/da/d2b 0 2024-05-09T06:38:23.320 INFO:tasks.workunit.client.0.smithi063.stdout:4/836: creat d7/da3/d28/d31/d71/d98/f10b x:0 0 0 2024-05-09T06:38:23.320 INFO:tasks.workunit.client.0.smithi063.stdout:4/837: fdatasync d7/da3/ffa 0 2024-05-09T06:38:23.323 INFO:tasks.workunit.client.0.smithi063.stdout:3/859: truncate d7/d22/d28/fc2 534256 0 2024-05-09T06:38:23.326 INFO:tasks.workunit.client.0.smithi063.stdout:4/838: creat d7/f10c x:0 0 0 2024-05-09T06:38:23.327 INFO:tasks.workunit.client.0.smithi063.stdout:8/927: getdents d9/da/d18/dc7/d58 0 2024-05-09T06:38:23.327 INFO:tasks.workunit.client.1.smithi193.stdout:0/420: write f8 [3273892,48426] 0 2024-05-09T06:38:23.328 INFO:tasks.workunit.client.1.smithi193.stdout:0/421: stat d18/d4c/d5f 0 2024-05-09T06:38:23.330 INFO:tasks.workunit.client.0.smithi063.stdout:4/839: mkdir d7/d50/d10d 0 2024-05-09T06:38:23.332 INFO:tasks.workunit.client.0.smithi063.stdout:8/928: stat d9/da/d18/ca0 0 2024-05-09T06:38:23.336 INFO:tasks.workunit.client.0.smithi063.stdout:4/840: mknod d7/df/d35/d5a/d60/d75/c10e 0 2024-05-09T06:38:23.337 INFO:tasks.workunit.client.0.smithi063.stdout:0/855: write dd/d1a/d2c/d51/d72/d9e/f80 [2101474,50436] 0 2024-05-09T06:38:23.337 INFO:tasks.workunit.client.0.smithi063.stdout:8/929: dwrite d9/da/d18/dc7/f113 [0,4194304] 0 2024-05-09T06:38:23.339 INFO:tasks.workunit.client.0.smithi063.stdout:8/930: stat d9/d3b/c131 0 2024-05-09T06:38:23.344 INFO:tasks.workunit.client.0.smithi063.stdout:3/860: dread d7/f1b [4194304,4194304] 0 2024-05-09T06:38:23.347 INFO:tasks.workunit.client.0.smithi063.stdout:0/856: truncate dd/d1a/d2c/d51/fd7 19488 0 2024-05-09T06:38:23.347 INFO:tasks.workunit.client.0.smithi063.stdout:0/857: chown dd/d1a/d2c/d42/cc6 318979 1 2024-05-09T06:38:23.347 INFO:tasks.workunit.client.0.smithi063.stdout:4/841: dread d7/da3/d28/d31/d32/d34/f44 [0,4194304] 0 2024-05-09T06:38:23.349 INFO:tasks.workunit.client.0.smithi063.stdout:4/842: readlink d7/df/d35/da5/ldd 0 2024-05-09T06:38:23.355 INFO:tasks.workunit.client.0.smithi063.stdout:3/861: link d7/da/d21/lc3 d7/d4c/d25/d10b/l128 0 2024-05-09T06:38:23.356 INFO:tasks.workunit.client.0.smithi063.stdout:3/862: stat d7/d4c/f69 0 2024-05-09T06:38:23.360 INFO:tasks.workunit.client.1.smithi193.stdout:7/436: mknod d6/d8/c93 0 2024-05-09T06:38:23.367 INFO:tasks.workunit.client.1.smithi193.stdout:8/460: fsync d6/d1c/d53/d56/f8c 0 2024-05-09T06:38:23.377 INFO:tasks.workunit.client.0.smithi063.stdout:3/863: unlink d7/da/d21/d30/fa3 0 2024-05-09T06:38:23.380 INFO:tasks.workunit.client.1.smithi193.stdout:7/437: truncate d6/dd/d53/f6a 743412 0 2024-05-09T06:38:23.383 INFO:tasks.workunit.client.0.smithi063.stdout:3/864: dwrite d7/d4c/d25/fd1 [0,4194304] 0 2024-05-09T06:38:23.383 INFO:tasks.workunit.client.1.smithi193.stdout:8/461: dread d6/d1c/d27/f4d [0,4194304] 0 2024-05-09T06:38:23.383 INFO:tasks.workunit.client.0.smithi063.stdout:3/865: dread - d7/da/d21/f124 zero size 2024-05-09T06:38:23.384 INFO:tasks.workunit.client.1.smithi193.stdout:8/462: read - d6/d1c/d53/d56/f8c zero size 2024-05-09T06:38:23.384 INFO:tasks.workunit.client.0.smithi063.stdout:3/866: truncate d7/d4c/ffe 317353 0 2024-05-09T06:38:23.387 INFO:tasks.workunit.client.1.smithi193.stdout:8/463: mknod d6/d1c/d18/d2f/c8e 0 2024-05-09T06:38:23.389 INFO:tasks.workunit.client.0.smithi063.stdout:3/867: fdatasync d7/da/d21/d30/f40 0 2024-05-09T06:38:23.392 INFO:tasks.workunit.client.1.smithi193.stdout:8/464: rename d6/d1c/d18/d66/l4c to d6/d1c/d53/l8f 0 2024-05-09T06:38:23.394 INFO:tasks.workunit.client.1.smithi193.stdout:7/438: link d6/d17/d7d/c8a d6/d8/d1a/d33/c94 0 2024-05-09T06:38:23.395 INFO:tasks.workunit.client.1.smithi193.stdout:8/465: symlink d6/d1c/d18/d2f/d40/d55/l90 0 2024-05-09T06:38:23.396 INFO:tasks.workunit.client.1.smithi193.stdout:7/439: stat d6/f16 0 2024-05-09T06:38:23.400 INFO:tasks.workunit.client.1.smithi193.stdout:8/466: link d6/f20 d6/d1c/d1b/d29/f91 0 2024-05-09T06:38:23.402 INFO:tasks.workunit.client.0.smithi063.stdout:7/813: stat d0/d15/d49/d54/d9f/c8d 0 2024-05-09T06:38:23.402 INFO:tasks.workunit.client.1.smithi193.stdout:7/440: dwrite d6/d8/d1a/d36/f57 [0,4194304] 0 2024-05-09T06:38:23.403 INFO:tasks.workunit.client.1.smithi193.stdout:8/467: rmdir d6/d1c/d1b/d5d/d84 39 2024-05-09T06:38:23.408 INFO:tasks.workunit.client.1.smithi193.stdout:8/468: symlink d6/l92 0 2024-05-09T06:38:23.409 INFO:tasks.workunit.client.1.smithi193.stdout:8/469: write d6/d1c/d18/f1d [2523107,43996] 0 2024-05-09T06:38:23.409 INFO:tasks.workunit.client.1.smithi193.stdout:7/441: chown d6/dd/d53/f6a 156795970 1 2024-05-09T06:38:23.410 INFO:tasks.workunit.client.1.smithi193.stdout:8/470: dread - d6/d1c/d1b/d5d/f6d zero size 2024-05-09T06:38:23.411 INFO:tasks.workunit.client.1.smithi193.stdout:8/471: chown d6/d1c/d53/d56/f6c 3 1 2024-05-09T06:38:23.412 INFO:tasks.workunit.client.1.smithi193.stdout:7/442: symlink d6/dd/l95 0 2024-05-09T06:38:23.414 INFO:tasks.workunit.client.1.smithi193.stdout:7/443: truncate d6/dd/d5e/d6b/f75 799859 0 2024-05-09T06:38:23.416 INFO:tasks.workunit.client.1.smithi193.stdout:8/472: rename d6/d1c/d53/d56 to d6/d1c/d18/d66/d93 0 2024-05-09T06:38:23.417 INFO:tasks.workunit.client.1.smithi193.stdout:8/473: readlink d6/l92 0 2024-05-09T06:38:23.417 INFO:tasks.workunit.client.1.smithi193.stdout:2/493: creat db/d25/fa9 x:0 0 0 2024-05-09T06:38:23.418 INFO:tasks.workunit.client.1.smithi193.stdout:7/444: creat d6/dd/d72/f96 x:0 0 0 2024-05-09T06:38:23.424 INFO:tasks.workunit.client.0.smithi063.stdout:5/803: creat d8/d19/f10f x:0 0 0 2024-05-09T06:38:23.424 INFO:tasks.workunit.client.1.smithi193.stdout:9/531: write dd/d1e/d66/d49/f70 [155393,42782] 0 2024-05-09T06:38:23.426 INFO:tasks.workunit.client.1.smithi193.stdout:7/445: dwrite d6/dd/f6c [4194304,4194304] 0 2024-05-09T06:38:23.428 INFO:tasks.workunit.client.1.smithi193.stdout:7/446: write d6/d8/d1a/d36/f50 [607709,77255] 0 2024-05-09T06:38:23.428 INFO:tasks.workunit.client.0.smithi063.stdout:5/804: unlink d8/d17/d18/d9c/fe8 0 2024-05-09T06:38:23.429 INFO:tasks.workunit.client.1.smithi193.stdout:7/447: write d6/dd/d70/f8e [259111,75235] 0 2024-05-09T06:38:23.433 INFO:tasks.workunit.client.1.smithi193.stdout:7/448: mknod d6/d17/d7d/c97 0 2024-05-09T06:38:23.437 INFO:tasks.workunit.client.1.smithi193.stdout:9/532: rename dd/d1e/d1f/d43/d5b/f9f to dd/df/d15/d98/fae 0 2024-05-09T06:38:23.439 INFO:tasks.workunit.client.1.smithi193.stdout:7/449: rmdir d6/d8/d29 39 2024-05-09T06:38:23.442 INFO:tasks.workunit.client.1.smithi193.stdout:9/533: symlink dd/d1e/d1f/d43/d5b/d8a/dad/laf 0 2024-05-09T06:38:23.444 INFO:tasks.workunit.client.1.smithi193.stdout:7/450: symlink d6/d8/l98 0 2024-05-09T06:38:23.445 INFO:tasks.workunit.client.0.smithi063.stdout:2/850: sync 2024-05-09T06:38:23.445 INFO:tasks.workunit.client.0.smithi063.stdout:1/777: sync 2024-05-09T06:38:23.447 INFO:tasks.workunit.client.1.smithi193.stdout:7/451: mkdir d6/d17/d99 0 2024-05-09T06:38:23.449 INFO:tasks.workunit.client.0.smithi063.stdout:9/820: write d0/d3b/f78 [939744,124119] 0 2024-05-09T06:38:23.450 INFO:tasks.workunit.client.0.smithi063.stdout:1/778: dread d9/d2a/d39/d5d/fc6 [0,4194304] 0 2024-05-09T06:38:23.451 INFO:tasks.workunit.client.1.smithi193.stdout:7/452: fsync d6/dd/d53/f58 0 2024-05-09T06:38:23.451 INFO:tasks.workunit.client.0.smithi063.stdout:2/851: dread dc/dd/d45/d5a/d6c/d8c/d9a/dd5/ff1 [0,4194304] 0 2024-05-09T06:38:23.452 INFO:tasks.workunit.client.1.smithi193.stdout:7/453: stat d6/f30 0 2024-05-09T06:38:23.456 INFO:tasks.workunit.client.0.smithi063.stdout:2/852: dwrite dc/dd/f64 [0,4194304] 0 2024-05-09T06:38:23.457 INFO:tasks.workunit.client.0.smithi063.stdout:2/853: write dc/d32/d33/fb0 [2894323,60042] 0 2024-05-09T06:38:23.461 INFO:tasks.workunit.client.0.smithi063.stdout:9/821: mkdir d0/d4/d57/d60/d76/d11e 0 2024-05-09T06:38:23.464 INFO:tasks.workunit.client.0.smithi063.stdout:6/844: write d2/d1f/f4c [2754453,51950] 0 2024-05-09T06:38:23.475 INFO:tasks.workunit.client.0.smithi063.stdout:1/779: dread d9/d10/f3e [0,4194304] 0 2024-05-09T06:38:23.475 INFO:tasks.workunit.client.0.smithi063.stdout:8/931: rename d9/da/d18/dc7/le3 to d9/da/d18/d24/d71/l147 0 2024-05-09T06:38:23.480 INFO:tasks.workunit.client.0.smithi063.stdout:3/868: rename d7/d70/d98/d86/f119 to d7/d22/d10a/f129 0 2024-05-09T06:38:23.481 INFO:tasks.workunit.client.1.smithi193.stdout:0/422: write d18/d4c/f3f [1176645,25808] 0 2024-05-09T06:38:23.481 INFO:tasks.workunit.client.0.smithi063.stdout:3/869: dread - d7/da/d2b/d66/fb4 zero size 2024-05-09T06:38:23.483 INFO:tasks.workunit.client.1.smithi193.stdout:0/423: mknod d18/d1d/d56/c89 0 2024-05-09T06:38:23.483 INFO:tasks.workunit.client.1.smithi193.stdout:0/424: readlink d18/l40 0 2024-05-09T06:38:23.484 INFO:tasks.workunit.client.0.smithi063.stdout:3/870: symlink d7/da/d2b/d66/dc0/l12a 0 2024-05-09T06:38:23.484 INFO:tasks.workunit.client.1.smithi193.stdout:0/425: fsync d18/d4c/d47/f58 0 2024-05-09T06:38:23.484 INFO:tasks.workunit.client.1.smithi193.stdout:0/426: fsync d18/d4c/d47/d54/f73 0 2024-05-09T06:38:23.485 INFO:tasks.workunit.client.1.smithi193.stdout:0/427: write d18/d1d/f30 [2098674,130475] 0 2024-05-09T06:38:23.488 INFO:tasks.workunit.client.0.smithi063.stdout:0/858: write dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/f6e [66986,65079] 0 2024-05-09T06:38:23.490 INFO:tasks.workunit.client.0.smithi063.stdout:4/843: truncate d7/fb 1486771 0 2024-05-09T06:38:23.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/871: symlink d7/da/l12b 0 2024-05-09T06:38:23.492 INFO:tasks.workunit.client.0.smithi063.stdout:3/872: write d7/f17 [5179030,40037] 0 2024-05-09T06:38:23.496 INFO:tasks.workunit.client.0.smithi063.stdout:0/859: creat dd/f10a x:0 0 0 2024-05-09T06:38:23.497 INFO:tasks.workunit.client.0.smithi063.stdout:0/860: write dd/d1a/d2c/d51/d72/d9e/d23/f109 [215807,111883] 0 2024-05-09T06:38:23.498 INFO:tasks.workunit.client.0.smithi063.stdout:3/873: creat d7/f12c x:0 0 0 2024-05-09T06:38:23.499 INFO:tasks.workunit.client.0.smithi063.stdout:0/861: chown dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fa6 1186 1 2024-05-09T06:38:23.499 INFO:tasks.workunit.client.0.smithi063.stdout:3/874: fdatasync d7/d22/dfc/f107 0 2024-05-09T06:38:23.500 INFO:tasks.workunit.client.0.smithi063.stdout:3/875: fdatasync d7/d4c/d25/fa0 0 2024-05-09T06:38:23.501 INFO:tasks.workunit.client.1.smithi193.stdout:5/403: write d0/d13/d2c/d37/f11 [2026165,123645] 0 2024-05-09T06:38:23.502 INFO:tasks.workunit.client.0.smithi063.stdout:3/876: write d7/da/d21/d30/d44/d48/f6f [3561664,34715] 0 2024-05-09T06:38:23.504 INFO:tasks.workunit.client.1.smithi193.stdout:5/404: fdatasync d0/d13/d2c/f4b 0 2024-05-09T06:38:23.505 INFO:tasks.workunit.client.0.smithi063.stdout:0/862: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/fdd [0,4194304] 0 2024-05-09T06:38:23.509 INFO:tasks.workunit.client.0.smithi063.stdout:0/863: creat dd/d1a/d2c/d51/d76/d97/f10b x:0 0 0 2024-05-09T06:38:23.510 INFO:tasks.workunit.client.0.smithi063.stdout:0/864: write dd/d1a/d2c/d51/d76/d97/fe7 [597661,116155] 0 2024-05-09T06:38:23.512 INFO:tasks.workunit.client.0.smithi063.stdout:3/877: dread d7/da/d2b/d66/dc0/f111 [0,4194304] 0 2024-05-09T06:38:23.513 INFO:tasks.workunit.client.1.smithi193.stdout:5/405: creat d0/f85 x:0 0 0 2024-05-09T06:38:23.514 INFO:tasks.workunit.client.0.smithi063.stdout:3/878: fdatasync d7/da/d2b/d66/d74/f92 0 2024-05-09T06:38:23.516 INFO:tasks.workunit.client.0.smithi063.stdout:0/865: rename dd/d1a/f25 to dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/f10c 0 2024-05-09T06:38:23.520 INFO:tasks.workunit.client.1.smithi193.stdout:5/406: dwrite d0/d13/d2c/d37/d5b/f70 [0,4194304] 0 2024-05-09T06:38:23.521 INFO:tasks.workunit.client.1.smithi193.stdout:5/407: readlink d0/d13/d2c/d37/d2b/l5e 0 2024-05-09T06:38:23.524 INFO:tasks.workunit.client.0.smithi063.stdout:3/879: creat d7/da/f12d x:0 0 0 2024-05-09T06:38:23.525 INFO:tasks.workunit.client.0.smithi063.stdout:7/814: write d0/d15/d4a/dc8/fe9 [2210397,66348] 0 2024-05-09T06:38:23.527 INFO:tasks.workunit.client.1.smithi193.stdout:5/408: link d0/d6d/l77 d0/d13/d2c/d4d/d67/l86 0 2024-05-09T06:38:23.528 INFO:tasks.workunit.client.0.smithi063.stdout:7/815: chown d0/d15/d49/f4c 322925 1 2024-05-09T06:38:23.528 INFO:tasks.workunit.client.0.smithi063.stdout:0/866: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fb6 [0,4194304] 0 2024-05-09T06:38:23.530 INFO:tasks.workunit.client.0.smithi063.stdout:3/880: symlink d7/da/d2b/d66/d74/l12e 0 2024-05-09T06:38:23.531 INFO:tasks.workunit.client.0.smithi063.stdout:7/816: mkdir d0/d15/d89/d113 0 2024-05-09T06:38:23.534 INFO:tasks.workunit.client.1.smithi193.stdout:5/409: mkdir d0/d13/d2c/d37/d2b/d47/d60/d87 0 2024-05-09T06:38:23.535 INFO:tasks.workunit.client.0.smithi063.stdout:0/867: fsync dd/d1a/d2c/d51/d72/d9e/d23/d33/f3e 0 2024-05-09T06:38:23.538 INFO:tasks.workunit.client.0.smithi063.stdout:3/881: mknod d7/da/d2b/d66/d74/d7e/c12f 0 2024-05-09T06:38:23.538 INFO:tasks.workunit.client.1.smithi193.stdout:2/494: fsync db/d25/d38/d40/f72 0 2024-05-09T06:38:23.540 INFO:tasks.workunit.client.1.smithi193.stdout:8/474: dwrite d6/fd [0,4194304] 0 2024-05-09T06:38:23.542 INFO:tasks.workunit.client.1.smithi193.stdout:8/475: write d6/d1c/d18/d2f/d40/d55/d7e/f8a [926935,41280] 0 2024-05-09T06:38:23.546 INFO:tasks.workunit.client.0.smithi063.stdout:5/805: dwrite d8/d17/f56 [0,4194304] 0 2024-05-09T06:38:23.551 INFO:tasks.workunit.client.1.smithi193.stdout:2/495: mkdir db/d37/d6d/d80/daa 0 2024-05-09T06:38:23.554 INFO:tasks.workunit.client.0.smithi063.stdout:3/882: creat d7/da/d21/d30/d44/dee/f130 x:0 0 0 2024-05-09T06:38:23.558 INFO:tasks.workunit.client.1.smithi193.stdout:9/534: dwrite dd/df/f6e [0,4194304] 0 2024-05-09T06:38:23.558 INFO:tasks.workunit.client.1.smithi193.stdout:8/476: rename d6/d1c/d1b/d29/c7b to d6/d1c/c94 0 2024-05-09T06:38:23.565 INFO:tasks.workunit.client.0.smithi063.stdout:5/806: dwrite d8/d17/f3d [4194304,4194304] 0 2024-05-09T06:38:23.567 INFO:tasks.workunit.client.0.smithi063.stdout:3/883: mknod d7/da/d21/c131 0 2024-05-09T06:38:23.568 INFO:tasks.workunit.client.1.smithi193.stdout:7/454: write d6/d8/f35 [4053746,26692] 0 2024-05-09T06:38:23.568 INFO:tasks.workunit.client.1.smithi193.stdout:9/535: truncate dd/df/d31/d7e/f32 3516398 0 2024-05-09T06:38:23.571 INFO:tasks.workunit.client.0.smithi063.stdout:3/884: read d7/da/d2b/d66/dc0/f6e [406594,103474] 0 2024-05-09T06:38:23.571 INFO:tasks.workunit.client.0.smithi063.stdout:6/845: write d2/f4d [977207,126304] 0 2024-05-09T06:38:23.572 INFO:tasks.workunit.client.1.smithi193.stdout:2/496: getdents db/df/d89 0 2024-05-09T06:38:23.573 INFO:tasks.workunit.client.0.smithi063.stdout:5/807: symlink d8/d17/d75/dc9/l110 0 2024-05-09T06:38:23.573 INFO:tasks.workunit.client.1.smithi193.stdout:5/410: dread d0/d13/d2c/d37/d5b/f6 [0,4194304] 0 2024-05-09T06:38:23.576 INFO:tasks.workunit.client.0.smithi063.stdout:2/854: dwrite dc/f27 [4194304,4194304] 0 2024-05-09T06:38:23.578 INFO:tasks.workunit.client.0.smithi063.stdout:7/817: link d0/d15/f26 d0/d15/d49/d4f/d66/d67/d86/f114 0 2024-05-09T06:38:23.579 INFO:tasks.workunit.client.1.smithi193.stdout:7/455: dread d6/dd/d7e/f82 [0,4194304] 0 2024-05-09T06:38:23.580 INFO:tasks.workunit.client.0.smithi063.stdout:8/932: write d9/da/d53/dbc/d102/f104 [1318297,33716] 0 2024-05-09T06:38:23.581 INFO:tasks.workunit.client.0.smithi063.stdout:6/846: dwrite d2/f4d [0,4194304] 0 2024-05-09T06:38:23.582 INFO:tasks.workunit.client.0.smithi063.stdout:8/933: chown d9/da/l3d 9130727 1 2024-05-09T06:38:23.584 INFO:tasks.workunit.client.0.smithi063.stdout:8/934: chown d9/da/d53/dbc/fc1 1480947 1 2024-05-09T06:38:23.589 INFO:tasks.workunit.client.0.smithi063.stdout:5/808: mkdir d8/d17/d75/dc9/d111 0 2024-05-09T06:38:23.590 INFO:tasks.workunit.client.1.smithi193.stdout:0/428: dwrite d18/d27/f34 [0,4194304] 0 2024-05-09T06:38:23.593 INFO:tasks.workunit.client.1.smithi193.stdout:4/540: sync 2024-05-09T06:38:23.593 INFO:tasks.workunit.client.1.smithi193.stdout:1/500: sync 2024-05-09T06:38:23.594 INFO:tasks.workunit.client.1.smithi193.stdout:6/441: sync 2024-05-09T06:38:23.594 INFO:tasks.workunit.client.1.smithi193.stdout:3/396: sync 2024-05-09T06:38:23.595 INFO:tasks.workunit.client.0.smithi063.stdout:2/855: rename dc/d36/l5f to dc/d36/d43/d46/d63/d9e/l101 0 2024-05-09T06:38:23.596 INFO:tasks.workunit.client.0.smithi063.stdout:2/856: readlink dc/d36/d43/d46/l4b 0 2024-05-09T06:38:23.600 INFO:tasks.workunit.client.1.smithi193.stdout:5/411: symlink d0/d13/d2c/d7e/l88 0 2024-05-09T06:38:23.600 INFO:tasks.workunit.client.1.smithi193.stdout:6/442: read d1/d25/d2c/d3c/d3e/d58/f62 [2018080,15307] 0 2024-05-09T06:38:23.601 INFO:tasks.workunit.client.0.smithi063.stdout:4/844: write d7/da3/d28/f7d [495142,125211] 0 2024-05-09T06:38:23.602 INFO:tasks.workunit.client.0.smithi063.stdout:7/818: creat d0/d15/d49/d54/d9f/dd2/df9/f115 x:0 0 0 2024-05-09T06:38:23.603 INFO:tasks.workunit.client.1.smithi193.stdout:9/536: link dd/l97 dd/d1e/d1f/d48/d55/d86/lb0 0 2024-05-09T06:38:23.603 INFO:tasks.workunit.client.0.smithi063.stdout:3/885: creat d7/d70/d98/dbb/d10d/f132 x:0 0 0 2024-05-09T06:38:23.605 INFO:tasks.workunit.client.1.smithi193.stdout:1/501: dwrite d4/d16/d60/f18 [4194304,4194304] 0 2024-05-09T06:38:23.606 INFO:tasks.workunit.client.0.smithi063.stdout:5/809: dread - d8/d17/d75/f7f zero size 2024-05-09T06:38:23.607 INFO:tasks.workunit.client.1.smithi193.stdout:1/502: chown d4/ff 0 1 2024-05-09T06:38:23.610 INFO:tasks.workunit.client.0.smithi063.stdout:3/886: dwrite d7/d70/d98/dbb/ff1 [0,4194304] 0 2024-05-09T06:38:23.612 INFO:tasks.workunit.client.1.smithi193.stdout:3/397: chown d1/db/d2f/d78/d86/c87 3999763 1 2024-05-09T06:38:23.613 INFO:tasks.workunit.client.1.smithi193.stdout:2/497: dread db/d25/d38/d40/f58 [0,4194304] 0 2024-05-09T06:38:23.620 INFO:tasks.workunit.client.0.smithi063.stdout:6/847: dread d2/d20/f8a [4194304,4194304] 0 2024-05-09T06:38:23.626 INFO:tasks.workunit.client.1.smithi193.stdout:9/537: write dd/f82 [434761,34064] 0 2024-05-09T06:38:23.626 INFO:tasks.workunit.client.1.smithi193.stdout:9/538: chown dd/c1a 116627 1 2024-05-09T06:38:23.635 INFO:tasks.workunit.client.1.smithi193.stdout:9/539: dwrite dd/d1e/d66/d49/f70 [0,4194304] 0 2024-05-09T06:38:23.637 INFO:tasks.workunit.client.0.smithi063.stdout:0/868: dwrite f2 [0,4194304] 0 2024-05-09T06:38:23.637 INFO:tasks.workunit.client.1.smithi193.stdout:9/540: chown dd/df/d15/d8c/f2a 176012972 1 2024-05-09T06:38:23.638 INFO:tasks.workunit.client.1.smithi193.stdout:7/456: dread d6/d8/d29/d2f/f4f [0,4194304] 0 2024-05-09T06:38:23.638 INFO:tasks.workunit.client.0.smithi063.stdout:9/822: sync 2024-05-09T06:38:23.638 INFO:tasks.workunit.client.0.smithi063.stdout:9/823: chown d0/d4/l50 13 1 2024-05-09T06:38:23.639 INFO:tasks.workunit.client.1.smithi193.stdout:9/541: stat dd/d1e/c67 0 2024-05-09T06:38:23.649 INFO:tasks.workunit.client.1.smithi193.stdout:2/498: creat db/d25/d38/fab x:0 0 0 2024-05-09T06:38:23.658 INFO:tasks.workunit.client.1.smithi193.stdout:3/398: dread d1/db/f18 [0,4194304] 0 2024-05-09T06:38:23.659 INFO:tasks.workunit.client.1.smithi193.stdout:3/399: fsync d1/db/f90 0 2024-05-09T06:38:23.660 INFO:tasks.workunit.client.1.smithi193.stdout:4/541: creat d1a/fa5 x:0 0 0 2024-05-09T06:38:23.662 INFO:tasks.workunit.client.1.smithi193.stdout:7/457: truncate d6/dd/f3f 5167397 0 2024-05-09T06:38:23.666 INFO:tasks.workunit.client.1.smithi193.stdout:7/458: dread d6/d8/d29/d2f/f4f [0,4194304] 0 2024-05-09T06:38:23.671 INFO:tasks.workunit.client.1.smithi193.stdout:2/499: symlink db/d37/d6d/lac 0 2024-05-09T06:38:23.673 INFO:tasks.workunit.client.1.smithi193.stdout:6/443: write d1/f70 [496439,33432] 0 2024-05-09T06:38:23.673 INFO:tasks.workunit.client.1.smithi193.stdout:3/400: chown d1/db/l68 28760158 1 2024-05-09T06:38:23.679 INFO:tasks.workunit.client.1.smithi193.stdout:0/429: write d18/d65/f79 [2559657,43495] 0 2024-05-09T06:38:23.681 INFO:tasks.workunit.client.0.smithi063.stdout:4/845: mkdir d7/d50/d10f 0 2024-05-09T06:38:23.682 INFO:tasks.workunit.client.1.smithi193.stdout:5/412: dwrite d0/d13/d2c/d37/d5b/d4e/f6b [0,4194304] 0 2024-05-09T06:38:23.682 INFO:tasks.workunit.client.1.smithi193.stdout:7/459: truncate d6/d8/d29/d2f/f4c 625543 0 2024-05-09T06:38:23.683 INFO:tasks.workunit.client.1.smithi193.stdout:7/460: read d6/d8/d29/d2f/f4f [2311925,121099] 0 2024-05-09T06:38:23.685 INFO:tasks.workunit.client.0.smithi063.stdout:7/819: rename d0/d15/d49/d54/d9f/ce6 to d0/d15/d4a/d68/c116 0 2024-05-09T06:38:23.688 INFO:tasks.workunit.client.1.smithi193.stdout:1/503: rmdir d4/d16/d35/d98 0 2024-05-09T06:38:23.689 INFO:tasks.workunit.client.1.smithi193.stdout:1/504: write d4/d14/f75 [4860818,102507] 0 2024-05-09T06:38:23.691 INFO:tasks.workunit.client.1.smithi193.stdout:1/505: write d4/d14/f51 [932257,21644] 0 2024-05-09T06:38:23.691 INFO:tasks.workunit.client.1.smithi193.stdout:2/500: symlink db/d37/d6d/d80/lad 0 2024-05-09T06:38:23.692 INFO:tasks.workunit.client.1.smithi193.stdout:2/501: fdatasync db/f28 0 2024-05-09T06:38:23.694 INFO:tasks.workunit.client.1.smithi193.stdout:3/401: mkdir d1/db/d2f/d78/d86/d94 0 2024-05-09T06:38:23.695 INFO:tasks.workunit.client.1.smithi193.stdout:4/542: mkdir d1a/da6 0 2024-05-09T06:38:23.698 INFO:tasks.workunit.client.0.smithi063.stdout:3/887: dwrite d7/d4c/f69 [0,4194304] 0 2024-05-09T06:38:23.701 INFO:tasks.workunit.client.0.smithi063.stdout:9/824: dread d0/d3b/da0/fde [0,4194304] 0 2024-05-09T06:38:23.703 INFO:tasks.workunit.client.0.smithi063.stdout:2/857: unlink dc/dd/d45/d5a/lb3 0 2024-05-09T06:38:23.705 INFO:tasks.workunit.client.0.smithi063.stdout:1/780: sync 2024-05-09T06:38:23.705 INFO:tasks.workunit.client.0.smithi063.stdout:8/935: sync 2024-05-09T06:38:23.706 INFO:tasks.workunit.client.0.smithi063.stdout:1/781: dread - d9/d2a/d3c/d4c/d83/d88/dfb/f107 zero size 2024-05-09T06:38:23.712 INFO:tasks.workunit.client.0.smithi063.stdout:7/820: dwrite d0/d15/d4a/d68/f9e [0,4194304] 0 2024-05-09T06:38:23.713 INFO:tasks.workunit.client.0.smithi063.stdout:6/848: unlink d2/d20/d4a/c8c 0 2024-05-09T06:38:23.714 INFO:tasks.workunit.client.0.smithi063.stdout:0/869: mknod dd/d1a/d2c/d51/d72/d9e/c10d 0 2024-05-09T06:38:23.718 INFO:tasks.workunit.client.0.smithi063.stdout:3/888: read d7/f11 [1537811,98162] 0 2024-05-09T06:38:23.721 INFO:tasks.workunit.client.1.smithi193.stdout:4/543: fsync d1a/d67/f84 0 2024-05-09T06:38:23.724 INFO:tasks.workunit.client.1.smithi193.stdout:9/542: dwrite dd/df/d31/f4c [0,4194304] 0 2024-05-09T06:38:23.725 INFO:tasks.workunit.client.0.smithi063.stdout:2/858: unlink dc/d6e/cf3 0 2024-05-09T06:38:23.727 INFO:tasks.workunit.client.1.smithi193.stdout:6/444: mknod d1/de/d11/d18/d94/c98 0 2024-05-09T06:38:23.729 INFO:tasks.workunit.client.1.smithi193.stdout:6/445: write d1/d25/d2c/d76/f7d [229118,21368] 0 2024-05-09T06:38:23.731 INFO:tasks.workunit.client.0.smithi063.stdout:4/846: symlink d7/d50/deb/l110 0 2024-05-09T06:38:23.732 INFO:tasks.workunit.client.0.smithi063.stdout:4/847: readlink d7/df/d35/l84 0 2024-05-09T06:38:23.734 INFO:tasks.workunit.client.0.smithi063.stdout:1/782: fsync d9/d2a/d35/d24/fac 0 2024-05-09T06:38:23.735 INFO:tasks.workunit.client.1.smithi193.stdout:0/430: symlink d18/d27/l8a 0 2024-05-09T06:38:23.739 INFO:tasks.workunit.client.0.smithi063.stdout:6/849: mknod d2/d4/d96/c113 0 2024-05-09T06:38:23.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/821: mkdir d0/d15/d89/d10c/de7/d117 0 2024-05-09T06:38:23.741 INFO:tasks.workunit.client.1.smithi193.stdout:2/502: symlink db/d9b/lae 0 2024-05-09T06:38:23.741 INFO:tasks.workunit.client.1.smithi193.stdout:8/477: dwrite d6/d1c/d1b/d29/f4f [0,4194304] 0 2024-05-09T06:38:23.742 INFO:tasks.workunit.client.0.smithi063.stdout:7/822: dread - d0/d15/d89/d10c/de3/ff8 zero size 2024-05-09T06:38:23.743 INFO:tasks.workunit.client.0.smithi063.stdout:5/810: dwrite d8/d17/fc1 [0,4194304] 0 2024-05-09T06:38:23.744 INFO:tasks.workunit.client.0.smithi063.stdout:5/811: chown d8/d17/d18/fc3 7272465 1 2024-05-09T06:38:23.750 INFO:tasks.workunit.client.0.smithi063.stdout:8/936: truncate d9/da/d18/dc7/d58/f8e 1939497 0 2024-05-09T06:38:23.753 INFO:tasks.workunit.client.1.smithi193.stdout:6/446: creat d1/d25/d2c/d3c/f99 x:0 0 0 2024-05-09T06:38:23.755 INFO:tasks.workunit.client.0.smithi063.stdout:4/848: mkdir d7/da3/d28/d31/d7b/d81/d111 0 2024-05-09T06:38:23.755 INFO:tasks.workunit.client.1.smithi193.stdout:0/431: stat cc 0 2024-05-09T06:38:23.759 INFO:tasks.workunit.client.0.smithi063.stdout:1/783: creat d9/d2a/d3c/d4c/d83/d88/dc2/f110 x:0 0 0 2024-05-09T06:38:23.760 INFO:tasks.workunit.client.1.smithi193.stdout:7/461: rename d6/d8/d1a/l79 to d6/d8/d29/l9a 0 2024-05-09T06:38:23.762 INFO:tasks.workunit.client.1.smithi193.stdout:7/462: write d6/d8/f35 [3584013,37459] 0 2024-05-09T06:38:23.766 INFO:tasks.workunit.client.0.smithi063.stdout:0/870: symlink dd/l10e 0 2024-05-09T06:38:23.766 INFO:tasks.workunit.client.0.smithi063.stdout:0/871: read - dd/d1a/d2c/d51/d72/d9e/d23/dbb/ffe zero size 2024-05-09T06:38:23.770 INFO:tasks.workunit.client.1.smithi193.stdout:4/544: mknod d1a/d6b/d60/ca7 0 2024-05-09T06:38:23.770 INFO:tasks.workunit.client.1.smithi193.stdout:8/478: dread d6/d1c/d18/f74 [0,4194304] 0 2024-05-09T06:38:23.774 INFO:tasks.workunit.client.0.smithi063.stdout:3/889: write d7/da/d2b/d66/d74/d7e/f7f [172874,476] 0 2024-05-09T06:38:23.776 INFO:tasks.workunit.client.1.smithi193.stdout:6/447: creat d1/d25/d2c/d81/d37/d4f/f9a x:0 0 0 2024-05-09T06:38:23.777 INFO:tasks.workunit.client.1.smithi193.stdout:6/448: readlink d1/d25/d4c/l7b 0 2024-05-09T06:38:23.777 INFO:tasks.workunit.client.0.smithi063.stdout:9/825: link d0/d8/d2d/l40 d0/d4/d26/d39/d48/d80/d84/dc2/d10f/l11f 0 2024-05-09T06:38:23.778 INFO:tasks.workunit.client.1.smithi193.stdout:6/449: dread - d1/d25/d2c/f75 zero size 2024-05-09T06:38:23.784 INFO:tasks.workunit.client.1.smithi193.stdout:5/413: getdents d0/d13/d2c/d37/d5b/d4e 0 2024-05-09T06:38:23.786 INFO:tasks.workunit.client.0.smithi063.stdout:4/849: mkdir d7/da3/da2/d112 0 2024-05-09T06:38:23.787 INFO:tasks.workunit.client.1.smithi193.stdout:1/506: rename d4/ff to d4/d16/f9b 0 2024-05-09T06:38:23.788 INFO:tasks.workunit.client.1.smithi193.stdout:1/507: read d4/d16/d60/f18 [6895102,8679] 0 2024-05-09T06:38:23.789 INFO:tasks.workunit.client.0.smithi063.stdout:5/812: write d8/d17/d18/f1f [883463,20576] 0 2024-05-09T06:38:23.790 INFO:tasks.workunit.client.0.smithi063.stdout:6/850: fsync d2/d1f/d6c/fb7 0 2024-05-09T06:38:23.791 INFO:tasks.workunit.client.0.smithi063.stdout:6/851: chown d2/d4/d7c/d42/def/cf9 4586069 1 2024-05-09T06:38:23.792 INFO:tasks.workunit.client.1.smithi193.stdout:7/463: rmdir d6/dd/d5e/d6b 39 2024-05-09T06:38:23.799 INFO:tasks.workunit.client.0.smithi063.stdout:0/872: rename dd/d60/fa0 to dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/f10f 0 2024-05-09T06:38:23.804 INFO:tasks.workunit.client.0.smithi063.stdout:3/890: fdatasync d7/da/d2b/d66/dc0/faa 0 2024-05-09T06:38:23.805 INFO:tasks.workunit.client.1.smithi193.stdout:9/543: creat dd/d1e/d1f/d48/d55/fb1 x:0 0 0 2024-05-09T06:38:23.810 INFO:tasks.workunit.client.1.smithi193.stdout:0/432: truncate d18/d1d/f1e 371222 0 2024-05-09T06:38:23.812 INFO:tasks.workunit.client.0.smithi063.stdout:8/937: rmdir d9/da/d45/de4 39 2024-05-09T06:38:23.813 INFO:tasks.workunit.client.1.smithi193.stdout:0/433: dread - d18/d27/f41 zero size 2024-05-09T06:38:23.813 INFO:tasks.workunit.client.1.smithi193.stdout:9/544: dwrite dd/df/d15/f8e [0,4194304] 0 2024-05-09T06:38:23.814 INFO:tasks.workunit.client.0.smithi063.stdout:8/938: dread d9/d41/dec/dc5/fd2 [0,4194304] 0 2024-05-09T06:38:23.814 INFO:tasks.workunit.client.1.smithi193.stdout:3/402: rename d1/db/d59/d63/c91 to d1/db/d2f/c95 0 2024-05-09T06:38:23.815 INFO:tasks.workunit.client.0.smithi063.stdout:4/850: creat d7/df/d77/f113 x:0 0 0 2024-05-09T06:38:23.817 INFO:tasks.workunit.client.0.smithi063.stdout:4/851: truncate d7/da3/d28/d31/d32/d34/ff5 619817 0 2024-05-09T06:38:23.820 INFO:tasks.workunit.client.0.smithi063.stdout:5/813: stat d8/l28 0 2024-05-09T06:38:23.822 INFO:tasks.workunit.client.1.smithi193.stdout:8/479: write d6/d1c/d1b/f75 [275956,97308] 0 2024-05-09T06:38:23.824 INFO:tasks.workunit.client.0.smithi063.stdout:7/823: dwrite d0/f10 [0,4194304] 0 2024-05-09T06:38:23.825 INFO:tasks.workunit.client.0.smithi063.stdout:9/826: write d0/d8/f69 [1527343,8749] 0 2024-05-09T06:38:23.827 INFO:tasks.workunit.client.0.smithi063.stdout:7/824: dread - d0/d15/d49/d4f/d66/df1/ffc zero size 2024-05-09T06:38:23.833 INFO:tasks.workunit.client.0.smithi063.stdout:5/814: dread d8/d53/f8b [0,4194304] 0 2024-05-09T06:38:23.834 INFO:tasks.workunit.client.1.smithi193.stdout:0/434: mkdir d18/d4c/d64/d8b 0 2024-05-09T06:38:23.837 INFO:tasks.workunit.client.0.smithi063.stdout:1/784: dwrite d9/d2a/d35/d1e/fa1 [0,4194304] 0 2024-05-09T06:38:23.837 INFO:tasks.workunit.client.1.smithi193.stdout:9/545: rmdir dd/df/d31/d7e/d37 39 2024-05-09T06:38:23.838 INFO:tasks.workunit.client.0.smithi063.stdout:1/785: stat d9/d2a/d35/d1e/d9d/d3f/d59/d62/f8b 0 2024-05-09T06:38:23.840 INFO:tasks.workunit.client.1.smithi193.stdout:6/450: write d1/de/d11/d20/f27 [726426,77000] 0 2024-05-09T06:38:23.842 INFO:tasks.workunit.client.0.smithi063.stdout:2/859: getdents dc/d36/d43/d46/d49/ddc 0 2024-05-09T06:38:23.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/860: write dc/d32/de5/ff2 [322036,107324] 0 2024-05-09T06:38:23.846 INFO:tasks.workunit.client.0.smithi063.stdout:8/939: rename d9/da/d51/c93 to d9/da/d51/c148 0 2024-05-09T06:38:23.848 INFO:tasks.workunit.client.1.smithi193.stdout:8/480: mkdir d6/d1c/d18/d66/d93/d95 0 2024-05-09T06:38:23.848 INFO:tasks.workunit.client.0.smithi063.stdout:4/852: rename d7/df/d35/cae to d7/da6/c114 0 2024-05-09T06:38:23.849 INFO:tasks.workunit.client.1.smithi193.stdout:0/435: readlink d18/d27/l3a 0 2024-05-09T06:38:23.849 INFO:tasks.workunit.client.1.smithi193.stdout:3/403: dwrite d1/db/d1b/f3b [0,4194304] 0 2024-05-09T06:38:23.849 INFO:tasks.workunit.client.0.smithi063.stdout:4/853: chown d7/da3/d28/d31/d32/f51 437 1 2024-05-09T06:38:23.850 INFO:tasks.workunit.client.0.smithi063.stdout:4/854: readlink d7/da3/d28/d31/d32/d34/db4/ld6 0 2024-05-09T06:38:23.850 INFO:tasks.workunit.client.1.smithi193.stdout:0/436: truncate d18/d65/f79 3103053 0 2024-05-09T06:38:23.851 INFO:tasks.workunit.client.1.smithi193.stdout:8/481: fdatasync d6/d1c/d18/f1d 0 2024-05-09T06:38:23.851 INFO:tasks.workunit.client.1.smithi193.stdout:3/404: chown d1/db/d2f 3208129 1 2024-05-09T06:38:23.853 INFO:tasks.workunit.client.1.smithi193.stdout:0/437: dread - d18/d4c/d47/d59/d76/f87 zero size 2024-05-09T06:38:23.853 INFO:tasks.workunit.client.1.smithi193.stdout:9/546: mkdir dd/d1e/d1f/d48/d55/d41/d68/db2 0 2024-05-09T06:38:23.854 INFO:tasks.workunit.client.1.smithi193.stdout:8/482: write d6/d5b/f64 [886262,34387] 0 2024-05-09T06:38:23.855 INFO:tasks.workunit.client.1.smithi193.stdout:0/438: fsync d18/f60 0 2024-05-09T06:38:23.856 INFO:tasks.workunit.client.1.smithi193.stdout:8/483: read d6/d1c/d18/f37 [2417497,53250] 0 2024-05-09T06:38:23.859 INFO:tasks.workunit.client.0.smithi063.stdout:0/873: mknod dd/d1a/d2c/d51/d72/d9e/d23/d31/dd2/c110 0 2024-05-09T06:38:23.860 INFO:tasks.workunit.client.1.smithi193.stdout:0/439: dread d18/d27/f34 [0,4194304] 0 2024-05-09T06:38:23.861 INFO:tasks.workunit.client.0.smithi063.stdout:1/786: creat d9/d2a/d3c/d4c/dc3/f111 x:0 0 0 2024-05-09T06:38:23.864 INFO:tasks.workunit.client.0.smithi063.stdout:0/874: dwrite dd/d60/d44/f69 [0,4194304] 0 2024-05-09T06:38:23.865 INFO:tasks.workunit.client.1.smithi193.stdout:6/451: symlink d1/de/d11/l9b 0 2024-05-09T06:38:23.866 INFO:tasks.workunit.client.0.smithi063.stdout:0/875: chown dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dc0/lc9 58067 1 2024-05-09T06:38:23.869 INFO:tasks.workunit.client.1.smithi193.stdout:1/508: rename d4/d16/f27 to d4/d16/d17/d1a/d3a/f9c 0 2024-05-09T06:38:23.870 INFO:tasks.workunit.client.0.smithi063.stdout:8/940: fdatasync d9/da/d18/dc7/d8f/f89 0 2024-05-09T06:38:23.871 INFO:tasks.workunit.client.1.smithi193.stdout:1/509: chown d4/d3c/d3e/c40 3139 1 2024-05-09T06:38:23.873 INFO:tasks.workunit.client.0.smithi063.stdout:6/852: link d2/fa9 d2/d1f/d6c/f114 0 2024-05-09T06:38:23.873 INFO:tasks.workunit.client.1.smithi193.stdout:8/484: readlink d6/d1c/d1b/d29/d57/l83 0 2024-05-09T06:38:23.875 INFO:tasks.workunit.client.1.smithi193.stdout:8/485: chown d6/d1c/d18/d2f/d40/c36 1179000443 1 2024-05-09T06:38:23.876 INFO:tasks.workunit.client.1.smithi193.stdout:3/405: link d1/db/d1b/f7a d1/d44/f96 0 2024-05-09T06:38:23.877 INFO:tasks.workunit.client.0.smithi063.stdout:9/827: symlink d0/d4/d7d/d105/d110/l120 0 2024-05-09T06:38:23.878 INFO:tasks.workunit.client.0.smithi063.stdout:9/828: chown d0/d4/d26/d47 48233 1 2024-05-09T06:38:23.880 INFO:tasks.workunit.client.1.smithi193.stdout:6/452: creat d1/d25/d2c/d76/d69/f9c x:0 0 0 2024-05-09T06:38:23.881 INFO:tasks.workunit.client.0.smithi063.stdout:3/891: creat d7/da/f133 x:0 0 0 2024-05-09T06:38:23.885 INFO:tasks.workunit.client.0.smithi063.stdout:1/787: rename d9/d2a/d3c/d4c/d83/de9/d10a to d9/d46/dfe/d94/d112 0 2024-05-09T06:38:23.887 INFO:tasks.workunit.client.0.smithi063.stdout:0/876: rmdir dd/d1a 39 2024-05-09T06:38:23.893 INFO:tasks.workunit.client.1.smithi193.stdout:8/486: creat d6/d1c/d1b/d3f/f96 x:0 0 0 2024-05-09T06:38:23.895 INFO:tasks.workunit.client.0.smithi063.stdout:4/855: creat d7/df/d35/d78/d109/f115 x:0 0 0 2024-05-09T06:38:23.895 INFO:tasks.workunit.client.1.smithi193.stdout:8/487: truncate d6/d1c/d1b/d3f/f96 1045746 0 2024-05-09T06:38:23.896 INFO:tasks.workunit.client.0.smithi063.stdout:4/856: truncate d7/df/d57/ff9 826987 0 2024-05-09T06:38:23.902 INFO:tasks.workunit.client.1.smithi193.stdout:6/453: rename d1/d25/f96 to d1/d25/f9d 0 2024-05-09T06:38:23.902 INFO:tasks.workunit.client.0.smithi063.stdout:5/815: creat d8/d17/d18/f112 x:0 0 0 2024-05-09T06:38:23.903 INFO:tasks.workunit.client.1.smithi193.stdout:7/464: rename d6/d8/d29/l9a to d6/d8/d29/d2f/l9b 0 2024-05-09T06:38:23.904 INFO:tasks.workunit.client.0.smithi063.stdout:2/861: write dc/dd/d45/f8a [1188281,110735] 0 2024-05-09T06:38:23.907 INFO:tasks.workunit.client.0.smithi063.stdout:3/892: symlink d7/da/de4/l134 0 2024-05-09T06:38:23.908 INFO:tasks.workunit.client.1.smithi193.stdout:3/406: unlink d1/db/d2f/c95 0 2024-05-09T06:38:23.911 INFO:tasks.workunit.client.1.smithi193.stdout:8/488: dread d6/d1c/d18/d66/f47 [0,4194304] 0 2024-05-09T06:38:23.912 INFO:tasks.workunit.client.1.smithi193.stdout:8/489: write d6/d5b/f8d [657919,17826] 0 2024-05-09T06:38:23.913 INFO:tasks.workunit.client.1.smithi193.stdout:6/454: mkdir d1/d25/d4c/d9e 0 2024-05-09T06:38:23.914 INFO:tasks.workunit.client.1.smithi193.stdout:0/440: write d18/f25 [4942169,66298] 0 2024-05-09T06:38:23.915 INFO:tasks.workunit.client.1.smithi193.stdout:0/441: chown f8 60424 1 2024-05-09T06:38:23.917 INFO:tasks.workunit.client.1.smithi193.stdout:4/545: rename d1a/d24/d73/d94/d7c/c48 to d1a/d22/d5d/ca8 0 2024-05-09T06:38:23.919 INFO:tasks.workunit.client.0.smithi063.stdout:6/853: fsync d2/d1f/d29/fb1 0 2024-05-09T06:38:23.920 INFO:tasks.workunit.client.1.smithi193.stdout:9/547: getdents dd/df/d15/d98 0 2024-05-09T06:38:23.922 INFO:tasks.workunit.client.0.smithi063.stdout:4/857: creat d7/d3d/d73/f116 x:0 0 0 2024-05-09T06:38:23.923 INFO:tasks.workunit.client.1.smithi193.stdout:7/465: mkdir d6/d8/d1a/d36/d4d/d60/d9c 0 2024-05-09T06:38:23.924 INFO:tasks.workunit.client.0.smithi063.stdout:7/825: getdents d0/d15/d49/d54/d9f/dd2/df9 0 2024-05-09T06:38:23.925 INFO:tasks.workunit.client.1.smithi193.stdout:3/407: truncate d1/db/d59/f43 1811816 0 2024-05-09T06:38:23.926 INFO:tasks.workunit.client.0.smithi063.stdout:5/816: chown d8/c50 1 1 2024-05-09T06:38:23.927 INFO:tasks.workunit.client.0.smithi063.stdout:5/817: chown d8/d35/db6 7 1 2024-05-09T06:38:23.928 INFO:tasks.workunit.client.0.smithi063.stdout:9/829: truncate d0/d4/d26/d39/d48/d80/d84/f10c 803227 0 2024-05-09T06:38:23.929 INFO:tasks.workunit.client.1.smithi193.stdout:6/455: fsync d1/de/d11/d2d/f2e 0 2024-05-09T06:38:23.930 INFO:tasks.workunit.client.1.smithi193.stdout:6/456: chown d1/c8 48558 1 2024-05-09T06:38:23.930 INFO:tasks.workunit.client.0.smithi063.stdout:2/862: write dc/d36/d43/d46/d49/ffd [4279994,48488] 0 2024-05-09T06:38:23.940 INFO:tasks.workunit.client.1.smithi193.stdout:4/546: dread d1a/d1f/f2c [0,4194304] 0 2024-05-09T06:38:23.940 INFO:tasks.workunit.client.1.smithi193.stdout:9/548: creat dd/df/d15/d98/fb3 x:0 0 0 2024-05-09T06:38:23.945 INFO:tasks.workunit.client.0.smithi063.stdout:6/854: mknod d2/d20/d26/d7f/dcf/c115 0 2024-05-09T06:38:23.946 INFO:tasks.workunit.client.0.smithi063.stdout:6/855: fsync d2/df/d91/f109 0 2024-05-09T06:38:23.947 INFO:tasks.workunit.client.0.smithi063.stdout:6/856: dread - d2/d1f/d29/d3f/d5c/f101 zero size 2024-05-09T06:38:23.951 INFO:tasks.workunit.client.0.smithi063.stdout:7/826: chown d0/d15/d49/d54/lc9 21 1 2024-05-09T06:38:23.951 INFO:tasks.workunit.client.0.smithi063.stdout:1/788: truncate d9/d2a/d35/d1e/d9d/f33 2349114 0 2024-05-09T06:38:23.952 INFO:tasks.workunit.client.1.smithi193.stdout:7/466: dread d6/d8/f44 [0,4194304] 0 2024-05-09T06:38:23.953 INFO:tasks.workunit.client.1.smithi193.stdout:7/467: write d6/dd/f40 [321809,97443] 0 2024-05-09T06:38:23.954 INFO:tasks.workunit.client.0.smithi063.stdout:5/818: fsync d8/f27 0 2024-05-09T06:38:23.956 INFO:tasks.workunit.client.0.smithi063.stdout:9/830: read d0/d4/d26/d39/f55 [3293067,121379] 0 2024-05-09T06:38:23.957 INFO:tasks.workunit.client.1.smithi193.stdout:1/510: rename d4/fc to d4/d16/d17/d1a/f9d 0 2024-05-09T06:38:23.959 INFO:tasks.workunit.client.1.smithi193.stdout:1/511: write d4/d16/d6b/d89/f99 [707972,68289] 0 2024-05-09T06:38:23.960 INFO:tasks.workunit.client.1.smithi193.stdout:3/408: write d1/db/f73 [596800,100607] 0 2024-05-09T06:38:23.961 INFO:tasks.workunit.client.1.smithi193.stdout:3/409: dread - d1/db/d59/f81 zero size 2024-05-09T06:38:23.962 INFO:tasks.workunit.client.1.smithi193.stdout:3/410: chown d1/c16 6419227 1 2024-05-09T06:38:23.963 INFO:tasks.workunit.client.0.smithi063.stdout:3/893: truncate d7/d22/d28/dab/fe0 202207 0 2024-05-09T06:38:23.963 INFO:tasks.workunit.client.1.smithi193.stdout:7/468: dread d6/d8/d1a/d36/f4e [0,4194304] 0 2024-05-09T06:38:23.964 INFO:tasks.workunit.client.1.smithi193.stdout:7/469: stat d6/d8/d1a/d36/f4e 0 2024-05-09T06:38:23.967 INFO:tasks.workunit.client.0.smithi063.stdout:8/941: rename d9/da/d45/fa4 to d9/f149 0 2024-05-09T06:38:23.968 INFO:tasks.workunit.client.1.smithi193.stdout:0/442: truncate d18/d4c/f3f 645121 0 2024-05-09T06:38:23.974 INFO:tasks.workunit.client.0.smithi063.stdout:4/858: symlink d7/d3d/d73/de5/l117 0 2024-05-09T06:38:23.976 INFO:tasks.workunit.client.0.smithi063.stdout:7/827: creat d0/d15/d4a/d68/f118 x:0 0 0 2024-05-09T06:38:23.980 INFO:tasks.workunit.client.0.smithi063.stdout:1/789: mkdir d9/d2a/d39/d5d/d75/d113 0 2024-05-09T06:38:23.984 INFO:tasks.workunit.client.0.smithi063.stdout:5/819: symlink d8/d19/d30/db1/l113 0 2024-05-09T06:38:23.991 INFO:tasks.workunit.client.0.smithi063.stdout:2/863: link f3 dc/d36/d43/d46/d49/ddc/f102 0 2024-05-09T06:38:23.994 INFO:tasks.workunit.client.0.smithi063.stdout:3/894: read d7/da/d21/d30/f57 [2368095,47951] 0 2024-05-09T06:38:23.995 INFO:tasks.workunit.client.0.smithi063.stdout:3/895: dread - d7/da/d21/d30/f40 zero size 2024-05-09T06:38:23.995 INFO:tasks.workunit.client.1.smithi193.stdout:4/547: fdatasync d1a/d22/f83 0 2024-05-09T06:38:23.997 INFO:tasks.workunit.client.1.smithi193.stdout:3/411: mkdir d1/db/d2f/d84/d97 0 2024-05-09T06:38:24.001 INFO:tasks.workunit.client.0.smithi063.stdout:6/857: mkdir d2/d1f/d29/d3f/d5c/d116 0 2024-05-09T06:38:24.007 INFO:tasks.workunit.client.0.smithi063.stdout:1/790: mkdir d9/d2a/d3c/dbc/d114 0 2024-05-09T06:38:24.008 INFO:tasks.workunit.client.0.smithi063.stdout:4/859: dread d7/da3/d28/d31/f94 [0,4194304] 0 2024-05-09T06:38:24.008 INFO:tasks.workunit.client.0.smithi063.stdout:4/860: readlink d7/df/d57/l87 0 2024-05-09T06:38:24.009 INFO:tasks.workunit.client.0.smithi063.stdout:1/791: write d9/d2a/d3c/d4c/d83/d88/dc2/fef [1111378,6125] 0 2024-05-09T06:38:24.009 INFO:tasks.workunit.client.0.smithi063.stdout:4/861: chown d7/da3/d28/d31/d71/d98/fa0 152562 1 2024-05-09T06:38:24.011 INFO:tasks.workunit.client.0.smithi063.stdout:4/862: write d7/da3/d28/d31/f107 [946176,93224] 0 2024-05-09T06:38:24.014 INFO:tasks.workunit.client.0.smithi063.stdout:0/877: rename dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/ca9 to dd/d1a/d2c/d42/c111 0 2024-05-09T06:38:24.020 INFO:tasks.workunit.client.1.smithi193.stdout:9/549: rmdir dd/df/d15/d98/da4 0 2024-05-09T06:38:24.021 INFO:tasks.workunit.client.1.smithi193.stdout:9/550: chown dd/df/l34 1449598829 1 2024-05-09T06:38:24.023 INFO:tasks.workunit.client.1.smithi193.stdout:1/512: truncate d4/d14/f75 1782382 0 2024-05-09T06:38:24.024 INFO:tasks.workunit.client.0.smithi063.stdout:1/792: rmdir d9/d2a/d3c 39 2024-05-09T06:38:24.024 INFO:tasks.workunit.client.0.smithi063.stdout:9/831: write d0/d3b/da0/f9c [3156708,88397] 0 2024-05-09T06:38:24.025 INFO:tasks.workunit.client.1.smithi193.stdout:4/548: creat d1a/d80/fa9 x:0 0 0 2024-05-09T06:38:24.026 INFO:tasks.workunit.client.1.smithi193.stdout:4/549: write d1a/d24/f78 [668505,76619] 0 2024-05-09T06:38:24.026 INFO:tasks.workunit.client.1.smithi193.stdout:3/412: symlink d1/d44/d7d/l98 0 2024-05-09T06:38:24.031 INFO:tasks.workunit.client.1.smithi193.stdout:6/457: rename d1/f4d to d1/d25/d2c/d81/f9f 0 2024-05-09T06:38:24.036 INFO:tasks.workunit.client.1.smithi193.stdout:3/413: dwrite d1/db/f73 [0,4194304] 0 2024-05-09T06:38:24.036 INFO:tasks.workunit.client.1.smithi193.stdout:4/550: dwrite d1a/d24/d73/d94/d7c/d5b/f7d [0,4194304] 0 2024-05-09T06:38:24.052 INFO:tasks.workunit.client.1.smithi193.stdout:9/551: dread fa [0,4194304] 0 2024-05-09T06:38:24.096 INFO:tasks.workunit.client.0.smithi063.stdout:5/820: sync 2024-05-09T06:38:24.096 INFO:tasks.workunit.client.0.smithi063.stdout:8/942: sync 2024-05-09T06:38:24.096 INFO:tasks.workunit.client.0.smithi063.stdout:3/896: sync 2024-05-09T06:38:24.127 INFO:tasks.workunit.client.0.smithi063.stdout:7/828: creat d0/d15/d49/f119 x:0 0 0 2024-05-09T06:38:24.129 INFO:tasks.workunit.client.1.smithi193.stdout:0/443: getdents d18/d4c/d5f 0 2024-05-09T06:38:24.132 INFO:tasks.workunit.client.0.smithi063.stdout:9/832: truncate d0/d3b/f56 489753 0 2024-05-09T06:38:24.134 INFO:tasks.workunit.client.0.smithi063.stdout:4/863: mknod d7/d50/d10f/c118 0 2024-05-09T06:38:24.136 INFO:tasks.workunit.client.1.smithi193.stdout:0/444: dwrite d18/d4c/d47/d59/d76/f87 [0,4194304] 0 2024-05-09T06:38:24.138 INFO:tasks.workunit.client.0.smithi063.stdout:6/858: dwrite d2/d20/d4a/f94 [0,4194304] 0 2024-05-09T06:38:24.139 INFO:tasks.workunit.client.1.smithi193.stdout:6/458: creat d1/de/d11/d18/d94/fa0 x:0 0 0 2024-05-09T06:38:24.141 INFO:tasks.workunit.client.0.smithi063.stdout:6/859: read d2/d4/d7c/fcc [770742,91616] 0 2024-05-09T06:38:24.142 INFO:tasks.workunit.client.1.smithi193.stdout:3/414: fdatasync d1/db/d59/f4e 0 2024-05-09T06:38:24.144 INFO:tasks.workunit.client.0.smithi063.stdout:5/821: chown d8/d19/d84/lfd 239828 1 2024-05-09T06:38:24.144 INFO:tasks.workunit.client.1.smithi193.stdout:1/513: mknod d4/c9e 0 2024-05-09T06:38:24.145 INFO:tasks.workunit.client.0.smithi063.stdout:3/897: creat d7/d22/dfc/f135 x:0 0 0 2024-05-09T06:38:24.148 INFO:tasks.workunit.client.0.smithi063.stdout:8/943: unlink d9/da/d34/fd5 0 2024-05-09T06:38:24.150 INFO:tasks.workunit.client.1.smithi193.stdout:7/470: rename d6/d8/d1a/d36/c83 to d6/dd/d7e/c9d 0 2024-05-09T06:38:24.152 INFO:tasks.workunit.client.0.smithi063.stdout:7/829: rename d0/d15/d4a/d50/l55 to d0/d15/d89/d10c/l11a 0 2024-05-09T06:38:24.153 INFO:tasks.workunit.client.1.smithi193.stdout:0/445: creat d18/d4c/d5f/f8c x:0 0 0 2024-05-09T06:38:24.153 INFO:tasks.workunit.client.1.smithi193.stdout:0/446: write f9 [7468605,64801] 0 2024-05-09T06:38:24.154 INFO:tasks.workunit.client.1.smithi193.stdout:0/447: stat d18/d4c/c43 0 2024-05-09T06:38:24.155 INFO:tasks.workunit.client.0.smithi063.stdout:4/864: unlink d7/df/f95 0 2024-05-09T06:38:24.158 INFO:tasks.workunit.client.0.smithi063.stdout:7/830: dwrite d0/d15/d49/d54/d9f/d97/d9b/d9d/fa8 [0,4194304] 0 2024-05-09T06:38:24.159 INFO:tasks.workunit.client.1.smithi193.stdout:6/459: creat d1/de/d11/d2d/d3a/d47/fa1 x:0 0 0 2024-05-09T06:38:24.159 INFO:tasks.workunit.client.1.smithi193.stdout:2/503: sync 2024-05-09T06:38:24.160 INFO:tasks.workunit.client.0.smithi063.stdout:7/831: stat d0/d15/d4a/d68/d10a 0 2024-05-09T06:38:24.160 INFO:tasks.workunit.client.1.smithi193.stdout:1/514: symlink d4/d16/d60/d4f/l9f 0 2024-05-09T06:38:24.160 INFO:tasks.workunit.client.1.smithi193.stdout:1/515: stat d4/c2c 0 2024-05-09T06:38:24.161 INFO:tasks.workunit.client.0.smithi063.stdout:6/860: symlink d2/d1f/l117 0 2024-05-09T06:38:24.162 INFO:tasks.workunit.client.0.smithi063.stdout:0/878: rmdir dd/d60/dcd/dfc 0 2024-05-09T06:38:24.167 INFO:tasks.workunit.client.0.smithi063.stdout:3/898: fdatasync d7/da/d2b/d66/dc0/faa 0 2024-05-09T06:38:24.170 INFO:tasks.workunit.client.1.smithi193.stdout:3/415: truncate d1/db/d2f/d84/f8d 272563 0 2024-05-09T06:38:24.177 INFO:tasks.workunit.client.0.smithi063.stdout:2/864: getdents dc/d36/d43/d46/d63/d79/d7d 0 2024-05-09T06:38:24.183 INFO:tasks.workunit.client.0.smithi063.stdout:1/793: dread d9/d2a/d35/f91 [0,4194304] 0 2024-05-09T06:38:24.184 INFO:tasks.workunit.client.1.smithi193.stdout:1/516: dread d4/d16/de/f28 [0,4194304] 0 2024-05-09T06:38:24.184 INFO:tasks.workunit.client.0.smithi063.stdout:7/832: rmdir d0/d15/d49 39 2024-05-09T06:38:24.185 INFO:tasks.workunit.client.0.smithi063.stdout:7/833: read - d0/d15/fcc zero size 2024-05-09T06:38:24.186 INFO:tasks.workunit.client.0.smithi063.stdout:7/834: chown d0/d15/d4a/d68/f118 710373 1 2024-05-09T06:38:24.186 INFO:tasks.workunit.client.1.smithi193.stdout:7/471: mknod d6/d8/c9e 0 2024-05-09T06:38:24.189 INFO:tasks.workunit.client.0.smithi063.stdout:0/879: fsync dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fa6 0 2024-05-09T06:38:24.190 INFO:tasks.workunit.client.1.smithi193.stdout:4/551: getdents d1a/d22/d5d 0 2024-05-09T06:38:24.192 INFO:tasks.workunit.client.1.smithi193.stdout:3/416: mknod d1/db/d2f/d84/c99 0 2024-05-09T06:38:24.195 INFO:tasks.workunit.client.0.smithi063.stdout:9/833: sync 2024-05-09T06:38:24.198 INFO:tasks.workunit.client.0.smithi063.stdout:5/822: dwrite d8/d19/d30/d46/f6c [0,4194304] 0 2024-05-09T06:38:24.198 INFO:tasks.workunit.client.1.smithi193.stdout:9/552: rename dd/d1e/d66/d49/l76 to dd/df/lb4 0 2024-05-09T06:38:24.201 INFO:tasks.workunit.client.0.smithi063.stdout:8/944: creat d9/da/d53/da3/de9/f14a x:0 0 0 2024-05-09T06:38:24.202 INFO:tasks.workunit.client.0.smithi063.stdout:4/865: write d7/da3/d28/d31/d32/d34/f80 [1844384,61750] 0 2024-05-09T06:38:24.203 INFO:tasks.workunit.client.0.smithi063.stdout:8/945: read d9/da/d53/fdf [63128,55595] 0 2024-05-09T06:38:24.203 INFO:tasks.workunit.client.0.smithi063.stdout:2/865: mknod dc/d36/d43/d46/d63/d9e/c103 0 2024-05-09T06:38:24.205 INFO:tasks.workunit.client.0.smithi063.stdout:2/866: read dc/f1b [7236510,114221] 0 2024-05-09T06:38:24.206 INFO:tasks.workunit.client.1.smithi193.stdout:7/472: mknod d6/dd/d7e/c9f 0 2024-05-09T06:38:24.207 INFO:tasks.workunit.client.1.smithi193.stdout:2/504: dwrite db/df/f20 [0,4194304] 0 2024-05-09T06:38:24.207 INFO:tasks.workunit.client.0.smithi063.stdout:8/946: dread d9/d3b/ffb [0,4194304] 0 2024-05-09T06:38:24.207 INFO:tasks.workunit.client.0.smithi063.stdout:2/867: write dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f86 [7880823,34844] 0 2024-05-09T06:38:24.209 INFO:tasks.workunit.client.1.smithi193.stdout:0/448: creat d18/d4c/f8d x:0 0 0 2024-05-09T06:38:24.210 INFO:tasks.workunit.client.0.smithi063.stdout:8/947: truncate d9/da/d18/dc7/d58/f121 636151 0 2024-05-09T06:38:24.211 INFO:tasks.workunit.client.1.smithi193.stdout:6/460: dwrite d1/d25/d2c/d3c/f49 [0,4194304] 0 2024-05-09T06:38:24.212 INFO:tasks.workunit.client.0.smithi063.stdout:8/948: truncate d9/da/d51/dc2/f11b 583117 0 2024-05-09T06:38:24.213 INFO:tasks.workunit.client.1.smithi193.stdout:6/461: write d1/f86 [387753,17769] 0 2024-05-09T06:38:24.214 INFO:tasks.workunit.client.1.smithi193.stdout:4/552: dread d1a/d1f/f2c [0,4194304] 0 2024-05-09T06:38:24.216 INFO:tasks.workunit.client.1.smithi193.stdout:3/417: creat d1/db/d2f/d84/f9a x:0 0 0 2024-05-09T06:38:24.217 INFO:tasks.workunit.client.0.smithi063.stdout:6/861: truncate d2/d1f/d29/de2/d4e/f6b 595183 0 2024-05-09T06:38:24.218 INFO:tasks.workunit.client.1.smithi193.stdout:1/517: rename d4/d16/de/c38 to d4/d16/d17/d1a/ca0 0 2024-05-09T06:38:24.219 INFO:tasks.workunit.client.1.smithi193.stdout:9/553: symlink dd/d1e/d66/lb5 0 2024-05-09T06:38:24.220 INFO:tasks.workunit.client.0.smithi063.stdout:0/880: fdatasync dd/d1a/d2c/d51/d72/d9e/d23/f105 0 2024-05-09T06:38:24.221 INFO:tasks.workunit.client.0.smithi063.stdout:3/899: mkdir d7/da/d21/d30/d44/d48/d11d/d136 0 2024-05-09T06:38:24.223 INFO:tasks.workunit.client.0.smithi063.stdout:8/949: dwrite d9/d41/dec/dc5/f136 [0,4194304] 0 2024-05-09T06:38:24.227 INFO:tasks.workunit.client.1.smithi193.stdout:0/449: chown d18/d4c/d47/d5e/d70/f83 3 1 2024-05-09T06:38:24.228 INFO:tasks.workunit.client.0.smithi063.stdout:4/866: rmdir d7/df/d35/d5a/d60 39 2024-05-09T06:38:24.231 INFO:tasks.workunit.client.1.smithi193.stdout:4/553: fsync f10 0 2024-05-09T06:38:24.231 INFO:tasks.workunit.client.0.smithi063.stdout:0/881: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fb6 [0,4194304] 0 2024-05-09T06:38:24.231 INFO:tasks.workunit.client.1.smithi193.stdout:4/554: chown d1a/d98 1086 1 2024-05-09T06:38:24.232 INFO:tasks.workunit.client.1.smithi193.stdout:4/555: fsync d1a/d6b/d60/f95 0 2024-05-09T06:38:24.234 INFO:tasks.workunit.client.1.smithi193.stdout:3/418: mknod d1/db/d1b/c9b 0 2024-05-09T06:38:24.237 INFO:tasks.workunit.client.1.smithi193.stdout:1/518: rmdir d4/d14 39 2024-05-09T06:38:24.239 INFO:tasks.workunit.client.1.smithi193.stdout:3/419: read d1/db/d1b/f7a [2757264,43831] 0 2024-05-09T06:38:24.241 INFO:tasks.workunit.client.1.smithi193.stdout:3/420: dread - d1/db/f90 zero size 2024-05-09T06:38:24.244 INFO:tasks.workunit.client.1.smithi193.stdout:7/473: symlink d6/dd/d5e/la0 0 2024-05-09T06:38:24.245 INFO:tasks.workunit.client.1.smithi193.stdout:1/519: dwrite d4/d16/d6b/d85/f86 [0,4194304] 0 2024-05-09T06:38:24.247 INFO:tasks.workunit.client.1.smithi193.stdout:7/474: write d6/d8/d1a/d33/f3e [2804551,65785] 0 2024-05-09T06:38:24.248 INFO:tasks.workunit.client.0.smithi063.stdout:6/862: creat d2/d9a/f118 x:0 0 0 2024-05-09T06:38:24.250 INFO:tasks.workunit.client.1.smithi193.stdout:0/450: dread d18/d27/f34 [0,4194304] 0 2024-05-09T06:38:24.254 INFO:tasks.workunit.client.1.smithi193.stdout:4/556: creat d1a/d6b/d60/faa x:0 0 0 2024-05-09T06:38:24.254 INFO:tasks.workunit.client.0.smithi063.stdout:5/823: write d8/d17/faa [3912079,46750] 0 2024-05-09T06:38:24.259 INFO:tasks.workunit.client.0.smithi063.stdout:3/900: dread d7/d22/d28/f8c [0,4194304] 0 2024-05-09T06:38:24.259 INFO:tasks.workunit.client.1.smithi193.stdout:9/554: mknod dd/d1e/d1f/d48/d55/d41/d68/db2/cb6 0 2024-05-09T06:38:24.259 INFO:tasks.workunit.client.1.smithi193.stdout:0/451: dwrite d18/d4c/d5f/f7c [0,4194304] 0 2024-05-09T06:38:24.267 INFO:tasks.workunit.client.1.smithi193.stdout:0/452: dwrite d18/d4c/d47/d54/f68 [0,4194304] 0 2024-05-09T06:38:24.269 INFO:tasks.workunit.client.1.smithi193.stdout:1/520: truncate d4/d16/d17/f29 6162102 0 2024-05-09T06:38:24.275 INFO:tasks.workunit.client.0.smithi063.stdout:0/882: chown dd/d1a/d2c/d51/d72/d9e/c8d 3 1 2024-05-09T06:38:24.276 INFO:tasks.workunit.client.1.smithi193.stdout:9/555: mkdir dd/df/d15/db7 0 2024-05-09T06:38:24.277 INFO:tasks.workunit.client.0.smithi063.stdout:4/867: dwrite d7/df/d1a/fe2 [0,4194304] 0 2024-05-09T06:38:24.278 INFO:tasks.workunit.client.0.smithi063.stdout:0/883: dread - dd/d1a/d50/ffb zero size 2024-05-09T06:38:24.280 INFO:tasks.workunit.client.0.smithi063.stdout:1/794: rename d9/d2a/d2e/df4/lfd to d9/d2a/d3c/d4c/dc3/l115 0 2024-05-09T06:38:24.280 INFO:tasks.workunit.client.0.smithi063.stdout:2/868: sync 2024-05-09T06:38:24.281 INFO:tasks.workunit.client.1.smithi193.stdout:0/453: fsync d18/d1d/f44 0 2024-05-09T06:38:24.283 INFO:tasks.workunit.client.1.smithi193.stdout:0/454: chown d18/f57 390 1 2024-05-09T06:38:24.285 INFO:tasks.workunit.client.1.smithi193.stdout:1/521: creat d4/d16/d60/d4f/d72/fa1 x:0 0 0 2024-05-09T06:38:24.287 INFO:tasks.workunit.client.1.smithi193.stdout:4/557: mkdir d1a/d67/d82/dab 0 2024-05-09T06:38:24.289 INFO:tasks.workunit.client.0.smithi063.stdout:8/950: dread d9/da/d18/d24/f84 [0,4194304] 0 2024-05-09T06:38:24.290 INFO:tasks.workunit.client.1.smithi193.stdout:7/475: write d6/d17/f37 [5623778,15528] 0 2024-05-09T06:38:24.291 INFO:tasks.workunit.client.0.smithi063.stdout:5/824: truncate d8/d65/f83 1637467 0 2024-05-09T06:38:24.292 INFO:tasks.workunit.client.0.smithi063.stdout:9/834: rmdir d0/d4/d26/d39/d48/d80/d84/dcd 0 2024-05-09T06:38:24.293 INFO:tasks.workunit.client.0.smithi063.stdout:3/901: symlink d7/d70/d98/d5b/l137 0 2024-05-09T06:38:24.294 INFO:tasks.workunit.client.1.smithi193.stdout:0/455: creat d18/d4c/d47/f8e x:0 0 0 2024-05-09T06:38:24.296 INFO:tasks.workunit.client.0.smithi063.stdout:4/868: symlink d7/df/d35/da5/l119 0 2024-05-09T06:38:24.297 INFO:tasks.workunit.client.1.smithi193.stdout:1/522: readlink d4/d14/l49 0 2024-05-09T06:38:24.297 INFO:tasks.workunit.client.0.smithi063.stdout:4/869: dread - d7/d3d/fa7 zero size 2024-05-09T06:38:24.298 INFO:tasks.workunit.client.1.smithi193.stdout:4/558: rename l15 to d1a/d67/lac 0 2024-05-09T06:38:24.298 INFO:tasks.workunit.client.1.smithi193.stdout:4/559: chown d1a/d1f/f3e 852414004 1 2024-05-09T06:38:24.299 INFO:tasks.workunit.client.0.smithi063.stdout:4/870: truncate d7/da3/d28/d31/d32/d34/f10a 765897 0 2024-05-09T06:38:24.300 INFO:tasks.workunit.client.0.smithi063.stdout:1/795: readlink d9/d2a/d35/d1e/l49 0 2024-05-09T06:38:24.301 INFO:tasks.workunit.client.0.smithi063.stdout:1/796: chown d9/d2a/d3c/d4c/d83/ccb 813909 1 2024-05-09T06:38:24.302 INFO:tasks.workunit.client.1.smithi193.stdout:7/476: mknod d6/dd/d70/ca1 0 2024-05-09T06:38:24.302 INFO:tasks.workunit.client.0.smithi063.stdout:1/797: dread - d9/d2a/d3c/d4c/d83/d88/dc2/dd0/fd6 zero size 2024-05-09T06:38:24.304 INFO:tasks.workunit.client.0.smithi063.stdout:7/835: creat d0/d15/d49/d54/d9f/d97/d9b/f11b x:0 0 0 2024-05-09T06:38:24.304 INFO:tasks.workunit.client.1.smithi193.stdout:9/556: write dd/f73 [1383351,57499] 0 2024-05-09T06:38:24.307 INFO:tasks.workunit.client.0.smithi063.stdout:2/869: mkdir dc/d36/d43/d46/d49/dbb/dec/d104 0 2024-05-09T06:38:24.311 INFO:tasks.workunit.client.1.smithi193.stdout:4/560: fsync d1a/f1b 0 2024-05-09T06:38:24.315 INFO:tasks.workunit.client.1.smithi193.stdout:0/456: mknod d18/d27/c8f 0 2024-05-09T06:38:24.315 INFO:tasks.workunit.client.1.smithi193.stdout:0/457: chown d18/d4c/d47/d59 56 1 2024-05-09T06:38:24.316 INFO:tasks.workunit.client.1.smithi193.stdout:0/458: write d18/f60 [2863144,28544] 0 2024-05-09T06:38:24.317 INFO:tasks.workunit.client.1.smithi193.stdout:9/557: truncate dd/d1e/f27 100700 0 2024-05-09T06:38:24.318 INFO:tasks.workunit.client.0.smithi063.stdout:0/884: write dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/fbd [1094374,117669] 0 2024-05-09T06:38:24.318 INFO:tasks.workunit.client.1.smithi193.stdout:9/558: fdatasync dd/d1e/d1f/d61/f85 0 2024-05-09T06:38:24.319 INFO:tasks.workunit.client.0.smithi063.stdout:9/835: mkdir d0/d4/d7d/d105/d121 0 2024-05-09T06:38:24.320 INFO:tasks.workunit.client.0.smithi063.stdout:9/836: chown d0/d4/d26/d39/d48/d80/d84/fc4 135788 1 2024-05-09T06:38:24.320 INFO:tasks.workunit.client.1.smithi193.stdout:1/523: creat d4/d16/d60/d4f/d72/d96/fa2 x:0 0 0 2024-05-09T06:38:24.326 INFO:tasks.workunit.client.0.smithi063.stdout:6/863: truncate d2/d4/f92 2433680 0 2024-05-09T06:38:24.326 INFO:tasks.workunit.client.0.smithi063.stdout:6/864: write d2/d20/f44 [3479833,51782] 0 2024-05-09T06:38:24.327 INFO:tasks.workunit.client.0.smithi063.stdout:8/951: write d9/da/d51/f10b [511024,102937] 0 2024-05-09T06:38:24.330 INFO:tasks.workunit.client.1.smithi193.stdout:0/459: dread f8 [0,4194304] 0 2024-05-09T06:38:24.330 INFO:tasks.workunit.client.1.smithi193.stdout:7/477: link d6/d8/d1a/d33/f87 d6/dd/d70/fa2 0 2024-05-09T06:38:24.332 INFO:tasks.workunit.client.0.smithi063.stdout:1/798: write d9/d2a/d3c/d4c/d83/d88/dc2/dd0/ff3 [570203,106452] 0 2024-05-09T06:38:24.333 INFO:tasks.workunit.client.0.smithi063.stdout:6/865: dwrite d2/d1f/d29/d3f/d5c/f101 [0,4194304] 0 2024-05-09T06:38:24.334 INFO:tasks.workunit.client.0.smithi063.stdout:6/866: read - d2/d1f/d29/de2/d4e/feb zero size 2024-05-09T06:38:24.335 INFO:tasks.workunit.client.0.smithi063.stdout:7/836: mkdir d0/dcb/d11c 0 2024-05-09T06:38:24.336 INFO:tasks.workunit.client.0.smithi063.stdout:3/902: write d7/da/fe [4016228,30347] 0 2024-05-09T06:38:24.343 INFO:tasks.workunit.client.1.smithi193.stdout:4/561: rename d1a/d1f/l90 to d1a/d24/d73/d94/d7c/lad 0 2024-05-09T06:38:24.346 INFO:tasks.workunit.client.1.smithi193.stdout:0/460: creat d18/d4c/d47/d54/f90 x:0 0 0 2024-05-09T06:38:24.347 INFO:tasks.workunit.client.0.smithi063.stdout:0/885: creat dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dcb/f112 x:0 0 0 2024-05-09T06:38:24.349 INFO:tasks.workunit.client.1.smithi193.stdout:7/478: rename d6/dd/d7e/f82 to d6/dd/d7e/fa3 0 2024-05-09T06:38:24.351 INFO:tasks.workunit.client.0.smithi063.stdout:9/837: creat d0/d4/d26/d39/d42/f122 x:0 0 0 2024-05-09T06:38:24.351 INFO:tasks.workunit.client.1.smithi193.stdout:0/461: dread d18/f60 [0,4194304] 0 2024-05-09T06:38:24.351 INFO:tasks.workunit.client.1.smithi193.stdout:0/462: fdatasync d18/f61 0 2024-05-09T06:38:24.355 INFO:tasks.workunit.client.0.smithi063.stdout:2/870: dwrite dc/dd/d2d/d60/f67 [4194304,4194304] 0 2024-05-09T06:38:24.363 INFO:tasks.workunit.client.0.smithi063.stdout:5/825: dwrite d8/f68 [4194304,4194304] 0 2024-05-09T06:38:24.364 INFO:tasks.workunit.client.0.smithi063.stdout:0/886: dread dd/d1a/d2c/d51/d72/d9e/d23/d33/d8f/f93 [0,4194304] 0 2024-05-09T06:38:24.365 INFO:tasks.workunit.client.0.smithi063.stdout:1/799: mknod d9/d2a/d35/d1e/d9d/d3f/c116 0 2024-05-09T06:38:24.367 INFO:tasks.workunit.client.1.smithi193.stdout:0/463: creat d18/d4c/d47/d59/f91 x:0 0 0 2024-05-09T06:38:24.368 INFO:tasks.workunit.client.0.smithi063.stdout:6/867: creat d2/d12/f119 x:0 0 0 2024-05-09T06:38:24.369 INFO:tasks.workunit.client.1.smithi193.stdout:9/559: creat dd/d1e/d1f/d48/d55/fb8 x:0 0 0 2024-05-09T06:38:24.371 INFO:tasks.workunit.client.0.smithi063.stdout:7/837: fsync d0/d15/d49/f5c 0 2024-05-09T06:38:24.371 INFO:tasks.workunit.client.1.smithi193.stdout:9/560: chown dd/d1e/d1f/d43/fa6 6985209 1 2024-05-09T06:38:24.373 INFO:tasks.workunit.client.1.smithi193.stdout:1/524: creat d4/d16/d17/d1a/d3a/fa3 x:0 0 0 2024-05-09T06:38:24.374 INFO:tasks.workunit.client.0.smithi063.stdout:4/871: dwrite d7/df/d35/d5a/f97 [0,4194304] 0 2024-05-09T06:38:24.378 INFO:tasks.workunit.client.0.smithi063.stdout:9/838: symlink d0/d4/d26/d39/d48/d80/d84/dc2/l123 0 2024-05-09T06:38:24.378 INFO:tasks.workunit.client.0.smithi063.stdout:9/839: chown d0/d3b/da0/dcf/fd2 20 1 2024-05-09T06:38:24.379 INFO:tasks.workunit.client.0.smithi063.stdout:9/840: chown d0/d8/da6/db0/db5/f106 136 1 2024-05-09T06:38:24.381 INFO:tasks.workunit.client.1.smithi193.stdout:0/464: dread d18/d1d/f30 [0,4194304] 0 2024-05-09T06:38:24.382 INFO:tasks.workunit.client.1.smithi193.stdout:0/465: readlink d18/d4c/d47/d55/l7a 0 2024-05-09T06:38:24.389 INFO:tasks.workunit.client.0.smithi063.stdout:0/887: unlink dd/d1a/d2c/d51/d72/d9e/d23/d89/f8b 0 2024-05-09T06:38:24.390 INFO:tasks.workunit.client.1.smithi193.stdout:5/414: sync 2024-05-09T06:38:24.393 INFO:tasks.workunit.client.0.smithi063.stdout:6/868: mknod d2/d1f/d29/de2/c11a 0 2024-05-09T06:38:24.394 INFO:tasks.workunit.client.1.smithi193.stdout:5/415: dread d0/d13/d2c/f4a [0,4194304] 0 2024-05-09T06:38:24.394 INFO:tasks.workunit.client.0.smithi063.stdout:6/869: fdatasync d2/d12/f119 0 2024-05-09T06:38:24.394 INFO:tasks.workunit.client.0.smithi063.stdout:6/870: chown d2/d1f/d29/d3f 333830 1 2024-05-09T06:38:24.399 INFO:tasks.workunit.client.0.smithi063.stdout:0/888: dread dd/d1a/d2c/d51/d72/d9e/f64 [0,4194304] 0 2024-05-09T06:38:24.402 INFO:tasks.workunit.client.1.smithi193.stdout:5/416: dwrite d0/d13/d2c/f3f [0,4194304] 0 2024-05-09T06:38:24.406 INFO:tasks.workunit.client.0.smithi063.stdout:4/872: truncate d7/da3/f23 1196385 0 2024-05-09T06:38:24.410 INFO:tasks.workunit.client.0.smithi063.stdout:9/841: dread d0/d4/d26/d39/d91/fa7 [0,4194304] 0 2024-05-09T06:38:24.410 INFO:tasks.workunit.client.0.smithi063.stdout:8/952: link d9/d41/dec/db8/ff5 d9/da/d53/dbc/d102/f14b 0 2024-05-09T06:38:24.412 INFO:tasks.workunit.client.0.smithi063.stdout:6/871: dread d2/d12/f34 [0,4194304] 0 2024-05-09T06:38:24.412 INFO:tasks.workunit.client.0.smithi063.stdout:8/953: fsync d9/d3b/df8/d12f/f145 0 2024-05-09T06:38:24.415 INFO:tasks.workunit.client.0.smithi063.stdout:3/903: truncate d7/d22/f47 3878928 0 2024-05-09T06:38:24.423 INFO:tasks.workunit.client.1.smithi193.stdout:9/561: mkdir dd/d1e/d66/db9 0 2024-05-09T06:38:24.425 INFO:tasks.workunit.client.0.smithi063.stdout:2/871: truncate dc/d36/d43/d46/d63/d9e/feb 3570265 0 2024-05-09T06:38:24.429 INFO:tasks.workunit.client.1.smithi193.stdout:7/479: creat d6/d8/d1a/fa4 x:0 0 0 2024-05-09T06:38:24.431 INFO:tasks.workunit.client.1.smithi193.stdout:0/466: rmdir d18/d1d/d56 39 2024-05-09T06:38:24.434 INFO:tasks.workunit.client.0.smithi063.stdout:9/842: unlink d0/d4/d26/d39/d91/c115 0 2024-05-09T06:38:24.436 INFO:tasks.workunit.client.0.smithi063.stdout:6/872: mkdir d2/df/d59/d99/d9f/d11b 0 2024-05-09T06:38:24.438 INFO:tasks.workunit.client.0.smithi063.stdout:5/826: link d8/d17/d75/f7f d8/d65/dbd/f114 0 2024-05-09T06:38:24.440 INFO:tasks.workunit.client.0.smithi063.stdout:3/904: creat d7/d70/d98/dbb/f138 x:0 0 0 2024-05-09T06:38:24.446 INFO:tasks.workunit.client.0.smithi063.stdout:1/800: rename d9/d46/dfe/f96 to d9/d2a/d35/f117 0 2024-05-09T06:38:24.447 INFO:tasks.workunit.client.1.smithi193.stdout:1/525: dwrite d4/d14/f44 [4194304,4194304] 0 2024-05-09T06:38:24.450 INFO:tasks.workunit.client.0.smithi063.stdout:2/872: mkdir dc/dd/d45/d5a/d6c/d8f/dbf/d105 0 2024-05-09T06:38:24.453 INFO:tasks.workunit.client.0.smithi063.stdout:7/838: link d0/f69 d0/d15/d49/d4f/d66/d67/f11d 0 2024-05-09T06:38:24.456 INFO:tasks.workunit.client.0.smithi063.stdout:4/873: mkdir d7/da3/d28/d31/d7b/df2/d11a 0 2024-05-09T06:38:24.459 INFO:tasks.workunit.client.0.smithi063.stdout:7/839: dwrite d0/d15/d4a/d50/f2e [4194304,4194304] 0 2024-05-09T06:38:24.460 INFO:tasks.workunit.client.1.smithi193.stdout:7/480: unlink d6/d8/d1a/l25 0 2024-05-09T06:38:24.463 INFO:tasks.workunit.client.0.smithi063.stdout:8/954: symlink d9/da/d51/dc2/d119/d11f/l14c 0 2024-05-09T06:38:24.464 INFO:tasks.workunit.client.0.smithi063.stdout:4/874: dwrite d7/da3/d28/d31/d32/d34/f80 [0,4194304] 0 2024-05-09T06:38:24.464 INFO:tasks.workunit.client.1.smithi193.stdout:1/526: symlink d4/la4 0 2024-05-09T06:38:24.464 INFO:tasks.workunit.client.0.smithi063.stdout:4/875: readlink d7/df/d35/l84 0 2024-05-09T06:38:24.465 INFO:tasks.workunit.client.1.smithi193.stdout:5/417: dwrite d0/d13/d2c/d37/d5b/d4e/f58 [0,4194304] 0 2024-05-09T06:38:24.468 INFO:tasks.workunit.client.1.smithi193.stdout:7/481: rmdir d6/dd/d70 39 2024-05-09T06:38:24.469 INFO:tasks.workunit.client.0.smithi063.stdout:5/827: dread d8/f1d [4194304,4194304] 0 2024-05-09T06:38:24.471 INFO:tasks.workunit.client.1.smithi193.stdout:0/467: link d18/d4c/d64/l6e d18/d65/d82/l92 0 2024-05-09T06:38:24.472 INFO:tasks.workunit.client.0.smithi063.stdout:3/905: mknod d7/da/d2b/d66/dc0/da4/c139 0 2024-05-09T06:38:24.477 INFO:tasks.workunit.client.0.smithi063.stdout:0/889: rename dd/d1a/d2c/d51/d72/d9e/d23/d31/dd2 to dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4/d113 0 2024-05-09T06:38:24.480 INFO:tasks.workunit.client.0.smithi063.stdout:1/801: unlink d9/d10/f22 0 2024-05-09T06:38:24.486 INFO:tasks.workunit.client.0.smithi063.stdout:2/873: rmdir dc/d36/d43/d46/d49/dbb/dec 39 2024-05-09T06:38:24.491 INFO:tasks.workunit.client.1.smithi193.stdout:5/418: mkdir d0/d13/d2c/d37/d5b/d89 0 2024-05-09T06:38:24.492 INFO:tasks.workunit.client.1.smithi193.stdout:5/419: chown d0/d13/d2c/d37/d5b/l27 1 1 2024-05-09T06:38:24.493 INFO:tasks.workunit.client.0.smithi063.stdout:9/843: unlink d0/lfe 0 2024-05-09T06:38:24.499 INFO:tasks.workunit.client.1.smithi193.stdout:0/468: dread d18/d1d/f3d [0,4194304] 0 2024-05-09T06:38:24.499 INFO:tasks.workunit.client.0.smithi063.stdout:6/873: mknod d2/d1f/d29/d3f/d5c/d116/c11c 0 2024-05-09T06:38:24.501 INFO:tasks.workunit.client.1.smithi193.stdout:5/420: symlink d0/d13/d2c/l8a 0 2024-05-09T06:38:24.502 INFO:tasks.workunit.client.1.smithi193.stdout:7/482: truncate d6/d8/d1a/d36/f57 3063595 0 2024-05-09T06:38:24.504 INFO:tasks.workunit.client.1.smithi193.stdout:1/527: dwrite d4/f13 [0,4194304] 0 2024-05-09T06:38:24.504 INFO:tasks.workunit.client.0.smithi063.stdout:5/828: truncate d8/d53/dca/f23 8481954 0 2024-05-09T06:38:24.505 INFO:tasks.workunit.client.1.smithi193.stdout:5/421: write d0/d13/d2c/d37/d5b/d4e/d5d/f61 [470411,91378] 0 2024-05-09T06:38:24.506 INFO:tasks.workunit.client.1.smithi193.stdout:1/528: chown d4/d16/d17/d1a/d5b/c69 245946 1 2024-05-09T06:38:24.507 INFO:tasks.workunit.client.0.smithi063.stdout:3/906: chown d7/da/fb 19430845 1 2024-05-09T06:38:24.510 INFO:tasks.workunit.client.1.smithi193.stdout:0/469: mknod d18/d1d/d56/c93 0 2024-05-09T06:38:24.513 INFO:tasks.workunit.client.0.smithi063.stdout:5/829: dwrite d8/d17/d75/dc9/dd5/d96/ff2 [0,4194304] 0 2024-05-09T06:38:24.513 INFO:tasks.workunit.client.0.smithi063.stdout:2/874: symlink dc/dd/d45/d5a/d6c/l106 0 2024-05-09T06:38:24.515 INFO:tasks.workunit.client.1.smithi193.stdout:7/483: symlink d6/dd/d5e/la5 0 2024-05-09T06:38:24.516 INFO:tasks.workunit.client.1.smithi193.stdout:1/529: dwrite d4/d16/d60/d4f/d72/f7a [0,4194304] 0 2024-05-09T06:38:24.526 INFO:tasks.workunit.client.0.smithi063.stdout:7/840: mknod d0/d15/d49/c11e 0 2024-05-09T06:38:24.526 INFO:tasks.workunit.client.0.smithi063.stdout:8/955: mknod d9/da/d18/dc7/c14d 0 2024-05-09T06:38:24.527 INFO:tasks.workunit.client.0.smithi063.stdout:0/890: dwrite dd/d1a/d50/f7a [0,4194304] 0 2024-05-09T06:38:24.528 INFO:tasks.workunit.client.0.smithi063.stdout:8/956: readlink d9/da/d18/dc7/d8f/lbe 0 2024-05-09T06:38:24.538 INFO:tasks.workunit.client.0.smithi063.stdout:5/830: fsync d8/d65/d9e/fb8 0 2024-05-09T06:38:24.538 INFO:tasks.workunit.client.1.smithi193.stdout:5/422: getdents d0/d13/d2c/d37/d5b/d4e/d5d 0 2024-05-09T06:38:24.539 INFO:tasks.workunit.client.0.smithi063.stdout:2/875: creat dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f107 x:0 0 0 2024-05-09T06:38:24.541 INFO:tasks.workunit.client.1.smithi193.stdout:0/470: getdents d18/d4c/d47/d54 0 2024-05-09T06:38:24.542 INFO:tasks.workunit.client.0.smithi063.stdout:5/831: read d8/d17/d18/f1f [861368,100248] 0 2024-05-09T06:38:24.544 INFO:tasks.workunit.client.0.smithi063.stdout:6/874: link d2/d4/d7c/d7e/f87 d2/df/d59/d99/d9f/d11b/f11d 0 2024-05-09T06:38:24.545 INFO:tasks.workunit.client.0.smithi063.stdout:6/875: fdatasync d2/df/f5d 0 2024-05-09T06:38:24.546 INFO:tasks.workunit.client.1.smithi193.stdout:0/471: creat d18/d4c/d47/d5e/d70/f94 x:0 0 0 2024-05-09T06:38:24.547 INFO:tasks.workunit.client.1.smithi193.stdout:0/472: stat d18/c19 0 2024-05-09T06:38:24.547 INFO:tasks.workunit.client.0.smithi063.stdout:8/957: mkdir d9/da/d34/d14e 0 2024-05-09T06:38:24.548 INFO:tasks.workunit.client.1.smithi193.stdout:7/484: link d6/dd/d5e/d6b/l91 d6/la6 0 2024-05-09T06:38:24.548 INFO:tasks.workunit.client.1.smithi193.stdout:0/473: read f9 [5250128,21304] 0 2024-05-09T06:38:24.548 INFO:tasks.workunit.client.1.smithi193.stdout:7/485: fsync d6/dd/d72/f8c 0 2024-05-09T06:38:24.549 INFO:tasks.workunit.client.1.smithi193.stdout:0/474: stat d18/l32 0 2024-05-09T06:38:24.551 INFO:tasks.workunit.client.1.smithi193.stdout:5/423: creat d0/d13/d2c/d37/d2b/d47/d60/d75/f8b x:0 0 0 2024-05-09T06:38:24.553 INFO:tasks.workunit.client.1.smithi193.stdout:5/424: write d0/d13/d2c/d37/d5b/f21 [57539,37780] 0 2024-05-09T06:38:24.553 INFO:tasks.workunit.client.0.smithi063.stdout:3/907: mknod d7/da/d21/d30/d44/d48/d11d/d136/c13a 0 2024-05-09T06:38:24.554 INFO:tasks.workunit.client.1.smithi193.stdout:5/425: dread - d0/d13/d2c/d37/d74/f80 zero size 2024-05-09T06:38:24.555 INFO:tasks.workunit.client.0.smithi063.stdout:1/802: link d9/d46/dfe/d94/lb5 d9/d2a/d35/d1e/d9d/d3f/d59/d62/l118 0 2024-05-09T06:38:24.555 INFO:tasks.workunit.client.0.smithi063.stdout:9/844: write d0/f71 [4163738,45151] 0 2024-05-09T06:38:24.556 INFO:tasks.workunit.client.1.smithi193.stdout:7/486: mknod d6/d17/d7d/ca7 0 2024-05-09T06:38:24.561 INFO:tasks.workunit.client.1.smithi193.stdout:1/530: write d4/d14/f1b [294358,83316] 0 2024-05-09T06:38:24.563 INFO:tasks.workunit.client.1.smithi193.stdout:5/426: dread d0/d13/d2c/d37/d5b/f32 [0,4194304] 0 2024-05-09T06:38:24.564 INFO:tasks.workunit.client.1.smithi193.stdout:2/505: sync 2024-05-09T06:38:24.564 INFO:tasks.workunit.client.1.smithi193.stdout:8/490: sync 2024-05-09T06:38:24.564 INFO:tasks.workunit.client.0.smithi063.stdout:7/841: symlink d0/d15/d49/d4f/l11f 0 2024-05-09T06:38:24.565 INFO:tasks.workunit.client.0.smithi063.stdout:2/876: dwrite dc/d36/f80 [4194304,4194304] 0 2024-05-09T06:38:24.569 INFO:tasks.workunit.client.1.smithi193.stdout:0/475: unlink d18/d4c/f3f 0 2024-05-09T06:38:24.570 INFO:tasks.workunit.client.1.smithi193.stdout:0/476: stat d18/l1a 0 2024-05-09T06:38:24.573 INFO:tasks.workunit.client.0.smithi063.stdout:3/908: mknod d7/d4c/dcf/c13b 0 2024-05-09T06:38:24.573 INFO:tasks.workunit.client.1.smithi193.stdout:1/531: dread d4/d16/d35/f48 [0,4194304] 0 2024-05-09T06:38:24.574 INFO:tasks.workunit.client.1.smithi193.stdout:5/427: creat d0/d13/d2c/d37/d5b/d4e/f8c x:0 0 0 2024-05-09T06:38:24.575 INFO:tasks.workunit.client.0.smithi063.stdout:3/909: chown d7/d4c/d25/d10b/l101 60569 1 2024-05-09T06:38:24.576 INFO:tasks.workunit.client.1.smithi193.stdout:2/506: write db/d37/d6d/d80/f17 [3650913,110819] 0 2024-05-09T06:38:24.577 INFO:tasks.workunit.client.0.smithi063.stdout:0/891: dwrite dd/d1a/d2c/d51/d76/d97/fa5 [0,4194304] 0 2024-05-09T06:38:24.578 INFO:tasks.workunit.client.1.smithi193.stdout:2/507: read - db/d25/d38/d40/d8f/f93 zero size 2024-05-09T06:38:24.579 INFO:tasks.workunit.client.1.smithi193.stdout:8/491: rename d6/d1c/d18/d2f/d40/l32 to d6/d1c/d18/d66/d93/d95/l97 0 2024-05-09T06:38:24.580 INFO:tasks.workunit.client.1.smithi193.stdout:8/492: stat d6/d1c/d18/c28 0 2024-05-09T06:38:24.581 INFO:tasks.workunit.client.1.smithi193.stdout:0/477: stat d18/l3b 0 2024-05-09T06:38:24.584 INFO:tasks.workunit.client.1.smithi193.stdout:5/428: dread d0/d13/d2c/d37/d5b/d4e/d5d/f69 [0,4194304] 0 2024-05-09T06:38:24.587 INFO:tasks.workunit.client.0.smithi063.stdout:6/876: mkdir d2/df/d91/d11e 0 2024-05-09T06:38:24.589 INFO:tasks.workunit.client.1.smithi193.stdout:1/532: creat d4/d16/d60/d4f/fa5 x:0 0 0 2024-05-09T06:38:24.589 INFO:tasks.workunit.client.1.smithi193.stdout:1/533: readlink d4/d16/d60/d4f/l9f 0 2024-05-09T06:38:24.593 INFO:tasks.workunit.client.0.smithi063.stdout:8/958: mknod d9/da/d53/c14f 0 2024-05-09T06:38:24.593 INFO:tasks.workunit.client.0.smithi063.stdout:9/845: write d0/d3b/fbc [426033,123906] 0 2024-05-09T06:38:24.594 INFO:tasks.workunit.client.0.smithi063.stdout:6/877: dwrite d2/d1f/d29/de2/d4e/feb [0,4194304] 0 2024-05-09T06:38:24.595 INFO:tasks.workunit.client.0.smithi063.stdout:9/846: readlink d0/da5/lff 0 2024-05-09T06:38:24.595 INFO:tasks.workunit.client.0.smithi063.stdout:8/959: chown d9/da/d18/dc7/d58 3 1 2024-05-09T06:38:24.596 INFO:tasks.workunit.client.0.smithi063.stdout:4/876: sync 2024-05-09T06:38:24.597 INFO:tasks.workunit.client.1.smithi193.stdout:2/508: chown db/d25/d38/d40/d8b/d77/l9a 44 1 2024-05-09T06:38:24.599 INFO:tasks.workunit.client.1.smithi193.stdout:8/493: symlink d6/d1c/d18/d2f/d40/d55/d7e/l98 0 2024-05-09T06:38:24.600 INFO:tasks.workunit.client.0.smithi063.stdout:6/878: read - d2/df/d59/d99/d9f/f111 zero size 2024-05-09T06:38:24.600 INFO:tasks.workunit.client.0.smithi063.stdout:3/910: symlink d7/da/d21/d30/d44/d48/d11d/d136/l13c 0 2024-05-09T06:38:24.600 INFO:tasks.workunit.client.0.smithi063.stdout:7/842: dread d0/d15/fd5 [0,4194304] 0 2024-05-09T06:38:24.606 INFO:tasks.workunit.client.0.smithi063.stdout:0/892: rmdir dd/d1a/d2c/d51/d100 39 2024-05-09T06:38:24.607 INFO:tasks.workunit.client.1.smithi193.stdout:7/487: getdents d6/dd/d5e 0 2024-05-09T06:38:24.607 INFO:tasks.workunit.client.0.smithi063.stdout:5/832: rename d8/d17/da2 to d8/d19/d115 0 2024-05-09T06:38:24.607 INFO:tasks.workunit.client.1.smithi193.stdout:7/488: readlink d6/l81 0 2024-05-09T06:38:24.611 INFO:tasks.workunit.client.0.smithi063.stdout:8/960: mknod d9/d37/d106/c150 0 2024-05-09T06:38:24.612 INFO:tasks.workunit.client.0.smithi063.stdout:2/877: dwrite dc/dd/d45/d5a/d6c/d8c/d9a/dd5/fde [0,4194304] 0 2024-05-09T06:38:24.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/847: creat d0/d4/d26/d39/d48/d80/d84/dc2/f124 x:0 0 0 2024-05-09T06:38:24.616 INFO:tasks.workunit.client.0.smithi063.stdout:4/877: symlink d7/da3/d28/d31/dbf/l11b 0 2024-05-09T06:38:24.622 INFO:tasks.workunit.client.0.smithi063.stdout:1/803: dwrite d9/d2a/d39/f5e [0,4194304] 0 2024-05-09T06:38:24.625 INFO:tasks.workunit.client.0.smithi063.stdout:4/878: dread d7/df/f21 [4194304,4194304] 0 2024-05-09T06:38:24.628 INFO:tasks.workunit.client.0.smithi063.stdout:3/911: dread d7/d70/d98/d5b/f8f [0,4194304] 0 2024-05-09T06:38:24.651 INFO:tasks.workunit.client.0.smithi063.stdout:0/893: unlink dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dcb/f112 0 2024-05-09T06:38:24.658 INFO:tasks.workunit.client.0.smithi063.stdout:1/804: creat d9/d46/dfe/d94/d112/f119 x:0 0 0 2024-05-09T06:38:24.662 INFO:tasks.workunit.client.0.smithi063.stdout:8/961: dread d9/da/d51/dc2/fd9 [0,4194304] 0 2024-05-09T06:38:24.668 INFO:tasks.workunit.client.0.smithi063.stdout:3/912: symlink d7/da/l13d 0 2024-05-09T06:38:24.669 INFO:tasks.workunit.client.0.smithi063.stdout:8/962: truncate d9/d3b/f68 2205095 0 2024-05-09T06:38:24.669 INFO:tasks.workunit.client.0.smithi063.stdout:7/843: link d0/d15/d89/d10c/dfb/c10e d0/d15/d89/d113/c120 0 2024-05-09T06:38:24.671 INFO:tasks.workunit.client.0.smithi063.stdout:0/894: link dd/fde dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4/df1/f114 0 2024-05-09T06:38:24.672 INFO:tasks.workunit.client.0.smithi063.stdout:0/895: readlink dd/d1a/d50/ld1 0 2024-05-09T06:38:24.673 INFO:tasks.workunit.client.0.smithi063.stdout:6/879: getdents d2/d4 0 2024-05-09T06:38:24.673 INFO:tasks.workunit.client.0.smithi063.stdout:6/880: chown d2/d20/d26/fde 18417 1 2024-05-09T06:38:24.674 INFO:tasks.workunit.client.0.smithi063.stdout:6/881: write d2/df/f50 [3158089,57261] 0 2024-05-09T06:38:24.676 INFO:tasks.workunit.client.0.smithi063.stdout:8/963: creat d9/da/d18/dc7/dd3/df1/f151 x:0 0 0 2024-05-09T06:38:24.677 INFO:tasks.workunit.client.0.smithi063.stdout:7/844: mknod d0/d15/d89/c121 0 2024-05-09T06:38:24.678 INFO:tasks.workunit.client.0.smithi063.stdout:1/805: creat d9/d2a/d35/f11a x:0 0 0 2024-05-09T06:38:24.679 INFO:tasks.workunit.client.0.smithi063.stdout:0/896: unlink dd/d1a/d2c/d51/d72/d9e/d23/d89/fc4 0 2024-05-09T06:38:24.680 INFO:tasks.workunit.client.0.smithi063.stdout:0/897: readlink dd/l8e 0 2024-05-09T06:38:24.685 INFO:tasks.workunit.client.0.smithi063.stdout:6/882: dread - d2/df/d91/ddf/fe1 zero size 2024-05-09T06:38:24.686 INFO:tasks.workunit.client.0.smithi063.stdout:7/845: creat d0/d15/d4a/d68/d10a/f122 x:0 0 0 2024-05-09T06:38:24.687 INFO:tasks.workunit.client.0.smithi063.stdout:0/898: rename dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dcb/lf3 to dd/d1a/d2c/l115 0 2024-05-09T06:38:24.691 INFO:tasks.workunit.client.0.smithi063.stdout:0/899: write dd/d1a/f35 [792024,113361] 0 2024-05-09T06:38:24.691 INFO:tasks.workunit.client.0.smithi063.stdout:7/846: dwrite d0/d15/d4a/d68/f9e [0,4194304] 0 2024-05-09T06:38:24.693 INFO:tasks.workunit.client.0.smithi063.stdout:6/883: rename d2/d4/d96/c10d to d2/df/d91/c11f 0 2024-05-09T06:38:24.694 INFO:tasks.workunit.client.0.smithi063.stdout:6/884: dread - d2/d9a/f118 zero size 2024-05-09T06:38:24.695 INFO:tasks.workunit.client.0.smithi063.stdout:6/885: chown d2/d20/d26/d7f 38338 1 2024-05-09T06:38:24.695 INFO:tasks.workunit.client.0.smithi063.stdout:7/847: rmdir d0/d15/d49/d4f/d66/ddd 39 2024-05-09T06:38:24.696 INFO:tasks.workunit.client.1.smithi193.stdout:5/429: truncate d0/d13/d2c/d37/d5b/d4e/f57 226548 0 2024-05-09T06:38:24.696 INFO:tasks.workunit.client.1.smithi193.stdout:8/494: mknod d6/d1c/d1b/c99 0 2024-05-09T06:38:24.697 INFO:tasks.workunit.client.1.smithi193.stdout:1/534: truncate d4/d14/f1b 37137 0 2024-05-09T06:38:24.698 INFO:tasks.workunit.client.0.smithi063.stdout:6/886: readlink d2/d20/d26/l43 0 2024-05-09T06:38:24.698 INFO:tasks.workunit.client.0.smithi063.stdout:6/887: fdatasync d2/d12/f10a 0 2024-05-09T06:38:24.699 INFO:tasks.workunit.client.0.smithi063.stdout:6/888: readlink d2/d1f/d6c/lb5 0 2024-05-09T06:38:24.706 INFO:tasks.workunit.client.0.smithi063.stdout:7/848: mkdir d0/d15/d89/d10c/de7/d123 0 2024-05-09T06:38:24.707 INFO:tasks.workunit.client.0.smithi063.stdout:2/878: truncate dc/dd/d45/d5a/fdd 125541 0 2024-05-09T06:38:24.707 INFO:tasks.workunit.client.0.smithi063.stdout:6/889: dwrite d2/d4/d7c/d7e/ff0 [0,4194304] 0 2024-05-09T06:38:24.707 INFO:tasks.workunit.client.1.smithi193.stdout:7/489: dread d6/dd/f89 [0,4194304] 0 2024-05-09T06:38:24.710 INFO:tasks.workunit.client.0.smithi063.stdout:2/879: read dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f86 [6604666,22353] 0 2024-05-09T06:38:24.711 INFO:tasks.workunit.client.1.smithi193.stdout:1/535: mknod d4/d76/ca6 0 2024-05-09T06:38:24.713 INFO:tasks.workunit.client.1.smithi193.stdout:8/495: mknod d6/d1c/d18/d2f/d6a/c9a 0 2024-05-09T06:38:24.716 INFO:tasks.workunit.client.0.smithi063.stdout:6/890: rename d2/d12/f10a to d2/d4/d7c/d42/f120 0 2024-05-09T06:38:24.717 INFO:tasks.workunit.client.1.smithi193.stdout:5/430: dread d0/f7 [0,4194304] 0 2024-05-09T06:38:24.718 INFO:tasks.workunit.client.1.smithi193.stdout:7/490: rename d6/d8/d1a/d36/d4d/d74/c90 to d6/dd/d5e/ca8 0 2024-05-09T06:38:24.724 INFO:tasks.workunit.client.1.smithi193.stdout:8/496: mknod d6/d1c/d18/d66/d93/d95/c9b 0 2024-05-09T06:38:24.725 INFO:tasks.workunit.client.0.smithi063.stdout:2/880: truncate dc/d36/d43/d46/fcc 967785 0 2024-05-09T06:38:24.730 INFO:tasks.workunit.client.1.smithi193.stdout:8/497: fsync d6/d1c/d18/d66/f47 0 2024-05-09T06:38:24.730 INFO:tasks.workunit.client.0.smithi063.stdout:5/833: write d8/d17/d75/dc9/dd5/fbe [203922,10781] 0 2024-05-09T06:38:24.734 INFO:tasks.workunit.client.1.smithi193.stdout:8/498: dread d6/fd [0,4194304] 0 2024-05-09T06:38:24.736 INFO:tasks.workunit.client.0.smithi063.stdout:9/848: write d0/d4/d7d/f82 [3752172,86149] 0 2024-05-09T06:38:24.737 INFO:tasks.workunit.client.0.smithi063.stdout:2/881: symlink dc/dd/d2d/l108 0 2024-05-09T06:38:24.744 INFO:tasks.workunit.client.1.smithi193.stdout:8/499: rmdir d6/d1c/d1b/d29 39 2024-05-09T06:38:24.744 INFO:tasks.workunit.client.1.smithi193.stdout:8/500: fdatasync d6/d5b/f8d 0 2024-05-09T06:38:24.746 INFO:tasks.workunit.client.0.smithi063.stdout:4/879: truncate d7/df/d35/d5a/f97 775841 0 2024-05-09T06:38:24.746 INFO:tasks.workunit.client.0.smithi063.stdout:9/849: read d0/d3b/da0/dcf/fd2 [1292642,49399] 0 2024-05-09T06:38:24.747 INFO:tasks.workunit.client.0.smithi063.stdout:9/850: write d0/d4/d7d/fb3 [4552378,124182] 0 2024-05-09T06:38:24.748 INFO:tasks.workunit.client.0.smithi063.stdout:2/882: mknod dc/d36/d43/d46/ded/c109 0 2024-05-09T06:38:24.752 INFO:tasks.workunit.client.1.smithi193.stdout:8/501: dwrite d6/d1c/d18/d66/f47 [0,4194304] 0 2024-05-09T06:38:24.754 INFO:tasks.workunit.client.0.smithi063.stdout:5/834: symlink d8/d19/d115/deb/l116 0 2024-05-09T06:38:24.762 INFO:tasks.workunit.client.0.smithi063.stdout:3/913: dwrite d7/da/d2b/d66/dc0/fb3 [0,4194304] 0 2024-05-09T06:38:24.763 INFO:tasks.workunit.client.0.smithi063.stdout:9/851: creat d0/d3b/da0/dcf/f125 x:0 0 0 2024-05-09T06:38:24.764 INFO:tasks.workunit.client.0.smithi063.stdout:2/883: mknod dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/c10a 0 2024-05-09T06:38:24.765 INFO:tasks.workunit.client.0.smithi063.stdout:1/806: write d9/d2a/d2e/fcc [853568,113676] 0 2024-05-09T06:38:24.766 INFO:tasks.workunit.client.0.smithi063.stdout:2/884: chown dc/d6e/cc8 536816 1 2024-05-09T06:38:24.767 INFO:tasks.workunit.client.1.smithi193.stdout:8/502: creat d6/d1c/d18/f9c x:0 0 0 2024-05-09T06:38:24.771 INFO:tasks.workunit.client.0.smithi063.stdout:9/852: creat d0/d3b/f126 x:0 0 0 2024-05-09T06:38:24.772 INFO:tasks.workunit.client.0.smithi063.stdout:1/807: dwrite d9/d2a/d3c/d4c/d83/d88/fcd [0,4194304] 0 2024-05-09T06:38:24.774 INFO:tasks.workunit.client.0.smithi063.stdout:1/808: write d9/d2a/d35/d1e/fa1 [19345,128855] 0 2024-05-09T06:38:24.775 INFO:tasks.workunit.client.0.smithi063.stdout:1/809: chown d9/d2a/d35/d1e/f8e 18963987 1 2024-05-09T06:38:24.775 INFO:tasks.workunit.client.1.smithi193.stdout:8/503: write d6/d1c/f34 [673557,46143] 0 2024-05-09T06:38:24.776 INFO:tasks.workunit.client.0.smithi063.stdout:5/835: fsync d8/d53/dca/f42 0 2024-05-09T06:38:24.777 INFO:tasks.workunit.client.0.smithi063.stdout:5/836: write d8/d35/f64 [1413757,12940] 0 2024-05-09T06:38:24.778 INFO:tasks.workunit.client.1.smithi193.stdout:6/462: sync 2024-05-09T06:38:24.780 INFO:tasks.workunit.client.0.smithi063.stdout:8/964: sync 2024-05-09T06:38:24.782 INFO:tasks.workunit.client.0.smithi063.stdout:1/810: dread d9/d2a/d3c/d4c/d83/d88/dc2/dd0/ff3 [0,4194304] 0 2024-05-09T06:38:24.787 INFO:tasks.workunit.client.1.smithi193.stdout:8/504: truncate d6/d1c/d53/f63 9070154 0 2024-05-09T06:38:24.788 INFO:tasks.workunit.client.1.smithi193.stdout:8/505: chown d6/d1c/d18/d2f/d40/c69 31 1 2024-05-09T06:38:24.788 INFO:tasks.workunit.client.0.smithi063.stdout:7/849: dwrite d0/f18 [0,4194304] 0 2024-05-09T06:38:24.792 INFO:tasks.workunit.client.0.smithi063.stdout:9/853: creat d0/d4/d57/f127 x:0 0 0 2024-05-09T06:38:24.792 INFO:tasks.workunit.client.0.smithi063.stdout:9/854: stat d0/d3b/da0/f9c 0 2024-05-09T06:38:24.794 INFO:tasks.workunit.client.0.smithi063.stdout:6/891: rename d2/d4/d7c/d42/f120 to d2/d1f/d29/de2/d4e/f121 0 2024-05-09T06:38:24.799 INFO:tasks.workunit.client.1.smithi193.stdout:8/506: creat d6/d1c/d53/f9d x:0 0 0 2024-05-09T06:38:24.800 INFO:tasks.workunit.client.1.smithi193.stdout:8/507: readlink d6/d1c/d18/l35 0 2024-05-09T06:38:24.800 INFO:tasks.workunit.client.1.smithi193.stdout:8/508: chown d6/d1c/d1b/d5d 0 1 2024-05-09T06:38:24.801 INFO:tasks.workunit.client.1.smithi193.stdout:1/536: write d4/f9 [603215,100184] 0 2024-05-09T06:38:24.803 INFO:tasks.workunit.client.1.smithi193.stdout:8/509: write d6/d1c/d18/d2f/f39 [2142628,24538] 0 2024-05-09T06:38:24.806 INFO:tasks.workunit.client.1.smithi193.stdout:6/463: mknod d1/d25/d2c/ca2 0 2024-05-09T06:38:24.806 INFO:tasks.workunit.client.1.smithi193.stdout:7/491: dread - d6/dd/d5e/f77 zero size 2024-05-09T06:38:24.814 INFO:tasks.workunit.client.1.smithi193.stdout:1/537: symlink d4/d16/d60/d4f/d72/d7f/la7 0 2024-05-09T06:38:24.819 INFO:tasks.workunit.client.1.smithi193.stdout:6/464: mknod d1/d25/d2c/d81/d37/d4f/ca3 0 2024-05-09T06:38:24.821 INFO:tasks.workunit.client.1.smithi193.stdout:5/431: dwrite d0/d13/d2c/d37/d2b/d47/f63 [0,4194304] 0 2024-05-09T06:38:24.826 INFO:tasks.workunit.client.0.smithi063.stdout:2/885: rmdir dc/d36/d43/d46/d49/dbb/dec 39 2024-05-09T06:38:24.827 INFO:tasks.workunit.client.1.smithi193.stdout:1/538: dread - d4/d16/f5f zero size 2024-05-09T06:38:24.829 INFO:tasks.workunit.client.0.smithi063.stdout:0/900: sync 2024-05-09T06:38:24.829 INFO:tasks.workunit.client.1.smithi193.stdout:1/539: fdatasync d4/d16/d17/d1a/d3a/d3f/f55 0 2024-05-09T06:38:24.830 INFO:tasks.workunit.client.0.smithi063.stdout:0/901: stat dd/d1a/d2c/d51/d72/d9e/d23/d46/lf0 0 2024-05-09T06:38:24.831 INFO:tasks.workunit.client.1.smithi193.stdout:1/540: write d4/d16/d60/d4f/d72/f7a [778638,88091] 0 2024-05-09T06:38:24.832 INFO:tasks.workunit.client.1.smithi193.stdout:1/541: chown d4/d16/d60/d4f/d72/d96 363384995 1 2024-05-09T06:38:24.837 INFO:tasks.workunit.client.0.smithi063.stdout:6/892: unlink d2/d20/la6 0 2024-05-09T06:38:24.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/880: dwrite d7/f39 [4194304,4194304] 0 2024-05-09T06:38:24.840 INFO:tasks.workunit.client.1.smithi193.stdout:8/510: dread d6/d1c/f1e [0,4194304] 0 2024-05-09T06:38:24.843 INFO:tasks.workunit.client.0.smithi063.stdout:3/914: write d7/d22/d28/dab/fe0 [749855,8320] 0 2024-05-09T06:38:24.845 INFO:tasks.workunit.client.1.smithi193.stdout:5/432: creat d0/d13/d2c/d37/d5b/d89/f8d x:0 0 0 2024-05-09T06:38:24.852 INFO:tasks.workunit.client.1.smithi193.stdout:5/433: rename d0/d13/d2c/d37/d5b/l5a to d0/d13/d2c/d37/d74/l8e 0 2024-05-09T06:38:24.854 INFO:tasks.workunit.client.1.smithi193.stdout:5/434: truncate d0/d13/d2c/d37/d74/f80 408519 0 2024-05-09T06:38:24.856 INFO:tasks.workunit.client.1.smithi193.stdout:5/435: write d0/d13/d2c/d37/d5b/d4e/f6b [746912,18771] 0 2024-05-09T06:38:24.859 INFO:tasks.workunit.client.1.smithi193.stdout:1/542: getdents d4/d3c 0 2024-05-09T06:38:24.860 INFO:tasks.workunit.client.0.smithi063.stdout:5/837: write d8/d17/d75/dc9/dd5/d96/fc5 [1756513,3050] 0 2024-05-09T06:38:24.861 INFO:tasks.workunit.client.0.smithi063.stdout:4/881: creat d7/da3/d28/d31/d32/f11c x:0 0 0 2024-05-09T06:38:24.862 INFO:tasks.workunit.client.1.smithi193.stdout:1/543: read d4/d16/d17/d1a/f1f [2704450,25145] 0 2024-05-09T06:38:24.865 INFO:tasks.workunit.client.0.smithi063.stdout:3/915: rename d7/da/d21/d30/d44/d48/ld9 to d7/da/d2b/d66/d74/l13e 0 2024-05-09T06:38:24.866 INFO:tasks.workunit.client.1.smithi193.stdout:5/436: fsync d0/d13/d2c/d37/d5b/f6 0 2024-05-09T06:38:24.867 INFO:tasks.workunit.client.0.smithi063.stdout:2/886: mkdir dc/d36/d43/dda/d10b 0 2024-05-09T06:38:24.868 INFO:tasks.workunit.client.0.smithi063.stdout:8/965: dwrite d9/da/d53/fa2 [0,4194304] 0 2024-05-09T06:38:24.868 INFO:tasks.workunit.client.0.smithi063.stdout:1/811: write d9/d2a/d35/d1e/f70 [533290,28240] 0 2024-05-09T06:38:24.868 INFO:tasks.workunit.client.1.smithi193.stdout:7/492: write d6/d8/fa [3400364,103734] 0 2024-05-09T06:38:24.869 INFO:tasks.workunit.client.0.smithi063.stdout:1/812: dread - d9/d46/dfe/d94/d112/f10c zero size 2024-05-09T06:38:24.869 INFO:tasks.workunit.client.1.smithi193.stdout:7/493: fdatasync d6/dd/f6c 0 2024-05-09T06:38:24.872 INFO:tasks.workunit.client.1.smithi193.stdout:6/465: write d1/f42 [1283732,34845] 0 2024-05-09T06:38:24.873 INFO:tasks.workunit.client.0.smithi063.stdout:7/850: creat d0/d15/d49/d54/d9f/d97/f124 x:0 0 0 2024-05-09T06:38:24.875 INFO:tasks.workunit.client.0.smithi063.stdout:7/851: stat d0/d15/d49/d4f/l11f 0 2024-05-09T06:38:24.877 INFO:tasks.workunit.client.0.smithi063.stdout:1/813: dwrite d9/d2a/d39/f105 [0,4194304] 0 2024-05-09T06:38:24.885 INFO:tasks.workunit.client.1.smithi193.stdout:8/511: dwrite d6/d1c/d18/d2f/d40/f2c [0,4194304] 0 2024-05-09T06:38:24.887 INFO:tasks.workunit.client.1.smithi193.stdout:3/421: sync 2024-05-09T06:38:24.888 INFO:tasks.workunit.client.0.smithi063.stdout:4/882: unlink d7/d3d/faa 0 2024-05-09T06:38:24.889 INFO:tasks.workunit.client.0.smithi063.stdout:9/855: truncate d0/d4/d26/d39/d48/d80/fad 2226332 0 2024-05-09T06:38:24.889 INFO:tasks.workunit.client.1.smithi193.stdout:7/494: dread d6/d8/f55 [0,4194304] 0 2024-05-09T06:38:24.891 INFO:tasks.workunit.client.1.smithi193.stdout:7/495: write d6/f30 [3956311,22388] 0 2024-05-09T06:38:24.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/466: mkdir d1/de/d11/d2d/da4 0 2024-05-09T06:38:24.899 INFO:tasks.workunit.client.1.smithi193.stdout:3/422: dwrite d1/d44/f47 [0,4194304] 0 2024-05-09T06:38:24.902 INFO:tasks.workunit.client.1.smithi193.stdout:7/496: fsync d6/f30 0 2024-05-09T06:38:24.906 INFO:tasks.workunit.client.1.smithi193.stdout:7/497: dread d6/d8/f44 [0,4194304] 0 2024-05-09T06:38:24.906 INFO:tasks.workunit.client.0.smithi063.stdout:0/902: creat dd/d1a/f116 x:0 0 0 2024-05-09T06:38:24.909 INFO:tasks.workunit.client.0.smithi063.stdout:2/887: dwrite dc/fd8 [0,4194304] 0 2024-05-09T06:38:24.909 INFO:tasks.workunit.client.1.smithi193.stdout:1/544: link d4/c9e d4/d3c/d3e/ca8 0 2024-05-09T06:38:24.911 INFO:tasks.workunit.client.0.smithi063.stdout:7/852: chown d0/d15/d49/d54/c62 280 1 2024-05-09T06:38:24.916 INFO:tasks.workunit.client.0.smithi063.stdout:1/814: mknod d9/d2a/d3c/d4c/d83/d88/dc2/dd0/c11b 0 2024-05-09T06:38:24.922 INFO:tasks.workunit.client.1.smithi193.stdout:8/512: rmdir d6/d1c/d1b/d29/d57 39 2024-05-09T06:38:24.923 INFO:tasks.workunit.client.0.smithi063.stdout:4/883: mknod d7/df/d35/d5a/dad/de9/c11d 0 2024-05-09T06:38:24.925 INFO:tasks.workunit.client.0.smithi063.stdout:9/856: symlink d0/d3b/d63/l128 0 2024-05-09T06:38:24.929 INFO:tasks.workunit.client.1.smithi193.stdout:8/513: rmdir d6/d1c/d18/d2f/d40/d5e 39 2024-05-09T06:38:24.932 INFO:tasks.workunit.client.0.smithi063.stdout:0/903: read dd/d1a/f2e [3276501,97467] 0 2024-05-09T06:38:24.934 INFO:tasks.workunit.client.1.smithi193.stdout:7/498: rename d6/dd/c71 to d6/dd/d5e/ca9 0 2024-05-09T06:38:24.935 INFO:tasks.workunit.client.0.smithi063.stdout:6/893: getdents d2/d20/d26/d7f/dcf 0 2024-05-09T06:38:24.936 INFO:tasks.workunit.client.1.smithi193.stdout:3/423: dwrite d1/d44/f96 [4194304,4194304] 0 2024-05-09T06:38:24.938 INFO:tasks.workunit.client.1.smithi193.stdout:1/545: getdents d4/d16/d6b/d85 0 2024-05-09T06:38:24.941 INFO:tasks.workunit.client.0.smithi063.stdout:1/815: mkdir d9/d2a/d3c/dbc/d11c 0 2024-05-09T06:38:24.942 INFO:tasks.workunit.client.1.smithi193.stdout:7/499: write d6/dd/d70/f8e [276804,107743] 0 2024-05-09T06:38:24.943 INFO:tasks.workunit.client.0.smithi063.stdout:5/838: creat d8/d19/d30/d74/f117 x:0 0 0 2024-05-09T06:38:24.944 INFO:tasks.workunit.client.0.smithi063.stdout:5/839: truncate d8/d19/d30/f102 16271 0 2024-05-09T06:38:24.945 INFO:tasks.workunit.client.0.smithi063.stdout:4/884: mknod d7/da3/d28/d31/d32/d34/c11e 0 2024-05-09T06:38:24.946 INFO:tasks.workunit.client.0.smithi063.stdout:4/885: chown d7/da3/d28/d31/d7b/d81/c8d 137064783 1 2024-05-09T06:38:24.947 INFO:tasks.workunit.client.1.smithi193.stdout:3/424: rename d1/db/d2f/f33 to d1/d44/d7d/f9c 0 2024-05-09T06:38:24.952 INFO:tasks.workunit.client.0.smithi063.stdout:3/916: link f1 d7/d22/f13f 0 2024-05-09T06:38:24.953 INFO:tasks.workunit.client.1.smithi193.stdout:7/500: rmdir d6/d17/d7d 39 2024-05-09T06:38:24.954 INFO:tasks.workunit.client.0.smithi063.stdout:8/966: link d9/da/d18/dc7/d58/ccf d9/da/d18/d24/d71/d11d/c152 0 2024-05-09T06:38:24.955 INFO:tasks.workunit.client.1.smithi193.stdout:1/546: rename d4/d16/d17/d1a/d3a/d3f/f95 to d4/d16/d35/fa9 0 2024-05-09T06:38:24.956 INFO:tasks.workunit.client.1.smithi193.stdout:1/547: fsync d4/d16/d17/d1a/d3a/d3f/f68 0 2024-05-09T06:38:24.957 INFO:tasks.workunit.client.0.smithi063.stdout:2/888: dwrite dc/d36/d43/d46/ded/ff0 [0,4194304] 0 2024-05-09T06:38:24.957 INFO:tasks.workunit.client.1.smithi193.stdout:1/548: readlink d4/d16/d60/d4f/l9f 0 2024-05-09T06:38:24.958 INFO:tasks.workunit.client.1.smithi193.stdout:3/425: fdatasync d1/d2d/f45 0 2024-05-09T06:38:24.958 INFO:tasks.workunit.client.0.smithi063.stdout:2/889: readlink dc/dd/l4a 0 2024-05-09T06:38:24.960 INFO:tasks.workunit.client.0.smithi063.stdout:7/853: dwrite d0/d15/d49/d4f/d66/f107 [0,4194304] 0 2024-05-09T06:38:24.963 INFO:tasks.workunit.client.0.smithi063.stdout:9/857: dread d0/d4/d26/f49 [0,4194304] 0 2024-05-09T06:38:24.967 INFO:tasks.workunit.client.0.smithi063.stdout:5/840: mknod d8/d35/dd6/c118 0 2024-05-09T06:38:24.968 INFO:tasks.workunit.client.0.smithi063.stdout:5/841: stat d8/d19/d30/d74/ff1 0 2024-05-09T06:38:24.968 INFO:tasks.workunit.client.0.smithi063.stdout:6/894: dwrite d2/d1f/d29/d3f/d5c/d64/f10b [0,4194304] 0 2024-05-09T06:38:24.969 INFO:tasks.workunit.client.1.smithi193.stdout:3/426: creat d1/db/d2f/f9d x:0 0 0 2024-05-09T06:38:24.970 INFO:tasks.workunit.client.1.smithi193.stdout:3/427: chown d1/f4 939 1 2024-05-09T06:38:24.975 INFO:tasks.workunit.client.1.smithi193.stdout:1/549: rename d4/d16/c62 to d4/d3c/caa 0 2024-05-09T06:38:24.977 INFO:tasks.workunit.client.0.smithi063.stdout:3/917: mkdir d7/d22/dfc/d140 0 2024-05-09T06:38:24.979 INFO:tasks.workunit.client.0.smithi063.stdout:0/904: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d89/f8a [0,4194304] 0 2024-05-09T06:38:24.981 INFO:tasks.workunit.client.1.smithi193.stdout:1/550: read d4/d16/f42 [1715364,65156] 0 2024-05-09T06:38:24.981 INFO:tasks.workunit.client.0.smithi063.stdout:0/905: write dd/d1a/d2c/d51/d76/d97/fe7 [1712119,21522] 0 2024-05-09T06:38:24.983 INFO:tasks.workunit.client.1.smithi193.stdout:7/501: rename d6/dd/d72 to d6/d17/d7d/daa 0 2024-05-09T06:38:24.984 INFO:tasks.workunit.client.0.smithi063.stdout:8/967: chown d9/da/d18/dc7/c7a 1246138 1 2024-05-09T06:38:24.988 INFO:tasks.workunit.client.1.smithi193.stdout:1/551: symlink d4/d16/d60/d4f/lab 0 2024-05-09T06:38:24.991 INFO:tasks.workunit.client.0.smithi063.stdout:2/890: dwrite dc/dd/d2d/d60/dd0/d90/fef [0,4194304] 0 2024-05-09T06:38:24.993 INFO:tasks.workunit.client.0.smithi063.stdout:1/816: rename d9/d2a/fd8 to d9/d2a/d3c/dbc/d11c/f11d 0 2024-05-09T06:38:24.995 INFO:tasks.workunit.client.1.smithi193.stdout:1/552: symlink d4/d16/d6b/d89/lac 0 2024-05-09T06:38:24.997 INFO:tasks.workunit.client.0.smithi063.stdout:5/842: mkdir d8/d65/d9e/d119 0 2024-05-09T06:38:24.999 INFO:tasks.workunit.client.0.smithi063.stdout:5/843: read d8/d19/f1a [4342184,125666] 0 2024-05-09T06:38:25.003 INFO:tasks.workunit.client.0.smithi063.stdout:6/895: mknod d2/d1f/d29/d3f/c122 0 2024-05-09T06:38:25.003 INFO:tasks.workunit.client.0.smithi063.stdout:3/918: creat d7/d22/f141 x:0 0 0 2024-05-09T06:38:25.003 INFO:tasks.workunit.client.0.smithi063.stdout:8/968: dread - d9/d41/dec/da1/fa7 zero size 2024-05-09T06:38:25.003 INFO:tasks.workunit.client.0.smithi063.stdout:3/919: chown d7/d70/d98 59565218 1 2024-05-09T06:38:25.004 INFO:tasks.workunit.client.1.smithi193.stdout:3/428: dread d1/d2d/d51/f6a [0,4194304] 0 2024-05-09T06:38:25.004 INFO:tasks.workunit.client.0.smithi063.stdout:3/920: truncate d7/d22/f141 394342 0 2024-05-09T06:38:25.005 INFO:tasks.workunit.client.0.smithi063.stdout:7/854: mkdir d0/d15/d89/d10c/de7/d117/d125 0 2024-05-09T06:38:25.006 INFO:tasks.workunit.client.0.smithi063.stdout:3/921: truncate d7/d70/d98/dbb/ff1 4593260 0 2024-05-09T06:38:25.007 INFO:tasks.workunit.client.0.smithi063.stdout:2/891: write dc/d36/f57 [1238651,38691] 0 2024-05-09T06:38:25.008 INFO:tasks.workunit.client.1.smithi193.stdout:3/429: fsync d1/d2d/d51/f6a 0 2024-05-09T06:38:25.009 INFO:tasks.workunit.client.1.smithi193.stdout:3/430: chown d1/c16 431 1 2024-05-09T06:38:25.010 INFO:tasks.workunit.client.0.smithi063.stdout:4/886: write d7/da3/d28/d31/d32/d34/f59 [5201679,95418] 0 2024-05-09T06:38:25.011 INFO:tasks.workunit.client.0.smithi063.stdout:9/858: creat d0/d109/f129 x:0 0 0 2024-05-09T06:38:25.011 INFO:tasks.workunit.client.0.smithi063.stdout:4/887: dread - d7/da3/d28/d31/d7b/fbe zero size 2024-05-09T06:38:25.012 INFO:tasks.workunit.client.0.smithi063.stdout:9/859: stat d0/d4/d57/d60/df5 0 2024-05-09T06:38:25.016 INFO:tasks.workunit.client.0.smithi063.stdout:5/844: creat d8/d65/f11a x:0 0 0 2024-05-09T06:38:25.020 INFO:tasks.workunit.client.1.smithi193.stdout:3/431: truncate d1/db/d2f/d84/f8d 1013103 0 2024-05-09T06:38:25.022 INFO:tasks.workunit.client.0.smithi063.stdout:0/906: truncate dd/d1a/d2c/d51/d72/d9e/d23/d33/d48/d75/fa6 2248727 0 2024-05-09T06:38:25.024 INFO:tasks.workunit.client.0.smithi063.stdout:6/896: creat d2/d4/d7c/f123 x:0 0 0 2024-05-09T06:38:25.029 INFO:tasks.workunit.client.1.smithi193.stdout:3/432: dwrite d1/d44/f47 [4194304,4194304] 0 2024-05-09T06:38:25.029 INFO:tasks.workunit.client.1.smithi193.stdout:7/502: dwrite d6/d8/d1a/d36/f4e [0,4194304] 0 2024-05-09T06:38:25.029 INFO:tasks.workunit.client.1.smithi193.stdout:3/433: fsync d1/f1d 0 2024-05-09T06:38:25.033 INFO:tasks.workunit.client.1.smithi193.stdout:7/503: fdatasync d6/d8/d29/f7a 0 2024-05-09T06:38:25.040 INFO:tasks.workunit.client.0.smithi063.stdout:1/817: link d9/d46/f6e d9/d2a/d3c/d4a/db8/f11e 0 2024-05-09T06:38:25.041 INFO:tasks.workunit.client.0.smithi063.stdout:4/888: symlink d7/d3d/dca/l11f 0 2024-05-09T06:38:25.042 INFO:tasks.workunit.client.0.smithi063.stdout:2/892: dread dc/f21 [0,4194304] 0 2024-05-09T06:38:25.045 INFO:tasks.workunit.client.1.smithi193.stdout:1/553: dwrite d4/d16/d17/d1a/f1e [0,4194304] 0 2024-05-09T06:38:25.047 INFO:tasks.workunit.client.1.smithi193.stdout:7/504: chown d6/d8/d29/d2f/f4c 311016891 1 2024-05-09T06:38:25.050 INFO:tasks.workunit.client.1.smithi193.stdout:3/434: getdents d1/db/d2f/d84/d97 0 2024-05-09T06:38:25.054 INFO:tasks.workunit.client.0.smithi063.stdout:0/907: symlink dd/d60/d44/d90/da4/l117 0 2024-05-09T06:38:25.055 INFO:tasks.workunit.client.0.smithi063.stdout:0/908: dread - dd/f10a zero size 2024-05-09T06:38:25.056 INFO:tasks.workunit.client.1.smithi193.stdout:3/435: dread d1/d2d/d51/f6a [0,4194304] 0 2024-05-09T06:38:25.058 INFO:tasks.workunit.client.1.smithi193.stdout:7/505: symlink d6/dd/d70/lab 0 2024-05-09T06:38:25.058 INFO:tasks.workunit.client.0.smithi063.stdout:3/922: write d7/da/d21/d30/f57 [3995701,39792] 0 2024-05-09T06:38:25.059 INFO:tasks.workunit.client.0.smithi063.stdout:8/969: mknod d9/da/d18/d24/c153 0 2024-05-09T06:38:25.059 INFO:tasks.workunit.client.1.smithi193.stdout:1/554: dwrite d4/d16/d35/fa9 [0,4194304] 0 2024-05-09T06:38:25.061 INFO:tasks.workunit.client.1.smithi193.stdout:9/562: sync 2024-05-09T06:38:25.062 INFO:tasks.workunit.client.1.smithi193.stdout:2/509: sync 2024-05-09T06:38:25.062 INFO:tasks.workunit.client.1.smithi193.stdout:0/478: sync 2024-05-09T06:38:25.062 INFO:tasks.workunit.client.1.smithi193.stdout:4/562: sync 2024-05-09T06:38:25.063 INFO:tasks.workunit.client.0.smithi063.stdout:4/889: mknod d7/df/d35/da5/c120 0 2024-05-09T06:38:25.063 INFO:tasks.workunit.client.0.smithi063.stdout:4/890: stat d7/df/l5c 0 2024-05-09T06:38:25.063 INFO:tasks.workunit.client.1.smithi193.stdout:7/506: readlink d6/l1d 0 2024-05-09T06:38:25.064 INFO:tasks.workunit.client.0.smithi063.stdout:2/893: mknod dc/d36/d43/d46/d49/ddc/c10c 0 2024-05-09T06:38:25.064 INFO:tasks.workunit.client.1.smithi193.stdout:9/563: write dd/df/d15/f9e [532570,83344] 0 2024-05-09T06:38:25.064 INFO:tasks.workunit.client.1.smithi193.stdout:1/555: mknod d4/d16/d60/d4f/d59/cad 0 2024-05-09T06:38:25.064 INFO:tasks.workunit.client.0.smithi063.stdout:2/894: chown dc/d36/d43/d46/d63/d79/d7d/le4 1263410260 1 2024-05-09T06:38:25.066 INFO:tasks.workunit.client.1.smithi193.stdout:2/510: truncate db/d25/d38/d5b/f88 979356 0 2024-05-09T06:38:25.066 INFO:tasks.workunit.client.0.smithi063.stdout:9/860: truncate d0/d4/d7d/d105/d110/fbe 642167 0 2024-05-09T06:38:25.068 INFO:tasks.workunit.client.1.smithi193.stdout:3/436: dread d1/f11 [0,4194304] 0 2024-05-09T06:38:25.069 INFO:tasks.workunit.client.1.smithi193.stdout:4/563: rmdir d1a/d80 39 2024-05-09T06:38:25.071 INFO:tasks.workunit.client.1.smithi193.stdout:0/479: symlink d18/d4c/d47/d5e/l95 0 2024-05-09T06:38:25.071 INFO:tasks.workunit.client.1.smithi193.stdout:0/480: stat d18/d27/c8f 0 2024-05-09T06:38:25.073 INFO:tasks.workunit.client.1.smithi193.stdout:7/507: mknod d6/d8/d1a/d36/cac 0 2024-05-09T06:38:25.073 INFO:tasks.workunit.client.1.smithi193.stdout:0/481: write d18/d27/f41 [127572,89012] 0 2024-05-09T06:38:25.075 INFO:tasks.workunit.client.0.smithi063.stdout:0/909: creat dd/d60/d44/f118 x:0 0 0 2024-05-09T06:38:25.076 INFO:tasks.workunit.client.1.smithi193.stdout:0/482: dread d18/d4c/d47/d54/f68 [4194304,4194304] 0 2024-05-09T06:38:25.077 INFO:tasks.workunit.client.0.smithi063.stdout:5/845: dwrite d8/d35/fad [0,4194304] 0 2024-05-09T06:38:25.078 INFO:tasks.workunit.client.0.smithi063.stdout:5/846: chown d8/d17/d75/fdb 46 1 2024-05-09T06:38:25.079 INFO:tasks.workunit.client.0.smithi063.stdout:6/897: link d2/d1f/d6c/f105 d2/d1f/f124 0 2024-05-09T06:38:25.079 INFO:tasks.workunit.client.0.smithi063.stdout:5/847: stat d8/d53/dca/l33 0 2024-05-09T06:38:25.080 INFO:tasks.workunit.client.1.smithi193.stdout:1/556: truncate d4/d16/f42 447885 0 2024-05-09T06:38:25.082 INFO:tasks.workunit.client.0.smithi063.stdout:3/923: rename d7/da/d21/lb2 to d7/d4c/l142 0 2024-05-09T06:38:25.084 INFO:tasks.workunit.client.1.smithi193.stdout:2/511: rename db/d25/d38/d5b/l74 to db/d25/d38/d61/laf 0 2024-05-09T06:38:25.086 INFO:tasks.workunit.client.0.smithi063.stdout:8/970: stat d9/da/d18/fda 0 2024-05-09T06:38:25.088 INFO:tasks.workunit.client.1.smithi193.stdout:3/437: symlink d1/db/d2f/l9e 0 2024-05-09T06:38:25.089 INFO:tasks.workunit.client.0.smithi063.stdout:1/818: mkdir d9/d2a/d35/d1e/d9d/d11f 0 2024-05-09T06:38:25.090 INFO:tasks.workunit.client.0.smithi063.stdout:1/819: read - d9/d46/f63 zero size 2024-05-09T06:38:25.090 INFO:tasks.workunit.client.1.smithi193.stdout:0/483: dread d18/d4c/d47/d59/d76/f77 [0,4194304] 0 2024-05-09T06:38:25.091 INFO:tasks.workunit.client.1.smithi193.stdout:4/564: creat d1a/d24/d73/d94/fae x:0 0 0 2024-05-09T06:38:25.092 INFO:tasks.workunit.client.0.smithi063.stdout:4/891: mkdir d7/da3/d28/d31/d32/d34/d121 0 2024-05-09T06:38:25.094 INFO:tasks.workunit.client.1.smithi193.stdout:9/564: write dd/d1e/d3c/f7d [289938,54449] 0 2024-05-09T06:38:25.095 INFO:tasks.workunit.client.1.smithi193.stdout:9/565: dread - dd/df/d31/d7e/d71/fac zero size 2024-05-09T06:38:25.096 INFO:tasks.workunit.client.0.smithi063.stdout:9/861: creat d0/d4/d26/d39/d48/d80/d84/dc2/d10f/f12a x:0 0 0 2024-05-09T06:38:25.096 INFO:tasks.workunit.client.0.smithi063.stdout:9/862: dread - d0/d4/d26/d39/d91/f11b zero size 2024-05-09T06:38:25.098 INFO:tasks.workunit.client.1.smithi193.stdout:1/557: symlink d4/d16/lae 0 2024-05-09T06:38:25.098 INFO:tasks.workunit.client.0.smithi063.stdout:6/898: mknod d2/d1f/d6c/c125 0 2024-05-09T06:38:25.100 INFO:tasks.workunit.client.1.smithi193.stdout:3/438: creat d1/db/d59/f9f x:0 0 0 2024-05-09T06:38:25.103 INFO:tasks.workunit.client.0.smithi063.stdout:0/910: rename dd/d1a/d2c/d51/d72/d9e/d23/d33 to dd/d60/d119 0 2024-05-09T06:38:25.103 INFO:tasks.workunit.client.1.smithi193.stdout:2/512: dread db/d25/f49 [0,4194304] 0 2024-05-09T06:38:25.103 INFO:tasks.workunit.client.1.smithi193.stdout:0/484: truncate fe 3463130 0 2024-05-09T06:38:25.104 INFO:tasks.workunit.client.1.smithi193.stdout:2/513: chown db/d37/d6d/d80/d60/d7d/da8 1793850 1 2024-05-09T06:38:25.105 INFO:tasks.workunit.client.1.smithi193.stdout:4/565: creat d1a/d24/d73/d94/faf x:0 0 0 2024-05-09T06:38:25.109 INFO:tasks.workunit.client.1.smithi193.stdout:7/508: mknod d6/dd/d5e/cad 0 2024-05-09T06:38:25.117 INFO:tasks.workunit.client.1.smithi193.stdout:9/566: rename dd/d1e/d1f/d48/d55/d41/ca9 to dd/d1e/d1f/d43/dab/cba 0 2024-05-09T06:38:25.126 INFO:tasks.workunit.client.1.smithi193.stdout:0/485: dread d18/d4c/d47/d54/f68 [0,4194304] 0 2024-05-09T06:38:25.126 INFO:tasks.workunit.client.1.smithi193.stdout:2/514: creat db/d37/d6d/d80/fb0 x:0 0 0 2024-05-09T06:38:25.126 INFO:tasks.workunit.client.1.smithi193.stdout:0/486: chown d18/d65/l67 948201 1 2024-05-09T06:38:25.127 INFO:tasks.workunit.client.0.smithi063.stdout:6/899: chown d2/d1f/d73/d88 0 1 2024-05-09T06:38:25.127 INFO:tasks.workunit.client.0.smithi063.stdout:6/900: chown d2/d4/d7c/d7e/cae 2053025 1 2024-05-09T06:38:25.127 INFO:tasks.workunit.client.0.smithi063.stdout:3/924: mkdir d7/d143 0 2024-05-09T06:38:25.129 INFO:tasks.workunit.client.0.smithi063.stdout:3/925: dread d7/da/d2b/f43 [0,4194304] 0 2024-05-09T06:38:25.130 INFO:tasks.workunit.client.1.smithi193.stdout:4/566: dread d1a/d24/d73/d94/f52 [0,4194304] 0 2024-05-09T06:38:25.131 INFO:tasks.workunit.client.0.smithi063.stdout:3/926: write d7/d22/d28/fec [14432,88325] 0 2024-05-09T06:38:25.133 INFO:tasks.workunit.client.1.smithi193.stdout:2/515: dwrite db/d25/d38/d40/f72 [8388608,4194304] 0 2024-05-09T06:38:25.133 INFO:tasks.workunit.client.0.smithi063.stdout:2/895: truncate dc/d36/f80 2692313 0 2024-05-09T06:38:25.134 INFO:tasks.workunit.client.1.smithi193.stdout:2/516: readlink db/d4b/l9e 0 2024-05-09T06:38:25.136 INFO:tasks.workunit.client.1.smithi193.stdout:2/517: chown db/d25/d38/l8d 127949 1 2024-05-09T06:38:25.136 INFO:tasks.workunit.client.1.smithi193.stdout:9/567: mkdir dd/d1e/d93/dbb 0 2024-05-09T06:38:25.143 INFO:tasks.workunit.client.0.smithi063.stdout:9/863: dwrite d0/d4/d57/d60/d76/fe1 [0,4194304] 0 2024-05-09T06:38:25.144 INFO:tasks.workunit.client.1.smithi193.stdout:0/487: fsync d18/d4c/d47/d5e/d70/f83 0 2024-05-09T06:38:25.144 INFO:tasks.workunit.client.0.smithi063.stdout:9/864: readlink d0/d43/lac 0 2024-05-09T06:38:25.147 INFO:tasks.workunit.client.0.smithi063.stdout:7/855: sync 2024-05-09T06:38:25.148 INFO:tasks.workunit.client.0.smithi063.stdout:8/971: creat d9/f154 x:0 0 0 2024-05-09T06:38:25.152 INFO:tasks.workunit.client.0.smithi063.stdout:1/820: creat d9/d2a/d3c/d4c/f120 x:0 0 0 2024-05-09T06:38:25.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/567: dread d1a/f64 [0,4194304] 0 2024-05-09T06:38:25.156 INFO:tasks.workunit.client.1.smithi193.stdout:4/568: stat d1a/d22/d5d/ca8 0 2024-05-09T06:38:25.157 INFO:tasks.workunit.client.0.smithi063.stdout:5/848: link d8/c15 d8/d19/d30/d46/c11b 0 2024-05-09T06:38:25.157 INFO:tasks.workunit.client.0.smithi063.stdout:5/849: chown d8/d17/d18/c72 3171 1 2024-05-09T06:38:25.159 INFO:tasks.workunit.client.0.smithi063.stdout:6/901: write d2/d1f/d29/fb1 [3747027,36680] 0 2024-05-09T06:38:25.160 INFO:tasks.workunit.client.0.smithi063.stdout:6/902: write d2/d20/d26/fca [2227533,76108] 0 2024-05-09T06:38:25.161 INFO:tasks.workunit.client.1.smithi193.stdout:5/437: sync 2024-05-09T06:38:25.163 INFO:tasks.workunit.client.0.smithi063.stdout:7/856: creat d0/d15/d89/d10c/dfb/f126 x:0 0 0 2024-05-09T06:38:25.167 INFO:tasks.workunit.client.1.smithi193.stdout:1/558: rename d4/d14/l49 to d4/d16/d6b/laf 0 2024-05-09T06:38:25.168 INFO:tasks.workunit.client.1.smithi193.stdout:1/559: fdatasync d4/d16/f47 0 2024-05-09T06:38:25.171 INFO:tasks.workunit.client.0.smithi063.stdout:2/896: write dc/f1f [2907754,61327] 0 2024-05-09T06:38:25.176 INFO:tasks.workunit.client.0.smithi063.stdout:9/865: write d0/d4/d26/d39/d48/d80/d84/faa [520404,90363] 0 2024-05-09T06:38:25.176 INFO:tasks.workunit.client.0.smithi063.stdout:9/866: chown d0/d4/d57/d60/df5 921866153 1 2024-05-09T06:38:25.177 INFO:tasks.workunit.client.1.smithi193.stdout:5/438: dread d0/d13/d2c/d37/d5b/d4e/f57 [0,4194304] 0 2024-05-09T06:38:25.178 INFO:tasks.workunit.client.1.smithi193.stdout:3/439: rename d1/db/f14 to d1/d44/fa0 0 2024-05-09T06:38:25.179 INFO:tasks.workunit.client.1.smithi193.stdout:5/439: dread - d0/d13/d2c/d37/d5b/d4e/f7a zero size 2024-05-09T06:38:25.180 INFO:tasks.workunit.client.1.smithi193.stdout:5/440: chown d0/d13/d2c/d37/d2b/d47/d6f 1434713 1 2024-05-09T06:38:25.181 INFO:tasks.workunit.client.1.smithi193.stdout:9/568: dread dd/d1e/d1f/d48/f5a [0,4194304] 0 2024-05-09T06:38:25.181 INFO:tasks.workunit.client.0.smithi063.stdout:3/927: link d7/d4c/c3d d7/da/d2b/d66/d74/d7e/d97/c144 0 2024-05-09T06:38:25.182 INFO:tasks.workunit.client.0.smithi063.stdout:3/928: chown d7/da/d21/f100 5723560 1 2024-05-09T06:38:25.182 INFO:tasks.workunit.client.1.smithi193.stdout:9/569: write dd/df/d15/f9e [999732,76474] 0 2024-05-09T06:38:25.185 INFO:tasks.workunit.client.1.smithi193.stdout:2/518: link db/d37/d6d/d80/l3b db/d4b/d56/lb1 0 2024-05-09T06:38:25.186 INFO:tasks.workunit.client.1.smithi193.stdout:2/519: stat db/d25/d38/l8d 0 2024-05-09T06:38:25.187 INFO:tasks.workunit.client.1.smithi193.stdout:7/509: link d6/dd/l4a d6/lae 0 2024-05-09T06:38:25.189 INFO:tasks.workunit.client.0.smithi063.stdout:5/850: mknod d8/d17/dfe/c11c 0 2024-05-09T06:38:25.192 INFO:tasks.workunit.client.0.smithi063.stdout:9/867: symlink d0/d4/d26/d39/d91/l12b 0 2024-05-09T06:38:25.193 INFO:tasks.workunit.client.0.smithi063.stdout:9/868: write d0/d4/d57/fe8 [297896,5884] 0 2024-05-09T06:38:25.196 INFO:tasks.workunit.client.1.smithi193.stdout:1/560: symlink d4/d16/d17/d1a/d3a/lb0 0 2024-05-09T06:38:25.197 INFO:tasks.workunit.client.0.smithi063.stdout:6/903: truncate d2/d20/d26/d7f/dcf/fda 3963266 0 2024-05-09T06:38:25.197 INFO:tasks.workunit.client.1.smithi193.stdout:1/561: write d4/d76/f80 [15381,126289] 0 2024-05-09T06:38:25.198 INFO:tasks.workunit.client.1.smithi193.stdout:9/570: creat dd/df/d15/d8c/fbc x:0 0 0 2024-05-09T06:38:25.202 INFO:tasks.workunit.client.1.smithi193.stdout:2/520: mkdir db/d25/d38/d40/db2 0 2024-05-09T06:38:25.202 INFO:tasks.workunit.client.1.smithi193.stdout:5/441: write d0/d13/d2c/d37/d5b/d4e/f57 [98653,85158] 0 2024-05-09T06:38:25.204 INFO:tasks.workunit.client.0.smithi063.stdout:1/821: rename d9/d46/dfe/d94/ce4 to d9/d2a/d3c/d4c/d83/c121 0 2024-05-09T06:38:25.205 INFO:tasks.workunit.client.0.smithi063.stdout:1/822: read - d9/d2a/d3c/d4c/dc3/f111 zero size 2024-05-09T06:38:25.206 INFO:tasks.workunit.client.1.smithi193.stdout:4/569: rename d1a/d22/c4c to d1a/d24/d73/d94/cb0 0 2024-05-09T06:38:25.207 INFO:tasks.workunit.client.0.smithi063.stdout:7/857: chown d0/d15/d49/d4f/d66/d67/d86/ff0 35 1 2024-05-09T06:38:25.209 INFO:tasks.workunit.client.0.smithi063.stdout:5/851: mkdir d8/d17/d11d 0 2024-05-09T06:38:25.211 INFO:tasks.workunit.client.1.smithi193.stdout:3/440: mkdir d1/db/da1 0 2024-05-09T06:38:25.213 INFO:tasks.workunit.client.0.smithi063.stdout:2/897: truncate dc/dd/d45/d5a/fdd 599755 0 2024-05-09T06:38:25.213 INFO:tasks.workunit.client.1.smithi193.stdout:3/441: write d1/db/d59/f52 [5160647,30481] 0 2024-05-09T06:38:25.214 INFO:tasks.workunit.client.0.smithi063.stdout:3/929: write d7/f11 [909729,108167] 0 2024-05-09T06:38:25.214 INFO:tasks.workunit.client.0.smithi063.stdout:4/892: sync 2024-05-09T06:38:25.216 INFO:tasks.workunit.client.1.smithi193.stdout:5/442: dread d0/d13/d2c/d37/d2b/d47/d66/f15 [0,4194304] 0 2024-05-09T06:38:25.217 INFO:tasks.workunit.client.1.smithi193.stdout:5/443: readlink d0/d13/d2c/d37/d2b/d47/l49 0 2024-05-09T06:38:25.218 INFO:tasks.workunit.client.1.smithi193.stdout:9/571: unlink dd/df/d15/d98/fa1 0 2024-05-09T06:38:25.220 INFO:tasks.workunit.client.0.smithi063.stdout:2/898: dwrite dc/dd/d2d/d60/fac [0,4194304] 0 2024-05-09T06:38:25.220 INFO:tasks.workunit.client.1.smithi193.stdout:2/521: readlink db/d25/l8e 0 2024-05-09T06:38:25.221 INFO:tasks.workunit.client.0.smithi063.stdout:6/904: rmdir d2/d4/d7c 39 2024-05-09T06:38:25.223 INFO:tasks.workunit.client.1.smithi193.stdout:4/570: rmdir d1a/d24/d73/d94/d7c/d5b 39 2024-05-09T06:38:25.225 INFO:tasks.workunit.client.1.smithi193.stdout:4/571: dread d1a/d22/d43/f75 [0,4194304] 0 2024-05-09T06:38:25.228 INFO:tasks.workunit.client.0.smithi063.stdout:7/858: mknod d0/dcb/c127 0 2024-05-09T06:38:25.233 INFO:tasks.workunit.client.0.smithi063.stdout:5/852: dread d8/d17/f8d [4194304,4194304] 0 2024-05-09T06:38:25.234 INFO:tasks.workunit.client.0.smithi063.stdout:4/893: dread - d7/df/d77/fe4 zero size 2024-05-09T06:38:25.236 INFO:tasks.workunit.client.0.smithi063.stdout:5/853: dread d8/d65/d9e/fb8 [0,4194304] 0 2024-05-09T06:38:25.242 INFO:tasks.workunit.client.0.smithi063.stdout:4/894: dwrite d7/da3/d28/d31/d32/f51 [0,4194304] 0 2024-05-09T06:38:25.242 INFO:tasks.workunit.client.1.smithi193.stdout:5/444: dread d0/d13/d2c/d37/d2b/d47/d66/f83 [0,4194304] 0 2024-05-09T06:38:25.243 INFO:tasks.workunit.client.0.smithi063.stdout:9/869: dwrite d0/d4/d7d/fdc [0,4194304] 0 2024-05-09T06:38:25.244 INFO:tasks.workunit.client.1.smithi193.stdout:9/572: rename dd/d1e/d1f/d43 to dd/d1e/d66/d49/dbd 0 2024-05-09T06:38:25.245 INFO:tasks.workunit.client.1.smithi193.stdout:9/573: chown dd/d1e/d66/f80 4537655 1 2024-05-09T06:38:25.245 INFO:tasks.workunit.client.0.smithi063.stdout:2/899: fdatasync dc/dd/d45/d5a/d6c/d8c/dad/faf 0 2024-05-09T06:38:25.246 INFO:tasks.workunit.client.0.smithi063.stdout:2/900: chown dc/d36/d43 3 1 2024-05-09T06:38:25.249 INFO:tasks.workunit.client.0.smithi063.stdout:0/911: sync 2024-05-09T06:38:25.249 INFO:tasks.workunit.client.0.smithi063.stdout:6/905: fdatasync d2/d12/f76 0 2024-05-09T06:38:25.250 INFO:tasks.workunit.client.1.smithi193.stdout:2/522: fdatasync db/d25/d38/d61/f65 0 2024-05-09T06:38:25.251 INFO:tasks.workunit.client.0.smithi063.stdout:1/823: symlink d9/d46/l122 0 2024-05-09T06:38:25.252 INFO:tasks.workunit.client.0.smithi063.stdout:1/824: chown d9/d2a/d35/d1e/d9d/f82 1106248158 1 2024-05-09T06:38:25.255 INFO:tasks.workunit.client.0.smithi063.stdout:6/906: dwrite d2/d1f/d29/d3f/d5c/d64/fee [0,4194304] 0 2024-05-09T06:38:25.256 INFO:tasks.workunit.client.1.smithi193.stdout:4/572: creat d1a/d6b/d60/fb1 x:0 0 0 2024-05-09T06:38:25.256 INFO:tasks.workunit.client.1.smithi193.stdout:1/562: dwrite d4/d14/f1b [0,4194304] 0 2024-05-09T06:38:25.257 INFO:tasks.workunit.client.0.smithi063.stdout:3/930: creat d7/da/d2b/d66/d74/d7e/d114/f145 x:0 0 0 2024-05-09T06:38:25.258 INFO:tasks.workunit.client.1.smithi193.stdout:7/510: link d6/dd/ff d6/d8/d1a/d36/d4d/d60/d9c/faf 0 2024-05-09T06:38:25.260 INFO:tasks.workunit.client.1.smithi193.stdout:5/445: rmdir d0/d13/d2c/d37/d74 39 2024-05-09T06:38:25.262 INFO:tasks.workunit.client.0.smithi063.stdout:5/854: mkdir d8/d53/dca/d11e 0 2024-05-09T06:38:25.266 INFO:tasks.workunit.client.1.smithi193.stdout:5/446: chown d0/f29 44521 1 2024-05-09T06:38:25.267 INFO:tasks.workunit.client.0.smithi063.stdout:9/870: chown d0/d8/d2d/f108 28 1 2024-05-09T06:38:25.268 INFO:tasks.workunit.client.1.smithi193.stdout:4/573: chown d1a/d1f/f3e 0 1 2024-05-09T06:38:25.269 INFO:tasks.workunit.client.1.smithi193.stdout:3/442: link d1/d44/f5f d1/db/da1/fa2 0 2024-05-09T06:38:25.269 INFO:tasks.workunit.client.0.smithi063.stdout:5/855: dwrite d8/d17/d10b/f100 [0,4194304] 0 2024-05-09T06:38:25.270 INFO:tasks.workunit.client.0.smithi063.stdout:4/895: creat d7/da3/d28/d31/d32/d34/db4/dcc/de1/f122 x:0 0 0 2024-05-09T06:38:25.271 INFO:tasks.workunit.client.0.smithi063.stdout:0/912: dread dd/d1a/d2c/d51/d72/d9e/d23/d46/fdf [4194304,4194304] 0 2024-05-09T06:38:25.273 INFO:tasks.workunit.client.1.smithi193.stdout:2/523: dwrite db/d25/d38/d40/d8f/f93 [0,4194304] 0 2024-05-09T06:38:25.274 INFO:tasks.workunit.client.1.smithi193.stdout:3/443: write d1/d2d/d51/f7e [80073,50915] 0 2024-05-09T06:38:25.277 INFO:tasks.workunit.client.1.smithi193.stdout:7/511: unlink d6/d8/l98 0 2024-05-09T06:38:25.281 INFO:tasks.workunit.client.1.smithi193.stdout:1/563: dwrite d4/d16/d17/d1a/f9d [4194304,4194304] 0 2024-05-09T06:38:25.282 INFO:tasks.workunit.client.1.smithi193.stdout:7/512: chown d6/d8/d1a/f8f 292923401 1 2024-05-09T06:38:25.282 INFO:tasks.workunit.client.1.smithi193.stdout:2/524: dread - db/d37/d6d/d80/fa6 zero size 2024-05-09T06:38:25.292 INFO:tasks.workunit.client.1.smithi193.stdout:4/574: read d1a/d24/f41 [909337,114815] 0 2024-05-09T06:38:25.294 INFO:tasks.workunit.client.1.smithi193.stdout:3/444: creat d1/db/d2f/fa3 x:0 0 0 2024-05-09T06:38:25.297 INFO:tasks.workunit.client.1.smithi193.stdout:1/564: dwrite d4/d16/d60/d4f/fa5 [0,4194304] 0 2024-05-09T06:38:25.298 INFO:tasks.workunit.client.1.smithi193.stdout:1/565: stat d4/d16/de/c34 0 2024-05-09T06:38:25.299 INFO:tasks.workunit.client.0.smithi063.stdout:9/871: fdatasync d0/d3b/da0/dcf/ffc 0 2024-05-09T06:38:25.299 INFO:tasks.workunit.client.1.smithi193.stdout:5/447: dread d0/d13/d2c/d4d/d67/f76 [0,4194304] 0 2024-05-09T06:38:25.302 INFO:tasks.workunit.client.0.smithi063.stdout:5/856: symlink d8/d65/d9e/l11f 0 2024-05-09T06:38:25.303 INFO:tasks.workunit.client.1.smithi193.stdout:2/525: mkdir db/d37/d6d/d80/d60/d7d/db3 0 2024-05-09T06:38:25.304 INFO:tasks.workunit.client.0.smithi063.stdout:4/896: symlink d7/df/d35/d78/l123 0 2024-05-09T06:38:25.305 INFO:tasks.workunit.client.1.smithi193.stdout:2/526: truncate db/f21 5014125 0 2024-05-09T06:38:25.306 INFO:tasks.workunit.client.0.smithi063.stdout:9/872: dread d0/d4/d57/d60/d76/fe1 [0,4194304] 0 2024-05-09T06:38:25.308 INFO:tasks.workunit.client.1.smithi193.stdout:9/574: write dd/d1e/d1f/f2f [2817853,55180] 0 2024-05-09T06:38:25.309 INFO:tasks.workunit.client.1.smithi193.stdout:1/566: dread d4/f13 [4194304,4194304] 0 2024-05-09T06:38:25.311 INFO:tasks.workunit.client.0.smithi063.stdout:8/972: sync 2024-05-09T06:38:25.311 INFO:tasks.workunit.client.0.smithi063.stdout:3/931: dread d7/da/fe [0,4194304] 0 2024-05-09T06:38:25.311 INFO:tasks.workunit.client.1.smithi193.stdout:4/575: rename d1a/d1f/f3e to d1a/d22/d43/fb2 0 2024-05-09T06:38:25.312 INFO:tasks.workunit.client.0.smithi063.stdout:3/932: chown d7/da/d2b/la1 662229 1 2024-05-09T06:38:25.312 INFO:tasks.workunit.client.1.smithi193.stdout:3/445: dwrite d1/d2d/d51/f6a [0,4194304] 0 2024-05-09T06:38:25.318 INFO:tasks.workunit.client.1.smithi193.stdout:5/448: mknod d0/d13/d2c/d37/d2b/c8f 0 2024-05-09T06:38:25.322 INFO:tasks.workunit.client.0.smithi063.stdout:7/859: rename d0/d15/d49/d4f/d66/d67/c88 to d0/d15/d49/d54/d9f/d97/c128 0 2024-05-09T06:38:25.323 INFO:tasks.workunit.client.1.smithi193.stdout:2/527: stat db/df/c64 0 2024-05-09T06:38:25.324 INFO:tasks.workunit.client.1.smithi193.stdout:5/449: dread d0/d13/d2c/d37/d5b/d4e/f78 [0,4194304] 0 2024-05-09T06:38:25.327 INFO:tasks.workunit.client.1.smithi193.stdout:1/567: mkdir d4/d16/d17/d1a/d3a/d4a/db1 0 2024-05-09T06:38:25.329 INFO:tasks.workunit.client.1.smithi193.stdout:9/575: rename dd/d1e/d1f/d48/l87 to dd/d1e/d66/d49/lbe 0 2024-05-09T06:38:25.334 INFO:tasks.workunit.client.0.smithi063.stdout:2/901: dwrite dc/d36/ff4 [0,4194304] 0 2024-05-09T06:38:25.339 INFO:tasks.workunit.client.0.smithi063.stdout:8/973: dread d9/da/d45/f8d [0,4194304] 0 2024-05-09T06:38:25.342 INFO:tasks.workunit.client.0.smithi063.stdout:0/913: link dd/d60/d119/d48/d75/fb6 dd/d60/d119/d8f/f11a 0 2024-05-09T06:38:25.343 INFO:tasks.workunit.client.0.smithi063.stdout:0/914: stat dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/fbd 0 2024-05-09T06:38:25.343 INFO:tasks.workunit.client.1.smithi193.stdout:5/450: fdatasync d0/d13/d2c/f4a 0 2024-05-09T06:38:25.348 INFO:tasks.workunit.client.0.smithi063.stdout:5/857: write d8/d17/d75/dc9/dd5/fba [920234,69995] 0 2024-05-09T06:38:25.349 INFO:tasks.workunit.client.1.smithi193.stdout:9/576: creat dd/d1e/d3c/fbf x:0 0 0 2024-05-09T06:38:25.351 INFO:tasks.workunit.client.0.smithi063.stdout:1/825: rename d9/d2a/d35/d1e/f8e to d9/d2a/d3c/d4c/d83/d88/dc2/f123 0 2024-05-09T06:38:25.353 INFO:tasks.workunit.client.1.smithi193.stdout:4/576: unlink d1a/d24/d73/d94/d7c/d5b/c9e 0 2024-05-09T06:38:25.355 INFO:tasks.workunit.client.1.smithi193.stdout:3/446: dwrite d1/db/d2f/f66 [0,4194304] 0 2024-05-09T06:38:25.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:25 smithi063 ceph-mon[27171]: pgmap v15: 65 pgs: 65 active+clean; 2.5 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 69 MiB/s rd, 198 MiB/s wr, 399 op/s 2024-05-09T06:38:25.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:25 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:25.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:25 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:25.358 INFO:tasks.workunit.client.0.smithi063.stdout:9/873: dwrite d0/f24 [4194304,4194304] 0 2024-05-09T06:38:25.358 INFO:tasks.workunit.client.0.smithi063.stdout:1/826: dread d9/d2a/d3c/d4c/d83/d88/fcd [0,4194304] 0 2024-05-09T06:38:25.359 INFO:tasks.workunit.client.0.smithi063.stdout:1/827: chown d9/d46/dfe/d94 9863 1 2024-05-09T06:38:25.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/528: dwrite db/d25/d38/d40/f58 [0,4194304] 0 2024-05-09T06:38:25.360 INFO:tasks.workunit.client.0.smithi063.stdout:9/874: dread d0/d4/d26/d39/d91/fa7 [0,4194304] 0 2024-05-09T06:38:25.363 INFO:tasks.workunit.client.1.smithi193.stdout:5/451: rmdir d0/d13/d2c/d37/d2b/d47/d60/d75 39 2024-05-09T06:38:25.374 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:25 smithi193 ceph-mon[35141]: pgmap v15: 65 pgs: 65 active+clean; 2.5 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 69 MiB/s rd, 198 MiB/s wr, 399 op/s 2024-05-09T06:38:25.374 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:25 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:25.374 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:25 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:25.374 INFO:tasks.workunit.client.1.smithi193.stdout:2/529: dwrite db/d25/d38/d40/f58 [4194304,4194304] 0 2024-05-09T06:38:25.375 INFO:tasks.workunit.client.1.smithi193.stdout:9/577: symlink dd/df/d15/d8c/lc0 0 2024-05-09T06:38:25.377 INFO:tasks.workunit.client.0.smithi063.stdout:0/915: mkdir dd/d60/dcd/d11b 0 2024-05-09T06:38:25.383 INFO:tasks.workunit.client.0.smithi063.stdout:6/907: getdents d2/d1f/d29 0 2024-05-09T06:38:25.387 INFO:tasks.workunit.client.0.smithi063.stdout:7/860: dread d0/d15/d49/d4f/d66/ddd/fdf [0,4194304] 0 2024-05-09T06:38:25.395 INFO:tasks.workunit.client.0.smithi063.stdout:9/875: dread d0/d4/d57/d60/d76/fa9 [0,4194304] 0 2024-05-09T06:38:25.396 INFO:tasks.workunit.client.0.smithi063.stdout:1/828: fsync d9/d2a/d3c/d4c/d83/d88/dc2/fe0 0 2024-05-09T06:38:25.401 INFO:tasks.workunit.client.0.smithi063.stdout:8/974: write d9/da/d18/d24/fdb [4262380,127491] 0 2024-05-09T06:38:25.403 INFO:tasks.workunit.client.0.smithi063.stdout:4/897: dwrite d7/df/d77/f8e [0,4194304] 0 2024-05-09T06:38:25.403 INFO:tasks.workunit.client.1.smithi193.stdout:1/568: dwrite d4/d14/f75 [0,4194304] 0 2024-05-09T06:38:25.404 INFO:tasks.workunit.client.1.smithi193.stdout:1/569: chown d4/d16/d17/c4c 87593932 1 2024-05-09T06:38:25.409 INFO:tasks.workunit.client.1.smithi193.stdout:9/578: dread dd/df/d31/d7e/f60 [0,4194304] 0 2024-05-09T06:38:25.409 INFO:tasks.workunit.client.0.smithi063.stdout:2/902: rename dc/d32/c6f to dc/d36/c10d 0 2024-05-09T06:38:25.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/903: fsync dc/dd/d2d/d60/f67 0 2024-05-09T06:38:25.412 INFO:tasks.workunit.client.1.smithi193.stdout:1/570: dread d4/d16/d17/d1a/f1e [0,4194304] 0 2024-05-09T06:38:25.414 INFO:tasks.workunit.client.0.smithi063.stdout:5/858: truncate d8/d19/d30/d46/f6c 3064528 0 2024-05-09T06:38:25.416 INFO:tasks.workunit.client.0.smithi063.stdout:5/859: dread - d8/d53/dca/f76 zero size 2024-05-09T06:38:25.418 INFO:tasks.workunit.client.0.smithi063.stdout:9/876: read d0/d4/d26/fae [981544,128702] 0 2024-05-09T06:38:25.419 INFO:tasks.workunit.client.1.smithi193.stdout:9/579: dwrite dd/df/d15/d98/fb3 [0,4194304] 0 2024-05-09T06:38:25.422 INFO:tasks.workunit.client.1.smithi193.stdout:9/580: stat dd/d1e/d1f/d48/d55/d86 0 2024-05-09T06:38:25.424 INFO:tasks.workunit.client.0.smithi063.stdout:9/877: dwrite d0/d4/d57/dba/dd5/fdb [0,4194304] 0 2024-05-09T06:38:25.427 INFO:tasks.workunit.client.0.smithi063.stdout:3/933: sync 2024-05-09T06:38:25.428 INFO:tasks.workunit.client.0.smithi063.stdout:7/861: write d0/d15/f20 [1586498,22785] 0 2024-05-09T06:38:25.429 INFO:tasks.workunit.client.0.smithi063.stdout:3/934: chown d7/d22/dfc/f135 13391 1 2024-05-09T06:38:25.430 INFO:tasks.workunit.client.0.smithi063.stdout:0/916: rename dd/d1a/d2c/d42/fd8 to dd/d60/d119/d8f/dfd/f11c 0 2024-05-09T06:38:25.431 INFO:tasks.workunit.client.0.smithi063.stdout:6/908: dwrite d2/d1f/d29/d3f/d5c/d64/f6f [0,4194304] 0 2024-05-09T06:38:25.438 INFO:tasks.workunit.client.0.smithi063.stdout:2/904: unlink dc/d36/f57 0 2024-05-09T06:38:25.441 INFO:tasks.workunit.client.0.smithi063.stdout:5/860: mkdir d8/d17/d10b/dda/d120 0 2024-05-09T06:38:25.443 INFO:tasks.workunit.client.0.smithi063.stdout:0/917: dread dd/d1a/d2c/d51/d72/d9e/d23/d31/fb9 [0,4194304] 0 2024-05-09T06:38:25.443 INFO:tasks.workunit.client.1.smithi193.stdout:5/452: unlink d0/d13/d2c/l73 0 2024-05-09T06:38:25.444 INFO:tasks.workunit.client.0.smithi063.stdout:2/905: dwrite dc/f27 [4194304,4194304] 0 2024-05-09T06:38:25.446 INFO:tasks.workunit.client.0.smithi063.stdout:0/918: write dd/d1a/d2c/d51/d76/d97/fe7 [1538230,88047] 0 2024-05-09T06:38:25.451 INFO:tasks.workunit.client.0.smithi063.stdout:9/878: unlink d0/d3b/d63/l128 0 2024-05-09T06:38:25.451 INFO:tasks.workunit.client.0.smithi063.stdout:0/919: truncate dd/d1a/d2c/d51/d72/fed 417231 0 2024-05-09T06:38:25.452 INFO:tasks.workunit.client.1.smithi193.stdout:3/447: mkdir d1/db/da4 0 2024-05-09T06:38:25.453 INFO:tasks.workunit.client.0.smithi063.stdout:0/920: chown dd/l27 1 1 2024-05-09T06:38:25.453 INFO:tasks.workunit.client.0.smithi063.stdout:4/898: dwrite d7/da3/d28/f79 [4194304,4194304] 0 2024-05-09T06:38:25.454 INFO:tasks.workunit.client.1.smithi193.stdout:8/514: sync 2024-05-09T06:38:25.455 INFO:tasks.workunit.client.1.smithi193.stdout:6/467: sync 2024-05-09T06:38:25.459 INFO:tasks.workunit.client.1.smithi193.stdout:6/468: write d1/d25/d2c/f91 [949146,76639] 0 2024-05-09T06:38:25.461 INFO:tasks.workunit.client.1.smithi193.stdout:3/448: dwrite d1/db/f73 [0,4194304] 0 2024-05-09T06:38:25.462 INFO:tasks.workunit.client.0.smithi063.stdout:0/921: dwrite dd/d1a/f35 [0,4194304] 0 2024-05-09T06:38:25.463 INFO:tasks.workunit.client.1.smithi193.stdout:3/449: chown d1/db/l1e 26010107 1 2024-05-09T06:38:25.466 INFO:tasks.workunit.client.1.smithi193.stdout:3/450: truncate d1/db/d1b/f7a 8427182 0 2024-05-09T06:38:25.468 INFO:tasks.workunit.client.0.smithi063.stdout:6/909: creat d2/df/d59/d99/d9f/f126 x:0 0 0 2024-05-09T06:38:25.469 INFO:tasks.workunit.client.1.smithi193.stdout:9/581: fsync dd/df/d31/d7e/f35 0 2024-05-09T06:38:25.473 INFO:tasks.workunit.client.1.smithi193.stdout:5/453: mkdir d0/d13/d2c/d37/d2b/d6a/d90 0 2024-05-09T06:38:25.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/530: mkdir db/d37/d6d/d80/daa/db4 0 2024-05-09T06:38:25.477 INFO:tasks.workunit.client.1.smithi193.stdout:2/531: stat db/f44 0 2024-05-09T06:38:25.478 INFO:tasks.workunit.client.1.smithi193.stdout:4/577: write d1a/d22/f56 [562935,59881] 0 2024-05-09T06:38:25.478 INFO:tasks.workunit.client.0.smithi063.stdout:5/861: truncate d8/d19/f1a 1132191 0 2024-05-09T06:38:25.487 INFO:tasks.workunit.client.0.smithi063.stdout:7/862: dwrite d0/d15/f48 [0,4194304] 0 2024-05-09T06:38:25.494 INFO:tasks.workunit.client.1.smithi193.stdout:3/451: creat d1/d2d/d51/fa5 x:0 0 0 2024-05-09T06:38:25.499 INFO:tasks.workunit.client.1.smithi193.stdout:4/578: dread fa [0,4194304] 0 2024-05-09T06:38:25.500 INFO:tasks.workunit.client.0.smithi063.stdout:9/879: truncate d0/d4/d26/d39/d48/d80/fc3 960385 0 2024-05-09T06:38:25.500 INFO:tasks.workunit.client.0.smithi063.stdout:9/880: stat d0/d4/d26/d39/d48/l85 0 2024-05-09T06:38:25.504 INFO:tasks.workunit.client.0.smithi063.stdout:8/975: rename d9/d41/dec/da1/cbf to d9/da/d51/dc2/d127/c155 0 2024-05-09T06:38:25.504 INFO:tasks.workunit.client.1.smithi193.stdout:1/571: write d4/d16/d17/f29 [6909580,32485] 0 2024-05-09T06:38:25.506 INFO:tasks.workunit.client.0.smithi063.stdout:0/922: unlink dd/d60/d119/c56 0 2024-05-09T06:38:25.507 INFO:tasks.workunit.client.1.smithi193.stdout:6/469: mknod d1/de/d11/d2d/da4/ca5 0 2024-05-09T06:38:25.510 INFO:tasks.workunit.client.1.smithi193.stdout:3/452: rename d1/db/d59/d63/l77 to d1/d2d/d51/la6 0 2024-05-09T06:38:25.511 INFO:tasks.workunit.client.1.smithi193.stdout:2/532: write db/d4b/f86 [89164,53] 0 2024-05-09T06:38:25.512 INFO:tasks.workunit.client.1.smithi193.stdout:9/582: dread dd/df/d15/d8c/f2a [0,4194304] 0 2024-05-09T06:38:25.513 INFO:tasks.workunit.client.0.smithi063.stdout:6/910: symlink d2/d1f/d29/de2/l127 0 2024-05-09T06:38:25.514 INFO:tasks.workunit.client.0.smithi063.stdout:5/862: mkdir d8/d17/d75/dc9/dd5/d96/d121 0 2024-05-09T06:38:25.517 INFO:tasks.workunit.client.1.smithi193.stdout:1/572: mknod d4/d16/d6b/d89/cb2 0 2024-05-09T06:38:25.518 INFO:tasks.workunit.client.0.smithi063.stdout:1/829: getdents d9/d2a/d3c/d4c/dc3 0 2024-05-09T06:38:25.518 INFO:tasks.workunit.client.0.smithi063.stdout:1/830: dread - d9/d2a/d3c/d4a/fe5 zero size 2024-05-09T06:38:25.520 INFO:tasks.workunit.client.1.smithi193.stdout:4/579: rename d1a/d6b/d60/f91 to d1a/d24/d73/d94/d7c/d5b/fb3 0 2024-05-09T06:38:25.521 INFO:tasks.workunit.client.0.smithi063.stdout:7/863: mknod d0/dcb/c129 0 2024-05-09T06:38:25.522 INFO:tasks.workunit.client.1.smithi193.stdout:2/533: creat db/df/d87/fb5 x:0 0 0 2024-05-09T06:38:25.525 INFO:tasks.workunit.client.1.smithi193.stdout:3/453: symlink d1/db/d2f/d78/d86/la7 0 2024-05-09T06:38:25.525 INFO:tasks.workunit.client.1.smithi193.stdout:9/583: dread dd/df/d31/d7e/f62 [0,4194304] 0 2024-05-09T06:38:25.526 INFO:tasks.workunit.client.0.smithi063.stdout:3/935: rmdir d7/d143 0 2024-05-09T06:38:25.526 INFO:tasks.workunit.client.1.smithi193.stdout:8/515: link d6/d5b/c68 d6/d1c/d27/c9e 0 2024-05-09T06:38:25.529 INFO:tasks.workunit.client.0.smithi063.stdout:9/881: creat d0/d4/d7d/d105/f12c x:0 0 0 2024-05-09T06:38:25.530 INFO:tasks.workunit.client.1.smithi193.stdout:1/573: rmdir d4/d16/d17/d1a/d3a/d4a 39 2024-05-09T06:38:25.538 INFO:tasks.workunit.client.0.smithi063.stdout:9/882: dwrite d0/d3b/da0/dcf/f125 [0,4194304] 0 2024-05-09T06:38:25.538 INFO:tasks.workunit.client.0.smithi063.stdout:0/923: creat dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/f11d x:0 0 0 2024-05-09T06:38:25.538 INFO:tasks.workunit.client.0.smithi063.stdout:9/883: chown d0/d4/d26/d47/cb6 1735 1 2024-05-09T06:38:25.539 INFO:tasks.workunit.client.1.smithi193.stdout:4/580: dread - d1a/d80/fa9 zero size 2024-05-09T06:38:25.539 INFO:tasks.workunit.client.1.smithi193.stdout:6/470: rename d1/d25/d2c/f91 to d1/de/fa6 0 2024-05-09T06:38:25.539 INFO:tasks.workunit.client.1.smithi193.stdout:6/471: read - d1/d25/d2c/d3c/f93 zero size 2024-05-09T06:38:25.545 INFO:tasks.workunit.client.0.smithi063.stdout:6/911: rename d2/d1f/d29/d3f/l16 to d2/dbf/l128 0 2024-05-09T06:38:25.547 INFO:tasks.workunit.client.1.smithi193.stdout:3/454: fsync d1/db/f18 0 2024-05-09T06:38:25.548 INFO:tasks.workunit.client.0.smithi063.stdout:2/906: symlink dc/d36/d43/d46/d49/dbb/dec/d104/l10e 0 2024-05-09T06:38:25.548 INFO:tasks.workunit.client.0.smithi063.stdout:4/899: write d7/df/d35/d5a/d60/f63 [1951154,31241] 0 2024-05-09T06:38:25.550 INFO:tasks.workunit.client.1.smithi193.stdout:9/584: mkdir dd/d1e/d66/d49/dc1 0 2024-05-09T06:38:25.551 INFO:tasks.workunit.client.0.smithi063.stdout:8/976: dread d9/da/d18/f63 [0,4194304] 0 2024-05-09T06:38:25.555 INFO:tasks.workunit.client.0.smithi063.stdout:8/977: dread d9/da/d51/dc2/fd9 [0,4194304] 0 2024-05-09T06:38:25.563 INFO:tasks.workunit.client.1.smithi193.stdout:2/534: dwrite db/d37/d6d/d80/f24 [0,4194304] 0 2024-05-09T06:38:25.566 INFO:tasks.workunit.client.1.smithi193.stdout:8/516: write d6/d1c/d1b/f52 [675509,82188] 0 2024-05-09T06:38:25.566 INFO:tasks.workunit.client.1.smithi193.stdout:2/535: read db/df/f10 [1661764,120655] 0 2024-05-09T06:38:25.568 INFO:tasks.workunit.client.1.smithi193.stdout:8/517: stat d6/d1c/d53/f71 0 2024-05-09T06:38:25.568 INFO:tasks.workunit.client.1.smithi193.stdout:2/536: write db/d4b/f5d [4206844,60869] 0 2024-05-09T06:38:25.569 INFO:tasks.workunit.client.1.smithi193.stdout:8/518: chown d6/d1c/d18/c5a 5 1 2024-05-09T06:38:25.571 INFO:tasks.workunit.client.0.smithi063.stdout:3/936: dwrite d7/da/d21/f6b [0,4194304] 0 2024-05-09T06:38:25.572 INFO:tasks.workunit.client.1.smithi193.stdout:3/455: creat d1/db/d2f/d78/d86/fa8 x:0 0 0 2024-05-09T06:38:25.575 INFO:tasks.workunit.client.0.smithi063.stdout:4/900: mkdir d7/df/d35/d5a/d60/d124 0 2024-05-09T06:38:25.576 INFO:tasks.workunit.client.0.smithi063.stdout:4/901: chown d7/df/d35/dd9/ffc 87 1 2024-05-09T06:38:25.579 INFO:tasks.workunit.client.1.smithi193.stdout:9/585: rename dd/d1e/d66 to dd/d1e/dc2 0 2024-05-09T06:38:25.579 INFO:tasks.workunit.client.0.smithi063.stdout:2/907: creat dc/dd/d2d/d60/dd0/d90/f10f x:0 0 0 2024-05-09T06:38:25.580 INFO:tasks.workunit.client.0.smithi063.stdout:2/908: fsync dc/dd/d2d/d60/dd0/d90/dc1/fe1 0 2024-05-09T06:38:25.580 INFO:tasks.workunit.client.0.smithi063.stdout:1/831: symlink d9/d2a/d35/d1e/d9d/l124 0 2024-05-09T06:38:25.582 INFO:tasks.workunit.client.0.smithi063.stdout:2/909: chown dc/dd/d45/d5a/d6c/d8f/dbd 2838985 1 2024-05-09T06:38:25.582 INFO:tasks.workunit.client.1.smithi193.stdout:1/574: chown d4/d16/d17/d1a/d3a/d4a/c5a 54689999 1 2024-05-09T06:38:25.584 INFO:tasks.workunit.client.0.smithi063.stdout:7/864: fdatasync d0/d15/d49/d54/fa5 0 2024-05-09T06:38:25.585 INFO:tasks.workunit.client.1.smithi193.stdout:6/472: symlink d1/d25/d4c/d9e/la7 0 2024-05-09T06:38:25.588 INFO:tasks.workunit.client.1.smithi193.stdout:6/473: dread d1/f86 [0,4194304] 0 2024-05-09T06:38:25.589 INFO:tasks.workunit.client.0.smithi063.stdout:7/865: dwrite d0/f10 [4194304,4194304] 0 2024-05-09T06:38:25.617 INFO:tasks.workunit.client.0.smithi063.stdout:5/863: sync 2024-05-09T06:38:25.617 INFO:tasks.workunit.client.0.smithi063.stdout:5/864: readlink d8/d19/d30/d74/lea 0 2024-05-09T06:38:25.631 INFO:tasks.workunit.client.0.smithi063.stdout:5/865: dread d8/d17/d75/fcf [0,4194304] 0 2024-05-09T06:38:25.676 INFO:tasks.workunit.client.0.smithi063.stdout:8/978: read - d9/da/d18/fc4 zero size 2024-05-09T06:38:25.679 INFO:tasks.workunit.client.1.smithi193.stdout:4/581: truncate d1a/d22/d43/f8b 183953 0 2024-05-09T06:38:25.681 INFO:tasks.workunit.client.0.smithi063.stdout:3/937: symlink d7/d22/d28/dab/l146 0 2024-05-09T06:38:25.682 INFO:tasks.workunit.client.1.smithi193.stdout:8/519: rename d6/d1c/d1b/f52 to d6/d1c/d18/d66/d93/d95/f9f 0 2024-05-09T06:38:25.684 INFO:tasks.workunit.client.1.smithi193.stdout:3/456: creat d1/db/d59/fa9 x:0 0 0 2024-05-09T06:38:25.685 INFO:tasks.workunit.client.0.smithi063.stdout:3/938: dread f1 [0,4194304] 0 2024-05-09T06:38:25.686 INFO:tasks.workunit.client.0.smithi063.stdout:4/902: unlink d7/da3/d28/c40 0 2024-05-09T06:38:25.688 INFO:tasks.workunit.client.1.smithi193.stdout:1/575: mkdir d4/d16/d6b/d89/db3 0 2024-05-09T06:38:25.689 INFO:tasks.workunit.client.1.smithi193.stdout:1/576: read d4/d16/d6b/d89/f99 [813755,79109] 0 2024-05-09T06:38:25.691 INFO:tasks.workunit.client.1.smithi193.stdout:2/537: mkdir db/df/db6 0 2024-05-09T06:38:25.692 INFO:tasks.workunit.client.1.smithi193.stdout:4/582: read d1a/d22/f83 [342574,34509] 0 2024-05-09T06:38:25.693 INFO:tasks.workunit.client.1.smithi193.stdout:4/583: dread - d1a/d80/fa9 zero size 2024-05-09T06:38:25.695 INFO:tasks.workunit.client.0.smithi063.stdout:7/866: dread - d0/d15/d49/d54/d9f/fd9 zero size 2024-05-09T06:38:25.696 INFO:tasks.workunit.client.0.smithi063.stdout:7/867: read d0/d15/f1b [7450,36703] 0 2024-05-09T06:38:25.697 INFO:tasks.workunit.client.1.smithi193.stdout:6/474: rename d1/f9 to d1/de/d11/d2d/fa8 0 2024-05-09T06:38:25.698 INFO:tasks.workunit.client.1.smithi193.stdout:6/475: read - d1/de/d11/d18/f6d zero size 2024-05-09T06:38:25.699 INFO:tasks.workunit.client.0.smithi063.stdout:2/910: dread dc/f9d [0,4194304] 0 2024-05-09T06:38:25.699 INFO:tasks.workunit.client.1.smithi193.stdout:6/476: chown d1/d25/d2c/d3c/d3e/l8e 24668205 1 2024-05-09T06:38:25.713 INFO:tasks.workunit.client.1.smithi193.stdout:2/538: mknod db/d25/d38/d40/d8f/cb7 0 2024-05-09T06:38:25.715 INFO:tasks.workunit.client.1.smithi193.stdout:4/584: creat d1a/d24/d73/d94/fb4 x:0 0 0 2024-05-09T06:38:25.716 INFO:tasks.workunit.client.1.smithi193.stdout:4/585: stat d1a/d80/fa9 0 2024-05-09T06:38:25.716 INFO:tasks.workunit.client.1.smithi193.stdout:4/586: read - d1a/d80/fa9 zero size 2024-05-09T06:38:25.717 INFO:tasks.workunit.client.1.smithi193.stdout:8/520: fdatasync d6/d1c/f3d 0 2024-05-09T06:38:25.720 INFO:tasks.workunit.client.1.smithi193.stdout:6/477: symlink d1/d25/d2c/d81/d37/la9 0 2024-05-09T06:38:25.723 INFO:tasks.workunit.client.1.smithi193.stdout:6/478: dread d1/d25/d2c/d76/f7d [0,4194304] 0 2024-05-09T06:38:25.727 INFO:tasks.workunit.client.1.smithi193.stdout:2/539: creat db/df/d54/fb8 x:0 0 0 2024-05-09T06:38:25.728 INFO:tasks.workunit.client.1.smithi193.stdout:1/577: dwrite d4/d16/d17/d1a/d3a/d3f/f65 [0,4194304] 0 2024-05-09T06:38:25.732 INFO:tasks.workunit.client.1.smithi193.stdout:4/587: creat d1a/d24/d73/fb5 x:0 0 0 2024-05-09T06:38:25.738 INFO:tasks.workunit.client.1.smithi193.stdout:6/479: dread - d1/d25/d2c/d76/f57 zero size 2024-05-09T06:38:25.742 INFO:tasks.workunit.client.1.smithi193.stdout:4/588: symlink d1a/d1f/lb6 0 2024-05-09T06:38:25.743 INFO:tasks.workunit.client.1.smithi193.stdout:4/589: write d1a/f1b [2215037,79282] 0 2024-05-09T06:38:25.749 INFO:tasks.workunit.client.1.smithi193.stdout:3/457: getdents d1/db/d59 0 2024-05-09T06:38:25.749 INFO:tasks.workunit.client.1.smithi193.stdout:1/578: dread d4/d16/d17/d1a/f52 [0,4194304] 0 2024-05-09T06:38:25.751 INFO:tasks.workunit.client.1.smithi193.stdout:6/480: dread d1/de/d11/f5d [0,4194304] 0 2024-05-09T06:38:25.756 INFO:tasks.workunit.client.1.smithi193.stdout:8/521: dread d6/d1c/d53/f60 [0,4194304] 0 2024-05-09T06:38:25.757 INFO:tasks.workunit.client.1.smithi193.stdout:8/522: chown d6/d1c/d18/d2f/d40 283346 1 2024-05-09T06:38:25.759 INFO:tasks.workunit.client.1.smithi193.stdout:3/458: dread - d1/db/da1/fa2 zero size 2024-05-09T06:38:25.759 INFO:tasks.workunit.client.1.smithi193.stdout:8/523: read d6/d1c/d53/f63 [1172894,59331] 0 2024-05-09T06:38:25.761 INFO:tasks.workunit.client.1.smithi193.stdout:1/579: unlink d4/d3c/c53 0 2024-05-09T06:38:25.762 INFO:tasks.workunit.client.1.smithi193.stdout:1/580: readlink d4/d16/d60/d4f/l9f 0 2024-05-09T06:38:25.762 INFO:tasks.workunit.client.1.smithi193.stdout:1/581: read d4/f13 [5966196,73717] 0 2024-05-09T06:38:25.764 INFO:tasks.workunit.client.1.smithi193.stdout:4/590: creat d1a/d22/d8e/fb7 x:0 0 0 2024-05-09T06:38:25.769 INFO:tasks.workunit.client.1.smithi193.stdout:3/459: mknod d1/db/d59/d63/caa 0 2024-05-09T06:38:25.771 INFO:tasks.workunit.client.1.smithi193.stdout:8/524: fsync d6/d1c/d18/d66/d93/d95/f9f 0 2024-05-09T06:38:25.772 INFO:tasks.workunit.client.1.smithi193.stdout:6/481: symlink d1/d25/d2c/d81/laa 0 2024-05-09T06:38:25.776 INFO:tasks.workunit.client.1.smithi193.stdout:1/582: rename d4/d16/d6b/d85/f94 to d4/d16/d17/d1a/d3a/d4a/db1/fb4 0 2024-05-09T06:38:25.776 INFO:tasks.workunit.client.1.smithi193.stdout:2/540: truncate db/f21 4961579 0 2024-05-09T06:38:25.778 INFO:tasks.workunit.client.1.smithi193.stdout:8/525: dwrite d6/d5b/f64 [0,4194304] 0 2024-05-09T06:38:25.782 INFO:tasks.workunit.client.1.smithi193.stdout:3/460: creat d1/db/d2f/fab x:0 0 0 2024-05-09T06:38:25.785 INFO:tasks.workunit.client.1.smithi193.stdout:2/541: dwrite db/d37/d6d/d80/fa6 [0,4194304] 0 2024-05-09T06:38:25.792 INFO:tasks.workunit.client.1.smithi193.stdout:0/488: sync 2024-05-09T06:38:25.793 INFO:tasks.workunit.client.1.smithi193.stdout:2/542: dread db/d25/d38/d40/d8f/f93 [0,4194304] 0 2024-05-09T06:38:25.793 INFO:tasks.workunit.client.1.smithi193.stdout:6/482: creat d1/d25/d2c/d81/d37/d4f/fab x:0 0 0 2024-05-09T06:38:25.795 INFO:tasks.workunit.client.1.smithi193.stdout:4/591: rename d1a/d22/f83 to d1a/d98/fb8 0 2024-05-09T06:38:25.799 INFO:tasks.workunit.client.1.smithi193.stdout:1/583: symlink d4/d16/d6b/d6e/lb5 0 2024-05-09T06:38:25.800 INFO:tasks.workunit.client.1.smithi193.stdout:1/584: chown d4/d76 0 1 2024-05-09T06:38:25.801 INFO:tasks.workunit.client.1.smithi193.stdout:1/585: write d4/d14/f2e [1116142,50212] 0 2024-05-09T06:38:25.801 INFO:tasks.workunit.client.1.smithi193.stdout:8/526: fdatasync d6/d1c/d18/f74 0 2024-05-09T06:38:25.803 INFO:tasks.workunit.client.1.smithi193.stdout:8/527: fsync d6/d1c/d18/d2f/d40/f2c 0 2024-05-09T06:38:25.807 INFO:tasks.workunit.client.0.smithi063.stdout:5/866: rmdir d8/d65/d9e 39 2024-05-09T06:38:25.807 INFO:tasks.workunit.client.0.smithi063.stdout:8/979: rmdir d9/da/d18/d24/d71/d11d 39 2024-05-09T06:38:25.807 INFO:tasks.workunit.client.0.smithi063.stdout:8/980: fsync d9/f154 0 2024-05-09T06:38:25.811 INFO:tasks.workunit.client.0.smithi063.stdout:3/939: creat d7/da/d21/d30/d44/dee/d10c/f147 x:0 0 0 2024-05-09T06:38:25.811 INFO:tasks.workunit.client.1.smithi193.stdout:7/513: sync 2024-05-09T06:38:25.811 INFO:tasks.workunit.client.1.smithi193.stdout:5/454: sync 2024-05-09T06:38:25.815 INFO:tasks.workunit.client.0.smithi063.stdout:4/903: mknod d7/d3d/c125 0 2024-05-09T06:38:25.816 INFO:tasks.workunit.client.1.smithi193.stdout:0/489: dread d18/d4c/f4b [0,4194304] 0 2024-05-09T06:38:25.817 INFO:tasks.workunit.client.1.smithi193.stdout:0/490: readlink d18/d27/l3a 0 2024-05-09T06:38:25.817 INFO:tasks.workunit.client.0.smithi063.stdout:1/832: creat d9/d2a/d3c/dbc/d114/f125 x:0 0 0 2024-05-09T06:38:25.820 INFO:tasks.workunit.client.1.smithi193.stdout:0/491: dread d18/f57 [0,4194304] 0 2024-05-09T06:38:25.822 INFO:tasks.workunit.client.0.smithi063.stdout:7/868: read - d0/d15/d89/d10c/de3/ff5 zero size 2024-05-09T06:38:25.827 INFO:tasks.workunit.client.1.smithi193.stdout:1/586: fsync d4/d16/d17/d1a/f52 0 2024-05-09T06:38:25.828 INFO:tasks.workunit.client.1.smithi193.stdout:6/483: dread d1/de/d11/f5c [0,4194304] 0 2024-05-09T06:38:25.828 INFO:tasks.workunit.client.1.smithi193.stdout:1/587: chown d4/d16/d6b/d6e/l81 1 1 2024-05-09T06:38:25.832 INFO:tasks.workunit.client.0.smithi063.stdout:8/981: symlink d9/da/d51/dc2/d127/l156 0 2024-05-09T06:38:25.834 INFO:tasks.workunit.client.0.smithi063.stdout:9/884: rename d0/l12 to d0/d4/d26/d39/d3e/l12d 0 2024-05-09T06:38:25.835 INFO:tasks.workunit.client.1.smithi193.stdout:2/543: truncate db/f39 3478516 0 2024-05-09T06:38:25.837 INFO:tasks.workunit.client.0.smithi063.stdout:3/940: unlink d7/d70/d98/dbb/c123 0 2024-05-09T06:38:25.839 INFO:tasks.workunit.client.1.smithi193.stdout:7/514: truncate d6/dd/d5e/d6b/f88 309997 0 2024-05-09T06:38:25.843 INFO:tasks.workunit.client.1.smithi193.stdout:6/484: dread d1/de/f13 [0,4194304] 0 2024-05-09T06:38:25.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/911: unlink dc/f99 0 2024-05-09T06:38:25.845 INFO:tasks.workunit.client.0.smithi063.stdout:5/867: mkdir d8/d17/d11d/d122 0 2024-05-09T06:38:25.847 INFO:tasks.workunit.client.0.smithi063.stdout:8/982: creat d9/d41/dec/db8/f157 x:0 0 0 2024-05-09T06:38:25.852 INFO:tasks.workunit.client.0.smithi063.stdout:6/912: rename d2/df/d91/dd8/dd9/ff7 to d2/df/d91/dd8/dd9/d10e/f129 0 2024-05-09T06:38:25.855 INFO:tasks.workunit.client.1.smithi193.stdout:7/515: rmdir d6/d8/d1a/d33 39 2024-05-09T06:38:25.857 INFO:tasks.workunit.client.0.smithi063.stdout:6/913: dread d2/df/d59/d99/ff8 [0,4194304] 0 2024-05-09T06:38:25.858 INFO:tasks.workunit.client.1.smithi193.stdout:5/455: creat d0/d13/d2c/d37/d74/f91 x:0 0 0 2024-05-09T06:38:25.860 INFO:tasks.workunit.client.1.smithi193.stdout:3/461: rename d1/db/c76 to d1/db/cac 0 2024-05-09T06:38:25.860 INFO:tasks.workunit.client.1.smithi193.stdout:3/462: readlink d1/d2d/d51/l62 0 2024-05-09T06:38:25.862 INFO:tasks.workunit.client.0.smithi063.stdout:2/912: symlink dc/d36/d43/l110 0 2024-05-09T06:38:25.864 INFO:tasks.workunit.client.1.smithi193.stdout:6/485: mkdir d1/de/d11/d18/d94/dac 0 2024-05-09T06:38:25.865 INFO:tasks.workunit.client.1.smithi193.stdout:6/486: chown d1/d25/d2c/d3c/d3e/d58/c8a 534175 1 2024-05-09T06:38:25.866 INFO:tasks.workunit.client.1.smithi193.stdout:6/487: rename d1 to d1/d25/d2c/d3c/dad 22 2024-05-09T06:38:25.868 INFO:tasks.workunit.client.0.smithi063.stdout:7/869: rename d0/d15/d89/d10c/de3 to d0/d15/d89/d10c/de7/d12a 0 2024-05-09T06:38:25.869 INFO:tasks.workunit.client.0.smithi063.stdout:7/870: read d0/d15/d49/d54/d9f/d97/fde [34442,67114] 0 2024-05-09T06:38:25.870 INFO:tasks.workunit.client.1.smithi193.stdout:1/588: link d4/d16/d6b/d85/f86 d4/d16/d60/d4f/d72/d7f/fb6 0 2024-05-09T06:38:25.870 INFO:tasks.workunit.client.1.smithi193.stdout:8/528: write d6/d1c/d27/f43 [2497816,13791] 0 2024-05-09T06:38:25.871 INFO:tasks.workunit.client.1.smithi193.stdout:8/529: read - d6/d1c/d18/d66/d93/f8c zero size 2024-05-09T06:38:25.872 INFO:tasks.workunit.client.1.smithi193.stdout:2/544: symlink db/d37/d6d/d80/d60/d7d/db3/lb9 0 2024-05-09T06:38:25.873 INFO:tasks.workunit.client.1.smithi193.stdout:8/530: fsync d6/d1c/d1b/d5d/f6d 0 2024-05-09T06:38:25.873 INFO:tasks.workunit.client.0.smithi063.stdout:7/871: dread d0/d15/d49/d4f/d66/f107 [0,4194304] 0 2024-05-09T06:38:25.874 INFO:tasks.workunit.client.1.smithi193.stdout:7/516: creat d6/d8/d1a/d36/d4d/fb0 x:0 0 0 2024-05-09T06:38:25.878 INFO:tasks.workunit.client.0.smithi063.stdout:0/924: truncate dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/f10f 425575 0 2024-05-09T06:38:25.879 INFO:tasks.workunit.client.1.smithi193.stdout:2/545: dwrite db/d4b/f90 [4194304,4194304] 0 2024-05-09T06:38:25.881 INFO:tasks.workunit.client.1.smithi193.stdout:2/546: readlink db/d37/d6d/d80/d60/d7d/db3/lb9 0 2024-05-09T06:38:25.881 INFO:tasks.workunit.client.1.smithi193.stdout:3/463: truncate d1/d44/f60 13600 0 2024-05-09T06:38:25.883 INFO:tasks.workunit.client.0.smithi063.stdout:4/904: sync 2024-05-09T06:38:25.883 INFO:tasks.workunit.client.1.smithi193.stdout:3/464: truncate d1/d2d/d51/f6a 5218859 0 2024-05-09T06:38:25.883 INFO:tasks.workunit.client.0.smithi063.stdout:1/833: creat d9/d46/dfe/f126 x:0 0 0 2024-05-09T06:38:25.884 INFO:tasks.workunit.client.1.smithi193.stdout:3/465: chown d1/db/d59/c71 88045982 1 2024-05-09T06:38:25.884 INFO:tasks.workunit.client.1.smithi193.stdout:3/466: dread - d1/db/d2f/fa3 zero size 2024-05-09T06:38:25.888 INFO:tasks.workunit.client.0.smithi063.stdout:8/983: mkdir d9/da/d18/d24/d71/d13b/d158 0 2024-05-09T06:38:25.888 INFO:tasks.workunit.client.0.smithi063.stdout:1/834: dread d9/d2a/d39/f5e [0,4194304] 0 2024-05-09T06:38:25.888 INFO:tasks.workunit.client.0.smithi063.stdout:9/885: write d0/d4/d26/d39/d48/d80/d84/fc7 [471108,66465] 0 2024-05-09T06:38:25.891 INFO:tasks.workunit.client.0.smithi063.stdout:5/868: rename d8/d17/d10b/dda/d120 to d8/d53/dca/d11e/d123 0 2024-05-09T06:38:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:1/589: creat d4/d16/d17/d1a/d5b/fb7 x:0 0 0 2024-05-09T06:38:25.901 INFO:tasks.workunit.client.1.smithi193.stdout:4/592: dwrite d1a/d98/fb8 [0,4194304] 0 2024-05-09T06:38:25.902 INFO:tasks.workunit.client.0.smithi063.stdout:7/872: rmdir d0/d15/d49/d54 39 2024-05-09T06:38:25.903 INFO:tasks.workunit.client.1.smithi193.stdout:4/593: chown d1a/d80/fa9 703 1 2024-05-09T06:38:25.904 INFO:tasks.workunit.client.1.smithi193.stdout:7/517: read d6/d8/d1a/d33/f3e [1056610,35035] 0 2024-05-09T06:38:25.904 INFO:tasks.workunit.client.1.smithi193.stdout:5/456: mknod d0/c92 0 2024-05-09T06:38:25.905 INFO:tasks.workunit.client.1.smithi193.stdout:4/594: dread - d1a/d24/d73/d94/f8f zero size 2024-05-09T06:38:25.907 INFO:tasks.workunit.client.0.smithi063.stdout:3/941: creat d7/d70/d98/f148 x:0 0 0 2024-05-09T06:38:25.907 INFO:tasks.workunit.client.0.smithi063.stdout:7/873: dwrite d0/f11 [0,4194304] 0 2024-05-09T06:38:25.913 INFO:tasks.workunit.client.0.smithi063.stdout:6/914: chown d2/df/d59/d99/d9f/fba 4925236 1 2024-05-09T06:38:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:4/595: dwrite d1a/d80/fa9 [0,4194304] 0 2024-05-09T06:38:25.919 INFO:tasks.workunit.client.0.smithi063.stdout:8/984: symlink d9/d41/dec/db8/l159 0 2024-05-09T06:38:25.921 INFO:tasks.workunit.client.1.smithi193.stdout:1/590: dread d4/d16/d6b/d85/f86 [0,4194304] 0 2024-05-09T06:38:25.925 INFO:tasks.workunit.client.1.smithi193.stdout:8/531: symlink d6/d1c/la0 0 2024-05-09T06:38:25.926 INFO:tasks.workunit.client.1.smithi193.stdout:5/457: unlink d0/d6d/f81 0 2024-05-09T06:38:25.926 INFO:tasks.workunit.client.0.smithi063.stdout:1/835: rmdir d9/d2a/d35/d1e/d9d/d3f/d59 39 2024-05-09T06:38:25.928 INFO:tasks.workunit.client.0.smithi063.stdout:1/836: truncate d9/d46/dfe/d94/d112/f119 942838 0 2024-05-09T06:38:25.928 INFO:tasks.workunit.client.0.smithi063.stdout:1/837: chown d9/d2a/d39/l40 144852 1 2024-05-09T06:38:25.929 INFO:tasks.workunit.client.0.smithi063.stdout:1/838: readlink d9/d2a/d35/d1e/d9d/l124 0 2024-05-09T06:38:25.930 INFO:tasks.workunit.client.0.smithi063.stdout:9/886: rmdir d0/d4/d26/d39/d3e/d66 39 2024-05-09T06:38:25.946 INFO:tasks.workunit.client.0.smithi063.stdout:3/942: rmdir d7/da/d21/d30/d44/d48 39 2024-05-09T06:38:25.947 INFO:tasks.workunit.client.1.smithi193.stdout:2/547: dwrite db/df/f69 [0,4194304] 0 2024-05-09T06:38:25.948 INFO:tasks.workunit.client.1.smithi193.stdout:2/548: chown db/df/d87 119558601 1 2024-05-09T06:38:25.948 INFO:tasks.workunit.client.1.smithi193.stdout:6/488: truncate d1/de/d11/d20/f27 653367 0 2024-05-09T06:38:25.951 INFO:tasks.workunit.client.1.smithi193.stdout:2/549: write db/d25/d38/d40/d8b/d77/fa7 [518776,114078] 0 2024-05-09T06:38:25.952 INFO:tasks.workunit.client.1.smithi193.stdout:6/489: chown d1/d25/f9d 9378204 1 2024-05-09T06:38:25.952 INFO:tasks.workunit.client.1.smithi193.stdout:6/490: chown d1/d25/d2c/d3c/f99 6322 1 2024-05-09T06:38:25.952 INFO:tasks.workunit.client.0.smithi063.stdout:0/925: rename dd/d60/d44/d90/fce to dd/d1a/d2c/d51/d72/f11e 0 2024-05-09T06:38:25.953 INFO:tasks.workunit.client.0.smithi063.stdout:0/926: chown dd/d1a/d2c/d42 2301 1 2024-05-09T06:38:25.953 INFO:tasks.workunit.client.1.smithi193.stdout:6/491: chown d1/de/d11/f5d 2599042 1 2024-05-09T06:38:25.954 INFO:tasks.workunit.client.0.smithi063.stdout:0/927: chown dd/d1a/d2c/d51/d72/d9e/dec 62557052 1 2024-05-09T06:38:25.956 INFO:tasks.workunit.client.0.smithi063.stdout:6/915: symlink d2/d20/d4a/l12a 0 2024-05-09T06:38:25.957 INFO:tasks.workunit.client.1.smithi193.stdout:1/591: creat d4/d16/d60/fb8 x:0 0 0 2024-05-09T06:38:25.957 INFO:tasks.workunit.client.1.smithi193.stdout:1/592: dread - d4/d16/d6b/d6e/f78 zero size 2024-05-09T06:38:25.958 INFO:tasks.workunit.client.0.smithi063.stdout:4/905: mknod d7/df/d35/c126 0 2024-05-09T06:38:25.959 INFO:tasks.workunit.client.0.smithi063.stdout:4/906: fsync d7/df/d35/d78/d109/f115 0 2024-05-09T06:38:25.960 INFO:tasks.workunit.client.0.smithi063.stdout:4/907: read - d7/df/d77/f113 zero size 2024-05-09T06:38:25.963 INFO:tasks.workunit.client.0.smithi063.stdout:8/985: truncate d9/da/d18/d24/f42 3408825 0 2024-05-09T06:38:25.964 INFO:tasks.workunit.client.1.smithi193.stdout:3/467: truncate d1/d44/f47 7638541 0 2024-05-09T06:38:25.965 INFO:tasks.workunit.client.1.smithi193.stdout:7/518: creat d6/d8/d1a/d36/d4d/d60/d9c/fb1 x:0 0 0 2024-05-09T06:38:25.968 INFO:tasks.workunit.client.0.smithi063.stdout:9/887: mkdir d0/d4/d7d/d12e 0 2024-05-09T06:38:25.970 INFO:tasks.workunit.client.0.smithi063.stdout:5/869: truncate d8/d17/d18/f2d 1919473 0 2024-05-09T06:38:25.974 INFO:tasks.workunit.client.0.smithi063.stdout:9/888: dwrite d0/d8/da6/db0/db5/f106 [0,4194304] 0 2024-05-09T06:38:25.975 INFO:tasks.workunit.client.0.smithi063.stdout:9/889: chown d0/d4/d57/d60 10173429 1 2024-05-09T06:38:25.976 INFO:tasks.workunit.client.1.smithi193.stdout:2/550: mkdir db/df/d54/dba 0 2024-05-09T06:38:25.978 INFO:tasks.workunit.client.1.smithi193.stdout:2/551: read db/d25/d38/d61/f73 [3219477,14884] 0 2024-05-09T06:38:25.979 INFO:tasks.workunit.client.0.smithi063.stdout:3/943: unlink d7/da/d2b/d66/d74/d7e/fca 0 2024-05-09T06:38:25.979 INFO:tasks.workunit.client.1.smithi193.stdout:6/492: symlink d1/d25/d2c/d3c/lae 0 2024-05-09T06:38:25.980 INFO:tasks.workunit.client.0.smithi063.stdout:3/944: stat d7/da/d2b/d66/dc0/l76 0 2024-05-09T06:38:25.980 INFO:tasks.workunit.client.0.smithi063.stdout:3/945: chown d7/d70/d98/d2d/d36 179566959 1 2024-05-09T06:38:25.981 INFO:tasks.workunit.client.0.smithi063.stdout:0/928: truncate dd/d1a/d2c/d51/d72/fd3 1305452 0 2024-05-09T06:38:25.981 INFO:tasks.workunit.client.1.smithi193.stdout:1/593: rename d4/d16/d60/c67 to d4/d16/d6b/d89/db3/cb9 0 2024-05-09T06:38:25.998 INFO:tasks.workunit.client.1.smithi193.stdout:1/594: dread d4/d16/d60/d4f/f61 [0,4194304] 0 2024-05-09T06:38:26.012 INFO:tasks.workunit.client.1.smithi193.stdout:9/586: sync 2024-05-09T06:38:26.017 INFO:tasks.workunit.client.1.smithi193.stdout:8/532: dwrite d6/d1c/f1e [0,4194304] 0 2024-05-09T06:38:26.019 INFO:tasks.workunit.client.0.smithi063.stdout:1/839: dwrite d9/d2a/d35/d1e/fd1 [0,4194304] 0 2024-05-09T06:38:26.021 INFO:tasks.workunit.client.0.smithi063.stdout:2/913: getdents dc/dd/d45/d5a/d6c/d8f 0 2024-05-09T06:38:26.027 INFO:tasks.workunit.client.1.smithi193.stdout:4/596: truncate d1a/d80/f9b 4130878 0 2024-05-09T06:38:26.030 INFO:tasks.workunit.client.0.smithi063.stdout:5/870: creat d8/d35/dd6/f124 x:0 0 0 2024-05-09T06:38:26.037 INFO:tasks.workunit.client.1.smithi193.stdout:9/587: mknod dd/d1e/d1f/d61/cc3 0 2024-05-09T06:38:26.038 INFO:tasks.workunit.client.0.smithi063.stdout:7/874: creat d0/d15/d49/d54/d9f/d106/f12b x:0 0 0 2024-05-09T06:38:26.038 INFO:tasks.workunit.client.1.smithi193.stdout:9/588: chown dd/d1e/d1f 474 1 2024-05-09T06:38:26.040 INFO:tasks.workunit.client.1.smithi193.stdout:4/597: dwrite d1a/d24/d73/d94/f8f [0,4194304] 0 2024-05-09T06:38:26.042 INFO:tasks.workunit.client.0.smithi063.stdout:9/890: readlink d0/d4/l89 0 2024-05-09T06:38:26.042 INFO:tasks.workunit.client.0.smithi063.stdout:4/908: write d7/df/f12 [4374761,55354] 0 2024-05-09T06:38:26.043 INFO:tasks.workunit.client.0.smithi063.stdout:9/891: chown d0/d3b/f126 62 1 2024-05-09T06:38:26.044 INFO:tasks.workunit.client.0.smithi063.stdout:4/909: write d7/df/d35/dd9/f103 [817930,127553] 0 2024-05-09T06:38:26.044 INFO:tasks.workunit.client.1.smithi193.stdout:5/458: link d0/d13/d2c/d37/d5b/f32 d0/d13/d2c/d4d/d67/f93 0 2024-05-09T06:38:26.047 INFO:tasks.workunit.client.0.smithi063.stdout:3/946: unlink d7/da/f133 0 2024-05-09T06:38:26.053 INFO:tasks.workunit.client.0.smithi063.stdout:3/947: read d7/d70/d98/fe9 [10036,117493] 0 2024-05-09T06:38:26.060 INFO:tasks.workunit.client.0.smithi063.stdout:1/840: rename d9/d2a/d3c/d4a/db8/ddf/lf9 to d9/d2a/d3c/d4a/l127 0 2024-05-09T06:38:26.062 INFO:tasks.workunit.client.1.smithi193.stdout:9/589: read dd/d1e/dc2/d49/f64 [1728015,65630] 0 2024-05-09T06:38:26.062 INFO:tasks.workunit.client.0.smithi063.stdout:8/986: mknod d9/da/d18/d24/d71/c15a 0 2024-05-09T06:38:26.064 INFO:tasks.workunit.client.1.smithi193.stdout:7/519: dwrite d6/d8/d1a/f27 [0,4194304] 0 2024-05-09T06:38:26.066 INFO:tasks.workunit.client.0.smithi063.stdout:0/929: write dd/d1a/f78 [1318833,20075] 0 2024-05-09T06:38:26.066 INFO:tasks.workunit.client.0.smithi063.stdout:1/841: dwrite d9/d2a/d35/d1e/fd1 [0,4194304] 0 2024-05-09T06:38:26.068 INFO:tasks.workunit.client.0.smithi063.stdout:1/842: dread - d9/d2a/d35/d24/d68/f104 zero size 2024-05-09T06:38:26.070 INFO:tasks.workunit.client.0.smithi063.stdout:6/916: dread d2/d4/d7c/d42/f61 [0,4194304] 0 2024-05-09T06:38:26.072 INFO:tasks.workunit.client.0.smithi063.stdout:8/987: dwrite d9/da/d45/d114/d139/f9c [0,4194304] 0 2024-05-09T06:38:26.073 INFO:tasks.workunit.client.1.smithi193.stdout:8/533: dread d6/d1c/d18/d2f/f44 [0,4194304] 0 2024-05-09T06:38:26.074 INFO:tasks.workunit.client.1.smithi193.stdout:7/520: dwrite d6/dd/f40 [4194304,4194304] 0 2024-05-09T06:38:26.075 INFO:tasks.workunit.client.0.smithi063.stdout:7/875: creat d0/d15/d89/f12c x:0 0 0 2024-05-09T06:38:26.076 INFO:tasks.workunit.client.1.smithi193.stdout:6/493: creat d1/d25/d2c/d81/faf x:0 0 0 2024-05-09T06:38:26.077 INFO:tasks.workunit.client.0.smithi063.stdout:7/876: write d0/d15/d4a/d50/f2e [8909677,22231] 0 2024-05-09T06:38:26.081 INFO:tasks.workunit.client.1.smithi193.stdout:1/595: rename d4/d16/d6b/d89/lac to d4/d16/d17/lba 0 2024-05-09T06:38:26.084 INFO:tasks.workunit.client.0.smithi063.stdout:4/910: creat d7/da3/d28/d31/dbf/f127 x:0 0 0 2024-05-09T06:38:26.085 INFO:tasks.workunit.client.1.smithi193.stdout:4/598: write fa [563322,79227] 0 2024-05-09T06:38:26.088 INFO:tasks.workunit.client.0.smithi063.stdout:7/877: dwrite d0/d15/d4a/d68/d10a/f122 [0,4194304] 0 2024-05-09T06:38:26.096 INFO:tasks.workunit.client.1.smithi193.stdout:1/596: creat d4/d16/d17/d1a/fbb x:0 0 0 2024-05-09T06:38:26.098 INFO:tasks.workunit.client.1.smithi193.stdout:9/590: mkdir dd/df/dc4 0 2024-05-09T06:38:26.100 INFO:tasks.workunit.client.1.smithi193.stdout:9/591: truncate dd/d1e/d3c/fbf 642205 0 2024-05-09T06:38:26.103 INFO:tasks.workunit.client.0.smithi063.stdout:9/892: rmdir d0/d4/d26/d39/d3e/d66 39 2024-05-09T06:38:26.109 INFO:tasks.workunit.client.1.smithi193.stdout:4/599: symlink d1a/d67/d82/dab/lb9 0 2024-05-09T06:38:26.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/988: dread d9/da/d18/d24/f27 [0,4194304] 0 2024-05-09T06:38:26.113 INFO:tasks.workunit.client.1.smithi193.stdout:5/459: rename d0/d13/d2c/d37/l42 to d0/l94 0 2024-05-09T06:38:26.120 INFO:tasks.workunit.client.1.smithi193.stdout:8/534: rename c2 to d6/d1c/d1b/d5d/d84/d87/ca1 0 2024-05-09T06:38:26.122 INFO:tasks.workunit.client.0.smithi063.stdout:6/917: mkdir d2/d1f/d29/de2/d4e/d12b 0 2024-05-09T06:38:26.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/843: write d9/d10/f1c [1499232,4058] 0 2024-05-09T06:38:26.123 INFO:tasks.workunit.client.1.smithi193.stdout:6/494: dread d1/de/ff [4194304,4194304] 0 2024-05-09T06:38:26.123 INFO:tasks.workunit.client.0.smithi063.stdout:1/844: stat d9/d2a/d3c/d4c/d83/d88/dc2/dd0/f9b 0 2024-05-09T06:38:26.124 INFO:tasks.workunit.client.0.smithi063.stdout:4/911: getdents d7/da3/d28/d31/d7b/df7 0 2024-05-09T06:38:26.126 INFO:tasks.workunit.client.1.smithi193.stdout:1/597: creat d4/d16/d17/fbc x:0 0 0 2024-05-09T06:38:26.126 INFO:tasks.workunit.client.1.smithi193.stdout:7/521: write d6/dd/d53/f6a [507194,110041] 0 2024-05-09T06:38:26.128 INFO:tasks.workunit.client.0.smithi063.stdout:7/878: mkdir d0/d15/d4a/d12d 0 2024-05-09T06:38:26.129 INFO:tasks.workunit.client.0.smithi063.stdout:1/845: dwrite d9/d2a/d35/d1e/fd1 [0,4194304] 0 2024-05-09T06:38:26.132 INFO:tasks.workunit.client.0.smithi063.stdout:2/914: rename dc/fd8 to dc/d36/d43/d46/d49/dbb/f111 0 2024-05-09T06:38:26.133 INFO:tasks.workunit.client.0.smithi063.stdout:2/915: chown dc/f2a 649650098 1 2024-05-09T06:38:26.134 INFO:tasks.workunit.client.0.smithi063.stdout:5/871: getdents d8/d35/dd6 0 2024-05-09T06:38:26.135 INFO:tasks.workunit.client.1.smithi193.stdout:8/535: chown d6/d5b/c68 506 1 2024-05-09T06:38:26.139 INFO:tasks.workunit.client.1.smithi193.stdout:9/592: rename dd/df/d15/f56 to dd/d1e/dc2/d49/dbd/d5b/d8a/fc5 0 2024-05-09T06:38:26.143 INFO:tasks.workunit.client.1.smithi193.stdout:6/495: creat d1/d25/d4c/d9e/fb0 x:0 0 0 2024-05-09T06:38:26.144 INFO:tasks.workunit.client.0.smithi063.stdout:7/879: fdatasync d0/d15/d49/d4f/d66/f71 0 2024-05-09T06:38:26.144 INFO:tasks.workunit.client.0.smithi063.stdout:7/880: dread - d0/d15/f109 zero size 2024-05-09T06:38:26.148 INFO:tasks.workunit.client.0.smithi063.stdout:3/948: rename d7/d22/dfc/f107 to d7/d70/d98/dbb/f149 0 2024-05-09T06:38:26.150 INFO:tasks.workunit.client.0.smithi063.stdout:0/930: sync 2024-05-09T06:38:26.150 INFO:tasks.workunit.client.0.smithi063.stdout:4/912: dread d7/da3/d28/d31/d32/d34/f80 [0,4194304] 0 2024-05-09T06:38:26.154 INFO:tasks.workunit.client.0.smithi063.stdout:2/916: mkdir dc/d32/d112 0 2024-05-09T06:38:26.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/600: creat d1a/d22/fba x:0 0 0 2024-05-09T06:38:26.154 INFO:tasks.workunit.client.1.smithi193.stdout:8/536: read d6/f21 [2278690,104228] 0 2024-05-09T06:38:26.158 INFO:tasks.workunit.client.1.smithi193.stdout:7/522: dread d6/dd/f47 [0,4194304] 0 2024-05-09T06:38:26.160 INFO:tasks.workunit.client.1.smithi193.stdout:9/593: mkdir dd/d1e/d1f/d61/dc6 0 2024-05-09T06:38:26.160 INFO:tasks.workunit.client.0.smithi063.stdout:5/872: dread d8/d17/d18/f2d [0,4194304] 0 2024-05-09T06:38:26.163 INFO:tasks.workunit.client.0.smithi063.stdout:1/846: mknod d9/d2a/d35/d24/c128 0 2024-05-09T06:38:26.165 INFO:tasks.workunit.client.0.smithi063.stdout:7/881: fsync d0/d15/d4a/fb4 0 2024-05-09T06:38:26.167 INFO:tasks.workunit.client.1.smithi193.stdout:1/598: creat d4/d3c/d3e/d7c/fbd x:0 0 0 2024-05-09T06:38:26.169 INFO:tasks.workunit.client.1.smithi193.stdout:1/599: read d4/f9 [608404,71438] 0 2024-05-09T06:38:26.169 INFO:tasks.workunit.client.1.smithi193.stdout:4/601: symlink d1a/d24/d73/d94/d7c/d5b/lbb 0 2024-05-09T06:38:26.171 INFO:tasks.workunit.client.1.smithi193.stdout:6/496: dwrite d1/de/fa6 [0,4194304] 0 2024-05-09T06:38:26.171 INFO:tasks.workunit.client.0.smithi063.stdout:9/893: rename d0/d3b/da0/fde to d0/d43/f12f 0 2024-05-09T06:38:26.174 INFO:tasks.workunit.client.0.smithi063.stdout:3/949: unlink d7/da/d21/f6b 0 2024-05-09T06:38:26.177 INFO:tasks.workunit.client.0.smithi063.stdout:4/913: fsync d7/d50/fa1 0 2024-05-09T06:38:26.177 INFO:tasks.workunit.client.1.smithi193.stdout:9/594: dread dd/df/d31/d7e/d37/f7c [0,4194304] 0 2024-05-09T06:38:26.178 INFO:tasks.workunit.client.1.smithi193.stdout:4/602: dwrite d1a/d24/d73/d94/f54 [0,4194304] 0 2024-05-09T06:38:26.178 INFO:tasks.workunit.client.1.smithi193.stdout:5/460: getdents d0/d13/d2c/d37/d5b/d89 0 2024-05-09T06:38:26.181 INFO:tasks.workunit.client.1.smithi193.stdout:4/603: truncate d1a/d67/f6c 567929 0 2024-05-09T06:38:26.182 INFO:tasks.workunit.client.1.smithi193.stdout:9/595: write dd/d1e/d93/fa3 [1403371,32454] 0 2024-05-09T06:38:26.182 INFO:tasks.workunit.client.1.smithi193.stdout:4/604: stat d1a/d24/d73/d94/fb4 0 2024-05-09T06:38:26.183 INFO:tasks.workunit.client.0.smithi063.stdout:5/873: mkdir d8/d19/d30/d74/d77/d125 0 2024-05-09T06:38:26.184 INFO:tasks.workunit.client.1.smithi193.stdout:4/605: truncate d1a/d1f/f20 4812682 0 2024-05-09T06:38:26.186 INFO:tasks.workunit.client.1.smithi193.stdout:1/600: symlink d4/d76/lbe 0 2024-05-09T06:38:26.191 INFO:tasks.workunit.client.0.smithi063.stdout:2/917: dread dc/dd/f13 [0,4194304] 0 2024-05-09T06:38:26.191 INFO:tasks.workunit.client.1.smithi193.stdout:8/537: mkdir d6/d1c/d18/da2 0 2024-05-09T06:38:26.192 INFO:tasks.workunit.client.1.smithi193.stdout:8/538: write d6/d5b/f8d [605565,72055] 0 2024-05-09T06:38:26.195 INFO:tasks.workunit.client.0.smithi063.stdout:2/918: dread dc/d36/d43/d46/ded/ff0 [0,4194304] 0 2024-05-09T06:38:26.195 INFO:tasks.workunit.client.1.smithi193.stdout:6/497: rename d1/d25/d2c/d3c/d3e to d1/d25/d2c/d3c/db1 0 2024-05-09T06:38:26.196 INFO:tasks.workunit.client.0.smithi063.stdout:0/931: write dd/d60/d119/d48/f57 [591954,122324] 0 2024-05-09T06:38:26.196 INFO:tasks.workunit.client.0.smithi063.stdout:2/919: truncate dc/dd/d45/fe3 1909887 0 2024-05-09T06:38:26.199 INFO:tasks.workunit.client.0.smithi063.stdout:9/894: dread - d0/d4/d57/d60/fe0 zero size 2024-05-09T06:38:26.200 INFO:tasks.workunit.client.0.smithi063.stdout:9/895: truncate d0/d109/f129 1035350 0 2024-05-09T06:38:26.201 INFO:tasks.workunit.client.0.smithi063.stdout:1/847: write d9/d2a/d35/f29 [5979874,62868] 0 2024-05-09T06:38:26.205 INFO:tasks.workunit.client.0.smithi063.stdout:3/950: creat d7/d70/d98/d5b/f14a x:0 0 0 2024-05-09T06:38:26.206 INFO:tasks.workunit.client.0.smithi063.stdout:3/951: chown d7/d22/dfc/f11e 640365 1 2024-05-09T06:38:26.207 INFO:tasks.workunit.client.0.smithi063.stdout:3/952: stat d7/d22/lb5 0 2024-05-09T06:38:26.207 INFO:tasks.workunit.client.1.smithi193.stdout:9/596: write dd/d1e/dc2/f80 [8236921,17506] 0 2024-05-09T06:38:26.209 INFO:tasks.workunit.client.1.smithi193.stdout:5/461: write d0/d13/f3d [1903482,21386] 0 2024-05-09T06:38:26.210 INFO:tasks.workunit.client.0.smithi063.stdout:7/882: dwrite d0/d15/d89/d10c/fd7 [0,4194304] 0 2024-05-09T06:38:26.211 INFO:tasks.workunit.client.1.smithi193.stdout:1/601: truncate d4/d16/d60/d4f/f61 3204998 0 2024-05-09T06:38:26.215 INFO:tasks.workunit.client.1.smithi193.stdout:5/462: truncate d0/d13/d2c/d37/d2b/d47/d66/d3a/f52 4788548 0 2024-05-09T06:38:26.218 INFO:tasks.workunit.client.0.smithi063.stdout:0/932: mknod dd/d1a/d50/c11f 0 2024-05-09T06:38:26.222 INFO:tasks.workunit.client.0.smithi063.stdout:8/989: rename d9/da/d18/d24/d71/l147 to d9/da/d18/dc7/dd3/l15b 0 2024-05-09T06:38:26.225 INFO:tasks.workunit.client.0.smithi063.stdout:2/920: truncate dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/f7a 655753 0 2024-05-09T06:38:26.226 INFO:tasks.workunit.client.1.smithi193.stdout:8/539: rename d6/d1c/d53/f9d to d6/d1c/d1b/d5d/d84/d87/fa3 0 2024-05-09T06:38:26.228 INFO:tasks.workunit.client.0.smithi063.stdout:1/848: unlink d9/d2a/d35/f117 0 2024-05-09T06:38:26.230 INFO:tasks.workunit.client.0.smithi063.stdout:2/921: dwrite dc/dd/d2d/d60/f67 [4194304,4194304] 0 2024-05-09T06:38:26.230 INFO:tasks.workunit.client.0.smithi063.stdout:0/933: dread dd/d1a/f65 [0,4194304] 0 2024-05-09T06:38:26.231 INFO:tasks.workunit.client.1.smithi193.stdout:6/498: dwrite d1/fa [0,4194304] 0 2024-05-09T06:38:26.232 INFO:tasks.workunit.client.1.smithi193.stdout:5/463: fdatasync d0/d13/d2c/d37/d2b/d47/d60/d75/f8b 0 2024-05-09T06:38:26.232 INFO:tasks.workunit.client.0.smithi063.stdout:0/934: chown dd/d1a/d2c/d51/d72/d9e/c10d 14379214 1 2024-05-09T06:38:26.237 INFO:tasks.workunit.client.1.smithi193.stdout:4/606: link d1a/d24/f41 d1a/d67/d82/dab/fbc 0 2024-05-09T06:38:26.237 INFO:tasks.workunit.client.1.smithi193.stdout:4/607: fsync d1a/d22/f7a 0 2024-05-09T06:38:26.238 INFO:tasks.workunit.client.1.smithi193.stdout:1/602: symlink d4/d16/d17/d1a/d3a/lbf 0 2024-05-09T06:38:26.244 INFO:tasks.workunit.client.1.smithi193.stdout:5/464: fdatasync d0/d13/d2c/d37/d5b/d4e/f78 0 2024-05-09T06:38:26.246 INFO:tasks.workunit.client.1.smithi193.stdout:9/597: creat dd/df/fc7 x:0 0 0 2024-05-09T06:38:26.249 INFO:tasks.workunit.client.0.smithi063.stdout:1/849: rmdir d9/d2a/d3c/d4c 39 2024-05-09T06:38:26.251 INFO:tasks.workunit.client.0.smithi063.stdout:2/922: creat dc/d6e/d7f/f113 x:0 0 0 2024-05-09T06:38:26.252 INFO:tasks.workunit.client.1.smithi193.stdout:9/598: dwrite dd/df/f6e [0,4194304] 0 2024-05-09T06:38:26.254 INFO:tasks.workunit.client.1.smithi193.stdout:4/608: mkdir d1a/d24/d73/d94/dbd 0 2024-05-09T06:38:26.260 INFO:tasks.workunit.client.1.smithi193.stdout:1/603: unlink d4/d16/d60/d4f/d72/d7f/f93 0 2024-05-09T06:38:26.260 INFO:tasks.workunit.client.1.smithi193.stdout:9/599: dwrite fa [0,4194304] 0 2024-05-09T06:38:26.262 INFO:tasks.workunit.client.1.smithi193.stdout:9/600: dread - dd/d1e/d1f/d48/d55/fb8 zero size 2024-05-09T06:38:26.262 INFO:tasks.workunit.client.0.smithi063.stdout:2/923: dread dc/dd/fb5 [0,4194304] 0 2024-05-09T06:38:26.264 INFO:tasks.workunit.client.1.smithi193.stdout:9/601: write dd/df/d31/d7e/d37/f83 [2542265,73507] 0 2024-05-09T06:38:26.266 INFO:tasks.workunit.client.0.smithi063.stdout:4/914: rmdir d7/d3d/dca/de0 0 2024-05-09T06:38:26.268 INFO:tasks.workunit.client.1.smithi193.stdout:6/499: mkdir d1/de/d11/d18/d94/dac/db2 0 2024-05-09T06:38:26.270 INFO:tasks.workunit.client.0.smithi063.stdout:7/883: mknod d0/d15/d49/d4f/d66/d67/d86/d110/c12e 0 2024-05-09T06:38:26.273 INFO:tasks.workunit.client.0.smithi063.stdout:5/874: write d8/f16 [2786340,69890] 0 2024-05-09T06:38:26.274 INFO:tasks.workunit.client.1.smithi193.stdout:8/540: write d6/fd [1830559,120649] 0 2024-05-09T06:38:26.276 INFO:tasks.workunit.client.1.smithi193.stdout:5/465: creat d0/d13/d2c/d37/d5b/d89/f95 x:0 0 0 2024-05-09T06:38:26.280 INFO:tasks.workunit.client.1.smithi193.stdout:4/609: chown d1a/d22/d5d/f93 26 1 2024-05-09T06:38:26.281 INFO:tasks.workunit.client.1.smithi193.stdout:4/610: chown d1a/d6b/c51 5784 1 2024-05-09T06:38:26.282 INFO:tasks.workunit.client.0.smithi063.stdout:3/953: creat d7/d70/d98/f14b x:0 0 0 2024-05-09T06:38:26.283 INFO:tasks.workunit.client.0.smithi063.stdout:3/954: fdatasync d7/da/f12d 0 2024-05-09T06:38:26.287 INFO:tasks.workunit.client.1.smithi193.stdout:9/602: creat dd/d1e/d1f/d48/fc8 x:0 0 0 2024-05-09T06:38:26.289 INFO:tasks.workunit.client.1.smithi193.stdout:6/500: mkdir d1/d25/d2c/d81/d37/db3 0 2024-05-09T06:38:26.293 INFO:tasks.workunit.client.0.smithi063.stdout:5/875: creat d8/d19/d84/f126 x:0 0 0 2024-05-09T06:38:26.294 INFO:tasks.workunit.client.1.smithi193.stdout:5/466: creat d0/d13/d2c/d37/d2b/d47/d66/d3a/f96 x:0 0 0 2024-05-09T06:38:26.296 INFO:tasks.workunit.client.1.smithi193.stdout:4/611: creat d1a/d6b/d60/fbe x:0 0 0 2024-05-09T06:38:26.297 INFO:tasks.workunit.client.0.smithi063.stdout:1/850: creat d9/d2a/d3c/d4c/dc3/f129 x:0 0 0 2024-05-09T06:38:26.298 INFO:tasks.workunit.client.0.smithi063.stdout:1/851: write d9/d46/f54 [4795578,113395] 0 2024-05-09T06:38:26.299 INFO:tasks.workunit.client.1.smithi193.stdout:9/603: creat dd/d1e/dc2/d49/dbd/fc9 x:0 0 0 2024-05-09T06:38:26.301 INFO:tasks.workunit.client.1.smithi193.stdout:6/501: creat d1/d25/d2c/d81/d37/d4f/fb4 x:0 0 0 2024-05-09T06:38:26.301 INFO:tasks.workunit.client.1.smithi193.stdout:8/541: write d6/d1c/d53/f71 [853916,104111] 0 2024-05-09T06:38:26.302 INFO:tasks.workunit.client.1.smithi193.stdout:8/542: fsync d6/f23 0 2024-05-09T06:38:26.304 INFO:tasks.workunit.client.1.smithi193.stdout:5/467: unlink d0/d13/d2c/d37/d5b/d4e/f58 0 2024-05-09T06:38:26.305 INFO:tasks.workunit.client.0.smithi063.stdout:7/884: dwrite d0/fbe [8388608,4194304] 0 2024-05-09T06:38:26.306 INFO:tasks.workunit.client.0.smithi063.stdout:7/885: chown d0/d15/d89/f12c 33535532 1 2024-05-09T06:38:26.306 INFO:tasks.workunit.client.0.smithi063.stdout:7/886: readlink d0/d15/l41 0 2024-05-09T06:38:26.306 INFO:tasks.workunit.client.0.smithi063.stdout:5/876: creat d8/d19/d30/d74/d77/d10d/f127 x:0 0 0 2024-05-09T06:38:26.308 INFO:tasks.workunit.client.1.smithi193.stdout:1/604: creat d4/d16/d6b/fc0 x:0 0 0 2024-05-09T06:38:26.308 INFO:tasks.workunit.client.0.smithi063.stdout:5/877: truncate d8/d17/da3/fbf 1672054 0 2024-05-09T06:38:26.310 INFO:tasks.workunit.client.0.smithi063.stdout:6/918: rename d2/d20/l54 to d2/df/d91/l12c 0 2024-05-09T06:38:26.314 INFO:tasks.workunit.client.0.smithi063.stdout:2/924: creat dc/d36/d43/d46/f114 x:0 0 0 2024-05-09T06:38:26.315 INFO:tasks.workunit.client.1.smithi193.stdout:6/502: rmdir d1/de/d11/d18 39 2024-05-09T06:38:26.318 INFO:tasks.workunit.client.1.smithi193.stdout:4/612: write d1a/d1f/f2a [1708004,117621] 0 2024-05-09T06:38:26.318 INFO:tasks.workunit.client.0.smithi063.stdout:4/915: link d7/df/d35/c89 d7/df/d35/d78/c128 0 2024-05-09T06:38:26.321 INFO:tasks.workunit.client.1.smithi193.stdout:0/492: sync 2024-05-09T06:38:26.321 INFO:tasks.workunit.client.1.smithi193.stdout:2/552: sync 2024-05-09T06:38:26.321 INFO:tasks.workunit.client.1.smithi193.stdout:3/468: sync 2024-05-09T06:38:26.322 INFO:tasks.workunit.client.0.smithi063.stdout:9/896: rename d0/d4/d26/f36 to d0/d4/d57/d60/d76/d11e/f130 0 2024-05-09T06:38:26.324 INFO:tasks.workunit.client.0.smithi063.stdout:9/897: truncate d0/d4/d26/d39/d48/d80/d84/dd8/ddf/fe2 6349062 0 2024-05-09T06:38:26.324 INFO:tasks.workunit.client.0.smithi063.stdout:6/919: creat d2/d1f/d29/d3f/d5c/d64/f12d x:0 0 0 2024-05-09T06:38:26.326 INFO:tasks.workunit.client.1.smithi193.stdout:8/543: dwrite d6/d1c/d1b/d5d/d84/d87/fa3 [0,4194304] 0 2024-05-09T06:38:26.327 INFO:tasks.workunit.client.1.smithi193.stdout:1/605: creat d4/d16/d60/d4f/d72/fc1 x:0 0 0 2024-05-09T06:38:26.328 INFO:tasks.workunit.client.0.smithi063.stdout:1/852: read d9/d2a/d35/d1e/d9d/d3f/d59/d62/fa5 [2236581,67456] 0 2024-05-09T06:38:26.329 INFO:tasks.workunit.client.1.smithi193.stdout:9/604: symlink dd/df/d31/d7e/lca 0 2024-05-09T06:38:26.332 INFO:tasks.workunit.client.0.smithi063.stdout:5/878: write d8/d17/f3d [1150030,23940] 0 2024-05-09T06:38:26.334 INFO:tasks.workunit.client.0.smithi063.stdout:3/955: creat d7/da/d21/d30/d44/f14c x:0 0 0 2024-05-09T06:38:26.335 INFO:tasks.workunit.client.1.smithi193.stdout:4/613: creat d1a/d24/fbf x:0 0 0 2024-05-09T06:38:26.335 INFO:tasks.workunit.client.1.smithi193.stdout:2/553: dwrite db/d25/d38/f75 [8388608,4194304] 0 2024-05-09T06:38:26.336 INFO:tasks.workunit.client.0.smithi063.stdout:3/956: fdatasync d7/da/d21/d30/d44/dee/d10c/f126 0 2024-05-09T06:38:26.336 INFO:tasks.workunit.client.0.smithi063.stdout:4/916: symlink d7/da3/d28/d31/d32/d34/db4/dcc/de1/l129 0 2024-05-09T06:38:26.337 INFO:tasks.workunit.client.1.smithi193.stdout:4/614: dread - d1a/d24/d73/d94/faf zero size 2024-05-09T06:38:26.337 INFO:tasks.workunit.client.1.smithi193.stdout:5/468: mknod d0/d13/d2c/c97 0 2024-05-09T06:38:26.341 INFO:tasks.workunit.client.1.smithi193.stdout:3/469: creat d1/db/d2f/fad x:0 0 0 2024-05-09T06:38:26.342 INFO:tasks.workunit.client.0.smithi063.stdout:8/990: sync 2024-05-09T06:38:26.342 INFO:tasks.workunit.client.0.smithi063.stdout:0/935: sync 2024-05-09T06:38:26.344 INFO:tasks.workunit.client.0.smithi063.stdout:5/879: dwrite d8/d19/d30/d74/d77/d10d/f127 [0,4194304] 0 2024-05-09T06:38:26.344 INFO:tasks.workunit.client.1.smithi193.stdout:0/493: creat d18/d4c/d47/f96 x:0 0 0 2024-05-09T06:38:26.346 INFO:tasks.workunit.client.0.smithi063.stdout:1/853: chown d9/d2a/l73 2001393490 1 2024-05-09T06:38:26.346 INFO:tasks.workunit.client.0.smithi063.stdout:3/957: chown d7/d70/d98/fe9 787201 1 2024-05-09T06:38:26.347 INFO:tasks.workunit.client.1.smithi193.stdout:8/544: dread - d6/d1c/d1b/d29/d57/f8b zero size 2024-05-09T06:38:26.347 INFO:tasks.workunit.client.0.smithi063.stdout:0/936: chown dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/dd6/fee 121 1 2024-05-09T06:38:26.348 INFO:tasks.workunit.client.1.smithi193.stdout:8/545: chown d6/d1c/d18/d66/f62 9886789 1 2024-05-09T06:38:26.348 INFO:tasks.workunit.client.0.smithi063.stdout:5/880: chown d8/d17/d10b/fe9 22343676 1 2024-05-09T06:38:26.350 INFO:tasks.workunit.client.0.smithi063.stdout:5/881: stat d8/d19/d30/d74/fec 0 2024-05-09T06:38:26.351 INFO:tasks.workunit.client.1.smithi193.stdout:3/470: dread d1/d2d/d51/f29 [0,4194304] 0 2024-05-09T06:38:26.351 INFO:tasks.workunit.client.0.smithi063.stdout:4/917: dwrite d7/da3/d28/d31/d7b/fbe [0,4194304] 0 2024-05-09T06:38:26.351 INFO:tasks.workunit.client.1.smithi193.stdout:9/605: symlink dd/df/d15/d8c/lcb 0 2024-05-09T06:38:26.354 INFO:tasks.workunit.client.0.smithi063.stdout:8/991: mkdir d9/da/d18/dc7/d8f/d15c 0 2024-05-09T06:38:26.362 INFO:tasks.workunit.client.0.smithi063.stdout:4/918: dwrite d7/da3/d28/d31/d71/d98/f10b [0,4194304] 0 2024-05-09T06:38:26.362 INFO:tasks.workunit.client.1.smithi193.stdout:4/615: readlink d1a/l68 0 2024-05-09T06:38:26.364 INFO:tasks.workunit.client.0.smithi063.stdout:2/925: dwrite dc/d36/d43/d46/d63/d79/d7d/f84 [0,4194304] 0 2024-05-09T06:38:26.364 INFO:tasks.workunit.client.1.smithi193.stdout:5/469: readlink d0/d13/d2c/d4d/l50 0 2024-05-09T06:38:26.365 INFO:tasks.workunit.client.1.smithi193.stdout:1/606: dread d4/f32 [4194304,4194304] 0 2024-05-09T06:38:26.366 INFO:tasks.workunit.client.0.smithi063.stdout:9/898: write d0/d4/fd1 [815294,13102] 0 2024-05-09T06:38:26.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/882: dread - d8/d17/d75/fdb zero size 2024-05-09T06:38:26.368 INFO:tasks.workunit.client.1.smithi193.stdout:0/494: creat d18/d4c/d47/d54/f97 x:0 0 0 2024-05-09T06:38:26.369 INFO:tasks.workunit.client.0.smithi063.stdout:5/883: write d8/d35/dd6/f124 [944385,120095] 0 2024-05-09T06:38:26.373 INFO:tasks.workunit.client.0.smithi063.stdout:1/854: unlink d9/l19 0 2024-05-09T06:38:26.373 INFO:tasks.workunit.client.1.smithi193.stdout:0/495: dread - f16 zero size 2024-05-09T06:38:26.377 INFO:tasks.workunit.client.0.smithi063.stdout:9/899: dwrite d0/d4/d57/d60/f7c [0,4194304] 0 2024-05-09T06:38:26.378 INFO:tasks.workunit.client.1.smithi193.stdout:3/471: creat d1/db/d59/d63/fae x:0 0 0 2024-05-09T06:38:26.382 INFO:tasks.workunit.client.0.smithi063.stdout:2/926: creat dc/dd/d45/d5a/d6c/d8f/f115 x:0 0 0 2024-05-09T06:38:26.383 INFO:tasks.workunit.client.0.smithi063.stdout:2/927: write dc/dd/d45/f8a [598580,592] 0 2024-05-09T06:38:26.385 INFO:tasks.workunit.client.1.smithi193.stdout:6/503: rename d1/d25/f9d to d1/d25/d2c/d81/fb5 0 2024-05-09T06:38:26.386 INFO:tasks.workunit.client.1.smithi193.stdout:7/523: sync 2024-05-09T06:38:26.388 INFO:tasks.workunit.client.1.smithi193.stdout:2/554: mkdir db/dbb 0 2024-05-09T06:38:26.388 INFO:tasks.workunit.client.1.smithi193.stdout:6/504: read d1/d25/d2c/d81/fb5 [340624,123623] 0 2024-05-09T06:38:26.388 INFO:tasks.workunit.client.1.smithi193.stdout:7/524: write d6/d8/d1a/d36/f3b [3056273,37919] 0 2024-05-09T06:38:26.389 INFO:tasks.workunit.client.1.smithi193.stdout:2/555: dread - db/d25/d38/f9d zero size 2024-05-09T06:38:26.391 INFO:tasks.workunit.client.1.smithi193.stdout:6/505: write d1/d25/d2c/d3c/db1/d58/f84 [3498478,67432] 0 2024-05-09T06:38:26.393 INFO:tasks.workunit.client.1.smithi193.stdout:6/506: dread - d1/d25/d2c/d81/d37/d4f/fb4 zero size 2024-05-09T06:38:26.393 INFO:tasks.workunit.client.1.smithi193.stdout:5/470: mknod d0/d13/d2c/d4d/d67/c98 0 2024-05-09T06:38:26.394 INFO:tasks.workunit.client.0.smithi063.stdout:4/919: dread d7/da3/d28/d31/d32/f8c [0,4194304] 0 2024-05-09T06:38:26.395 INFO:tasks.workunit.client.0.smithi063.stdout:8/992: truncate d9/da/d18/dc7/d58/f8e 1737962 0 2024-05-09T06:38:26.396 INFO:tasks.workunit.client.0.smithi063.stdout:8/993: chown d9/da/cb1 263 1 2024-05-09T06:38:26.398 INFO:tasks.workunit.client.0.smithi063.stdout:1/855: creat d9/d2a/d35/d1e/d9d/d3f/d59/d62/f12a x:0 0 0 2024-05-09T06:38:26.399 INFO:tasks.workunit.client.0.smithi063.stdout:0/937: write dd/d60/d119/d48/d75/fa6 [2890590,6872] 0 2024-05-09T06:38:26.403 INFO:tasks.workunit.client.0.smithi063.stdout:5/884: write d8/d17/d75/dc9/dd5/fa7 [5819745,27762] 0 2024-05-09T06:38:26.407 INFO:tasks.workunit.client.0.smithi063.stdout:5/885: dread d8/d19/d30/d46/f5d [0,4194304] 0 2024-05-09T06:38:26.407 INFO:tasks.workunit.client.0.smithi063.stdout:7/887: sync 2024-05-09T06:38:26.412 INFO:tasks.workunit.client.1.smithi193.stdout:6/507: dread d1/d25/d2c/d3c/f49 [0,4194304] 0 2024-05-09T06:38:26.418 INFO:tasks.workunit.client.0.smithi063.stdout:8/994: mknod d9/da/d18/dc7/d58/c15d 0 2024-05-09T06:38:26.422 INFO:tasks.workunit.client.0.smithi063.stdout:3/958: link d7/d70/d98/d2d/l2e d7/d70/d98/d86/l14d 0 2024-05-09T06:38:26.425 INFO:tasks.workunit.client.0.smithi063.stdout:0/938: mknod dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4/d113/c120 0 2024-05-09T06:38:26.426 INFO:tasks.workunit.client.0.smithi063.stdout:2/928: dwrite dc/dd/d45/d5a/f5e [0,4194304] 0 2024-05-09T06:38:26.428 INFO:tasks.workunit.client.0.smithi063.stdout:2/929: truncate dc/dd/d2d/d60/dd0/ffe 766421 0 2024-05-09T06:38:26.438 INFO:tasks.workunit.client.0.smithi063.stdout:4/920: getdents d7/da3/d28/d31/d32/d34/d121 0 2024-05-09T06:38:26.439 INFO:tasks.workunit.client.0.smithi063.stdout:4/921: stat d7/df/d35/d5a/d60/c76 0 2024-05-09T06:38:26.440 INFO:tasks.workunit.client.0.smithi063.stdout:3/959: rename d7/da/d2b/f43 to d7/d70/d98/dbb/d10d/f14e 0 2024-05-09T06:38:26.441 INFO:tasks.workunit.client.0.smithi063.stdout:8/995: dread d9/d41/dec/db8/fd4 [0,4194304] 0 2024-05-09T06:38:26.441 INFO:tasks.workunit.client.0.smithi063.stdout:6/920: sync 2024-05-09T06:38:26.441 INFO:tasks.workunit.client.0.smithi063.stdout:3/960: chown d7/d70/d98/d2d 23752 1 2024-05-09T06:38:26.442 INFO:tasks.workunit.client.1.smithi193.stdout:8/546: mknod d6/d1c/d18/ca4 0 2024-05-09T06:38:26.442 INFO:tasks.workunit.client.0.smithi063.stdout:0/939: mknod dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4/de1/c121 0 2024-05-09T06:38:26.443 INFO:tasks.workunit.client.0.smithi063.stdout:6/921: chown d2/d1f/d29/de2 589407 1 2024-05-09T06:38:26.443 INFO:tasks.workunit.client.0.smithi063.stdout:9/900: creat d0/d4/d26/d39/f131 x:0 0 0 2024-05-09T06:38:26.444 INFO:tasks.workunit.client.0.smithi063.stdout:0/940: readlink dd/d60/d44/l83 0 2024-05-09T06:38:26.445 INFO:tasks.workunit.client.1.smithi193.stdout:3/472: read d1/db/f58 [201794,70572] 0 2024-05-09T06:38:26.445 INFO:tasks.workunit.client.1.smithi193.stdout:9/606: unlink dd/l97 0 2024-05-09T06:38:26.448 INFO:tasks.workunit.client.0.smithi063.stdout:5/886: creat d8/d65/d9e/f128 x:0 0 0 2024-05-09T06:38:26.448 INFO:tasks.workunit.client.0.smithi063.stdout:7/888: truncate d0/d15/d49/d4f/d66/d67/d6b/fc3 653980 0 2024-05-09T06:38:26.451 INFO:tasks.workunit.client.0.smithi063.stdout:2/930: rename dc/dd/d45/d5a/d6c/d8f/d98 to dc/dd/d45/d116 0 2024-05-09T06:38:26.452 INFO:tasks.workunit.client.0.smithi063.stdout:8/996: dwrite d9/da/d53/da3/de9/f14a [0,4194304] 0 2024-05-09T06:38:26.453 INFO:tasks.workunit.client.0.smithi063.stdout:3/961: mkdir d7/d70/d98/d2d/d36/db7/d14f 0 2024-05-09T06:38:26.453 INFO:tasks.workunit.client.0.smithi063.stdout:3/962: readlink d7/d70/ld3 0 2024-05-09T06:38:26.455 INFO:tasks.workunit.client.0.smithi063.stdout:1/856: symlink d9/d2a/d35/l12b 0 2024-05-09T06:38:26.461 INFO:tasks.workunit.client.1.smithi193.stdout:1/607: link d4/d16/d17/d1a/d3a/fa3 d4/d16/d6b/d6e/fc2 0 2024-05-09T06:38:26.466 INFO:tasks.workunit.client.0.smithi063.stdout:4/922: mkdir d7/da3/d28/d31/d7b/d12a 0 2024-05-09T06:38:26.467 INFO:tasks.workunit.client.1.smithi193.stdout:0/496: mknod d18/c98 0 2024-05-09T06:38:26.469 INFO:tasks.workunit.client.1.smithi193.stdout:3/473: symlink d1/d2d/d51/laf 0 2024-05-09T06:38:26.471 INFO:tasks.workunit.client.0.smithi063.stdout:7/889: rmdir d0/d15/d49/d54/d9f 39 2024-05-09T06:38:26.472 INFO:tasks.workunit.client.1.smithi193.stdout:9/607: read dd/f11 [2248706,39593] 0 2024-05-09T06:38:26.474 INFO:tasks.workunit.client.1.smithi193.stdout:2/556: symlink db/lbc 0 2024-05-09T06:38:26.476 INFO:tasks.workunit.client.0.smithi063.stdout:5/887: dread d8/d65/f83 [0,4194304] 0 2024-05-09T06:38:26.478 INFO:tasks.workunit.client.0.smithi063.stdout:2/931: truncate dc/dd/d2d/f3e 4027281 0 2024-05-09T06:38:26.479 INFO:tasks.workunit.client.0.smithi063.stdout:2/932: write dc/dd/d2d/fa0 [2791060,95935] 0 2024-05-09T06:38:26.479 INFO:tasks.workunit.client.0.smithi063.stdout:8/997: dread d9/da/d45/d114/d139/f52 [0,4194304] 0 2024-05-09T06:38:26.480 INFO:tasks.workunit.client.1.smithi193.stdout:2/557: dread db/d25/f49 [0,4194304] 0 2024-05-09T06:38:26.481 INFO:tasks.workunit.client.1.smithi193.stdout:1/608: dread d4/d16/d35/fa9 [0,4194304] 0 2024-05-09T06:38:26.482 INFO:tasks.workunit.client.0.smithi063.stdout:3/963: creat d7/da/de4/de8/f150 x:0 0 0 2024-05-09T06:38:26.486 INFO:tasks.workunit.client.0.smithi063.stdout:1/857: chown d9/d2a/d35/d1e/d9d/d3f/d59/d62/d7e 4 1 2024-05-09T06:38:26.488 INFO:tasks.workunit.client.1.smithi193.stdout:5/471: write d0/d13/d2c/d37/d5b/d4e/f78 [1103424,78247] 0 2024-05-09T06:38:26.489 INFO:tasks.workunit.client.0.smithi063.stdout:9/901: write d0/d4/d26/d39/d91/fa7 [1647383,101552] 0 2024-05-09T06:38:26.489 INFO:tasks.workunit.client.0.smithi063.stdout:2/933: dwrite dc/dd/d45/d5a/d6c/d8f/dbf/fd9 [0,4194304] 0 2024-05-09T06:38:26.489 INFO:tasks.workunit.client.1.smithi193.stdout:5/472: chown d0/d13/d2c/d37/c31 3539 1 2024-05-09T06:38:26.490 INFO:tasks.workunit.client.0.smithi063.stdout:8/998: dread d9/da/d18/d24/f27 [0,4194304] 0 2024-05-09T06:38:26.492 INFO:tasks.workunit.client.0.smithi063.stdout:5/888: mkdir d8/d19/d115/deb/d129 0 2024-05-09T06:38:26.492 INFO:tasks.workunit.client.0.smithi063.stdout:8/999: dread - d9/d41/dec/db8/f157 zero size 2024-05-09T06:38:26.494 INFO:tasks.workunit.client.1.smithi193.stdout:0/497: fsync d18/d1d/f1f 0 2024-05-09T06:38:26.498 INFO:tasks.workunit.client.1.smithi193.stdout:3/474: truncate d1/db/f6e 5213824 0 2024-05-09T06:38:26.498 INFO:tasks.workunit.client.1.smithi193.stdout:0/498: write d18/d4c/d47/d5e/f80 [953714,52560] 0 2024-05-09T06:38:26.499 INFO:tasks.workunit.client.1.smithi193.stdout:5/473: dwrite d0/d13/d2c/d37/d2b/d47/d66/d3a/f52 [0,4194304] 0 2024-05-09T06:38:26.503 INFO:tasks.workunit.client.0.smithi063.stdout:6/922: getdents d2/d89 0 2024-05-09T06:38:26.504 INFO:tasks.workunit.client.0.smithi063.stdout:6/923: chown d2/d1f 1 1 2024-05-09T06:38:26.504 INFO:tasks.workunit.client.1.smithi193.stdout:4/616: link d1a/d22/d43/c4f d1a/cc0 0 2024-05-09T06:38:26.504 INFO:tasks.workunit.client.0.smithi063.stdout:6/924: fsync d2/d20/d26/fde 0 2024-05-09T06:38:26.508 INFO:tasks.workunit.client.1.smithi193.stdout:7/525: link d6/d8/d29/c73 d6/d8/d1a/d36/d4d/d60/cb2 0 2024-05-09T06:38:26.508 INFO:tasks.workunit.client.1.smithi193.stdout:0/499: dwrite d18/d27/f41 [0,4194304] 0 2024-05-09T06:38:26.510 INFO:tasks.workunit.client.0.smithi063.stdout:1/858: write d9/d2a/d35/d1e/d9d/fe8 [604376,31492] 0 2024-05-09T06:38:26.511 INFO:tasks.workunit.client.0.smithi063.stdout:1/859: readlink d9/d2a/d35/d1e/d9d/l124 0 2024-05-09T06:38:26.516 INFO:tasks.workunit.client.0.smithi063.stdout:5/889: read - d8/d19/d30/ff0 zero size 2024-05-09T06:38:26.517 INFO:tasks.workunit.client.1.smithi193.stdout:8/547: truncate d6/d5b/f64 2478642 0 2024-05-09T06:38:26.520 INFO:tasks.workunit.client.1.smithi193.stdout:6/508: mknod d1/de/d11/d18/cb6 0 2024-05-09T06:38:26.526 INFO:tasks.workunit.client.0.smithi063.stdout:9/902: symlink d0/d4/d57/d60/d76/l132 0 2024-05-09T06:38:26.527 INFO:tasks.workunit.client.1.smithi193.stdout:9/608: dwrite dd/df/d31/d7e/d37/f7c [0,4194304] 0 2024-05-09T06:38:26.528 INFO:tasks.workunit.client.0.smithi063.stdout:2/934: mknod dc/d36/d43/d46/d63/c117 0 2024-05-09T06:38:26.528 INFO:tasks.workunit.client.0.smithi063.stdout:2/935: chown dc/dd/d2d/d60/dd0 15594 1 2024-05-09T06:38:26.528 INFO:tasks.workunit.client.0.smithi063.stdout:2/936: stat dc/d32/lc9 0 2024-05-09T06:38:26.529 INFO:tasks.workunit.client.1.smithi193.stdout:9/609: stat dd/d1e/dc2/d49/dc1 0 2024-05-09T06:38:26.530 INFO:tasks.workunit.client.0.smithi063.stdout:5/890: mknod d8/d17/d75/dc9/dd5/c12a 0 2024-05-09T06:38:26.532 INFO:tasks.workunit.client.1.smithi193.stdout:0/500: creat d18/d4c/d64/f99 x:0 0 0 2024-05-09T06:38:26.534 INFO:tasks.workunit.client.1.smithi193.stdout:2/558: mkdir db/d37/d6d/d80/daa/db4/dbd 0 2024-05-09T06:38:26.535 INFO:tasks.workunit.client.0.smithi063.stdout:6/925: mkdir d2/d4/d12e 0 2024-05-09T06:38:26.538 INFO:tasks.workunit.client.0.smithi063.stdout:4/923: dwrite d7/d50/fc4 [0,4194304] 0 2024-05-09T06:38:26.540 INFO:tasks.workunit.client.1.smithi193.stdout:8/548: mkdir d6/d1c/d1b/d5d/d84/d87/da5 0 2024-05-09T06:38:26.542 INFO:tasks.workunit.client.0.smithi063.stdout:1/860: truncate d9/fe 64095 0 2024-05-09T06:38:26.543 INFO:tasks.workunit.client.1.smithi193.stdout:6/509: creat d1/d25/fb7 x:0 0 0 2024-05-09T06:38:26.544 INFO:tasks.workunit.client.0.smithi063.stdout:6/926: dwrite d2/d1f/f4c [0,4194304] 0 2024-05-09T06:38:26.549 INFO:tasks.workunit.client.1.smithi193.stdout:3/475: mknod d1/cb0 0 2024-05-09T06:38:26.551 INFO:tasks.workunit.client.0.smithi063.stdout:3/964: getdents d7/da/de4/de8 0 2024-05-09T06:38:26.552 INFO:tasks.workunit.client.0.smithi063.stdout:9/903: dwrite d0/d4/d57/d60/d76/fa9 [0,4194304] 0 2024-05-09T06:38:26.554 INFO:tasks.workunit.client.0.smithi063.stdout:5/891: dread d8/d17/f3d [0,4194304] 0 2024-05-09T06:38:26.559 INFO:tasks.workunit.client.0.smithi063.stdout:1/861: dwrite d9/d2a/d39/d5d/d75/fcf [0,4194304] 0 2024-05-09T06:38:26.559 INFO:tasks.workunit.client.1.smithi193.stdout:5/474: fdatasync d0/d13/d2c/d4d/d67/f93 0 2024-05-09T06:38:26.560 INFO:tasks.workunit.client.1.smithi193.stdout:4/617: rename d1a/d22/d43/f8b to d1a/d67/fc1 0 2024-05-09T06:38:26.561 INFO:tasks.workunit.client.0.smithi063.stdout:9/904: dread d0/d4/d57/d60/f7c [0,4194304] 0 2024-05-09T06:38:26.562 INFO:tasks.workunit.client.1.smithi193.stdout:0/501: symlink d18/d4c/d64/d66/l9a 0 2024-05-09T06:38:26.562 INFO:tasks.workunit.client.0.smithi063.stdout:4/924: creat d7/d3d/d73/f12b x:0 0 0 2024-05-09T06:38:26.567 INFO:tasks.workunit.client.0.smithi063.stdout:6/927: mknod d2/d1f/d29/c12f 0 2024-05-09T06:38:26.568 INFO:tasks.workunit.client.1.smithi193.stdout:1/609: link d4/d16/d17/d1a/c22 d4/d16/d6b/d6e/cc3 0 2024-05-09T06:38:26.568 INFO:tasks.workunit.client.1.smithi193.stdout:8/549: creat d6/d1c/d18/d66/d93/fa6 x:0 0 0 2024-05-09T06:38:26.569 INFO:tasks.workunit.client.0.smithi063.stdout:0/941: sync 2024-05-09T06:38:26.569 INFO:tasks.workunit.client.1.smithi193.stdout:6/510: rmdir d1/de/d11/d20 39 2024-05-09T06:38:26.569 INFO:tasks.workunit.client.1.smithi193.stdout:9/610: dread dd/d1e/dc2/d49/f51 [0,4194304] 0 2024-05-09T06:38:26.569 INFO:tasks.workunit.client.1.smithi193.stdout:6/511: stat d1/d25/d4c 0 2024-05-09T06:38:26.573 INFO:tasks.workunit.client.1.smithi193.stdout:1/610: dwrite d4/d14/f75 [0,4194304] 0 2024-05-09T06:38:26.579 INFO:tasks.workunit.client.1.smithi193.stdout:5/475: symlink d0/d13/d2c/d37/d2b/d47/d66/d3a/l99 0 2024-05-09T06:38:26.580 INFO:tasks.workunit.client.1.smithi193.stdout:3/476: dread d1/f6d [0,4194304] 0 2024-05-09T06:38:26.581 INFO:tasks.workunit.client.0.smithi063.stdout:5/892: creat d8/d19/d30/db1/f12b x:0 0 0 2024-05-09T06:38:26.581 INFO:tasks.workunit.client.1.smithi193.stdout:2/559: rename db/df/d54/c81 to db/d25/d38/d5b/cbe 0 2024-05-09T06:38:26.582 INFO:tasks.workunit.client.1.smithi193.stdout:5/476: read d0/f5c [314087,90026] 0 2024-05-09T06:38:26.583 INFO:tasks.workunit.client.0.smithi063.stdout:4/925: creat d7/d3d/d73/de5/f12c x:0 0 0 2024-05-09T06:38:26.584 INFO:tasks.workunit.client.1.smithi193.stdout:2/560: write db/d37/d6d/d80/f24 [4886863,105343] 0 2024-05-09T06:38:26.587 INFO:tasks.workunit.client.1.smithi193.stdout:8/550: rmdir d6/d1c/d18/d2f/d6a 39 2024-05-09T06:38:26.589 INFO:tasks.workunit.client.0.smithi063.stdout:0/942: stat dd/d1a/d2c/d51/d100 0 2024-05-09T06:38:26.590 INFO:tasks.workunit.client.1.smithi193.stdout:6/512: creat d1/de/fb8 x:0 0 0 2024-05-09T06:38:26.590 INFO:tasks.workunit.client.0.smithi063.stdout:4/926: dwrite d7/df/d35/dd9/f103 [0,4194304] 0 2024-05-09T06:38:26.590 INFO:tasks.workunit.client.1.smithi193.stdout:6/513: fdatasync d1/d25/d2c/d76/d69/f9c 0 2024-05-09T06:38:26.591 INFO:tasks.workunit.client.1.smithi193.stdout:9/611: symlink dd/d1e/dc2/d49/dbd/d5b/d8a/lcc 0 2024-05-09T06:38:26.599 INFO:tasks.workunit.client.1.smithi193.stdout:1/611: dwrite d4/d14/f44 [0,4194304] 0 2024-05-09T06:38:26.603 INFO:tasks.workunit.client.1.smithi193.stdout:4/618: symlink d1a/d24/d73/d94/d7c/lc2 0 2024-05-09T06:38:26.604 INFO:tasks.workunit.client.0.smithi063.stdout:2/937: link dc/dd/d2d/f77 dc/dd/f118 0 2024-05-09T06:38:26.612 INFO:tasks.workunit.client.1.smithi193.stdout:2/561: creat db/d37/d6d/d80/fbf x:0 0 0 2024-05-09T06:38:26.616 INFO:tasks.workunit.client.0.smithi063.stdout:7/890: sync 2024-05-09T06:38:26.618 INFO:tasks.workunit.client.0.smithi063.stdout:3/965: truncate d7/d4c/d25/f6c 313955 0 2024-05-09T06:38:26.618 INFO:tasks.workunit.client.0.smithi063.stdout:1/862: creat d9/d2a/d35/d1e/f12c x:0 0 0 2024-05-09T06:38:26.618 INFO:tasks.workunit.client.0.smithi063.stdout:6/928: write d2/df/d91/ddf/fe1 [925071,33436] 0 2024-05-09T06:38:26.619 INFO:tasks.workunit.client.1.smithi193.stdout:9/612: read dd/df/d31/d7e/d37/f79 [56831,11790] 0 2024-05-09T06:38:26.620 INFO:tasks.workunit.client.0.smithi063.stdout:5/893: write d8/d17/d18/f2d [1426401,52121] 0 2024-05-09T06:38:26.621 INFO:tasks.workunit.client.0.smithi063.stdout:0/943: dread dd/d1a/d2c/d51/d76/d97/fa5 [0,4194304] 0 2024-05-09T06:38:26.621 INFO:tasks.workunit.client.0.smithi063.stdout:9/905: write d0/d8/ffa [894310,29126] 0 2024-05-09T06:38:26.622 INFO:tasks.workunit.client.1.smithi193.stdout:3/477: write d1/db/f19 [7402956,121222] 0 2024-05-09T06:38:26.625 INFO:tasks.workunit.client.1.smithi193.stdout:1/612: rename d4/d16/d60/d4f/d72/d96 to d4/d16/d6b/d89/dc4 0 2024-05-09T06:38:26.628 INFO:tasks.workunit.client.1.smithi193.stdout:0/502: link d18/d4c/d64/d66/l9a d18/d4c/d64/d74/l9b 0 2024-05-09T06:38:26.629 INFO:tasks.workunit.client.1.smithi193.stdout:5/477: write d0/d13/d2c/d37/d2b/d47/d66/f23 [2529223,103558] 0 2024-05-09T06:38:26.631 INFO:tasks.workunit.client.1.smithi193.stdout:5/478: read d0/d13/d2c/d37/f68 [307305,76121] 0 2024-05-09T06:38:26.633 INFO:tasks.workunit.client.0.smithi063.stdout:4/927: dwrite d7/d3d/fa7 [0,4194304] 0 2024-05-09T06:38:26.635 INFO:tasks.workunit.client.0.smithi063.stdout:4/928: chown d7/da3/d28/d31/d71/d98/fef 39329 1 2024-05-09T06:38:26.636 INFO:tasks.workunit.client.1.smithi193.stdout:8/551: dwrite d6/d1c/d1b/d29/f91 [0,4194304] 0 2024-05-09T06:38:26.637 INFO:tasks.workunit.client.0.smithi063.stdout:4/929: truncate d7/da3/d28/d31/d32/d34/ff5 841148 0 2024-05-09T06:38:26.640 INFO:tasks.workunit.client.1.smithi193.stdout:3/478: truncate d1/f6d 1018108 0 2024-05-09T06:38:26.641 INFO:tasks.workunit.client.1.smithi193.stdout:1/613: mkdir d4/d16/d17/d1a/dc5 0 2024-05-09T06:38:26.643 INFO:tasks.workunit.client.1.smithi193.stdout:8/552: dwrite d6/d1c/f1e [0,4194304] 0 2024-05-09T06:38:26.646 INFO:tasks.workunit.client.1.smithi193.stdout:4/619: mkdir d1a/d22/dc3 0 2024-05-09T06:38:26.649 INFO:tasks.workunit.client.1.smithi193.stdout:0/503: symlink d18/d4c/d47/l9c 0 2024-05-09T06:38:26.650 INFO:tasks.workunit.client.1.smithi193.stdout:0/504: readlink d18/d27/l8a 0 2024-05-09T06:38:26.651 INFO:tasks.workunit.client.1.smithi193.stdout:0/505: read - d18/d4c/d47/d54/f97 zero size 2024-05-09T06:38:26.659 INFO:tasks.workunit.client.1.smithi193.stdout:9/613: dwrite dd/df/d31/d7e/f60 [0,4194304] 0 2024-05-09T06:38:26.659 INFO:tasks.workunit.client.1.smithi193.stdout:5/479: write d0/d13/d2c/d37/d2b/d47/d66/f83 [3867914,122573] 0 2024-05-09T06:38:26.662 INFO:tasks.workunit.client.1.smithi193.stdout:6/514: rmdir d1/de/d11/d18/d94/dac/db2 0 2024-05-09T06:38:26.662 INFO:tasks.workunit.client.1.smithi193.stdout:0/506: dwrite d18/d4c/d5f/f8c [0,4194304] 0 2024-05-09T06:38:26.665 INFO:tasks.workunit.client.1.smithi193.stdout:6/515: truncate d1/d25/d2c/d3c/db1/f3f 4556539 0 2024-05-09T06:38:26.672 INFO:tasks.workunit.client.1.smithi193.stdout:4/620: chown d1a/d67/d82/dab/fbc 13803765 1 2024-05-09T06:38:26.674 INFO:tasks.workunit.client.1.smithi193.stdout:6/516: dwrite d1/d25/d2c/d3c/f99 [0,4194304] 0 2024-05-09T06:38:26.682 INFO:tasks.workunit.client.1.smithi193.stdout:9/614: fdatasync dd/d1e/f88 0 2024-05-09T06:38:26.684 INFO:tasks.workunit.client.1.smithi193.stdout:1/614: dread d4/d16/d17/d1a/f66 [0,4194304] 0 2024-05-09T06:38:26.685 INFO:tasks.workunit.client.1.smithi193.stdout:1/615: readlink d4/d76/lbe 0 2024-05-09T06:38:26.685 INFO:tasks.workunit.client.1.smithi193.stdout:2/562: truncate db/f1e 3853274 0 2024-05-09T06:38:26.686 INFO:tasks.workunit.client.1.smithi193.stdout:0/507: mkdir d18/d4c/d64/d66/d9d 0 2024-05-09T06:38:26.690 INFO:tasks.workunit.client.1.smithi193.stdout:4/621: symlink d1a/d22/d8e/lc4 0 2024-05-09T06:38:26.692 INFO:tasks.workunit.client.1.smithi193.stdout:9/615: readlink dd/df/d31/d7e/l94 0 2024-05-09T06:38:26.694 INFO:tasks.workunit.client.1.smithi193.stdout:9/616: dread - dd/d1e/dc2/d49/dbd/fc9 zero size 2024-05-09T06:38:26.694 INFO:tasks.workunit.client.1.smithi193.stdout:5/480: mkdir d0/d13/d2c/d37/d2b/d47/d60/d9a 0 2024-05-09T06:38:26.702 INFO:tasks.workunit.client.1.smithi193.stdout:2/563: mknod db/d37/d6d/cc0 0 2024-05-09T06:38:26.704 INFO:tasks.workunit.client.1.smithi193.stdout:5/481: dread d0/d13/f3d [0,4194304] 0 2024-05-09T06:38:26.706 INFO:tasks.workunit.client.1.smithi193.stdout:5/482: write d0/d13/d2c/d37/d2b/d47/d6f/f7d [59144,100410] 0 2024-05-09T06:38:26.708 INFO:tasks.workunit.client.1.smithi193.stdout:5/483: chown d0/d13/d2c/d37/d74/f91 1 1 2024-05-09T06:38:26.708 INFO:tasks.workunit.client.1.smithi193.stdout:3/479: rename d1/d2d/d51/c74 to d1/db/cb1 0 2024-05-09T06:38:26.709 INFO:tasks.workunit.client.1.smithi193.stdout:5/484: stat d0/d13/d2c/d37/d2b/d47/d66/f23 0 2024-05-09T06:38:26.709 INFO:tasks.workunit.client.1.smithi193.stdout:3/480: chown d1/db/d2f/fab 596246249 1 2024-05-09T06:38:26.710 INFO:tasks.workunit.client.1.smithi193.stdout:3/481: read - d1/db/d59/fa9 zero size 2024-05-09T06:38:26.712 INFO:tasks.workunit.client.1.smithi193.stdout:4/622: truncate d1a/d24/d73/d94/d7c/f32 2106548 0 2024-05-09T06:38:26.714 INFO:tasks.workunit.client.1.smithi193.stdout:4/623: readlink d1a/d24/d73/d94/d7c/d5b/lbb 0 2024-05-09T06:38:26.717 INFO:tasks.workunit.client.1.smithi193.stdout:3/482: dwrite d1/db/d59/f9f [0,4194304] 0 2024-05-09T06:38:26.719 INFO:tasks.workunit.client.1.smithi193.stdout:3/483: stat d1/d2d/d51 0 2024-05-09T06:38:26.719 INFO:tasks.workunit.client.1.smithi193.stdout:9/617: creat dd/d1e/fcd x:0 0 0 2024-05-09T06:38:26.727 INFO:tasks.workunit.client.0.smithi063.stdout:3/966: creat d7/da/d21/d30/f151 x:0 0 0 2024-05-09T06:38:26.727 INFO:tasks.workunit.client.1.smithi193.stdout:2/564: creat db/d25/d38/d5b/fc1 x:0 0 0 2024-05-09T06:38:26.730 INFO:tasks.workunit.client.1.smithi193.stdout:6/517: creat d1/d25/d2c/fb9 x:0 0 0 2024-05-09T06:38:26.731 INFO:tasks.workunit.client.0.smithi063.stdout:6/929: symlink d2/df/d59/d99/l130 0 2024-05-09T06:38:26.732 INFO:tasks.workunit.client.1.smithi193.stdout:5/485: rmdir d0/d13/d2c/d37/d2b/d47/d60 39 2024-05-09T06:38:26.736 INFO:tasks.workunit.client.0.smithi063.stdout:5/894: dread d8/d17/f6a [0,4194304] 0 2024-05-09T06:38:26.737 INFO:tasks.workunit.client.0.smithi063.stdout:0/944: rmdir dd/d1a/d2c/d42 39 2024-05-09T06:38:26.741 INFO:tasks.workunit.client.0.smithi063.stdout:9/906: creat d0/d4/d26/d39/d48/d80/d84/dd8/ddf/f133 x:0 0 0 2024-05-09T06:38:26.742 INFO:tasks.workunit.client.0.smithi063.stdout:5/895: dwrite d8/d19/d30/d74/f117 [0,4194304] 0 2024-05-09T06:38:26.743 INFO:tasks.workunit.client.0.smithi063.stdout:5/896: dread - d8/d17/d18/f112 zero size 2024-05-09T06:38:26.746 INFO:tasks.workunit.client.1.smithi193.stdout:2/565: creat db/d9b/fc2 x:0 0 0 2024-05-09T06:38:26.746 INFO:tasks.workunit.client.1.smithi193.stdout:5/486: dread d0/d13/d2c/d37/d5b/d4e/f78 [0,4194304] 0 2024-05-09T06:38:26.750 INFO:tasks.workunit.client.0.smithi063.stdout:3/967: symlink d7/da/d21/d30/l152 0 2024-05-09T06:38:26.753 INFO:tasks.workunit.client.0.smithi063.stdout:6/930: mknod d2/d1f/d29/de2/c131 0 2024-05-09T06:38:26.757 INFO:tasks.workunit.client.0.smithi063.stdout:2/938: rename dc/d36/d43/d46/fcc to dc/d36/d43/d46/d49/dbb/f119 0 2024-05-09T06:38:26.758 INFO:tasks.workunit.client.1.smithi193.stdout:1/616: dwrite d4/f37 [0,4194304] 0 2024-05-09T06:38:26.758 INFO:tasks.workunit.client.0.smithi063.stdout:2/939: chown dc/dd/d2d/d60/dd0/d90/fef 0 1 2024-05-09T06:38:26.761 INFO:tasks.workunit.client.0.smithi063.stdout:9/907: mknod d0/d4/d7d/d105/c134 0 2024-05-09T06:38:26.763 INFO:tasks.workunit.client.0.smithi063.stdout:5/897: unlink d8/d19/d30/db1/le7 0 2024-05-09T06:38:26.764 INFO:tasks.workunit.client.0.smithi063.stdout:5/898: chown d8/d65/dbd/f114 3 1 2024-05-09T06:38:26.765 INFO:tasks.workunit.client.1.smithi193.stdout:0/508: getdents d18/d4c/d47/d55 0 2024-05-09T06:38:26.765 INFO:tasks.workunit.client.1.smithi193.stdout:9/618: dwrite dd/d1e/d1f/d48/f5a [0,4194304] 0 2024-05-09T06:38:26.766 INFO:tasks.workunit.client.0.smithi063.stdout:4/930: write d7/f62 [129086,71106] 0 2024-05-09T06:38:26.769 INFO:tasks.workunit.client.0.smithi063.stdout:1/863: creat d9/d2a/d39/f12d x:0 0 0 2024-05-09T06:38:26.772 INFO:tasks.workunit.client.0.smithi063.stdout:6/931: readlink d2/df/d59/ld7 0 2024-05-09T06:38:26.774 INFO:tasks.workunit.client.1.smithi193.stdout:9/619: dwrite fa [4194304,4194304] 0 2024-05-09T06:38:26.774 INFO:tasks.workunit.client.0.smithi063.stdout:6/932: write d2/df/d91/f109 [431967,110884] 0 2024-05-09T06:38:26.775 INFO:tasks.workunit.client.1.smithi193.stdout:3/484: truncate d1/db/d2f/d78/f8b 2407813 0 2024-05-09T06:38:26.775 INFO:tasks.workunit.client.1.smithi193.stdout:3/485: readlink d1/db/d59/l3c 0 2024-05-09T06:38:26.777 INFO:tasks.workunit.client.0.smithi063.stdout:7/891: dwrite d0/d15/d49/d4f/d66/d67/d6b/fc3 [0,4194304] 0 2024-05-09T06:38:26.782 INFO:tasks.workunit.client.0.smithi063.stdout:5/899: symlink d8/d17/dfe/l12c 0 2024-05-09T06:38:26.783 INFO:tasks.workunit.client.0.smithi063.stdout:3/968: mkdir d7/d22/dfc/d140/d153 0 2024-05-09T06:38:26.784 INFO:tasks.workunit.client.1.smithi193.stdout:0/509: creat d18/d4c/d64/d66/f9e x:0 0 0 2024-05-09T06:38:26.784 INFO:tasks.workunit.client.0.smithi063.stdout:3/969: chown d7/d4c/d25/d10b/l128 1 1 2024-05-09T06:38:26.794 INFO:tasks.workunit.client.1.smithi193.stdout:1/617: symlink d4/lc6 0 2024-05-09T06:38:26.795 INFO:tasks.workunit.client.0.smithi063.stdout:2/940: mknod dc/d36/d43/d46/d49/dbb/c11a 0 2024-05-09T06:38:26.797 INFO:tasks.workunit.client.1.smithi193.stdout:2/566: rename db/d25/d38/fab to db/d25/d38/d40/d8b/fc3 0 2024-05-09T06:38:26.798 INFO:tasks.workunit.client.0.smithi063.stdout:5/900: creat d8/d17/d10b/f12d x:0 0 0 2024-05-09T06:38:26.799 INFO:tasks.workunit.client.0.smithi063.stdout:1/864: write d9/d2a/d39/d5d/f9a [572857,63880] 0 2024-05-09T06:38:26.802 INFO:tasks.workunit.client.1.smithi193.stdout:0/510: creat d18/d65/f9f x:0 0 0 2024-05-09T06:38:26.804 INFO:tasks.workunit.client.1.smithi193.stdout:6/518: getdents d1 0 2024-05-09T06:38:26.804 INFO:tasks.workunit.client.1.smithi193.stdout:9/620: write dd/d1e/f88 [333682,128850] 0 2024-05-09T06:38:26.805 INFO:tasks.workunit.client.0.smithi063.stdout:1/865: dread d9/d2a/d35/d1e/d9d/fe8 [0,4194304] 0 2024-05-09T06:38:26.806 INFO:tasks.workunit.client.0.smithi063.stdout:6/933: mkdir d2/df/dd2/dfe/d132 0 2024-05-09T06:38:26.807 INFO:tasks.workunit.client.1.smithi193.stdout:3/486: symlink d1/db/lb2 0 2024-05-09T06:38:26.807 INFO:tasks.workunit.client.0.smithi063.stdout:1/866: write d9/d2a/d3c/d4c/d83/d88/fe3 [2084765,107314] 0 2024-05-09T06:38:26.807 INFO:tasks.workunit.client.0.smithi063.stdout:7/892: getdents d0/d15/d89/d10c/de7/d117/d125 0 2024-05-09T06:38:26.808 INFO:tasks.workunit.client.1.smithi193.stdout:3/487: read - d1/db/d2f/f9d zero size 2024-05-09T06:38:26.808 INFO:tasks.workunit.client.1.smithi193.stdout:3/488: stat d1/d2d/d51/l28 0 2024-05-09T06:38:26.808 INFO:tasks.workunit.client.1.smithi193.stdout:4/624: getdents d1a/d6d 0 2024-05-09T06:38:26.809 INFO:tasks.workunit.client.0.smithi063.stdout:1/867: write d9/d2a/d39/f105 [2233898,12920] 0 2024-05-09T06:38:26.810 INFO:tasks.workunit.client.0.smithi063.stdout:0/945: rename dd/d60/d44/d90/cb7 to dd/d1a/d2c/d51/d72/d9e/c122 0 2024-05-09T06:38:26.810 INFO:tasks.workunit.client.0.smithi063.stdout:5/901: symlink d8/d17/l12e 0 2024-05-09T06:38:26.814 INFO:tasks.workunit.client.0.smithi063.stdout:6/934: symlink d2/d20/d95/dec/l133 0 2024-05-09T06:38:26.815 INFO:tasks.workunit.client.1.smithi193.stdout:9/621: mknod dd/d1e/dc2/d49/cce 0 2024-05-09T06:38:26.818 INFO:tasks.workunit.client.0.smithi063.stdout:7/893: symlink d0/d15/d89/d10c/de7/d117/l12f 0 2024-05-09T06:38:26.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/868: readlink d9/d46/dfe/d94/lb5 0 2024-05-09T06:38:26.821 INFO:tasks.workunit.client.0.smithi063.stdout:6/935: dwrite d2/d20/f44 [0,4194304] 0 2024-05-09T06:38:26.821 INFO:tasks.workunit.client.1.smithi193.stdout:4/625: dread f11 [0,4194304] 0 2024-05-09T06:38:26.824 INFO:tasks.workunit.client.0.smithi063.stdout:5/902: fdatasync d8/d17/d18/f51 0 2024-05-09T06:38:26.830 INFO:tasks.workunit.client.0.smithi063.stdout:3/970: mknod d7/da/d2b/d66/d74/c154 0 2024-05-09T06:38:26.831 INFO:tasks.workunit.client.0.smithi063.stdout:3/971: chown d7/d70/d98/d2d/d36/fcd 20248622 1 2024-05-09T06:38:26.832 INFO:tasks.workunit.client.1.smithi193.stdout:5/487: rename d0/d13/d2c/d37/f68 to d0/f9b 0 2024-05-09T06:38:26.835 INFO:tasks.workunit.client.0.smithi063.stdout:7/894: truncate d0/d15/d89/d10c/fb6 3612707 0 2024-05-09T06:38:26.835 INFO:tasks.workunit.client.1.smithi193.stdout:6/519: write d1/d25/d2c/d81/d37/f5b [158972,31032] 0 2024-05-09T06:38:26.836 INFO:tasks.workunit.client.1.smithi193.stdout:4/626: fsync d1a/d24/d73/d94/d7c/f89 0 2024-05-09T06:38:26.838 INFO:tasks.workunit.client.0.smithi063.stdout:9/908: rename d0/d3b/da0/c8e to d0/d8/d2d/c135 0 2024-05-09T06:38:26.841 INFO:tasks.workunit.client.0.smithi063.stdout:6/936: unlink d2/d1f/d29/de2/l8f 0 2024-05-09T06:38:26.841 INFO:tasks.workunit.client.1.smithi193.stdout:9/622: write dd/df/d31/d7e/d71/f84 [453681,63337] 0 2024-05-09T06:38:26.843 INFO:tasks.workunit.client.1.smithi193.stdout:1/618: rename d4/d16/d6b/d89/c8e to d4/d16/cc7 0 2024-05-09T06:38:26.845 INFO:tasks.workunit.client.0.smithi063.stdout:1/869: truncate d9/d2a/d3c/d4c/f92 405389 0 2024-05-09T06:38:26.847 INFO:tasks.workunit.client.1.smithi193.stdout:5/488: creat d0/d13/d2c/d37/d5b/d4e/d5d/f9c x:0 0 0 2024-05-09T06:38:26.848 INFO:tasks.workunit.client.1.smithi193.stdout:9/623: dwrite dd/d1e/d1f/d48/fc8 [0,4194304] 0 2024-05-09T06:38:26.849 INFO:tasks.workunit.client.1.smithi193.stdout:0/511: getdents d18/d4c/d47 0 2024-05-09T06:38:26.851 INFO:tasks.workunit.client.1.smithi193.stdout:9/624: write dd/d1e/d1f/f6f [165524,69524] 0 2024-05-09T06:38:26.853 INFO:tasks.workunit.client.0.smithi063.stdout:5/903: unlink d8/d17/d10b/f12d 0 2024-05-09T06:38:26.856 INFO:tasks.workunit.client.0.smithi063.stdout:3/972: creat d7/d22/d28/f155 x:0 0 0 2024-05-09T06:38:26.860 INFO:tasks.workunit.client.1.smithi193.stdout:6/520: symlink d1/d25/d2c/d76/d69/lba 0 2024-05-09T06:38:26.862 INFO:tasks.workunit.client.0.smithi063.stdout:2/941: rename dc/lb1 to dc/dd/d2d/d60/dd0/d90/dc1/l11b 0 2024-05-09T06:38:26.864 INFO:tasks.workunit.client.1.smithi193.stdout:3/489: getdents d1/db/d2f 0 2024-05-09T06:38:26.865 INFO:tasks.workunit.client.0.smithi063.stdout:9/909: creat d0/da5/f136 x:0 0 0 2024-05-09T06:38:26.869 INFO:tasks.workunit.client.1.smithi193.stdout:5/489: mkdir d0/d13/d2c/d37/d5b/d4e/d9d 0 2024-05-09T06:38:26.873 INFO:tasks.workunit.client.1.smithi193.stdout:4/627: dread f7 [4194304,4194304] 0 2024-05-09T06:38:26.874 INFO:tasks.workunit.client.1.smithi193.stdout:4/628: dread - d1a/d6b/d60/fbe zero size 2024-05-09T06:38:26.876 INFO:tasks.workunit.client.0.smithi063.stdout:3/973: creat d7/d4c/d25/f156 x:0 0 0 2024-05-09T06:38:26.877 INFO:tasks.workunit.client.0.smithi063.stdout:7/895: mknod d0/d15/d49/d4f/d66/c130 0 2024-05-09T06:38:26.879 INFO:tasks.workunit.client.1.smithi193.stdout:6/521: creat d1/d25/d2c/d3c/fbb x:0 0 0 2024-05-09T06:38:26.879 INFO:tasks.workunit.client.1.smithi193.stdout:6/522: stat d1/d25/d2c/d81/d37/d4f 0 2024-05-09T06:38:26.880 INFO:tasks.workunit.client.0.smithi063.stdout:2/942: creat dc/d36/d43/d46/ded/f11c x:0 0 0 2024-05-09T06:38:26.882 INFO:tasks.workunit.client.0.smithi063.stdout:0/946: getdents dd/d60/d44/d90/da4 0 2024-05-09T06:38:26.884 INFO:tasks.workunit.client.0.smithi063.stdout:6/937: fsync d2/d1f/d29/de2/d4e/f6b 0 2024-05-09T06:38:26.885 INFO:tasks.workunit.client.0.smithi063.stdout:5/904: dwrite d8/d65/d9e/ff5 [0,4194304] 0 2024-05-09T06:38:26.885 INFO:tasks.workunit.client.1.smithi193.stdout:5/490: dread d0/d13/d2c/d37/d5b/d4e/f78 [0,4194304] 0 2024-05-09T06:38:26.885 INFO:tasks.workunit.client.1.smithi193.stdout:9/625: symlink dd/d1e/d1f/d48/d55/d41/lcf 0 2024-05-09T06:38:26.885 INFO:tasks.workunit.client.0.smithi063.stdout:0/947: chown dd/d60/d119/d48/d75/l99 23327184 1 2024-05-09T06:38:26.886 INFO:tasks.workunit.client.1.smithi193.stdout:6/523: mkdir d1/de/dbc 0 2024-05-09T06:38:26.886 INFO:tasks.workunit.client.0.smithi063.stdout:6/938: dread - d2/df/d91/dd8/dd9/d10e/f129 zero size 2024-05-09T06:38:26.891 INFO:tasks.workunit.client.1.smithi193.stdout:3/490: rename d1/c40 to d1/d2d/cb3 0 2024-05-09T06:38:26.895 INFO:tasks.workunit.client.1.smithi193.stdout:1/619: getdents d4/d16/d17/d1a/d3a/d4a 0 2024-05-09T06:38:26.895 INFO:tasks.workunit.client.1.smithi193.stdout:1/620: stat d4 0 2024-05-09T06:38:26.898 INFO:tasks.workunit.client.1.smithi193.stdout:0/512: rename d18/d27/f49 to d18/d27/fa0 0 2024-05-09T06:38:26.901 INFO:tasks.workunit.client.1.smithi193.stdout:9/626: creat dd/d1e/dc2/db9/fd0 x:0 0 0 2024-05-09T06:38:26.902 INFO:tasks.workunit.client.1.smithi193.stdout:9/627: dread - dd/d1e/d1f/faa zero size 2024-05-09T06:38:26.902 INFO:tasks.workunit.client.1.smithi193.stdout:6/524: mkdir d1/d25/d2c/d81/d37/db3/dbd 0 2024-05-09T06:38:26.903 INFO:tasks.workunit.client.0.smithi063.stdout:6/939: write d2/d1f/d29/d3f/d5c/f9e [3506486,79232] 0 2024-05-09T06:38:26.903 INFO:tasks.workunit.client.0.smithi063.stdout:2/943: mkdir dc/d32/d112/d11d 0 2024-05-09T06:38:26.906 INFO:tasks.workunit.client.0.smithi063.stdout:6/940: write d2/d1f/d29/fb1 [3074903,75473] 0 2024-05-09T06:38:26.906 INFO:tasks.workunit.client.0.smithi063.stdout:5/905: getdents d8/d17/d18/df4 0 2024-05-09T06:38:26.907 INFO:tasks.workunit.client.0.smithi063.stdout:9/910: dwrite d0/d4/d26/d39/d3e/d66/fe3 [0,4194304] 0 2024-05-09T06:38:26.908 INFO:tasks.workunit.client.1.smithi193.stdout:9/628: creat dd/df/d31/fd1 x:0 0 0 2024-05-09T06:38:26.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/629: write dd/df/d31/d7e/d37/f83 [4919120,105762] 0 2024-05-09T06:38:26.911 INFO:tasks.workunit.client.1.smithi193.stdout:9/630: chown dd/d1e/dc2/d49/c77 562152063 1 2024-05-09T06:38:26.912 INFO:tasks.workunit.client.0.smithi063.stdout:7/896: dread d0/d15/d89/d10c/fb6 [0,4194304] 0 2024-05-09T06:38:26.913 INFO:tasks.workunit.client.0.smithi063.stdout:0/948: dread dd/f6b [0,4194304] 0 2024-05-09T06:38:26.913 INFO:tasks.workunit.client.0.smithi063.stdout:3/974: rename d7/da/d21/d30/d44/d48/d11d/d136 to d7/d157 0 2024-05-09T06:38:26.916 INFO:tasks.workunit.client.0.smithi063.stdout:5/906: fdatasync d8/d65/f80 0 2024-05-09T06:38:26.918 INFO:tasks.workunit.client.1.smithi193.stdout:9/631: dwrite dd/df/d31/f4c [4194304,4194304] 0 2024-05-09T06:38:26.920 INFO:tasks.workunit.client.1.smithi193.stdout:1/621: link d4/d16/d17/d1a/d3a/lb0 d4/d16/d60/d4f/lc8 0 2024-05-09T06:38:26.924 INFO:tasks.workunit.client.1.smithi193.stdout:6/525: dwrite d1/de/f53 [0,4194304] 0 2024-05-09T06:38:26.925 INFO:tasks.workunit.client.0.smithi063.stdout:0/949: rename dd/d1a/d50/f86 to dd/d60/f123 0 2024-05-09T06:38:26.926 INFO:tasks.workunit.client.0.smithi063.stdout:0/950: stat dd/d1a/d2c/c70 0 2024-05-09T06:38:26.927 INFO:tasks.workunit.client.1.smithi193.stdout:9/632: dwrite dd/f82 [0,4194304] 0 2024-05-09T06:38:26.928 INFO:tasks.workunit.client.0.smithi063.stdout:2/944: dread dc/d36/ff4 [0,4194304] 0 2024-05-09T06:38:26.929 INFO:tasks.workunit.client.1.smithi193.stdout:5/491: rename d0/l18 to d0/d13/l9e 0 2024-05-09T06:38:26.930 INFO:tasks.workunit.client.1.smithi193.stdout:9/633: write dd/df/d31/f4c [1398499,5607] 0 2024-05-09T06:38:26.932 INFO:tasks.workunit.client.1.smithi193.stdout:1/622: creat d4/d3c/fc9 x:0 0 0 2024-05-09T06:38:26.935 INFO:tasks.workunit.client.1.smithi193.stdout:6/526: symlink d1/de/lbe 0 2024-05-09T06:38:26.939 INFO:tasks.workunit.client.1.smithi193.stdout:0/513: getdents d18/d4c/d47/d5e 0 2024-05-09T06:38:26.939 INFO:tasks.workunit.client.1.smithi193.stdout:0/514: dread - f16 zero size 2024-05-09T06:38:26.940 INFO:tasks.workunit.client.0.smithi063.stdout:6/941: write d2/d1f/d29/d3f/d5c/f9d [573159,29354] 0 2024-05-09T06:38:26.940 INFO:tasks.workunit.client.1.smithi193.stdout:0/515: read - d18/d4c/d47/d5e/d70/f94 zero size 2024-05-09T06:38:26.942 INFO:tasks.workunit.client.0.smithi063.stdout:5/907: rename d8/d17/f6a to d8/d17/d75/dc9/d111/f12f 0 2024-05-09T06:38:26.948 INFO:tasks.workunit.client.0.smithi063.stdout:3/975: dwrite d7/da/fb [0,4194304] 0 2024-05-09T06:38:26.948 INFO:tasks.workunit.client.0.smithi063.stdout:2/945: creat dc/dd/d2d/d60/dd0/d90/dc1/f11e x:0 0 0 2024-05-09T06:38:26.949 INFO:tasks.workunit.client.0.smithi063.stdout:9/911: link d0/d4/d26/d47/cb6 d0/d43/c137 0 2024-05-09T06:38:26.949 INFO:tasks.workunit.client.0.smithi063.stdout:2/946: chown dc/dd/fb5 51664562 1 2024-05-09T06:38:26.950 INFO:tasks.workunit.client.0.smithi063.stdout:9/912: chown d0/d43/f4b 27151157 1 2024-05-09T06:38:26.950 INFO:tasks.workunit.client.0.smithi063.stdout:2/947: chown dc/d36/d43/d46/d49/ddc 206628 1 2024-05-09T06:38:26.951 INFO:tasks.workunit.client.0.smithi063.stdout:7/897: creat d0/d15/f131 x:0 0 0 2024-05-09T06:38:26.952 INFO:tasks.workunit.client.0.smithi063.stdout:4/931: sync 2024-05-09T06:38:26.953 INFO:tasks.workunit.client.1.smithi193.stdout:0/516: creat d18/d4c/d47/d54/fa1 x:0 0 0 2024-05-09T06:38:26.953 INFO:tasks.workunit.client.1.smithi193.stdout:7/526: sync 2024-05-09T06:38:26.955 INFO:tasks.workunit.client.1.smithi193.stdout:3/491: rename d1/db/d2f/d84/f8d to d1/db/d2f/d78/fb4 0 2024-05-09T06:38:26.959 INFO:tasks.workunit.client.0.smithi063.stdout:3/976: chown d7/d4c/cf7 126464696 1 2024-05-09T06:38:26.959 INFO:tasks.workunit.client.0.smithi063.stdout:3/977: readlink d7/d157/l13c 0 2024-05-09T06:38:26.962 INFO:tasks.workunit.client.0.smithi063.stdout:2/948: fsync dc/d36/f80 0 2024-05-09T06:38:26.963 INFO:tasks.workunit.client.1.smithi193.stdout:6/527: creat d1/de/d11/d20/fbf x:0 0 0 2024-05-09T06:38:26.964 INFO:tasks.workunit.client.0.smithi063.stdout:6/942: getdents d2/d1f/d29/de2/d4e/d12b 0 2024-05-09T06:38:26.965 INFO:tasks.workunit.client.0.smithi063.stdout:7/898: mknod d0/d15/d4a/d68/d10a/c132 0 2024-05-09T06:38:26.966 INFO:tasks.workunit.client.0.smithi063.stdout:7/899: chown d0/d15/d49/c11e 0 1 2024-05-09T06:38:26.967 INFO:tasks.workunit.client.1.smithi193.stdout:3/492: symlink d1/db/d2f/d78/lb5 0 2024-05-09T06:38:26.968 INFO:tasks.workunit.client.0.smithi063.stdout:5/908: creat d8/d17/d11d/d122/f130 x:0 0 0 2024-05-09T06:38:26.970 INFO:tasks.workunit.client.0.smithi063.stdout:3/978: fsync d7/d4c/d25/d10b/fc7 0 2024-05-09T06:38:26.970 INFO:tasks.workunit.client.0.smithi063.stdout:0/951: dread dd/d60/f123 [0,4194304] 0 2024-05-09T06:38:26.975 INFO:tasks.workunit.client.0.smithi063.stdout:3/979: chown d7/d4c/d25/fd1 3490309 1 2024-05-09T06:38:26.975 INFO:tasks.workunit.client.1.smithi193.stdout:9/634: truncate dd/d1e/f28 1380907 0 2024-05-09T06:38:26.975 INFO:tasks.workunit.client.1.smithi193.stdout:9/635: readlink dd/df/d31/d7e/d37/l46 0 2024-05-09T06:38:26.975 INFO:tasks.workunit.client.1.smithi193.stdout:5/492: dwrite d0/f9b [0,4194304] 0 2024-05-09T06:38:26.976 INFO:tasks.workunit.client.1.smithi193.stdout:6/528: rename d1/de/f2f to d1/d25/d2c/d76/fc0 0 2024-05-09T06:38:26.977 INFO:tasks.workunit.client.1.smithi193.stdout:0/517: write f16 [388895,71982] 0 2024-05-09T06:38:26.977 INFO:tasks.workunit.client.0.smithi063.stdout:2/949: mknod dc/dd/d2d/d60/dd0/d90/c11f 0 2024-05-09T06:38:26.979 INFO:tasks.workunit.client.1.smithi193.stdout:0/518: write d18/d4c/d47/d5e/f80 [554499,107321] 0 2024-05-09T06:38:26.979 INFO:tasks.workunit.client.1.smithi193.stdout:3/493: creat d1/db/d59/d63/fb6 x:0 0 0 2024-05-09T06:38:26.983 INFO:tasks.workunit.client.0.smithi063.stdout:4/932: creat d7/d50/d10d/f12d x:0 0 0 2024-05-09T06:38:26.983 INFO:tasks.workunit.client.0.smithi063.stdout:9/913: write d0/d3b/f56 [952934,93057] 0 2024-05-09T06:38:26.985 INFO:tasks.workunit.client.0.smithi063.stdout:1/870: sync 2024-05-09T06:38:26.986 INFO:tasks.workunit.client.0.smithi063.stdout:9/914: read d0/d109/f129 [1014530,12643] 0 2024-05-09T06:38:26.987 INFO:tasks.workunit.client.0.smithi063.stdout:9/915: write d0/d4/d7d/d105/f12c [886503,18390] 0 2024-05-09T06:38:26.988 INFO:tasks.workunit.client.1.smithi193.stdout:7/527: link d6/d8/d29/d2f/f8d d6/d8/d1a/fb3 0 2024-05-09T06:38:26.991 INFO:tasks.workunit.client.1.smithi193.stdout:3/494: rename d1/db/d59/f81 to d1/db/d2f/d84/fb7 0 2024-05-09T06:38:26.992 INFO:tasks.workunit.client.0.smithi063.stdout:0/952: truncate dd/d60/f123 1839280 0 2024-05-09T06:38:26.996 INFO:tasks.workunit.client.1.smithi193.stdout:5/493: symlink d0/d13/d2c/d37/d2b/d47/d60/d75/l9f 0 2024-05-09T06:38:26.996 INFO:tasks.workunit.client.1.smithi193.stdout:5/494: chown d0/f5c 66 1 2024-05-09T06:38:27.000 INFO:tasks.workunit.client.1.smithi193.stdout:0/519: dread d18/f61 [0,4194304] 0 2024-05-09T06:38:27.000 INFO:tasks.workunit.client.0.smithi063.stdout:2/950: truncate dc/d32/d33/f4c 749512 0 2024-05-09T06:38:27.002 INFO:tasks.workunit.client.0.smithi063.stdout:6/943: dread d2/d1f/d29/d3f/d5c/f70 [0,4194304] 0 2024-05-09T06:38:27.003 INFO:tasks.workunit.client.1.smithi193.stdout:0/520: dread d18/f61 [0,4194304] 0 2024-05-09T06:38:27.003 INFO:tasks.workunit.client.1.smithi193.stdout:0/521: fsync d18/d4c/d47/d54/fa1 0 2024-05-09T06:38:27.004 INFO:tasks.workunit.client.1.smithi193.stdout:0/522: read - d18/d4c/d47/f96 zero size 2024-05-09T06:38:27.006 INFO:tasks.workunit.client.0.smithi063.stdout:2/951: dwrite dc/dd/d45/fe3 [0,4194304] 0 2024-05-09T06:38:27.009 INFO:tasks.workunit.client.0.smithi063.stdout:5/909: creat d8/d19/d115/deb/d129/f131 x:0 0 0 2024-05-09T06:38:27.009 INFO:tasks.workunit.client.1.smithi193.stdout:7/528: write d6/dd/f47 [5632164,11564] 0 2024-05-09T06:38:27.010 INFO:tasks.workunit.client.0.smithi063.stdout:5/910: chown d8/d19/d30/d74/ff1 5618 1 2024-05-09T06:38:27.011 INFO:tasks.workunit.client.0.smithi063.stdout:1/871: unlink d9/d2a/d35/d1e/l38 0 2024-05-09T06:38:27.012 INFO:tasks.workunit.client.0.smithi063.stdout:5/911: truncate d8/d19/f10f 637770 0 2024-05-09T06:38:27.013 INFO:tasks.workunit.client.1.smithi193.stdout:3/495: fdatasync d1/db/d2f/d78/fb4 0 2024-05-09T06:38:27.013 INFO:tasks.workunit.client.0.smithi063.stdout:5/912: fdatasync d8/d17/d18/f8a 0 2024-05-09T06:38:27.017 INFO:tasks.workunit.client.0.smithi063.stdout:1/872: dread d9/d2a/d2e/fcc [0,4194304] 0 2024-05-09T06:38:27.019 INFO:tasks.workunit.client.1.smithi193.stdout:9/636: getdents dd/df/d15/d8c 0 2024-05-09T06:38:27.020 INFO:tasks.workunit.client.0.smithi063.stdout:1/873: chown d9/d2a/d3c/d4c/d83/d88/dfb/f107 0 1 2024-05-09T06:38:27.021 INFO:tasks.workunit.client.0.smithi063.stdout:0/953: mkdir dd/d60/d119/d124 0 2024-05-09T06:38:27.022 INFO:tasks.workunit.client.0.smithi063.stdout:5/913: dwrite d8/d17/d18/f2d [0,4194304] 0 2024-05-09T06:38:27.023 INFO:tasks.workunit.client.1.smithi193.stdout:0/523: read d18/f60 [3246919,62098] 0 2024-05-09T06:38:27.023 INFO:tasks.workunit.client.0.smithi063.stdout:3/980: getdents d7/d22/dfc/d140/d153 0 2024-05-09T06:38:27.023 INFO:tasks.workunit.client.1.smithi193.stdout:5/495: mkdir d0/d13/d2c/da0 0 2024-05-09T06:38:27.024 INFO:tasks.workunit.client.0.smithi063.stdout:0/954: chown dd/d1a/d2c/ca3 14142 1 2024-05-09T06:38:27.025 INFO:tasks.workunit.client.1.smithi193.stdout:7/529: symlink d6/d8/d1a/d33/lb4 0 2024-05-09T06:38:27.026 INFO:tasks.workunit.client.0.smithi063.stdout:6/944: dread d2/d1f/d29/d3f/d5c/d64/f6f [0,4194304] 0 2024-05-09T06:38:27.026 INFO:tasks.workunit.client.1.smithi193.stdout:7/530: write d6/f8b [277891,106655] 0 2024-05-09T06:38:27.029 INFO:tasks.workunit.client.1.smithi193.stdout:9/637: truncate dd/f29 1225792 0 2024-05-09T06:38:27.032 INFO:tasks.workunit.client.0.smithi063.stdout:9/916: dwrite d0/d4/d57/f9b [0,4194304] 0 2024-05-09T06:38:27.032 INFO:tasks.workunit.client.0.smithi063.stdout:2/952: rename dc/dd/d2d/d60/l69 to dc/dd/d45/d5a/d6c/d8f/dbf/l120 0 2024-05-09T06:38:27.032 INFO:tasks.workunit.client.1.smithi193.stdout:5/496: truncate d0/d13/d2c/d37/d5b/d4e/f78 4241817 0 2024-05-09T06:38:27.034 INFO:tasks.workunit.client.0.smithi063.stdout:9/917: truncate d0/d4/d26/d39/f131 603453 0 2024-05-09T06:38:27.034 INFO:tasks.workunit.client.0.smithi063.stdout:9/918: chown d0/d4/d26/d47 7709 1 2024-05-09T06:38:27.035 INFO:tasks.workunit.client.1.smithi193.stdout:7/531: readlink d6/d8/d1a/l39 0 2024-05-09T06:38:27.037 INFO:tasks.workunit.client.1.smithi193.stdout:0/524: dwrite d18/f4e [0,4194304] 0 2024-05-09T06:38:27.043 INFO:tasks.workunit.client.1.smithi193.stdout:7/532: creat d6/d17/d7d/fb5 x:0 0 0 2024-05-09T06:38:27.046 INFO:tasks.workunit.client.1.smithi193.stdout:0/525: dwrite d18/d4c/f38 [0,4194304] 0 2024-05-09T06:38:27.049 INFO:tasks.workunit.client.0.smithi063.stdout:5/914: symlink d8/d35/db6/l132 0 2024-05-09T06:38:27.056 INFO:tasks.workunit.client.0.smithi063.stdout:4/933: creat d7/df/d35/d5a/f12e x:0 0 0 2024-05-09T06:38:27.057 INFO:tasks.workunit.client.1.smithi193.stdout:3/496: write d1/d44/fa0 [335248,21309] 0 2024-05-09T06:38:27.058 INFO:tasks.workunit.client.0.smithi063.stdout:4/934: dread - d7/da3/d28/d31/d32/d34/db4/dcc/de1/f122 zero size 2024-05-09T06:38:27.059 INFO:tasks.workunit.client.0.smithi063.stdout:4/935: fsync d7/f10c 0 2024-05-09T06:38:27.061 INFO:tasks.workunit.client.0.smithi063.stdout:6/945: creat d2/df/d59/d99/d9f/f134 x:0 0 0 2024-05-09T06:38:27.063 INFO:tasks.workunit.client.1.smithi193.stdout:5/497: dread d0/d13/d2c/d37/d5b/d4e/d5d/f61 [0,4194304] 0 2024-05-09T06:38:27.064 INFO:tasks.workunit.client.0.smithi063.stdout:7/900: fdatasync d0/d15/d49/d54/d9f/d97/fdc 0 2024-05-09T06:38:27.065 INFO:tasks.workunit.client.1.smithi193.stdout:9/638: write dd/d1e/f28 [1669514,81711] 0 2024-05-09T06:38:27.066 INFO:tasks.workunit.client.1.smithi193.stdout:9/639: chown dd/d1e/dc2/d49/dbd/c81 7917402 1 2024-05-09T06:38:27.066 INFO:tasks.workunit.client.1.smithi193.stdout:3/497: mkdir d1/db/d59/db8 0 2024-05-09T06:38:27.067 INFO:tasks.workunit.client.1.smithi193.stdout:9/640: write fa [392752,107544] 0 2024-05-09T06:38:27.069 INFO:tasks.workunit.client.0.smithi063.stdout:1/874: dwrite d9/d2a/d35/d1e/d9d/d3f/d59/f79 [4194304,4194304] 0 2024-05-09T06:38:27.069 INFO:tasks.workunit.client.1.smithi193.stdout:0/526: dread d18/d4c/d5f/f6a [0,4194304] 0 2024-05-09T06:38:27.073 INFO:tasks.workunit.client.1.smithi193.stdout:9/641: creat dd/d1e/fd2 x:0 0 0 2024-05-09T06:38:27.074 INFO:tasks.workunit.client.0.smithi063.stdout:3/981: mknod d7/d4c/c158 0 2024-05-09T06:38:27.076 INFO:tasks.workunit.client.1.smithi193.stdout:5/498: symlink d0/d13/d2c/d37/d2b/d47/d60/la1 0 2024-05-09T06:38:27.077 INFO:tasks.workunit.client.0.smithi063.stdout:5/915: symlink d8/d17/d75/dc9/dd5/d96/l133 0 2024-05-09T06:38:27.079 INFO:tasks.workunit.client.1.smithi193.stdout:0/527: readlink d18/d4c/d64/l6e 0 2024-05-09T06:38:27.080 INFO:tasks.workunit.client.1.smithi193.stdout:0/528: truncate d18/d4c/d47/d54/fa1 328153 0 2024-05-09T06:38:27.083 INFO:tasks.workunit.client.0.smithi063.stdout:4/936: unlink d7/df/d35/d5a/d60/d75/c10e 0 2024-05-09T06:38:27.086 INFO:tasks.workunit.client.0.smithi063.stdout:6/946: symlink d2/d9a/l135 0 2024-05-09T06:38:27.086 INFO:tasks.workunit.client.0.smithi063.stdout:7/901: dread d0/d15/f4e [0,4194304] 0 2024-05-09T06:38:27.087 INFO:tasks.workunit.client.1.smithi193.stdout:0/529: dwrite f16 [0,4194304] 0 2024-05-09T06:38:27.087 INFO:tasks.workunit.client.1.smithi193.stdout:0/530: readlink d18/d1d/l5a 0 2024-05-09T06:38:27.090 INFO:tasks.workunit.client.0.smithi063.stdout:6/947: write d2/df/d59/d99/d9f/f126 [882572,87871] 0 2024-05-09T06:38:27.091 INFO:tasks.workunit.client.0.smithi063.stdout:9/919: dwrite d0/d4/d57/ff4 [0,4194304] 0 2024-05-09T06:38:27.092 INFO:tasks.workunit.client.0.smithi063.stdout:9/920: readlink d0/d4/d57/d60/d76/l132 0 2024-05-09T06:38:27.094 INFO:tasks.workunit.client.1.smithi193.stdout:5/499: mknod d0/d13/d2c/da0/ca2 0 2024-05-09T06:38:27.097 INFO:tasks.workunit.client.1.smithi193.stdout:9/642: write dd/d1e/dc2/d49/dbd/f8b [698607,63690] 0 2024-05-09T06:38:27.099 INFO:tasks.workunit.client.1.smithi193.stdout:0/531: rename cb to d18/d4c/d47/d5e/d70/ca2 0 2024-05-09T06:38:27.099 INFO:tasks.workunit.client.1.smithi193.stdout:3/498: link d1/db/d59/c71 d1/d2d/cb9 0 2024-05-09T06:38:27.102 INFO:tasks.workunit.client.1.smithi193.stdout:5/500: symlink d0/d13/d2c/d7e/la3 0 2024-05-09T06:38:27.103 INFO:tasks.workunit.client.0.smithi063.stdout:0/955: dwrite dd/d60/f123 [0,4194304] 0 2024-05-09T06:38:27.104 INFO:tasks.workunit.client.0.smithi063.stdout:9/921: dwrite d0/d4/d26/d39/f53 [4194304,4194304] 0 2024-05-09T06:38:27.104 INFO:tasks.workunit.client.1.smithi193.stdout:9/643: symlink dd/ld3 0 2024-05-09T06:38:27.105 INFO:tasks.workunit.client.1.smithi193.stdout:9/644: chown dd/d1e/d93/fa3 7268 1 2024-05-09T06:38:27.108 INFO:tasks.workunit.client.1.smithi193.stdout:0/532: rename d18/d1d/l5a to d18/d4c/d47/d5e/d70/la3 0 2024-05-09T06:38:27.113 INFO:tasks.workunit.client.0.smithi063.stdout:4/937: dread d7/da3/d28/d31/d32/d34/f59 [0,4194304] 0 2024-05-09T06:38:27.115 INFO:tasks.workunit.client.0.smithi063.stdout:2/953: truncate dc/d36/d43/d46/ded/ff0 3460335 0 2024-05-09T06:38:27.117 INFO:tasks.workunit.client.1.smithi193.stdout:0/533: dread d18/d1d/f1e [0,4194304] 0 2024-05-09T06:38:27.117 INFO:tasks.workunit.client.0.smithi063.stdout:6/948: unlink d2/d20/f9c 0 2024-05-09T06:38:27.118 INFO:tasks.workunit.client.1.smithi193.stdout:0/534: read d18/f28 [1176550,35217] 0 2024-05-09T06:38:27.119 INFO:tasks.workunit.client.1.smithi193.stdout:0/535: chown d18/d4c/l5d 271 1 2024-05-09T06:38:27.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:26 smithi193 ceph-mon[35141]: pgmap v16: 65 pgs: 65 active+clean; 2.5 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 69 MiB/s rd, 198 MiB/s wr, 399 op/s 2024-05-09T06:38:27.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:26 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:27.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:26 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:27.125 INFO:tasks.workunit.client.0.smithi063.stdout:5/916: truncate d8/d19/d30/d46/f6c 958842 0 2024-05-09T06:38:27.126 INFO:tasks.workunit.client.0.smithi063.stdout:3/982: write d7/d70/d98/f9f [650581,7219] 0 2024-05-09T06:38:27.128 INFO:tasks.workunit.client.0.smithi063.stdout:7/902: creat d0/d15/d89/d113/f133 x:0 0 0 2024-05-09T06:38:27.129 INFO:tasks.workunit.client.0.smithi063.stdout:1/875: dwrite d9/d2a/d35/d1e/faf [0,4194304] 0 2024-05-09T06:38:27.129 INFO:tasks.workunit.client.0.smithi063.stdout:3/983: write d7/d22/f47 [5177839,117545] 0 2024-05-09T06:38:27.131 INFO:tasks.workunit.client.1.smithi193.stdout:3/499: write d1/d44/f60 [608152,99868] 0 2024-05-09T06:38:27.131 INFO:tasks.workunit.client.0.smithi063.stdout:4/938: creat d7/da3/d28/d31/d71/d98/dff/f12f x:0 0 0 2024-05-09T06:38:27.132 INFO:tasks.workunit.client.1.smithi193.stdout:3/500: dread - d1/d2d/f45 zero size 2024-05-09T06:38:27.132 INFO:tasks.workunit.client.1.smithi193.stdout:0/536: rename f9 to d18/d4c/fa4 0 2024-05-09T06:38:27.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/954: mkdir dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d121 0 2024-05-09T06:38:27.135 INFO:tasks.workunit.client.1.smithi193.stdout:9/645: link dd/d1e/dc2/d49/dbd/d5b/l99 dd/d1e/d1f/d48/d55/d86/ld4 0 2024-05-09T06:38:27.136 INFO:tasks.workunit.client.0.smithi063.stdout:2/955: chown dc/d6e/cc6 6 1 2024-05-09T06:38:27.136 INFO:tasks.workunit.client.1.smithi193.stdout:9/646: dread - dd/d1e/dc2/d49/dbd/fc9 zero size 2024-05-09T06:38:27.137 INFO:tasks.workunit.client.1.smithi193.stdout:5/501: truncate d0/d13/d2c/d37/d5b/f21 1723053 0 2024-05-09T06:38:27.138 INFO:tasks.workunit.client.0.smithi063.stdout:2/956: chown dc/dd/d45/d5a/d6c/d8c/dad/le8 0 1 2024-05-09T06:38:27.138 INFO:tasks.workunit.client.1.smithi193.stdout:3/501: dread - d1/db/d1b/f70 zero size 2024-05-09T06:38:27.146 INFO:tasks.workunit.client.0.smithi063.stdout:5/917: mknod d8/d35/dd6/c134 0 2024-05-09T06:38:27.148 INFO:tasks.workunit.client.0.smithi063.stdout:9/922: write d0/d4/d26/d47/f7a [2868724,32104] 0 2024-05-09T06:38:27.148 INFO:tasks.workunit.client.0.smithi063.stdout:6/949: dread d2/d1f/d6c/fb7 [0,4194304] 0 2024-05-09T06:38:27.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/876: creat d9/d2a/d3c/d4a/f12e x:0 0 0 2024-05-09T06:38:27.150 INFO:tasks.workunit.client.0.smithi063.stdout:3/984: symlink d7/d157/l159 0 2024-05-09T06:38:27.151 INFO:tasks.workunit.client.0.smithi063.stdout:9/923: write d0/d4/d57/d60/d76/fa9 [2488244,74771] 0 2024-05-09T06:38:27.151 INFO:tasks.workunit.client.0.smithi063.stdout:7/903: mknod d0/d15/d49/d54/d9f/d97/d9b/d9d/c134 0 2024-05-09T06:38:27.154 INFO:tasks.workunit.client.0.smithi063.stdout:2/957: creat dc/dd/d45/d5a/d6c/d8f/dbf/f122 x:0 0 0 2024-05-09T06:38:27.155 INFO:tasks.workunit.client.0.smithi063.stdout:5/918: dread - d8/d17/d75/dc9/dd5/fee zero size 2024-05-09T06:38:27.156 INFO:tasks.workunit.client.0.smithi063.stdout:9/924: write d0/d3b/f56 [1747878,3918] 0 2024-05-09T06:38:27.156 INFO:tasks.workunit.client.0.smithi063.stdout:7/904: chown d0/d15/d49/d54/d9f/c8d 198693927 1 2024-05-09T06:38:27.158 INFO:tasks.workunit.client.0.smithi063.stdout:1/877: creat d9/d2a/d2e/df4/f12f x:0 0 0 2024-05-09T06:38:27.158 INFO:tasks.workunit.client.0.smithi063.stdout:4/939: dread d7/da3/d28/d31/d71/dbb/fd7 [0,4194304] 0 2024-05-09T06:38:27.159 INFO:tasks.workunit.client.0.smithi063.stdout:2/958: chown dc/dd/d45/d5a/d6c/d8c/dad/faf 416917 1 2024-05-09T06:38:27.159 INFO:tasks.workunit.client.0.smithi063.stdout:6/950: getdents d2/d1f/d29/de2/d4e/d12b 0 2024-05-09T06:38:27.161 INFO:tasks.workunit.client.0.smithi063.stdout:0/956: write dd/d1a/d2c/d42/fcf [131057,99914] 0 2024-05-09T06:38:27.165 INFO:tasks.workunit.client.0.smithi063.stdout:4/940: unlink d7/d50/d10f/c118 0 2024-05-09T06:38:27.166 INFO:tasks.workunit.client.0.smithi063.stdout:3/985: rename d7/da/d2b/la5 to d7/da/d21/d30/d44/d48/l15a 0 2024-05-09T06:38:27.171 INFO:tasks.workunit.client.0.smithi063.stdout:6/951: mknod d2/df/d59/da3/c136 0 2024-05-09T06:38:27.172 INFO:tasks.workunit.client.0.smithi063.stdout:4/941: read - d7/da3/d28/d31/d7b/fec zero size 2024-05-09T06:38:27.172 INFO:tasks.workunit.client.0.smithi063.stdout:6/952: chown d2/d20/f44 234775053 1 2024-05-09T06:38:27.173 INFO:tasks.workunit.client.0.smithi063.stdout:7/905: rename d0/l5a to d0/d15/d4a/d50/l135 0 2024-05-09T06:38:27.174 INFO:tasks.workunit.client.0.smithi063.stdout:2/959: creat dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d121/f123 x:0 0 0 2024-05-09T06:38:27.175 INFO:tasks.workunit.client.0.smithi063.stdout:0/957: creat dd/d60/dcd/de0/f125 x:0 0 0 2024-05-09T06:38:27.185 INFO:tasks.workunit.client.1.smithi193.stdout:0/537: rename d18/d27/d7d to d18/d4c/d64/d66/d9d/da5 0 2024-05-09T06:38:27.186 INFO:tasks.workunit.client.1.smithi193.stdout:3/502: fdatasync d1/f4 0 2024-05-09T06:38:27.189 INFO:tasks.workunit.client.1.smithi193.stdout:9/647: dwrite dd/df/d31/d7e/f60 [4194304,4194304] 0 2024-05-09T06:38:27.189 INFO:tasks.workunit.client.1.smithi193.stdout:3/503: fdatasync d1/db/d59/f52 0 2024-05-09T06:38:27.191 INFO:tasks.workunit.client.1.smithi193.stdout:0/538: rename f17 to d18/d4c/d64/d8b/fa6 0 2024-05-09T06:38:27.198 INFO:tasks.workunit.client.1.smithi193.stdout:5/502: dwrite d0/d13/d2c/d37/d5b/f32 [0,4194304] 0 2024-05-09T06:38:27.201 INFO:tasks.workunit.client.1.smithi193.stdout:9/648: dread dd/d1e/d1f/d48/fc8 [0,4194304] 0 2024-05-09T06:38:27.202 INFO:tasks.workunit.client.0.smithi063.stdout:2/960: rmdir dc/d36/d43/d46/d49 39 2024-05-09T06:38:27.203 INFO:tasks.workunit.client.1.smithi193.stdout:0/539: dwrite d18/d4c/d64/f85 [0,4194304] 0 2024-05-09T06:38:27.204 INFO:tasks.workunit.client.0.smithi063.stdout:5/919: write d8/f10 [3984235,107113] 0 2024-05-09T06:38:27.204 INFO:tasks.workunit.client.0.smithi063.stdout:9/925: truncate d0/d4/d57/ff4 2235150 0 2024-05-09T06:38:27.207 INFO:tasks.workunit.client.1.smithi193.stdout:9/649: mknod dd/d1e/dc2/cd5 0 2024-05-09T06:38:27.210 INFO:tasks.workunit.client.1.smithi193.stdout:0/540: chown d18/d4c/d47/d5e/d70/ca2 15 1 2024-05-09T06:38:27.210 INFO:tasks.workunit.client.1.smithi193.stdout:9/650: write dd/df/d31/d7e/f60 [5562031,89281] 0 2024-05-09T06:38:27.211 INFO:tasks.workunit.client.0.smithi063.stdout:0/958: dwrite dd/d1a/d2c/d51/d72/d9e/d23/dbb/ffe [0,4194304] 0 2024-05-09T06:38:27.211 INFO:tasks.workunit.client.1.smithi193.stdout:9/651: chown dd/df/d15 9529 1 2024-05-09T06:38:27.213 INFO:tasks.workunit.client.0.smithi063.stdout:4/942: creat d7/da3/f130 x:0 0 0 2024-05-09T06:38:27.213 INFO:tasks.workunit.client.0.smithi063.stdout:0/959: write dd/d1a/d2c/d51/d72/d9e/feb [1296812,103271] 0 2024-05-09T06:38:27.218 INFO:tasks.workunit.client.0.smithi063.stdout:5/920: creat d8/d17/d75/dc9/d111/f135 x:0 0 0 2024-05-09T06:38:27.219 INFO:tasks.workunit.client.1.smithi193.stdout:3/504: dwrite d1/d2d/d51/f29 [0,4194304] 0 2024-05-09T06:38:27.221 INFO:tasks.workunit.client.0.smithi063.stdout:2/961: truncate dc/d32/f42 2297103 0 2024-05-09T06:38:27.222 INFO:tasks.workunit.client.0.smithi063.stdout:2/962: fsync f6 0 2024-05-09T06:38:27.223 INFO:tasks.workunit.client.0.smithi063.stdout:2/963: fsync dc/dd/d45/d5a/d6c/d8f/dbf/f122 0 2024-05-09T06:38:27.223 INFO:tasks.workunit.client.1.smithi193.stdout:0/541: dwrite d18/d4c/d47/f8e [0,4194304] 0 2024-05-09T06:38:27.225 INFO:tasks.workunit.client.1.smithi193.stdout:5/503: fdatasync d0/d13/d2c/d4d/d67/f93 0 2024-05-09T06:38:27.226 INFO:tasks.workunit.client.1.smithi193.stdout:9/652: dread dd/d1e/dc2/d49/dbd/f8b [0,4194304] 0 2024-05-09T06:38:27.228 INFO:tasks.workunit.client.1.smithi193.stdout:9/653: truncate dd/d1e/d1f/d61/f85 588275 0 2024-05-09T06:38:27.233 INFO:tasks.workunit.client.1.smithi193.stdout:3/505: fsync d1/db/f18 0 2024-05-09T06:38:27.236 INFO:tasks.workunit.client.0.smithi063.stdout:7/906: rename d0/d15/d89/d113/c120 to d0/d15/d49/d4f/d66/c136 0 2024-05-09T06:38:27.236 INFO:tasks.workunit.client.1.smithi193.stdout:0/542: mkdir d18/d1d/d56/da7 0 2024-05-09T06:38:27.239 INFO:tasks.workunit.client.1.smithi193.stdout:5/504: read d0/d13/f3d [374445,56521] 0 2024-05-09T06:38:27.241 INFO:tasks.workunit.client.0.smithi063.stdout:6/953: dread d2/df/f14 [0,4194304] 0 2024-05-09T06:38:27.241 INFO:tasks.workunit.client.1.smithi193.stdout:5/505: truncate d0/d13/d2c/d37/d5b/f7b 296362 0 2024-05-09T06:38:27.242 INFO:tasks.workunit.client.0.smithi063.stdout:5/921: mknod d8/d17/d11d/c136 0 2024-05-09T06:38:27.242 INFO:tasks.workunit.client.0.smithi063.stdout:6/954: dread - d2/d1f/f124 zero size 2024-05-09T06:38:27.242 INFO:tasks.workunit.client.0.smithi063.stdout:6/955: readlink d2/d20/d4a/led 0 2024-05-09T06:38:27.243 INFO:tasks.workunit.client.1.smithi193.stdout:9/654: creat dd/d1e/dc2/d49/fd6 x:0 0 0 2024-05-09T06:38:27.244 INFO:tasks.workunit.client.1.smithi193.stdout:9/655: chown dd/df/d31/d7e/f4b 27 1 2024-05-09T06:38:27.246 INFO:tasks.workunit.client.1.smithi193.stdout:3/506: symlink d1/d2d/d51/lba 0 2024-05-09T06:38:27.249 INFO:tasks.workunit.client.0.smithi063.stdout:4/943: dwrite d7/da3/d28/d31/d61/fb6 [0,4194304] 0 2024-05-09T06:38:27.249 INFO:tasks.workunit.client.1.smithi193.stdout:5/506: mknod d0/d7c/ca4 0 2024-05-09T06:38:27.253 INFO:tasks.workunit.client.0.smithi063.stdout:2/964: mkdir dc/dd/d45/d5a/d6c/d8f/dbf/d105/d124 0 2024-05-09T06:38:27.254 INFO:tasks.workunit.client.1.smithi193.stdout:5/507: creat d0/d13/d2c/d37/d2b/d47/d60/d87/fa5 x:0 0 0 2024-05-09T06:38:27.257 INFO:tasks.workunit.client.0.smithi063.stdout:5/922: symlink d8/d19/d30/l137 0 2024-05-09T06:38:27.257 INFO:tasks.workunit.client.0.smithi063.stdout:9/926: write d0/d4/d7d/d105/d110/fbe [729118,13] 0 2024-05-09T06:38:27.259 INFO:tasks.workunit.client.1.smithi193.stdout:5/508: truncate d0/d13/d2c/d37/d5b/d4e/d5d/f69 1094498 0 2024-05-09T06:38:27.259 INFO:tasks.workunit.client.0.smithi063.stdout:7/907: dread d0/f32 [0,4194304] 0 2024-05-09T06:38:27.262 INFO:tasks.workunit.client.1.smithi193.stdout:5/509: symlink d0/d13/d2c/d37/d2b/d47/d66/d3a/la6 0 2024-05-09T06:38:27.264 INFO:tasks.workunit.client.1.smithi193.stdout:9/656: write dd/f11 [2189670,19486] 0 2024-05-09T06:38:27.265 INFO:tasks.workunit.client.0.smithi063.stdout:7/908: mkdir d0/d15/d49/d54/d9f/d106/d137 0 2024-05-09T06:38:27.269 INFO:tasks.workunit.client.1.smithi193.stdout:5/510: dwrite d0/d13/d2c/d37/d2b/d47/d66/f83 [0,4194304] 0 2024-05-09T06:38:27.271 INFO:tasks.workunit.client.0.smithi063.stdout:6/956: write d2/d1f/f5b [839108,63552] 0 2024-05-09T06:38:27.271 INFO:tasks.workunit.client.1.smithi193.stdout:9/657: read dd/f13 [1025033,40470] 0 2024-05-09T06:38:27.271 INFO:tasks.workunit.client.0.smithi063.stdout:2/965: dread dc/dd/f2f [0,4194304] 0 2024-05-09T06:38:27.272 INFO:tasks.workunit.client.0.smithi063.stdout:0/960: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d31/f5e [0,4194304] 0 2024-05-09T06:38:27.272 INFO:tasks.workunit.client.1.smithi193.stdout:3/507: dwrite d1/f8 [0,4194304] 0 2024-05-09T06:38:27.275 INFO:tasks.workunit.client.0.smithi063.stdout:7/909: truncate d0/d15/d4a/fb4 600115 0 2024-05-09T06:38:27.277 INFO:tasks.workunit.client.1.smithi193.stdout:5/511: rmdir d0/d13/d2c/d37/d2b/d47 39 2024-05-09T06:38:27.279 INFO:tasks.workunit.client.0.smithi063.stdout:4/944: write d7/d3d/fab [3869753,67229] 0 2024-05-09T06:38:27.280 INFO:tasks.workunit.client.0.smithi063.stdout:6/957: chown d2/d1f/d29/d3f/f82 25971 1 2024-05-09T06:38:27.281 INFO:tasks.workunit.client.1.smithi193.stdout:9/658: dwrite dd/d1e/d1f/d48/f5a [0,4194304] 0 2024-05-09T06:38:27.282 INFO:tasks.workunit.client.1.smithi193.stdout:9/659: chown dd/d1e/d1f/d48/d55/l91 715728 1 2024-05-09T06:38:27.284 INFO:tasks.workunit.client.1.smithi193.stdout:9/660: rename dd/d1e to dd/d1e/d1f/d61/dc6/dd7 22 2024-05-09T06:38:27.286 INFO:tasks.workunit.client.1.smithi193.stdout:9/661: write dd/df/d31/d7e/f60 [9287121,118248] 0 2024-05-09T06:38:27.286 INFO:tasks.workunit.client.1.smithi193.stdout:3/508: creat d1/d44/d7d/fbb x:0 0 0 2024-05-09T06:38:27.289 INFO:tasks.workunit.client.0.smithi063.stdout:2/966: mknod dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/c125 0 2024-05-09T06:38:27.291 INFO:tasks.workunit.client.0.smithi063.stdout:5/923: truncate d8/d17/f8d 7875207 0 2024-05-09T06:38:27.293 INFO:tasks.workunit.client.0.smithi063.stdout:7/910: fdatasync d0/d15/d49/d54/d9f/d97/fdc 0 2024-05-09T06:38:27.295 INFO:tasks.workunit.client.0.smithi063.stdout:9/927: truncate d0/d4/d57/d60/d76/fa9 1354822 0 2024-05-09T06:38:27.295 INFO:tasks.workunit.client.1.smithi193.stdout:9/662: dwrite dd/d1e/dc2/d49/dbd/fc9 [0,4194304] 0 2024-05-09T06:38:27.301 INFO:tasks.workunit.client.0.smithi063.stdout:2/967: rename dc/d36/d43/d46/d63/d79/d7d/d8d/d8e/d92/c125 to dc/dd/d45/d5a/d6c/d8c/d9a/dd5/c126 0 2024-05-09T06:38:27.303 INFO:tasks.workunit.client.0.smithi063.stdout:7/911: rmdir d0/d15/d49/d54/d9f/dd2 39 2024-05-09T06:38:27.303 INFO:tasks.workunit.client.0.smithi063.stdout:2/968: chown dc/dd/d2d/d60/dd0/d90/dc1/f11e 6 1 2024-05-09T06:38:27.304 INFO:tasks.workunit.client.1.smithi193.stdout:3/509: mknod d1/db/da1/cbc 0 2024-05-09T06:38:27.305 INFO:tasks.workunit.client.0.smithi063.stdout:2/969: write dc/dd/d45/d5a/d6c/d8f/f115 [244614,33618] 0 2024-05-09T06:38:27.306 INFO:tasks.workunit.client.1.smithi193.stdout:9/663: symlink dd/d1e/d1f/d61/dc6/ld8 0 2024-05-09T06:38:27.306 INFO:tasks.workunit.client.1.smithi193.stdout:5/512: dread d0/d13/d2c/d37/d5b/d4e/d5d/f69 [0,4194304] 0 2024-05-09T06:38:27.307 INFO:tasks.workunit.client.1.smithi193.stdout:9/664: write dd/d1e/dc2/d49/f70 [211927,40515] 0 2024-05-09T06:38:27.308 INFO:tasks.workunit.client.0.smithi063.stdout:0/961: write dd/d1a/f9f [1039582,25187] 0 2024-05-09T06:38:27.308 INFO:tasks.workunit.client.0.smithi063.stdout:6/958: getdents d2/df/d91/d11e 0 2024-05-09T06:38:27.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/513: write d0/d13/d2c/d37/d5b/d4e/f7a [32470,4475] 0 2024-05-09T06:38:27.311 INFO:tasks.workunit.client.0.smithi063.stdout:5/924: rename d8/d53/dca to d8/d35/db6/dc2/d138 0 2024-05-09T06:38:27.312 INFO:tasks.workunit.client.0.smithi063.stdout:5/925: write d8/d35/dd6/f124 [1565154,37931] 0 2024-05-09T06:38:27.313 INFO:tasks.workunit.client.0.smithi063.stdout:5/926: readlink d8/d35/db6/lc0 0 2024-05-09T06:38:27.313 INFO:tasks.workunit.client.0.smithi063.stdout:4/945: creat d7/da3/d28/d31/f131 x:0 0 0 2024-05-09T06:38:27.316 INFO:tasks.workunit.client.1.smithi193.stdout:9/665: mkdir dd/dd9 0 2024-05-09T06:38:27.316 INFO:tasks.workunit.client.0.smithi063.stdout:7/912: rmdir d0/d15/d49/d54 39 2024-05-09T06:38:27.320 INFO:tasks.workunit.client.1.smithi193.stdout:9/666: creat dd/d1e/d3c/fda x:0 0 0 2024-05-09T06:38:27.322 INFO:tasks.workunit.client.1.smithi193.stdout:5/514: mknod d0/d13/d2c/d37/d5b/ca7 0 2024-05-09T06:38:27.322 INFO:tasks.workunit.client.1.smithi193.stdout:5/515: chown d0/f7 456860793 1 2024-05-09T06:38:27.324 INFO:tasks.workunit.client.0.smithi063.stdout:6/959: unlink d2/d12/ce9 0 2024-05-09T06:38:27.328 INFO:tasks.workunit.client.1.smithi193.stdout:8/553: sync 2024-05-09T06:38:27.329 INFO:tasks.workunit.client.0.smithi063.stdout:0/962: dread dd/d1a/d2c/f43 [0,4194304] 0 2024-05-09T06:38:27.329 INFO:tasks.workunit.client.1.smithi193.stdout:9/667: symlink dd/d1e/d1f/ldb 0 2024-05-09T06:38:27.330 INFO:tasks.workunit.client.0.smithi063.stdout:4/946: unlink d7/d50/deb/l110 0 2024-05-09T06:38:27.331 INFO:tasks.workunit.client.1.smithi193.stdout:3/510: dread d1/db/f6e [0,4194304] 0 2024-05-09T06:38:27.334 INFO:tasks.workunit.client.0.smithi063.stdout:7/913: creat d0/d15/d49/d4f/d66/d67/d86/d110/f138 x:0 0 0 2024-05-09T06:38:27.335 INFO:tasks.workunit.client.1.smithi193.stdout:5/516: truncate d0/f29 414766 0 2024-05-09T06:38:27.336 INFO:tasks.workunit.client.0.smithi063.stdout:6/960: dread d2/d1f/d29/fb1 [0,4194304] 0 2024-05-09T06:38:27.336 INFO:tasks.workunit.client.0.smithi063.stdout:7/914: chown d0/d15/d4a/d50/l7c 3773 1 2024-05-09T06:38:27.337 INFO:tasks.workunit.client.1.smithi193.stdout:5/517: truncate d0/d13/d2c/d37/d5b/d4e/f7a 745282 0 2024-05-09T06:38:27.339 INFO:tasks.workunit.client.1.smithi193.stdout:9/668: creat dd/df/d15/fdc x:0 0 0 2024-05-09T06:38:27.341 INFO:tasks.workunit.client.0.smithi063.stdout:7/915: dwrite d0/d15/d4a/dc8/fe9 [0,4194304] 0 2024-05-09T06:38:27.343 INFO:tasks.workunit.client.1.smithi193.stdout:3/511: rename d1/db/d2f/fad to d1/d2d/d51/fbd 0 2024-05-09T06:38:27.349 INFO:tasks.workunit.client.0.smithi063.stdout:0/963: write dd/d1a/f2e [1951015,19575] 0 2024-05-09T06:38:27.350 INFO:tasks.workunit.client.0.smithi063.stdout:9/928: getdents d0/d4/d7d/d105 0 2024-05-09T06:38:27.352 INFO:tasks.workunit.client.1.smithi193.stdout:5/518: rename d0/f9b to d0/d13/d2c/d37/d5b/d4e/d9d/fa8 0 2024-05-09T06:38:27.353 INFO:tasks.workunit.client.0.smithi063.stdout:6/961: write d2/d1f/d29/de2/d4e/f121 [1032852,82904] 0 2024-05-09T06:38:27.354 INFO:tasks.workunit.client.0.smithi063.stdout:6/962: chown d2/d20/d4a 8156 1 2024-05-09T06:38:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:26 smithi063 ceph-mon[27171]: pgmap v16: 65 pgs: 65 active+clean; 2.5 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 69 MiB/s rd, 198 MiB/s wr, 399 op/s 2024-05-09T06:38:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:26 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:26 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:27.355 INFO:tasks.workunit.client.1.smithi193.stdout:8/554: creat d6/d1c/d1b/fa7 x:0 0 0 2024-05-09T06:38:27.357 INFO:tasks.workunit.client.0.smithi063.stdout:5/927: creat d8/d65/f139 x:0 0 0 2024-05-09T06:38:27.357 INFO:tasks.workunit.client.0.smithi063.stdout:5/928: fdatasync d8/d17/d75/dc9/dd5/fa7 0 2024-05-09T06:38:27.358 INFO:tasks.workunit.client.0.smithi063.stdout:5/929: chown d8/d35/db6/dc2/d138/f9f 248660 1 2024-05-09T06:38:27.359 INFO:tasks.workunit.client.1.smithi193.stdout:9/669: symlink dd/df/ldd 0 2024-05-09T06:38:27.363 INFO:tasks.workunit.client.0.smithi063.stdout:4/947: dwrite d7/df/d77/fe4 [0,4194304] 0 2024-05-09T06:38:27.364 INFO:tasks.workunit.client.1.smithi193.stdout:3/512: symlink d1/db/d2f/d84/d97/lbe 0 2024-05-09T06:38:27.365 INFO:tasks.workunit.client.0.smithi063.stdout:4/948: truncate d7/d3d/d73/f116 239994 0 2024-05-09T06:38:27.367 INFO:tasks.workunit.client.0.smithi063.stdout:6/963: symlink d2/d20/d26/l137 0 2024-05-09T06:38:27.368 INFO:tasks.workunit.client.0.smithi063.stdout:6/964: dread - d2/df/d59/d99/d9f/f134 zero size 2024-05-09T06:38:27.368 INFO:tasks.workunit.client.0.smithi063.stdout:4/949: read d7/da3/d28/d31/d32/d34/f10a [574334,92376] 0 2024-05-09T06:38:27.370 INFO:tasks.workunit.client.0.smithi063.stdout:9/929: write d0/d4/d57/d60/df5/f10b [161324,100395] 0 2024-05-09T06:38:27.373 INFO:tasks.workunit.client.0.smithi063.stdout:4/950: dread d7/df/d35/da5/fb9 [0,4194304] 0 2024-05-09T06:38:27.374 INFO:tasks.workunit.client.0.smithi063.stdout:4/951: chown d7/df/d57/l87 1997 1 2024-05-09T06:38:27.375 INFO:tasks.workunit.client.0.smithi063.stdout:1/878: sync 2024-05-09T06:38:27.375 INFO:tasks.workunit.client.0.smithi063.stdout:3/986: sync 2024-05-09T06:38:27.377 INFO:tasks.workunit.client.0.smithi063.stdout:1/879: readlink d9/d2a/d35/d1e/d9d/d3f/d59/d62/l118 0 2024-05-09T06:38:27.377 INFO:tasks.workunit.client.0.smithi063.stdout:0/964: mkdir dd/d1a/d2c/d51/d126 0 2024-05-09T06:38:27.378 INFO:tasks.workunit.client.1.smithi193.stdout:3/513: mkdir d1/db/d1b/dbf 0 2024-05-09T06:38:27.379 INFO:tasks.workunit.client.1.smithi193.stdout:8/555: unlink d6/d1c/d1b/d5d/d84/d87/ca1 0 2024-05-09T06:38:27.379 INFO:tasks.workunit.client.1.smithi193.stdout:9/670: write dd/df/d15/d8c/f50 [171609,51649] 0 2024-05-09T06:38:27.381 INFO:tasks.workunit.client.0.smithi063.stdout:6/965: symlink d2/d1f/l138 0 2024-05-09T06:38:27.381 INFO:tasks.workunit.client.0.smithi063.stdout:3/987: dread d7/da/d2b/d66/d74/d7e/f7f [0,4194304] 0 2024-05-09T06:38:27.382 INFO:tasks.workunit.client.1.smithi193.stdout:3/514: chown d1/db/d2f/d84/fb7 186859 1 2024-05-09T06:38:27.382 INFO:tasks.workunit.client.0.smithi063.stdout:6/966: chown d2/d1f/d6c/lb5 6 1 2024-05-09T06:38:27.388 INFO:tasks.workunit.client.1.smithi193.stdout:5/519: dwrite d0/d13/d2c/d37/d2b/d47/d66/f26 [0,4194304] 0 2024-05-09T06:38:27.388 INFO:tasks.workunit.client.1.smithi193.stdout:8/556: dwrite d6/fd [0,4194304] 0 2024-05-09T06:38:27.388 INFO:tasks.workunit.client.0.smithi063.stdout:4/952: mknod d7/d3d/dca/c132 0 2024-05-09T06:38:27.389 INFO:tasks.workunit.client.0.smithi063.stdout:3/988: dwrite d7/d22/d10a/f129 [0,4194304] 0 2024-05-09T06:38:27.398 INFO:tasks.workunit.client.0.smithi063.stdout:9/930: write d0/d4/d57/d60/df5/f102 [162448,89133] 0 2024-05-09T06:38:27.402 INFO:tasks.workunit.client.0.smithi063.stdout:6/967: symlink d2/d9a/l139 0 2024-05-09T06:38:27.404 INFO:tasks.workunit.client.0.smithi063.stdout:5/930: dwrite d8/f21 [0,4194304] 0 2024-05-09T06:38:27.404 INFO:tasks.workunit.client.0.smithi063.stdout:7/916: link d0/fc d0/d15/d4a/d50/d10f/f139 0 2024-05-09T06:38:27.414 INFO:tasks.workunit.client.0.smithi063.stdout:0/965: rename dd/f1d to dd/d1a/d2c/d51/d72/d9e/d23/f127 0 2024-05-09T06:38:27.417 INFO:tasks.workunit.client.0.smithi063.stdout:9/931: mkdir d0/d4/d26/d47/d138 0 2024-05-09T06:38:27.421 INFO:tasks.workunit.client.0.smithi063.stdout:5/931: creat d8/d53/f13a x:0 0 0 2024-05-09T06:38:27.425 INFO:tasks.workunit.client.0.smithi063.stdout:4/953: creat d7/df/d35/d5a/d60/d124/f133 x:0 0 0 2024-05-09T06:38:27.430 INFO:tasks.workunit.client.0.smithi063.stdout:1/880: dread d9/d2a/d35/f44 [0,4194304] 0 2024-05-09T06:38:27.430 INFO:tasks.workunit.client.0.smithi063.stdout:9/932: mknod d0/d4/d26/d39/d48/d80/d84/dc2/c139 0 2024-05-09T06:38:27.433 INFO:tasks.workunit.client.0.smithi063.stdout:5/932: mkdir d8/d17/d18/d9c/d13b 0 2024-05-09T06:38:27.435 INFO:tasks.workunit.client.1.smithi193.stdout:8/557: symlink d6/d1c/d18/d66/d93/d95/la8 0 2024-05-09T06:38:27.436 INFO:tasks.workunit.client.1.smithi193.stdout:8/558: write d6/d5b/f8d [1036595,81578] 0 2024-05-09T06:38:27.436 INFO:tasks.workunit.client.0.smithi063.stdout:5/933: write d8/d19/d115/deb/d129/f131 [257172,7225] 0 2024-05-09T06:38:27.437 INFO:tasks.workunit.client.0.smithi063.stdout:7/917: fsync d0/d15/d4a/d50/d10f/f139 0 2024-05-09T06:38:27.437 INFO:tasks.workunit.client.0.smithi063.stdout:3/989: dwrite f2 [4194304,4194304] 0 2024-05-09T06:38:27.440 INFO:tasks.workunit.client.1.smithi193.stdout:9/671: symlink dd/d1e/d93/dbb/lde 0 2024-05-09T06:38:27.440 INFO:tasks.workunit.client.0.smithi063.stdout:3/990: write d7/d70/d98/d2d/ffb [3483853,32610] 0 2024-05-09T06:38:27.441 INFO:tasks.workunit.client.0.smithi063.stdout:3/991: dread - d7/da/d21/d30/f151 zero size 2024-05-09T06:38:27.441 INFO:tasks.workunit.client.1.smithi193.stdout:3/515: symlink d1/db/d2f/d78/d86/d94/lc0 0 2024-05-09T06:38:27.442 INFO:tasks.workunit.client.1.smithi193.stdout:3/516: chown d1/db/d59/db8 383174 1 2024-05-09T06:38:27.444 INFO:tasks.workunit.client.0.smithi063.stdout:2/970: sync 2024-05-09T06:38:27.444 INFO:tasks.workunit.client.0.smithi063.stdout:2/971: readlink dc/l19 0 2024-05-09T06:38:27.445 INFO:tasks.workunit.client.0.smithi063.stdout:1/881: stat d9/d2a/d35/d24/d68/l84 0 2024-05-09T06:38:27.447 INFO:tasks.workunit.client.0.smithi063.stdout:9/933: creat d0/da5/f13a x:0 0 0 2024-05-09T06:38:27.447 INFO:tasks.workunit.client.0.smithi063.stdout:9/934: readlink d0/d8/lf9 0 2024-05-09T06:38:27.449 INFO:tasks.workunit.client.1.smithi193.stdout:9/672: truncate dd/df/d15/f96 1335742 0 2024-05-09T06:38:27.452 INFO:tasks.workunit.client.1.smithi193.stdout:3/517: creat d1/db/d2f/fc1 x:0 0 0 2024-05-09T06:38:27.452 INFO:tasks.workunit.client.0.smithi063.stdout:6/968: dwrite d2/d1f/d6c/fb7 [0,4194304] 0 2024-05-09T06:38:27.453 INFO:tasks.workunit.client.1.smithi193.stdout:9/673: mknod dd/d1e/d1f/d61/cdf 0 2024-05-09T06:38:27.460 INFO:tasks.workunit.client.1.smithi193.stdout:3/518: dwrite d1/db/d2f/f66 [0,4194304] 0 2024-05-09T06:38:27.466 INFO:tasks.workunit.client.1.smithi193.stdout:9/674: dwrite dd/d1e/dc2/f8d [0,4194304] 0 2024-05-09T06:38:27.466 INFO:tasks.workunit.client.1.smithi193.stdout:3/519: write d1/d2d/d51/f29 [3488931,40189] 0 2024-05-09T06:38:27.468 INFO:tasks.workunit.client.0.smithi063.stdout:9/935: stat d0/d4/d7d/d105/d110/l2f 0 2024-05-09T06:38:27.469 INFO:tasks.workunit.client.0.smithi063.stdout:4/954: write d7/d3d/f55 [1718612,95960] 0 2024-05-09T06:38:27.470 INFO:tasks.workunit.client.1.smithi193.stdout:8/559: dread d6/d1c/d18/d66/f62 [0,4194304] 0 2024-05-09T06:38:27.471 INFO:tasks.workunit.client.0.smithi063.stdout:4/955: chown d7/fb 12 1 2024-05-09T06:38:27.471 INFO:tasks.workunit.client.1.smithi193.stdout:8/560: chown d6/d5b/f8d 2224 1 2024-05-09T06:38:27.475 INFO:tasks.workunit.client.1.smithi193.stdout:3/520: creat d1/db/d59/fc2 x:0 0 0 2024-05-09T06:38:27.476 INFO:tasks.workunit.client.0.smithi063.stdout:5/934: symlink d8/d17/d18/d9c/d13b/l13c 0 2024-05-09T06:38:27.477 INFO:tasks.workunit.client.1.smithi193.stdout:9/675: symlink dd/d1e/d3c/le0 0 2024-05-09T06:38:27.479 INFO:tasks.workunit.client.0.smithi063.stdout:6/969: fsync d2/d1f/d29/d3f/d5c/f70 0 2024-05-09T06:38:27.484 INFO:tasks.workunit.client.1.smithi193.stdout:9/676: creat dd/d1e/dc2/db9/fe1 x:0 0 0 2024-05-09T06:38:27.484 INFO:tasks.workunit.client.0.smithi063.stdout:3/992: write d7/da/d2b/d66/d74/d7e/f45 [1067099,8991] 0 2024-05-09T06:38:27.486 INFO:tasks.workunit.client.1.smithi193.stdout:3/521: dread d1/d2d/d51/f7e [0,4194304] 0 2024-05-09T06:38:27.487 INFO:tasks.workunit.client.0.smithi063.stdout:3/993: dread d7/da/d2b/d66/dc0/faa [0,4194304] 0 2024-05-09T06:38:27.490 INFO:tasks.workunit.client.1.smithi193.stdout:8/561: creat d6/d1c/d18/d2f/d40/d55/d7e/fa9 x:0 0 0 2024-05-09T06:38:27.492 INFO:tasks.workunit.client.0.smithi063.stdout:5/935: mknod d8/d65/dbd/c13d 0 2024-05-09T06:38:27.492 INFO:tasks.workunit.client.1.smithi193.stdout:9/677: getdents dd/dd9 0 2024-05-09T06:38:27.494 INFO:tasks.workunit.client.0.smithi063.stdout:3/994: dwrite d7/da/d21/d30/d44/dee/d10c/f106 [0,4194304] 0 2024-05-09T06:38:27.495 INFO:tasks.workunit.client.0.smithi063.stdout:3/995: chown d7/da/d21/d30/d44 28 1 2024-05-09T06:38:27.495 INFO:tasks.workunit.client.0.smithi063.stdout:3/996: chown d7/da/d21/d30/d44/d48 16157 1 2024-05-09T06:38:27.496 INFO:tasks.workunit.client.1.smithi193.stdout:9/678: rmdir dd/df/d31/d7e/d37 39 2024-05-09T06:38:27.500 INFO:tasks.workunit.client.0.smithi063.stdout:2/972: link dc/f27 dc/d36/d43/d46/d63/d79/d7d/d8d/f127 0 2024-05-09T06:38:27.501 INFO:tasks.workunit.client.0.smithi063.stdout:6/970: dread d2/fa9 [0,4194304] 0 2024-05-09T06:38:27.501 INFO:tasks.workunit.client.1.smithi193.stdout:3/522: dread d1/db/d59/f52 [0,4194304] 0 2024-05-09T06:38:27.502 INFO:tasks.workunit.client.0.smithi063.stdout:7/918: creat d0/d15/d49/d54/d9f/d97/d9b/f13a x:0 0 0 2024-05-09T06:38:27.503 INFO:tasks.workunit.client.0.smithi063.stdout:4/956: link d7/da3/d28/d31/d32/f11c d7/d50/d10d/f134 0 2024-05-09T06:38:27.505 INFO:tasks.workunit.client.1.smithi193.stdout:9/679: dwrite dd/df/d31/d7e/d37/f7c [0,4194304] 0 2024-05-09T06:38:27.506 INFO:tasks.workunit.client.0.smithi063.stdout:0/966: rmdir dd/d1a/d2c/d51/d72/d9e/dec 0 2024-05-09T06:38:27.508 INFO:tasks.workunit.client.1.smithi193.stdout:3/523: dread d1/f6d [0,4194304] 0 2024-05-09T06:38:27.508 INFO:tasks.workunit.client.0.smithi063.stdout:5/936: link d8/d17/d75/dc9/dd5/fa7 d8/d19/d105/f13e 0 2024-05-09T06:38:27.509 INFO:tasks.workunit.client.1.smithi193.stdout:3/524: stat d1/d2d/d51/c3a 0 2024-05-09T06:38:27.510 INFO:tasks.workunit.client.1.smithi193.stdout:9/680: unlink dd/c1a 0 2024-05-09T06:38:27.511 INFO:tasks.workunit.client.0.smithi063.stdout:9/936: dwrite d0/d4/d26/d39/d3e/ff1 [0,4194304] 0 2024-05-09T06:38:27.512 INFO:tasks.workunit.client.1.smithi193.stdout:4/629: sync 2024-05-09T06:38:27.512 INFO:tasks.workunit.client.1.smithi193.stdout:2/567: sync 2024-05-09T06:38:27.512 INFO:tasks.workunit.client.0.smithi063.stdout:4/957: unlink d7/c25 0 2024-05-09T06:38:27.513 INFO:tasks.workunit.client.0.smithi063.stdout:5/937: write d8/d53/f13a [916926,97316] 0 2024-05-09T06:38:27.514 INFO:tasks.workunit.client.0.smithi063.stdout:3/997: dwrite d7/d70/d98/dbb/d10d/f132 [0,4194304] 0 2024-05-09T06:38:27.514 INFO:tasks.workunit.client.1.smithi193.stdout:9/681: dread - dd/d1e/dc2/db9/fd0 zero size 2024-05-09T06:38:27.514 INFO:tasks.workunit.client.1.smithi193.stdout:3/525: truncate d1/db/d59/d63/fae 870494 0 2024-05-09T06:38:27.514 INFO:tasks.workunit.client.1.smithi193.stdout:4/630: chown d1a/d24/f41 0 1 2024-05-09T06:38:27.515 INFO:tasks.workunit.client.1.smithi193.stdout:2/568: dread - db/d25/d38/f9d zero size 2024-05-09T06:38:27.516 INFO:tasks.workunit.client.1.smithi193.stdout:8/562: write d6/d1c/d1b/d29/f4f [1283739,26838] 0 2024-05-09T06:38:27.541 INFO:tasks.workunit.client.1.smithi193.stdout:9/682: dread dd/df/f6e [0,4194304] 0 2024-05-09T06:38:27.543 INFO:tasks.workunit.client.1.smithi193.stdout:4/631: dread d1a/d1f/f20 [0,4194304] 0 2024-05-09T06:38:27.605 INFO:tasks.workunit.client.0.smithi063.stdout:1/882: sync 2024-05-09T06:38:27.611 INFO:tasks.workunit.client.0.smithi063.stdout:5/938: mkdir d8/d65/d9e/d13f 0 2024-05-09T06:38:27.612 INFO:tasks.workunit.client.1.smithi193.stdout:3/526: creat d1/d44/d7d/fc3 x:0 0 0 2024-05-09T06:38:27.613 INFO:tasks.workunit.client.1.smithi193.stdout:2/569: mkdir db/d37/d6d/dc4 0 2024-05-09T06:38:27.614 INFO:tasks.workunit.client.1.smithi193.stdout:2/570: write db/f44 [4771194,94572] 0 2024-05-09T06:38:27.635 INFO:tasks.workunit.client.0.smithi063.stdout:2/973: sync 2024-05-09T06:38:27.691 INFO:tasks.workunit.client.1.smithi193.stdout:1/623: sync 2024-05-09T06:38:27.696 INFO:tasks.workunit.client.0.smithi063.stdout:3/998: dwrite d7/da/d2b/d66/dc0/fa2 [0,4194304] 0 2024-05-09T06:38:27.696 INFO:tasks.workunit.client.1.smithi193.stdout:9/683: mknod dd/d1e/d1f/d48/ce2 0 2024-05-09T06:38:27.699 INFO:tasks.workunit.client.0.smithi063.stdout:0/967: creat dd/d60/d44/d90/da4/f128 x:0 0 0 2024-05-09T06:38:27.700 INFO:tasks.workunit.client.1.smithi193.stdout:4/632: symlink d1a/d80/lc5 0 2024-05-09T06:38:27.700 INFO:tasks.workunit.client.0.smithi063.stdout:0/968: write dd/d60/d44/f61 [1641971,73786] 0 2024-05-09T06:38:27.701 INFO:tasks.workunit.client.1.smithi193.stdout:4/633: write d1a/d24/f40 [6608371,629] 0 2024-05-09T06:38:27.701 INFO:tasks.workunit.client.0.smithi063.stdout:9/937: write d0/f98 [520253,129631] 0 2024-05-09T06:38:27.703 INFO:tasks.workunit.client.0.smithi063.stdout:1/883: creat d9/d2a/d3c/dbc/d114/f130 x:0 0 0 2024-05-09T06:38:27.704 INFO:tasks.workunit.client.0.smithi063.stdout:9/938: write d0/d4/d26/d39/d48/d80/d84/fc7 [884257,10721] 0 2024-05-09T06:38:27.705 INFO:tasks.workunit.client.0.smithi063.stdout:1/884: write d9/d2a/d39/f105 [3969478,19616] 0 2024-05-09T06:38:27.705 INFO:tasks.workunit.client.0.smithi063.stdout:5/939: mknod d8/d35/db6/c140 0 2024-05-09T06:38:27.705 INFO:tasks.workunit.client.1.smithi193.stdout:8/563: dread d6/d1c/f12 [0,4194304] 0 2024-05-09T06:38:27.707 INFO:tasks.workunit.client.1.smithi193.stdout:2/571: rmdir db/d37/d6d/d80/d60/d7d/db3 39 2024-05-09T06:38:27.715 INFO:tasks.workunit.client.1.smithi193.stdout:3/527: creat d1/db/d1b/dbf/fc4 x:0 0 0 2024-05-09T06:38:27.715 INFO:tasks.workunit.client.0.smithi063.stdout:4/958: dwrite d7/df/d1a/f96 [0,4194304] 0 2024-05-09T06:38:27.717 INFO:tasks.workunit.client.0.smithi063.stdout:4/959: dread - d7/da3/d28/d31/d32/d34/db4/dcc/de1/f122 zero size 2024-05-09T06:38:27.720 INFO:tasks.workunit.client.1.smithi193.stdout:9/684: write dd/df/d31/d7e/f4b [974076,18687] 0 2024-05-09T06:38:27.727 INFO:tasks.workunit.client.1.smithi193.stdout:8/564: dread d6/d1c/d1b/d5d/d84/d87/fa3 [0,4194304] 0 2024-05-09T06:38:27.729 INFO:tasks.workunit.client.1.smithi193.stdout:3/528: unlink d1/d2d/l42 0 2024-05-09T06:38:27.730 INFO:tasks.workunit.client.1.smithi193.stdout:3/529: chown d1/db/d2f/d78/d86/la7 3 1 2024-05-09T06:38:27.734 INFO:tasks.workunit.client.1.smithi193.stdout:4/634: dwrite d1a/d22/d5d/f93 [0,4194304] 0 2024-05-09T06:38:27.737 INFO:tasks.workunit.client.1.smithi193.stdout:9/685: unlink dd/df/d31/d7e/l8f 0 2024-05-09T06:38:27.738 INFO:tasks.workunit.client.1.smithi193.stdout:3/530: dwrite d1/db/d59/d63/fae [0,4194304] 0 2024-05-09T06:38:27.743 INFO:tasks.workunit.client.1.smithi193.stdout:2/572: write db/d25/f95 [841754,124087] 0 2024-05-09T06:38:27.746 INFO:tasks.workunit.client.1.smithi193.stdout:9/686: rmdir dd/df/d31/d7e/d37 39 2024-05-09T06:38:27.747 INFO:tasks.workunit.client.1.smithi193.stdout:8/565: mkdir d6/d1c/d1b/d5d/d84/d87/da5/daa 0 2024-05-09T06:38:27.748 INFO:tasks.workunit.client.1.smithi193.stdout:8/566: fdatasync d6/d1c/d1b/d29/f91 0 2024-05-09T06:38:27.748 INFO:tasks.workunit.client.1.smithi193.stdout:3/531: rmdir d1/db/d1b/dbf 39 2024-05-09T06:38:27.752 INFO:tasks.workunit.client.1.smithi193.stdout:4/635: mkdir d1a/d24/d73/d94/d7c/dc6 0 2024-05-09T06:38:27.754 INFO:tasks.workunit.client.1.smithi193.stdout:9/687: creat dd/df/d31/d7e/d71/fe3 x:0 0 0 2024-05-09T06:38:27.755 INFO:tasks.workunit.client.1.smithi193.stdout:4/636: truncate d1a/d1f/f20 5014911 0 2024-05-09T06:38:27.757 INFO:tasks.workunit.client.1.smithi193.stdout:8/567: stat d6/d1c/c94 0 2024-05-09T06:38:27.757 INFO:tasks.workunit.client.1.smithi193.stdout:3/532: dwrite d1/db/d2f/fab [0,4194304] 0 2024-05-09T06:38:27.758 INFO:tasks.workunit.client.1.smithi193.stdout:3/533: chown d1/db/d1b/l6b 468 1 2024-05-09T06:38:27.758 INFO:tasks.workunit.client.1.smithi193.stdout:3/534: chown d1/db/da1 31530 1 2024-05-09T06:38:27.762 INFO:tasks.workunit.client.1.smithi193.stdout:4/637: read d1a/d6b/f77 [54528,86307] 0 2024-05-09T06:38:27.769 INFO:tasks.workunit.client.1.smithi193.stdout:8/568: dwrite d6/d1c/f34 [0,4194304] 0 2024-05-09T06:38:27.775 INFO:tasks.workunit.client.1.smithi193.stdout:2/573: dread db/d25/f48 [0,4194304] 0 2024-05-09T06:38:27.775 INFO:tasks.workunit.client.1.smithi193.stdout:8/569: dread d6/d1c/d18/d66/d93/f6c [0,4194304] 0 2024-05-09T06:38:27.775 INFO:tasks.workunit.client.1.smithi193.stdout:2/574: chown db/df/db6 262670233 1 2024-05-09T06:38:27.784 INFO:tasks.workunit.client.1.smithi193.stdout:9/688: rename dd/df/d31/d7e/d37/f79 to dd/df/d31/d7e/fe4 0 2024-05-09T06:38:27.787 INFO:tasks.workunit.client.1.smithi193.stdout:4/638: creat d1a/d24/d73/d94/dbd/fc7 x:0 0 0 2024-05-09T06:38:27.788 INFO:tasks.workunit.client.1.smithi193.stdout:4/639: chown d1a/d1f/f2a 678 1 2024-05-09T06:38:27.789 INFO:tasks.workunit.client.1.smithi193.stdout:4/640: dread - d1a/d6b/d60/fb1 zero size 2024-05-09T06:38:27.792 INFO:tasks.workunit.client.1.smithi193.stdout:9/689: rename dd/d1e/c67 to dd/df/d15/d8c/ce5 0 2024-05-09T06:38:27.794 INFO:tasks.workunit.client.1.smithi193.stdout:8/570: dread d6/f23 [0,4194304] 0 2024-05-09T06:38:27.800 INFO:tasks.workunit.client.1.smithi193.stdout:2/575: truncate db/df/f10 4002286 0 2024-05-09T06:38:27.802 INFO:tasks.workunit.client.1.smithi193.stdout:9/690: mkdir dd/df/d15/d8c/de6 0 2024-05-09T06:38:27.810 INFO:tasks.workunit.client.1.smithi193.stdout:2/576: dwrite db/d37/d6d/d80/d60/d7d/f7e [0,4194304] 0 2024-05-09T06:38:27.814 INFO:tasks.workunit.client.1.smithi193.stdout:3/535: dwrite d1/d44/d7d/f9c [0,4194304] 0 2024-05-09T06:38:27.815 INFO:tasks.workunit.client.0.smithi063.stdout:2/974: rename dc/dd/d45/d116 to dc/dd/d2d/d60/dd0/d7b/d128 0 2024-05-09T06:38:27.821 INFO:tasks.workunit.client.1.smithi193.stdout:9/691: dread dd/df/d15/f8e [0,4194304] 0 2024-05-09T06:38:27.821 INFO:tasks.workunit.client.0.smithi063.stdout:3/999: symlink d7/da/d21/d30/d44/d48/l15b 0 2024-05-09T06:38:27.826 INFO:tasks.workunit.client.1.smithi193.stdout:9/692: dread dd/f73 [0,4194304] 0 2024-05-09T06:38:27.828 INFO:tasks.workunit.client.1.smithi193.stdout:8/571: getdents d6/d1c 0 2024-05-09T06:38:27.831 INFO:tasks.workunit.client.1.smithi193.stdout:9/693: dread dd/d1e/dc2/d49/dbd/fc9 [0,4194304] 0 2024-05-09T06:38:27.835 INFO:tasks.workunit.client.0.smithi063.stdout:1/885: write d9/d2a/d39/f5e [4028564,126795] 0 2024-05-09T06:38:27.835 INFO:tasks.workunit.client.1.smithi193.stdout:9/694: truncate dd/df/d31/d7e/f35 499593 0 2024-05-09T06:38:27.837 INFO:tasks.workunit.client.0.smithi063.stdout:5/940: symlink d8/d19/d84/l141 0 2024-05-09T06:38:27.837 INFO:tasks.workunit.client.0.smithi063.stdout:1/886: write d9/d2a/d3c/d4c/dc3/f111 [96154,45343] 0 2024-05-09T06:38:27.841 INFO:tasks.workunit.client.0.smithi063.stdout:6/971: getdents d2/d20/d26 0 2024-05-09T06:38:27.845 INFO:tasks.workunit.client.0.smithi063.stdout:2/975: unlink dc/dd/d2d/d60/dd0/d90/l71 0 2024-05-09T06:38:27.845 INFO:tasks.workunit.client.0.smithi063.stdout:7/919: link d0/dcb/ce5 d0/d15/d49/d54/c13b 0 2024-05-09T06:38:27.846 INFO:tasks.workunit.client.0.smithi063.stdout:7/920: chown d0/d15/d89/d10c/de7/l112 120784 1 2024-05-09T06:38:27.848 INFO:tasks.workunit.client.0.smithi063.stdout:5/941: chown d8/d17/d18/f45 42152 1 2024-05-09T06:38:27.850 INFO:tasks.workunit.client.0.smithi063.stdout:1/887: chown d9/d2a/d35/l20 60488038 1 2024-05-09T06:38:27.851 INFO:tasks.workunit.client.1.smithi193.stdout:8/572: dread d6/d1c/d1b/d29/f91 [0,4194304] 0 2024-05-09T06:38:27.852 INFO:tasks.workunit.client.0.smithi063.stdout:2/976: creat dc/dd/d45/d5a/d6c/f129 x:0 0 0 2024-05-09T06:38:27.853 INFO:tasks.workunit.client.1.smithi193.stdout:8/573: rmdir d6/d1c/d1b/d5d/d84 39 2024-05-09T06:38:27.857 INFO:tasks.workunit.client.1.smithi193.stdout:3/536: dwrite d1/db/d1b/f70 [0,4194304] 0 2024-05-09T06:38:27.857 INFO:tasks.workunit.client.0.smithi063.stdout:7/921: dread - d0/d15/d49/d54/d9f/dd2/df9/f115 zero size 2024-05-09T06:38:27.858 INFO:tasks.workunit.client.1.smithi193.stdout:8/574: mknod d6/d1c/d18/d66/d93/cab 0 2024-05-09T06:38:27.859 INFO:tasks.workunit.client.0.smithi063.stdout:5/942: fsync d8/d17/f1c 0 2024-05-09T06:38:27.864 INFO:tasks.workunit.client.0.smithi063.stdout:6/972: mknod d2/d4/c13a 0 2024-05-09T06:38:27.865 INFO:tasks.workunit.client.1.smithi193.stdout:3/537: mknod d1/db/d2f/d84/cc5 0 2024-05-09T06:38:27.867 INFO:tasks.workunit.client.1.smithi193.stdout:2/577: truncate db/df/f10 821833 0 2024-05-09T06:38:27.868 INFO:tasks.workunit.client.1.smithi193.stdout:2/578: write db/d25/fa9 [188793,123603] 0 2024-05-09T06:38:27.870 INFO:tasks.workunit.client.0.smithi063.stdout:2/977: rename dc/dd/d45/d5a/ca7 to dc/d32/d112/d11d/c12a 0 2024-05-09T06:38:27.871 INFO:tasks.workunit.client.0.smithi063.stdout:2/978: chown dc/d32/de5/ff2 108 1 2024-05-09T06:38:27.871 INFO:tasks.workunit.client.1.smithi193.stdout:3/538: symlink d1/db/d2f/d78/lc6 0 2024-05-09T06:38:27.873 INFO:tasks.workunit.client.1.smithi193.stdout:9/695: truncate dd/d1e/d1f/d48/d55/d41/d68/f78 2205420 0 2024-05-09T06:38:27.873 INFO:tasks.workunit.client.1.smithi193.stdout:2/579: dread - db/df/f94 zero size 2024-05-09T06:38:27.874 INFO:tasks.workunit.client.0.smithi063.stdout:9/939: getdents d0/d109 0 2024-05-09T06:38:27.875 INFO:tasks.workunit.client.0.smithi063.stdout:0/969: write dd/d60/d119/d8f/dfd/f11c [878741,37980] 0 2024-05-09T06:38:27.876 INFO:tasks.workunit.client.0.smithi063.stdout:9/940: truncate d0/d4/d57/f127 846469 0 2024-05-09T06:38:27.876 INFO:tasks.workunit.client.1.smithi193.stdout:8/575: rename d6/d1c/d1b/d5d/d84/d87/fa3 to d6/d1c/d1b/d5d/d84/d87/da5/daa/fac 0 2024-05-09T06:38:27.877 INFO:tasks.workunit.client.0.smithi063.stdout:7/922: fdatasync d0/d15/f108 0 2024-05-09T06:38:27.878 INFO:tasks.workunit.client.1.smithi193.stdout:3/539: fsync d1/d2d/f45 0 2024-05-09T06:38:27.879 INFO:tasks.workunit.client.1.smithi193.stdout:3/540: chown d1/d2d/d51/c53 0 1 2024-05-09T06:38:27.880 INFO:tasks.workunit.client.1.smithi193.stdout:3/541: chown d1/d2d/f45 379 1 2024-05-09T06:38:27.881 INFO:tasks.workunit.client.1.smithi193.stdout:3/542: write d1/f8 [1320604,127023] 0 2024-05-09T06:38:27.881 INFO:tasks.workunit.client.1.smithi193.stdout:9/696: symlink dd/le7 0 2024-05-09T06:38:27.881 INFO:tasks.workunit.client.0.smithi063.stdout:9/941: dread d0/d4/d7d/d105/d110/fbe [0,4194304] 0 2024-05-09T06:38:27.883 INFO:tasks.workunit.client.0.smithi063.stdout:0/970: dwrite dd/d1a/d2c/d51/d72/d9e/d23/d31/d58/f6e [0,4194304] 0 2024-05-09T06:38:27.887 INFO:tasks.workunit.client.1.smithi193.stdout:8/576: creat d6/d1c/d18/d66/d93/fad x:0 0 0 2024-05-09T06:38:27.889 INFO:tasks.workunit.client.0.smithi063.stdout:2/979: creat dc/d36/d43/d46/d63/d9e/f12b x:0 0 0 2024-05-09T06:38:27.890 INFO:tasks.workunit.client.1.smithi193.stdout:1/624: sync 2024-05-09T06:38:27.890 INFO:tasks.workunit.client.1.smithi193.stdout:5/520: sync 2024-05-09T06:38:27.890 INFO:tasks.workunit.client.1.smithi193.stdout:0/543: sync 2024-05-09T06:38:27.890 INFO:tasks.workunit.client.1.smithi193.stdout:7/533: sync 2024-05-09T06:38:27.890 INFO:tasks.workunit.client.1.smithi193.stdout:6/529: sync 2024-05-09T06:38:27.892 INFO:tasks.workunit.client.1.smithi193.stdout:0/544: write d18/d65/f9f [442442,119525] 0 2024-05-09T06:38:27.892 INFO:tasks.workunit.client.0.smithi063.stdout:1/888: dwrite d9/d2a/d35/d24/d68/f98 [0,4194304] 0 2024-05-09T06:38:27.892 INFO:tasks.workunit.client.1.smithi193.stdout:6/530: truncate d1/d25/fb7 912125 0 2024-05-09T06:38:27.895 INFO:tasks.workunit.client.0.smithi063.stdout:1/889: read d9/d2a/d35/d1e/f27 [216973,13316] 0 2024-05-09T06:38:27.896 INFO:tasks.workunit.client.0.smithi063.stdout:1/890: readlink d9/d2a/d35/d1e/d9d/l6b 0 2024-05-09T06:38:27.897 INFO:tasks.workunit.client.1.smithi193.stdout:9/697: creat dd/df/d15/d8c/fe8 x:0 0 0 2024-05-09T06:38:27.903 INFO:tasks.workunit.client.0.smithi063.stdout:7/923: unlink d0/d15/f48 0 2024-05-09T06:38:27.903 INFO:tasks.workunit.client.1.smithi193.stdout:2/580: write db/d25/d38/d5b/f66 [591259,121283] 0 2024-05-09T06:38:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:4/960: dread d7/df/d35/d5a/f97 [0,4194304] 0 2024-05-09T06:38:27.907 INFO:tasks.workunit.client.0.smithi063.stdout:9/942: rename d0/d8/d2d/l40 to d0/d4/d26/d39/d3e/d66/l13b 0 2024-05-09T06:38:27.908 INFO:tasks.workunit.client.1.smithi193.stdout:1/625: write d4/d16/d17/d1a/d3a/d4a/db1/fb4 [441392,29752] 0 2024-05-09T06:38:27.909 INFO:tasks.workunit.client.1.smithi193.stdout:1/626: fsync d4/d16/d60/d4f/fa5 0 2024-05-09T06:38:27.910 INFO:tasks.workunit.client.1.smithi193.stdout:5/521: mkdir d0/d13/d2c/d37/d2b/d47/d60/d75/da9 0 2024-05-09T06:38:27.911 INFO:tasks.workunit.client.0.smithi063.stdout:1/891: stat d9/l5a 0 2024-05-09T06:38:27.914 INFO:tasks.workunit.client.1.smithi193.stdout:3/543: rename d1/db/d59/c71 to d1/db/d2f/d78/d86/d94/cc7 0 2024-05-09T06:38:27.918 INFO:tasks.workunit.client.0.smithi063.stdout:7/924: readlink d0/d15/d49/d54/l78 0 2024-05-09T06:38:27.918 INFO:tasks.workunit.client.0.smithi063.stdout:7/925: write d0/d15/d89/d10c/fd7 [3979883,77885] 0 2024-05-09T06:38:27.921 INFO:tasks.workunit.client.1.smithi193.stdout:1/627: mkdir d4/d16/de/dca 0 2024-05-09T06:38:27.923 INFO:tasks.workunit.client.0.smithi063.stdout:4/961: symlink d7/da3/d28/d31/d32/l135 0 2024-05-09T06:38:27.923 INFO:tasks.workunit.client.1.smithi193.stdout:0/545: mkdir d18/d1d/d56/da7/da8 0 2024-05-09T06:38:27.924 INFO:tasks.workunit.client.1.smithi193.stdout:0/546: fdatasync d18/d4c/d47/f96 0 2024-05-09T06:38:27.926 INFO:tasks.workunit.client.1.smithi193.stdout:9/698: mknod dd/df/d15/d8c/d7a/ce9 0 2024-05-09T06:38:27.930 INFO:tasks.workunit.client.0.smithi063.stdout:2/980: mkdir dc/d36/d43/d46/d12c 0 2024-05-09T06:38:27.931 INFO:tasks.workunit.client.1.smithi193.stdout:0/547: dwrite d18/d4c/f2b [0,4194304] 0 2024-05-09T06:38:27.932 INFO:tasks.workunit.client.0.smithi063.stdout:1/892: chown d9/d2a/d3c/cad 480092 1 2024-05-09T06:38:27.933 INFO:tasks.workunit.client.1.smithi193.stdout:8/577: rename d6/d1c/d1b/d5d/l85 to d6/d1c/d1b/d3f/lae 0 2024-05-09T06:38:27.937 INFO:tasks.workunit.client.1.smithi193.stdout:1/628: dread d4/d16/d60/d4f/d72/f7a [0,4194304] 0 2024-05-09T06:38:27.937 INFO:tasks.workunit.client.0.smithi063.stdout:7/926: dread d0/d15/d49/fbf [4194304,4194304] 0 2024-05-09T06:38:27.939 INFO:tasks.workunit.client.0.smithi063.stdout:5/943: getdents d8/d19/d84 0 2024-05-09T06:38:27.940 INFO:tasks.workunit.client.1.smithi193.stdout:9/699: unlink dd/d1e/dc2/db9/fd0 0 2024-05-09T06:38:27.942 INFO:tasks.workunit.client.0.smithi063.stdout:4/962: write d7/df/d77/f113 [650190,114801] 0 2024-05-09T06:38:27.947 INFO:tasks.workunit.client.0.smithi063.stdout:2/981: unlink dc/d6e/cfa 0 2024-05-09T06:38:27.948 INFO:tasks.workunit.client.1.smithi193.stdout:2/581: link db/d25/d38/d40/d8b/d77/l9a db/d4b/d56/lc5 0 2024-05-09T06:38:27.950 INFO:tasks.workunit.client.1.smithi193.stdout:8/578: fdatasync d6/d1c/d53/f60 0 2024-05-09T06:38:27.951 INFO:tasks.workunit.client.0.smithi063.stdout:7/927: mknod d0/d15/d49/d54/d9f/d106/c13c 0 2024-05-09T06:38:27.953 INFO:tasks.workunit.client.1.smithi193.stdout:1/629: creat d4/d16/d60/d4f/d59/fcb x:0 0 0 2024-05-09T06:38:27.954 INFO:tasks.workunit.client.0.smithi063.stdout:7/928: dread d0/d15/f4e [0,4194304] 0 2024-05-09T06:38:27.957 INFO:tasks.workunit.client.0.smithi063.stdout:0/971: rename dd/d1a/d2c/d51/d72/d9e/d23/d89/da8/dcb to dd/d1a/d2c/d51/d129 0 2024-05-09T06:38:27.958 INFO:tasks.workunit.client.0.smithi063.stdout:0/972: chown dd/d1a/d2c/d51/d72/c106 465210 1 2024-05-09T06:38:27.958 INFO:tasks.workunit.client.1.smithi193.stdout:9/700: mkdir dd/df/d15/d98/dea 0 2024-05-09T06:38:27.958 INFO:tasks.workunit.client.1.smithi193.stdout:9/701: fdatasync dd/d1e/f28 0 2024-05-09T06:38:27.959 INFO:tasks.workunit.client.1.smithi193.stdout:9/702: chown dd/df/d15/d8c/d7a/ce9 0 1 2024-05-09T06:38:27.960 INFO:tasks.workunit.client.0.smithi063.stdout:4/963: creat d7/df/d35/d78/f136 x:0 0 0 2024-05-09T06:38:27.960 INFO:tasks.workunit.client.1.smithi193.stdout:3/544: creat d1/d2d/fc8 x:0 0 0 2024-05-09T06:38:27.961 INFO:tasks.workunit.client.0.smithi063.stdout:9/943: creat d0/d4/d26/d39/d3e/f13c x:0 0 0 2024-05-09T06:38:27.961 INFO:tasks.workunit.client.0.smithi063.stdout:2/982: rmdir dc/d32/d33 39 2024-05-09T06:38:27.963 INFO:tasks.workunit.client.0.smithi063.stdout:7/929: truncate d0/d15/d49/d4f/d66/d67/d6b/f105 81246 0 2024-05-09T06:38:27.964 INFO:tasks.workunit.client.1.smithi193.stdout:1/630: truncate d4/d16/f5f 781496 0 2024-05-09T06:38:27.964 INFO:tasks.workunit.client.0.smithi063.stdout:5/944: dwrite d8/d65/f7a [0,4194304] 0 2024-05-09T06:38:27.966 INFO:tasks.workunit.client.0.smithi063.stdout:4/964: creat d7/d3d/d73/de5/f137 x:0 0 0 2024-05-09T06:38:27.967 INFO:tasks.workunit.client.1.smithi193.stdout:0/548: creat d18/d27/fa9 x:0 0 0 2024-05-09T06:38:27.969 INFO:tasks.workunit.client.1.smithi193.stdout:7/534: rename d6/d8/d29/d2f/c5c to d6/d8/d29/cb6 0 2024-05-09T06:38:27.972 INFO:tasks.workunit.client.0.smithi063.stdout:0/973: dwrite dd/f6b [0,4194304] 0 2024-05-09T06:38:27.973 INFO:tasks.workunit.client.0.smithi063.stdout:0/974: write dd/d1a/d2c/d51/d72/d9e/d23/d31/f4a [4619212,22868] 0 2024-05-09T06:38:27.973 INFO:tasks.workunit.client.0.smithi063.stdout:9/944: mknod d0/d4/d26/d39/d48/d80/d84/dc2/c13d 0 2024-05-09T06:38:27.974 INFO:tasks.workunit.client.0.smithi063.stdout:0/975: fdatasync dd/d1a/f9f 0 2024-05-09T06:38:27.976 INFO:tasks.workunit.client.1.smithi193.stdout:1/631: creat d4/d16/d35/fcc x:0 0 0 2024-05-09T06:38:27.977 INFO:tasks.workunit.client.0.smithi063.stdout:6/973: sync 2024-05-09T06:38:27.980 INFO:tasks.workunit.client.0.smithi063.stdout:6/974: chown d2/df/d59/d99/fc2 148928951 1 2024-05-09T06:38:27.982 INFO:tasks.workunit.client.0.smithi063.stdout:4/965: read - d7/da3/d8b/fbc zero size 2024-05-09T06:38:27.983 INFO:tasks.workunit.client.0.smithi063.stdout:9/945: dwrite d0/d4/d57/dba/dd5/fdb [0,4194304] 0 2024-05-09T06:38:27.986 INFO:tasks.workunit.client.1.smithi193.stdout:6/531: rename d1/d25/d2c/d81/d37/db3/dbd to d1/d25/dc1 0 2024-05-09T06:38:27.988 INFO:tasks.workunit.client.0.smithi063.stdout:1/893: getdents d9/d2a/d35 0 2024-05-09T06:38:27.989 INFO:tasks.workunit.client.1.smithi193.stdout:0/549: dwrite d18/d4c/d5f/f6a [0,4194304] 0 2024-05-09T06:38:27.989 INFO:tasks.workunit.client.1.smithi193.stdout:2/582: dread db/d25/f57 [0,4194304] 0 2024-05-09T06:38:27.990 INFO:tasks.workunit.client.0.smithi063.stdout:1/894: chown d9/d2a/d3c/dbc/d114/f130 0 1 2024-05-09T06:38:27.991 INFO:tasks.workunit.client.1.smithi193.stdout:7/535: rmdir d6/d8/d1a/d36/d4d/d60/d9c 39 2024-05-09T06:38:27.995 INFO:tasks.workunit.client.1.smithi193.stdout:1/632: mkdir d4/d16/d60/d4f/dcd 0 2024-05-09T06:38:27.996 INFO:tasks.workunit.client.1.smithi193.stdout:1/633: stat d4/d16/d17/d1a/d3a/d3f/f68 0 2024-05-09T06:38:27.999 INFO:tasks.workunit.client.1.smithi193.stdout:9/703: write dd/d1e/d1f/d48/d55/d41/d68/f78 [1487366,125372] 0 2024-05-09T06:38:28.006 INFO:tasks.workunit.client.0.smithi063.stdout:0/976: mkdir dd/d60/dcd/d12a 0 2024-05-09T06:38:28.007 INFO:tasks.workunit.client.1.smithi193.stdout:1/634: dwrite d4/d14/f2e [4194304,4194304] 0 2024-05-09T06:38:28.009 INFO:tasks.workunit.client.0.smithi063.stdout:5/945: dread f6 [0,4194304] 0 2024-05-09T06:38:28.010 INFO:tasks.workunit.client.0.smithi063.stdout:4/966: dread d7/da3/d28/d31/d32/f8c [0,4194304] 0 2024-05-09T06:38:28.040 INFO:tasks.workunit.client.1.smithi193.stdout:4/641: sync 2024-05-09T06:38:28.046 INFO:tasks.workunit.client.1.smithi193.stdout:4/642: dwrite d1a/d22/f56 [0,4194304] 0 2024-05-09T06:38:28.076 INFO:tasks.workunit.client.1.smithi193.stdout:5/522: sync 2024-05-09T06:38:28.084 INFO:tasks.workunit.client.0.smithi063.stdout:6/975: mknod d2/d4/d7c/dfc/c13b 0 2024-05-09T06:38:28.088 INFO:tasks.workunit.client.0.smithi063.stdout:9/946: creat d0/d3b/da0/dcf/f13e x:0 0 0 2024-05-09T06:38:28.092 INFO:tasks.workunit.client.0.smithi063.stdout:1/895: read d9/d10/f3e [5551191,86834] 0 2024-05-09T06:38:28.096 INFO:tasks.workunit.client.1.smithi193.stdout:9/704: write dd/d1e/dc2/d49/dbd/d5b/d8a/fc5 [3287895,36603] 0 2024-05-09T06:38:28.104 INFO:tasks.workunit.client.0.smithi063.stdout:6/976: truncate d2/d1f/d6c/f71 4342936 0 2024-05-09T06:38:28.105 INFO:tasks.workunit.client.1.smithi193.stdout:4/643: mkdir d1a/d80/dc8 0 2024-05-09T06:38:28.106 INFO:tasks.workunit.client.0.smithi063.stdout:5/946: dread d8/d19/d115/fe1 [0,4194304] 0 2024-05-09T06:38:28.109 INFO:tasks.workunit.client.0.smithi063.stdout:2/983: creat dc/d36/d43/d46/d49/dbb/f12d x:0 0 0 2024-05-09T06:38:28.113 INFO:tasks.workunit.client.1.smithi193.stdout:2/583: mknod db/d37/d6d/d80/d60/d7d/db3/cc6 0 2024-05-09T06:38:28.114 INFO:tasks.workunit.client.0.smithi063.stdout:6/977: mknod d2/df/d59/d99/d9f/c13c 0 2024-05-09T06:38:28.115 INFO:tasks.workunit.client.0.smithi063.stdout:4/967: dread d7/d3d/f49 [0,4194304] 0 2024-05-09T06:38:28.117 INFO:tasks.workunit.client.0.smithi063.stdout:4/968: chown d7/da3/d28/d31/d32/d34/c54 3970173 1 2024-05-09T06:38:28.124 INFO:tasks.workunit.client.1.smithi193.stdout:5/523: getdents d0/d13/d2c/d37/d2b/d47/d60/d75/da9 0 2024-05-09T06:38:28.125 INFO:tasks.workunit.client.0.smithi063.stdout:5/947: dread d8/d19/f3f [0,4194304] 0 2024-05-09T06:38:28.126 INFO:tasks.workunit.client.1.smithi193.stdout:6/532: rmdir d1/d25/dc1 0 2024-05-09T06:38:28.128 INFO:tasks.workunit.client.0.smithi063.stdout:6/978: symlink d2/d4/d7c/d42/def/l13d 0 2024-05-09T06:38:28.130 INFO:tasks.workunit.client.1.smithi193.stdout:9/705: symlink dd/dd9/leb 0 2024-05-09T06:38:28.131 INFO:tasks.workunit.client.0.smithi063.stdout:4/969: unlink d7/da3/f65 0 2024-05-09T06:38:28.133 INFO:tasks.workunit.client.1.smithi193.stdout:8/579: rename d6/d1c/d18/d2f/c8e to d6/d1c/d18/caf 0 2024-05-09T06:38:28.135 INFO:tasks.workunit.client.0.smithi063.stdout:9/947: creat d0/d4/d26/d39/f13f x:0 0 0 2024-05-09T06:38:28.136 INFO:tasks.workunit.client.1.smithi193.stdout:9/706: dwrite dd/d1e/d1f/d48/d55/fb1 [0,4194304] 0 2024-05-09T06:38:28.139 INFO:tasks.workunit.client.1.smithi193.stdout:6/533: mkdir d1/de/d11/d2d/da4/dc2 0 2024-05-09T06:38:28.140 INFO:tasks.workunit.client.0.smithi063.stdout:4/970: dwrite d7/da3/d28/d31/d71/d98/f10b [4194304,4194304] 0 2024-05-09T06:38:28.142 INFO:tasks.workunit.client.0.smithi063.stdout:0/977: getdents dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4/d113 0 2024-05-09T06:38:28.144 INFO:tasks.workunit.client.0.smithi063.stdout:5/948: readlink d8/l36 0 2024-05-09T06:38:28.149 INFO:tasks.workunit.client.0.smithi063.stdout:5/949: dread f6 [0,4194304] 0 2024-05-09T06:38:28.150 INFO:tasks.workunit.client.0.smithi063.stdout:1/896: getdents d9/d2a/d2e 0 2024-05-09T06:38:28.151 INFO:tasks.workunit.client.1.smithi193.stdout:6/534: creat d1/d25/d2c/d81/d37/d4f/fc3 x:0 0 0 2024-05-09T06:38:28.151 INFO:tasks.workunit.client.0.smithi063.stdout:1/897: stat d9/d46/fdd 0 2024-05-09T06:38:28.151 INFO:tasks.workunit.client.1.smithi193.stdout:6/535: stat d1/d25/d2c/d81/c64 0 2024-05-09T06:38:28.152 INFO:tasks.workunit.client.0.smithi063.stdout:5/950: write d8/d19/d30/db1/f12b [168714,119225] 0 2024-05-09T06:38:28.152 INFO:tasks.workunit.client.1.smithi193.stdout:6/536: dread - d1/d25/d2c/fb9 zero size 2024-05-09T06:38:28.154 INFO:tasks.workunit.client.0.smithi063.stdout:4/971: creat d7/d50/deb/f138 x:0 0 0 2024-05-09T06:38:28.159 INFO:tasks.workunit.client.0.smithi063.stdout:9/948: link d0/d3b/da0/f9c d0/d8/da6/db0/d113/f140 0 2024-05-09T06:38:28.160 INFO:tasks.workunit.client.0.smithi063.stdout:1/898: creat d9/d2a/d3c/dbc/d114/f131 x:0 0 0 2024-05-09T06:38:28.161 INFO:tasks.workunit.client.0.smithi063.stdout:1/899: fdatasync d9/d2a/d3c/d4c/dc3/f111 0 2024-05-09T06:38:28.162 INFO:tasks.workunit.client.0.smithi063.stdout:5/951: creat d8/d17/d18/d9c/f142 x:0 0 0 2024-05-09T06:38:28.163 INFO:tasks.workunit.client.0.smithi063.stdout:4/972: mkdir d7/da3/d28/d31/d71/dbb/d139 0 2024-05-09T06:38:28.164 INFO:tasks.workunit.client.1.smithi193.stdout:6/537: creat d1/d25/d2c/fc4 x:0 0 0 2024-05-09T06:38:28.168 INFO:tasks.workunit.client.0.smithi063.stdout:1/900: read d9/d2a/d39/d5d/d75/fcf [1715205,122030] 0 2024-05-09T06:38:28.169 INFO:tasks.workunit.client.1.smithi193.stdout:6/538: dwrite d1/d25/d2c/fc4 [0,4194304] 0 2024-05-09T06:38:28.170 INFO:tasks.workunit.client.0.smithi063.stdout:4/973: creat d7/da3/d28/f13a x:0 0 0 2024-05-09T06:38:28.171 INFO:tasks.workunit.client.0.smithi063.stdout:5/952: dwrite d8/d19/d115/deb/d129/f131 [0,4194304] 0 2024-05-09T06:38:28.177 INFO:tasks.workunit.client.0.smithi063.stdout:5/953: creat d8/d19/d30/d74/d77/f143 x:0 0 0 2024-05-09T06:38:28.179 INFO:tasks.workunit.client.1.smithi193.stdout:6/539: getdents d1/d25/d2c/d81 0 2024-05-09T06:38:28.180 INFO:tasks.workunit.client.0.smithi063.stdout:4/974: dwrite d7/d3d/d73/de5/f137 [0,4194304] 0 2024-05-09T06:38:28.181 INFO:tasks.workunit.client.1.smithi193.stdout:6/540: dread - d1/d25/f46 zero size 2024-05-09T06:38:28.182 INFO:tasks.workunit.client.1.smithi193.stdout:6/541: chown d1/d25/d2c/d3c/f99 1872598 1 2024-05-09T06:38:28.187 INFO:tasks.workunit.client.0.smithi063.stdout:5/954: creat d8/d19/d105/f144 x:0 0 0 2024-05-09T06:38:28.187 INFO:tasks.workunit.client.0.smithi063.stdout:5/955: stat d8/d35/c5e 0 2024-05-09T06:38:28.188 INFO:tasks.workunit.client.1.smithi193.stdout:6/542: mkdir d1/d25/d4c/d9e/dc5 0 2024-05-09T06:38:28.190 INFO:tasks.workunit.client.0.smithi063.stdout:4/975: mknod d7/da3/d28/d31/c13b 0 2024-05-09T06:38:28.193 INFO:tasks.workunit.client.0.smithi063.stdout:9/949: dread d0/f5b [0,4194304] 0 2024-05-09T06:38:28.194 INFO:tasks.workunit.client.0.smithi063.stdout:5/956: truncate d8/d19/f1a 549074 0 2024-05-09T06:38:28.196 INFO:tasks.workunit.client.0.smithi063.stdout:5/957: creat d8/d17/d18/d9c/f145 x:0 0 0 2024-05-09T06:38:28.198 INFO:tasks.workunit.client.0.smithi063.stdout:5/958: chown d8/d19/d30/d74/d77/l90 5682 1 2024-05-09T06:38:28.201 INFO:tasks.workunit.client.0.smithi063.stdout:5/959: unlink d8/d17/fc1 0 2024-05-09T06:38:28.202 INFO:tasks.workunit.client.0.smithi063.stdout:9/950: truncate d0/d4/d26/d39/d48/d80/fad 156003 0 2024-05-09T06:38:28.204 INFO:tasks.workunit.client.0.smithi063.stdout:4/976: getdents d7/da3/d28/d31/d32/d34/db4/dcc 0 2024-05-09T06:38:28.210 INFO:tasks.workunit.client.0.smithi063.stdout:4/977: creat d7/da3/da2/f13c x:0 0 0 2024-05-09T06:38:28.210 INFO:tasks.workunit.client.0.smithi063.stdout:5/960: dwrite d8/d17/da3/fbf [0,4194304] 0 2024-05-09T06:38:28.212 INFO:tasks.workunit.client.0.smithi063.stdout:4/978: fdatasync d7/d3d/d73/fd4 0 2024-05-09T06:38:28.213 INFO:tasks.workunit.client.0.smithi063.stdout:5/961: symlink d8/d53/l146 0 2024-05-09T06:38:28.215 INFO:tasks.workunit.client.0.smithi063.stdout:4/979: creat d7/da3/d28/d31/d71/d98/dff/f13d x:0 0 0 2024-05-09T06:38:28.215 INFO:tasks.workunit.client.0.smithi063.stdout:5/962: chown d8/d35/dd6/c118 3255 1 2024-05-09T06:38:28.217 INFO:tasks.workunit.client.0.smithi063.stdout:4/980: chown d7/df/d77/l7a 12 1 2024-05-09T06:38:28.221 INFO:tasks.workunit.client.1.smithi193.stdout:0/550: write d18/d1d/f44 [323679,69518] 0 2024-05-09T06:38:28.222 INFO:tasks.workunit.client.0.smithi063.stdout:5/963: dread d8/d65/f80 [0,4194304] 0 2024-05-09T06:38:28.225 INFO:tasks.workunit.client.1.smithi193.stdout:0/551: truncate d18/f61 3819672 0 2024-05-09T06:38:28.226 INFO:tasks.workunit.client.0.smithi063.stdout:4/981: dwrite d7/d3d/f55 [0,4194304] 0 2024-05-09T06:38:28.228 INFO:tasks.workunit.client.1.smithi193.stdout:0/552: creat d18/d4c/d47/d59/faa x:0 0 0 2024-05-09T06:38:28.231 INFO:tasks.workunit.client.1.smithi193.stdout:7/536: dwrite d6/d8/d1a/d33/f3e [0,4194304] 0 2024-05-09T06:38:28.231 INFO:tasks.workunit.client.1.smithi193.stdout:0/553: rmdir d18/d4c/d47/d59/d76 39 2024-05-09T06:38:28.232 INFO:tasks.workunit.client.1.smithi193.stdout:0/554: chown d18/d4c 3 1 2024-05-09T06:38:28.233 INFO:tasks.workunit.client.1.smithi193.stdout:0/555: truncate d18/d4c/d5f/f7c 4661562 0 2024-05-09T06:38:28.234 INFO:tasks.workunit.client.1.smithi193.stdout:0/556: chown d18/d65/f79 56 1 2024-05-09T06:38:28.236 INFO:tasks.workunit.client.1.smithi193.stdout:1/635: write d4/d16/de/f12 [2885379,79708] 0 2024-05-09T06:38:28.237 INFO:tasks.workunit.client.1.smithi193.stdout:0/557: mknod d18/d1d/d56/da7/cab 0 2024-05-09T06:38:28.238 INFO:tasks.workunit.client.1.smithi193.stdout:0/558: readlink d18/d4c/d47/l9c 0 2024-05-09T06:38:28.245 INFO:tasks.workunit.client.1.smithi193.stdout:0/559: dwrite d18/d4c/d47/f96 [0,4194304] 0 2024-05-09T06:38:28.247 INFO:tasks.workunit.client.1.smithi193.stdout:0/560: write d18/d4c/f38 [3543059,90690] 0 2024-05-09T06:38:28.255 INFO:tasks.workunit.client.1.smithi193.stdout:3/545: rename d1/c6 to d1/db/d59/cc9 0 2024-05-09T06:38:28.255 INFO:tasks.workunit.client.1.smithi193.stdout:4/644: write f17 [585784,20682] 0 2024-05-09T06:38:28.256 INFO:tasks.workunit.client.1.smithi193.stdout:3/546: chown d1/db/l68 50 1 2024-05-09T06:38:28.258 INFO:tasks.workunit.client.1.smithi193.stdout:4/645: read d1a/d67/f84 [3982868,53592] 0 2024-05-09T06:38:28.259 INFO:tasks.workunit.client.1.smithi193.stdout:7/537: rename d6/dd/d7e/c9d to d6/d8/d29/d2f/cb7 0 2024-05-09T06:38:28.261 INFO:tasks.workunit.client.1.smithi193.stdout:3/547: dread d1/db/d59/f52 [0,4194304] 0 2024-05-09T06:38:28.261 INFO:tasks.workunit.client.1.smithi193.stdout:3/548: chown f0 2529811 1 2024-05-09T06:38:28.263 INFO:tasks.workunit.client.1.smithi193.stdout:1/636: rename d4/d16/de/c34 to d4/d16/d6b/d6e/cce 0 2024-05-09T06:38:28.266 INFO:tasks.workunit.client.1.smithi193.stdout:7/538: mknod d6/dd/d5e/cb8 0 2024-05-09T06:38:28.268 INFO:tasks.workunit.client.1.smithi193.stdout:2/584: dwrite db/df/d54/f7c [0,4194304] 0 2024-05-09T06:38:28.274 INFO:tasks.workunit.client.1.smithi193.stdout:3/549: creat d1/db/d1b/fca x:0 0 0 2024-05-09T06:38:28.275 INFO:tasks.workunit.client.1.smithi193.stdout:7/539: dwrite d6/dd/d53/f6a [0,4194304] 0 2024-05-09T06:38:28.279 INFO:tasks.workunit.client.1.smithi193.stdout:0/561: rename d18/d4c/d5f/f7c to d18/d1d/fac 0 2024-05-09T06:38:28.280 INFO:tasks.workunit.client.0.smithi063.stdout:2/984: dwrite dc/dd/d2d/f3e [0,4194304] 0 2024-05-09T06:38:28.282 INFO:tasks.workunit.client.1.smithi193.stdout:5/524: write d0/d13/d2c/f4b [598415,839] 0 2024-05-09T06:38:28.285 INFO:tasks.workunit.client.1.smithi193.stdout:7/540: write d6/d8/f44 [1562254,74821] 0 2024-05-09T06:38:28.286 INFO:tasks.workunit.client.0.smithi063.stdout:2/985: mkdir dc/dd/d2d/d60/dd0/d90/d12e 0 2024-05-09T06:38:28.289 INFO:tasks.workunit.client.1.smithi193.stdout:0/562: dwrite d18/d4c/f38 [0,4194304] 0 2024-05-09T06:38:28.289 INFO:tasks.workunit.client.1.smithi193.stdout:7/541: read d6/d8/f55 [336645,98194] 0 2024-05-09T06:38:28.292 INFO:tasks.workunit.client.0.smithi063.stdout:6/979: write d2/d20/d95/dec/ff6 [497189,66213] 0 2024-05-09T06:38:28.292 INFO:tasks.workunit.client.1.smithi193.stdout:1/637: rename d4/d16/d17/d1a/f52 to d4/d16/d60/fcf 0 2024-05-09T06:38:28.293 INFO:tasks.workunit.client.0.smithi063.stdout:6/980: stat d2/d1f/d29/d3f/d5c/d64/f10b 0 2024-05-09T06:38:28.293 INFO:tasks.workunit.client.1.smithi193.stdout:3/550: mknod d1/d2d/ccb 0 2024-05-09T06:38:28.295 INFO:tasks.workunit.client.1.smithi193.stdout:7/542: chown d6/d8/d29/d2f/f4f 1 1 2024-05-09T06:38:28.295 INFO:tasks.workunit.client.0.smithi063.stdout:2/986: mkdir dc/d36/d43/d12f 0 2024-05-09T06:38:28.297 INFO:tasks.workunit.client.0.smithi063.stdout:2/987: truncate dc/dd/d2d/d60/dd0/d90/dc1/f11e 369382 0 2024-05-09T06:38:28.297 INFO:tasks.workunit.client.1.smithi193.stdout:8/580: write d6/d1c/d53/f60 [1298858,100155] 0 2024-05-09T06:38:28.300 INFO:tasks.workunit.client.0.smithi063.stdout:0/978: write dd/d60/fe5 [741426,47231] 0 2024-05-09T06:38:28.301 INFO:tasks.workunit.client.1.smithi193.stdout:0/563: dwrite d18/d4c/d47/d59/f91 [0,4194304] 0 2024-05-09T06:38:28.305 INFO:tasks.workunit.client.1.smithi193.stdout:2/585: dread db/df/f69 [0,4194304] 0 2024-05-09T06:38:28.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/988: rename dc/d36/d43/d46/d63/d79/d7d/d8d/f127 to dc/dd/d45/f130 0 2024-05-09T06:38:28.308 INFO:tasks.workunit.client.0.smithi063.stdout:6/981: creat d2/f13e x:0 0 0 2024-05-09T06:38:28.311 INFO:tasks.workunit.client.0.smithi063.stdout:2/989: creat dc/dd/d2d/d60/dd0/d90/dc1/f131 x:0 0 0 2024-05-09T06:38:28.314 INFO:tasks.workunit.client.1.smithi193.stdout:5/525: symlink d0/d13/d2c/d37/d2b/d47/laa 0 2024-05-09T06:38:28.315 INFO:tasks.workunit.client.1.smithi193.stdout:8/581: readlink d6/d1c/d18/d2f/d40/d5e/l6b 0 2024-05-09T06:38:28.317 INFO:tasks.workunit.client.0.smithi063.stdout:2/990: symlink dc/d6e/l132 0 2024-05-09T06:38:28.318 INFO:tasks.workunit.client.1.smithi193.stdout:0/564: rmdir d18/d65/d82 39 2024-05-09T06:38:28.320 INFO:tasks.workunit.client.0.smithi063.stdout:1/901: sync 2024-05-09T06:38:28.321 INFO:tasks.workunit.client.0.smithi063.stdout:0/979: creat dd/d1a/d2c/d51/d72/d9e/d23/d89/f12b x:0 0 0 2024-05-09T06:38:28.321 INFO:tasks.workunit.client.1.smithi193.stdout:2/586: creat db/d37/d6d/d80/d60/fc7 x:0 0 0 2024-05-09T06:38:28.322 INFO:tasks.workunit.client.0.smithi063.stdout:0/980: dread - dd/d1a/d2c/d51/d72/d9e/d23/d89/f12b zero size 2024-05-09T06:38:28.322 INFO:tasks.workunit.client.0.smithi063.stdout:2/991: truncate dc/dd/d45/d5a/f66 2895245 0 2024-05-09T06:38:28.322 INFO:tasks.workunit.client.0.smithi063.stdout:0/981: dread - dd/d60/d44/d90/da4/f128 zero size 2024-05-09T06:38:28.323 INFO:tasks.workunit.client.1.smithi193.stdout:7/543: mkdir d6/d8/d29/db9 0 2024-05-09T06:38:28.323 INFO:tasks.workunit.client.0.smithi063.stdout:2/992: chown dc/d36/d43/lc2 192 1 2024-05-09T06:38:28.324 INFO:tasks.workunit.client.0.smithi063.stdout:6/982: creat d2/d20/f13f x:0 0 0 2024-05-09T06:38:28.325 INFO:tasks.workunit.client.0.smithi063.stdout:6/983: truncate d2/f13e 262245 0 2024-05-09T06:38:28.327 INFO:tasks.workunit.client.1.smithi193.stdout:5/526: rename d0/d13/d2c/d37/d5b/d4e/d9d/fa8 to d0/d13/d2c/d37/d2b/d47/d66/d3a/fab 0 2024-05-09T06:38:28.328 INFO:tasks.workunit.client.1.smithi193.stdout:8/582: creat d6/d1c/d18/d66/fb0 x:0 0 0 2024-05-09T06:38:28.329 INFO:tasks.workunit.client.0.smithi063.stdout:0/982: mkdir dd/d60/d44/d90/d12c 0 2024-05-09T06:38:28.334 INFO:tasks.workunit.client.1.smithi193.stdout:8/583: dwrite d6/d1c/d53/f60 [0,4194304] 0 2024-05-09T06:38:28.335 INFO:tasks.workunit.client.1.smithi193.stdout:4/646: creat d1a/fc9 x:0 0 0 2024-05-09T06:38:28.336 INFO:tasks.workunit.client.0.smithi063.stdout:6/984: creat d2/d12/f140 x:0 0 0 2024-05-09T06:38:28.337 INFO:tasks.workunit.client.1.smithi193.stdout:6/543: write d1/de/d11/d2d/d3a/f55 [1461103,60877] 0 2024-05-09T06:38:28.339 INFO:tasks.workunit.client.0.smithi063.stdout:0/983: mkdir dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4/d113/d12d 0 2024-05-09T06:38:28.341 INFO:tasks.workunit.client.1.smithi193.stdout:7/544: rmdir d6/d17/d7d/daa 39 2024-05-09T06:38:28.343 INFO:tasks.workunit.client.0.smithi063.stdout:1/902: creat d9/d46/dfe/f132 x:0 0 0 2024-05-09T06:38:28.345 INFO:tasks.workunit.client.1.smithi193.stdout:5/527: mkdir d0/d13/d2c/d7e/dac 0 2024-05-09T06:38:28.346 INFO:tasks.workunit.client.1.smithi193.stdout:5/528: fdatasync d0/d13/d2c/d37/d5b/d89/f95 0 2024-05-09T06:38:28.349 INFO:tasks.workunit.client.0.smithi063.stdout:1/903: rename d9/d2a/d35/d1e/fd5 to d9/d2a/d35/d1e/d9d/d3f/d59/d62/d7e/f133 0 2024-05-09T06:38:28.352 INFO:tasks.workunit.client.1.smithi193.stdout:8/584: rmdir d6/d1c/d18/d2f/d40/d55 39 2024-05-09T06:38:28.353 INFO:tasks.workunit.client.1.smithi193.stdout:2/587: creat db/df/d54/fc8 x:0 0 0 2024-05-09T06:38:28.355 INFO:tasks.workunit.client.1.smithi193.stdout:6/544: rmdir d1/d25/d2c/d3c 39 2024-05-09T06:38:28.359 INFO:tasks.workunit.client.0.smithi063.stdout:1/904: creat d9/d2a/d35/d1e/f134 x:0 0 0 2024-05-09T06:38:28.361 INFO:tasks.workunit.client.1.smithi193.stdout:5/529: mknod d0/d13/d2c/d37/d5b/d4e/cad 0 2024-05-09T06:38:28.361 INFO:tasks.workunit.client.0.smithi063.stdout:9/951: dwrite d0/d4/d7d/d105/d110/fbe [0,4194304] 0 2024-05-09T06:38:28.363 INFO:tasks.workunit.client.1.smithi193.stdout:0/565: rmdir d18/d1d/d56/da7/da8 0 2024-05-09T06:38:28.364 INFO:tasks.workunit.client.1.smithi193.stdout:4/647: getdents d1a/d22/d43/d9a 0 2024-05-09T06:38:28.365 INFO:tasks.workunit.client.0.smithi063.stdout:1/905: mknod d9/d2a/d35/d24/d68/c135 0 2024-05-09T06:38:28.367 INFO:tasks.workunit.client.1.smithi193.stdout:2/588: unlink db/d37/d6d/d80/d60/d7d/db3/lb9 0 2024-05-09T06:38:28.369 INFO:tasks.workunit.client.0.smithi063.stdout:9/952: unlink d0/d4/d26/d39/d48/d80/f11c 0 2024-05-09T06:38:28.373 INFO:tasks.workunit.client.0.smithi063.stdout:1/906: dread - d9/d2a/d3c/d4c/d83/d88/fe1 zero size 2024-05-09T06:38:28.374 INFO:tasks.workunit.client.1.smithi193.stdout:2/589: dwrite db/d25/d38/d5b/f88 [0,4194304] 0 2024-05-09T06:38:28.374 INFO:tasks.workunit.client.1.smithi193.stdout:5/530: mknod d0/d13/d2c/d37/d2b/d47/d60/d75/cae 0 2024-05-09T06:38:28.378 INFO:tasks.workunit.client.1.smithi193.stdout:0/566: rename d18/d4c/d47/d5e to d18/d4c/d64/d74/dad 0 2024-05-09T06:38:28.379 INFO:tasks.workunit.client.0.smithi063.stdout:9/953: chown d0/d4/d26/d47/fa8 0 1 2024-05-09T06:38:28.384 INFO:tasks.workunit.client.0.smithi063.stdout:7/930: link d0/c39 d0/d15/d49/d54/d9f/d97/c13d 0 2024-05-09T06:38:28.387 INFO:tasks.workunit.client.0.smithi063.stdout:1/907: creat d9/d46/dfe/d94/f136 x:0 0 0 2024-05-09T06:38:28.388 INFO:tasks.workunit.client.1.smithi193.stdout:0/567: dread d18/d4c/f45 [0,4194304] 0 2024-05-09T06:38:28.389 INFO:tasks.workunit.client.0.smithi063.stdout:5/964: dwrite d8/d35/db6/dc2/d138/f5f [0,4194304] 0 2024-05-09T06:38:28.391 INFO:tasks.workunit.client.0.smithi063.stdout:9/954: rename d0/d4/d26/d39/d91/fa7 to d0/d4/d57/dba/dd5/f141 0 2024-05-09T06:38:28.392 INFO:tasks.workunit.client.0.smithi063.stdout:4/982: truncate d7/df/f2b 2651551 0 2024-05-09T06:38:28.396 INFO:tasks.workunit.client.0.smithi063.stdout:1/908: dread d9/d2a/d3c/d4a/f67 [0,4194304] 0 2024-05-09T06:38:28.397 INFO:tasks.workunit.client.0.smithi063.stdout:5/965: unlink d8/d19/d84/lcb 0 2024-05-09T06:38:28.401 INFO:tasks.workunit.client.0.smithi063.stdout:7/931: rename d0/d15/d49/d4f/d66/ddd to d0/d15/d89/d10c/dfb/d13e 0 2024-05-09T06:38:28.402 INFO:tasks.workunit.client.0.smithi063.stdout:7/932: dread - d0/d15/d89/f12c zero size 2024-05-09T06:38:28.403 INFO:tasks.workunit.client.0.smithi063.stdout:1/909: dwrite d9/d2a/d39/d5d/f9a [0,4194304] 0 2024-05-09T06:38:28.406 INFO:tasks.workunit.client.0.smithi063.stdout:1/910: dread - d9/d2a/d3c/d4c/d83/d88/dc2/f110 zero size 2024-05-09T06:38:28.417 INFO:tasks.workunit.client.0.smithi063.stdout:4/983: read d7/df/d1a/f22 [1322784,79522] 0 2024-05-09T06:38:28.418 INFO:tasks.workunit.client.0.smithi063.stdout:4/984: write d7/df/d35/d5a/d60/fd8 [4749291,27080] 0 2024-05-09T06:38:28.419 INFO:tasks.workunit.client.0.smithi063.stdout:5/966: symlink d8/d17/d18/l147 0 2024-05-09T06:38:28.419 INFO:tasks.workunit.client.0.smithi063.stdout:4/985: stat d7/df/d77/fe4 0 2024-05-09T06:38:28.421 INFO:tasks.workunit.client.1.smithi193.stdout:3/551: getdents d1/d2d 0 2024-05-09T06:38:28.422 INFO:tasks.workunit.client.1.smithi193.stdout:3/552: readlink d1/d2d/d51/lba 0 2024-05-09T06:38:28.423 INFO:tasks.workunit.client.1.smithi193.stdout:3/553: write d1/db/d59/f52 [1906131,129038] 0 2024-05-09T06:38:28.423 INFO:tasks.workunit.client.0.smithi063.stdout:2/993: rmdir dc/d36 39 2024-05-09T06:38:28.425 INFO:tasks.workunit.client.1.smithi193.stdout:1/638: write d4/f87 [714452,26049] 0 2024-05-09T06:38:28.431 INFO:tasks.workunit.client.0.smithi063.stdout:7/933: creat d0/d15/f13f x:0 0 0 2024-05-09T06:38:28.432 INFO:tasks.workunit.client.0.smithi063.stdout:7/934: dread - d0/d15/d89/d10c/de7/d12a/ff5 zero size 2024-05-09T06:38:28.433 INFO:tasks.workunit.client.0.smithi063.stdout:2/994: chown dc/d36 39333646 1 2024-05-09T06:38:28.434 INFO:tasks.workunit.client.0.smithi063.stdout:2/995: chown dc/dd/d45/d5a/d6c/d8f/dbf/f122 0 1 2024-05-09T06:38:28.436 INFO:tasks.workunit.client.1.smithi193.stdout:0/568: creat d18/d4c/d47/d55/fae x:0 0 0 2024-05-09T06:38:28.438 INFO:tasks.workunit.client.0.smithi063.stdout:7/935: mknod d0/d15/d89/d10c/de7/d12a/c140 0 2024-05-09T06:38:28.438 INFO:tasks.workunit.client.1.smithi193.stdout:2/590: mknod db/df/d54/d63/cc9 0 2024-05-09T06:38:28.444 INFO:tasks.workunit.client.1.smithi193.stdout:6/545: link d1/de/d11/f68 d1/d25/d2c/d3c/fc6 0 2024-05-09T06:38:28.444 INFO:tasks.workunit.client.0.smithi063.stdout:2/996: rename dc/d36/d43/d46 to dc/dd/d45/d5a/d6c/d8c/d133 0 2024-05-09T06:38:28.446 INFO:tasks.workunit.client.1.smithi193.stdout:0/569: unlink d18/d4c/d47/f96 0 2024-05-09T06:38:28.448 INFO:tasks.workunit.client.0.smithi063.stdout:0/984: write dd/d1a/d2c/d51/d72/d9e/faf [723610,121249] 0 2024-05-09T06:38:28.449 INFO:tasks.workunit.client.0.smithi063.stdout:6/985: dwrite d2/d1f/d29/f84 [0,4194304] 0 2024-05-09T06:38:28.451 INFO:tasks.workunit.client.1.smithi193.stdout:2/591: dread db/d25/d38/d40/f58 [4194304,4194304] 0 2024-05-09T06:38:28.452 INFO:tasks.workunit.client.0.smithi063.stdout:7/936: mkdir d0/d15/d49/d4f/d66/df1/d141 0 2024-05-09T06:38:28.452 INFO:tasks.workunit.client.1.smithi193.stdout:7/545: write d6/f2e [4488181,18525] 0 2024-05-09T06:38:28.453 INFO:tasks.workunit.client.1.smithi193.stdout:1/639: symlink d4/d16/d17/d1a/dc5/ld0 0 2024-05-09T06:38:28.454 INFO:tasks.workunit.client.0.smithi063.stdout:0/985: creat dd/d1a/d2c/d51/d76/d97/f12e x:0 0 0 2024-05-09T06:38:28.459 INFO:tasks.workunit.client.1.smithi193.stdout:0/570: chown d18/d4c/fa4 60722207 1 2024-05-09T06:38:28.463 INFO:tasks.workunit.client.1.smithi193.stdout:5/531: getdents d0/d13/d2c/da0 0 2024-05-09T06:38:28.464 INFO:tasks.workunit.client.0.smithi063.stdout:2/997: dread dc/d32/d33/f4c [0,4194304] 0 2024-05-09T06:38:28.465 INFO:tasks.workunit.client.0.smithi063.stdout:0/986: creat dd/d60/d119/d8f/dfd/f12f x:0 0 0 2024-05-09T06:38:28.466 INFO:tasks.workunit.client.0.smithi063.stdout:0/987: fsync dd/d60/d119/d8f/fdd 0 2024-05-09T06:38:28.467 INFO:tasks.workunit.client.0.smithi063.stdout:0/988: stat dd/d1a/d2c/d51/d72/fca 0 2024-05-09T06:38:28.471 INFO:tasks.workunit.client.1.smithi193.stdout:1/640: symlink d4/d16/d6b/d89/dc4/ld1 0 2024-05-09T06:38:28.478 INFO:tasks.workunit.client.1.smithi193.stdout:7/546: rename d6/d17/d7d/fb5 to d6/d17/d7d/daa/fba 0 2024-05-09T06:38:28.481 INFO:tasks.workunit.client.0.smithi063.stdout:9/955: dwrite d0/d4/d26/d47/fa8 [0,4194304] 0 2024-05-09T06:38:28.481 INFO:tasks.workunit.client.1.smithi193.stdout:6/546: dread d1/de/d11/d2d/fa8 [0,4194304] 0 2024-05-09T06:38:28.483 INFO:tasks.workunit.client.1.smithi193.stdout:6/547: read d1/f6 [9888052,53990] 0 2024-05-09T06:38:28.484 INFO:tasks.workunit.client.1.smithi193.stdout:1/641: dread d4/f9 [0,4194304] 0 2024-05-09T06:38:28.485 INFO:tasks.workunit.client.1.smithi193.stdout:1/642: chown d4/d16/d60/d4f/d72/l8d 5 1 2024-05-09T06:38:28.485 INFO:tasks.workunit.client.1.smithi193.stdout:5/532: mknod d0/d13/d2c/d37/d2b/d47/d60/caf 0 2024-05-09T06:38:28.488 INFO:tasks.workunit.client.0.smithi063.stdout:9/956: symlink d0/d8/da6/l142 0 2024-05-09T06:38:28.489 INFO:tasks.workunit.client.1.smithi193.stdout:7/547: rename d6/d8/c93 to d6/dd/d70/cbb 0 2024-05-09T06:38:28.490 INFO:tasks.workunit.client.1.smithi193.stdout:0/571: link d18/d4c/fa4 d18/d4c/d64/d66/d9d/faf 0 2024-05-09T06:38:28.491 INFO:tasks.workunit.client.0.smithi063.stdout:1/911: truncate d9/d2a/d3c/d4c/d83/d88/dc2/fef 461288 0 2024-05-09T06:38:28.493 INFO:tasks.workunit.client.0.smithi063.stdout:1/912: truncate d9/d2a/d3c/d4c/dc3/f129 741290 0 2024-05-09T06:38:28.493 INFO:tasks.workunit.client.1.smithi193.stdout:6/548: unlink d1/d25/d2c/d3c/db1/d58/l77 0 2024-05-09T06:38:28.496 INFO:tasks.workunit.client.0.smithi063.stdout:9/957: rmdir d0/d4/d26/d39 39 2024-05-09T06:38:28.497 INFO:tasks.workunit.client.0.smithi063.stdout:9/958: write d0/d4/d26/d47/f7a [2518784,104199] 0 2024-05-09T06:38:28.497 INFO:tasks.workunit.client.1.smithi193.stdout:5/533: creat d0/d6d/fb0 x:0 0 0 2024-05-09T06:38:28.498 INFO:tasks.workunit.client.1.smithi193.stdout:4/648: truncate f17 1612973 0 2024-05-09T06:38:28.500 INFO:tasks.workunit.client.1.smithi193.stdout:1/643: mknod d4/d16/d17/d1a/d3a/d4a/db1/cd2 0 2024-05-09T06:38:28.501 INFO:tasks.workunit.client.0.smithi063.stdout:1/913: mknod d9/d46/dfe/d94/d112/c137 0 2024-05-09T06:38:28.504 INFO:tasks.workunit.client.0.smithi063.stdout:5/967: dwrite d8/d19/f109 [0,4194304] 0 2024-05-09T06:38:28.504 INFO:tasks.workunit.client.0.smithi063.stdout:1/914: write d9/d2a/d35/d1e/d9d/f82 [4728338,97516] 0 2024-05-09T06:38:28.506 INFO:tasks.workunit.client.0.smithi063.stdout:5/968: chown d8/d19/d30/d74/d77/dde/fdf 21812 1 2024-05-09T06:38:28.507 INFO:tasks.workunit.client.0.smithi063.stdout:1/915: truncate d9/d2a/d3c/d4c/f120 816400 0 2024-05-09T06:38:28.508 INFO:tasks.workunit.client.0.smithi063.stdout:9/959: mknod d0/d4/d57/d60/d76/d11e/c143 0 2024-05-09T06:38:28.509 INFO:tasks.workunit.client.1.smithi193.stdout:6/549: mkdir d1/d25/d4c/dc7 0 2024-05-09T06:38:28.510 INFO:tasks.workunit.client.0.smithi063.stdout:0/989: link dd/fb2 dd/f130 0 2024-05-09T06:38:28.511 INFO:tasks.workunit.client.1.smithi193.stdout:3/554: write d1/d2d/f7c [567100,85962] 0 2024-05-09T06:38:28.512 INFO:tasks.workunit.client.1.smithi193.stdout:5/534: write d0/f7 [848216,63971] 0 2024-05-09T06:38:28.513 INFO:tasks.workunit.client.1.smithi193.stdout:7/548: dread d6/d8/d1a/d36/f50 [0,4194304] 0 2024-05-09T06:38:28.514 INFO:tasks.workunit.client.1.smithi193.stdout:5/535: fdatasync d0/d13/d2c/d37/d2b/d47/f63 0 2024-05-09T06:38:28.518 INFO:tasks.workunit.client.1.smithi193.stdout:1/644: rename d4/d16/d60/d4f/d72/l8d to d4/d16/d35/ld3 0 2024-05-09T06:38:28.520 INFO:tasks.workunit.client.0.smithi063.stdout:6/986: write d2/d12/f24 [1747743,904] 0 2024-05-09T06:38:28.521 INFO:tasks.workunit.client.1.smithi193.stdout:4/649: dread d1a/d6d/fa3 [0,4194304] 0 2024-05-09T06:38:28.522 INFO:tasks.workunit.client.0.smithi063.stdout:9/960: dread - d0/d4/d26/d39/d42/f122 zero size 2024-05-09T06:38:28.523 INFO:tasks.workunit.client.1.smithi193.stdout:6/550: mkdir d1/d25/d4c/d9e/dc8 0 2024-05-09T06:38:28.525 INFO:tasks.workunit.client.1.smithi193.stdout:1/645: dwrite d4/d16/d17/d1a/d3a/d3f/f68 [0,4194304] 0 2024-05-09T06:38:28.527 INFO:tasks.workunit.client.1.smithi193.stdout:2/592: write db/d37/d6d/d80/f14 [587531,68564] 0 2024-05-09T06:38:28.528 INFO:tasks.workunit.client.0.smithi063.stdout:7/937: dwrite d0/d15/d49/f5b [0,4194304] 0 2024-05-09T06:38:28.530 INFO:tasks.workunit.client.0.smithi063.stdout:6/987: rename d2/d4/f1c to d2/d1f/d29/d3f/d5c/d64/f141 0 2024-05-09T06:38:28.540 INFO:tasks.workunit.client.0.smithi063.stdout:1/916: rename d9/d46/f54 to d9/d46/f138 0 2024-05-09T06:38:28.541 INFO:tasks.workunit.client.1.smithi193.stdout:4/650: dwrite d1a/d24/d73/d94/d7c/d5b/fb3 [0,4194304] 0 2024-05-09T06:38:28.542 INFO:tasks.workunit.client.0.smithi063.stdout:1/917: chown d9/d2a/d2e/df4/f12f 40637 1 2024-05-09T06:38:28.544 INFO:tasks.workunit.client.0.smithi063.stdout:7/938: read - d0/d15/d4a/d68/fe8 zero size 2024-05-09T06:38:28.544 INFO:tasks.workunit.client.1.smithi193.stdout:5/536: fsync d0/d13/d2c/d37/d5b/f21 0 2024-05-09T06:38:28.545 INFO:tasks.workunit.client.0.smithi063.stdout:2/998: dwrite dc/dd/d2d/f77 [0,4194304] 0 2024-05-09T06:38:28.546 INFO:tasks.workunit.client.0.smithi063.stdout:6/988: mknod d2/d4/d7c/d7e/c142 0 2024-05-09T06:38:28.547 INFO:tasks.workunit.client.0.smithi063.stdout:6/989: readlink d2/df/d59/d99/l130 0 2024-05-09T06:38:28.548 INFO:tasks.workunit.client.1.smithi193.stdout:2/593: mkdir db/df/d54/d63/dca 0 2024-05-09T06:38:28.557 INFO:tasks.workunit.client.0.smithi063.stdout:0/990: getdents dd/d60/dcd/d12a 0 2024-05-09T06:38:28.561 INFO:tasks.workunit.client.1.smithi193.stdout:0/572: write d18/d27/f34 [2038031,58406] 0 2024-05-09T06:38:28.568 INFO:tasks.workunit.client.1.smithi193.stdout:7/549: rename d6/d8/d1a/d33 to d6/dd/dbc 0 2024-05-09T06:38:28.569 INFO:tasks.workunit.client.0.smithi063.stdout:4/986: sync 2024-05-09T06:38:28.569 INFO:tasks.workunit.client.1.smithi193.stdout:7/550: chown d6/dd/dbc/c34 136285 1 2024-05-09T06:38:28.570 INFO:tasks.workunit.client.1.smithi193.stdout:7/551: truncate d6/dd/dbc/f87 858219 0 2024-05-09T06:38:28.571 INFO:tasks.workunit.client.1.smithi193.stdout:7/552: chown d6/d17/f52 0 1 2024-05-09T06:38:28.571 INFO:tasks.workunit.client.1.smithi193.stdout:0/573: mknod d18/d4c/d64/d74/dad/cb0 0 2024-05-09T06:38:28.571 INFO:tasks.workunit.client.0.smithi063.stdout:6/990: dread d2/d1f/f69 [0,4194304] 0 2024-05-09T06:38:28.572 INFO:tasks.workunit.client.0.smithi063.stdout:6/991: chown d2/d4/c3e 1254986907 1 2024-05-09T06:38:28.572 INFO:tasks.workunit.client.1.smithi193.stdout:7/553: write d6/d8/d1a/d36/f5b [2055203,128071] 0 2024-05-09T06:38:28.573 INFO:tasks.workunit.client.0.smithi063.stdout:9/961: write d0/f9f [3606237,36915] 0 2024-05-09T06:38:28.573 INFO:tasks.workunit.client.0.smithi063.stdout:1/918: dread d9/d2a/d3c/d4a/f7b [0,4194304] 0 2024-05-09T06:38:28.573 INFO:tasks.workunit.client.1.smithi193.stdout:5/537: creat d0/d13/d2c/d37/d2b/d6a/d90/fb1 x:0 0 0 2024-05-09T06:38:28.573 INFO:tasks.workunit.client.1.smithi193.stdout:7/554: fdatasync d6/d17/f37 0 2024-05-09T06:38:28.574 INFO:tasks.workunit.client.0.smithi063.stdout:6/992: read d2/d20/d4a/f94 [1648333,9674] 0 2024-05-09T06:38:28.575 INFO:tasks.workunit.client.0.smithi063.stdout:6/993: dread - d2/d4/fa1 zero size 2024-05-09T06:38:28.576 INFO:tasks.workunit.client.1.smithi193.stdout:2/594: mkdir db/d25/d38/d40/db2/dcb 0 2024-05-09T06:38:28.577 INFO:tasks.workunit.client.1.smithi193.stdout:6/551: write d1/de/d11/d2d/fa8 [323694,80320] 0 2024-05-09T06:38:28.577 INFO:tasks.workunit.client.0.smithi063.stdout:4/987: fsync f5 0 2024-05-09T06:38:28.579 INFO:tasks.workunit.client.1.smithi193.stdout:1/646: rename d4/d16/d60/d4f/l9f to d4/d3c/d3e/d7c/ld4 0 2024-05-09T06:38:28.580 INFO:tasks.workunit.client.1.smithi193.stdout:2/595: read db/d25/fa9 [173122,73430] 0 2024-05-09T06:38:28.580 INFO:tasks.workunit.client.1.smithi193.stdout:6/552: write d1/d25/d2c/fb9 [399588,87987] 0 2024-05-09T06:38:28.580 INFO:tasks.workunit.client.0.smithi063.stdout:1/919: dwrite d9/d2a/d35/f11a [0,4194304] 0 2024-05-09T06:38:28.583 INFO:tasks.workunit.client.1.smithi193.stdout:6/553: write d1/f70 [3260228,88610] 0 2024-05-09T06:38:28.584 INFO:tasks.workunit.client.1.smithi193.stdout:5/538: dwrite d0/d13/d2c/d37/d2b/d47/f63 [0,4194304] 0 2024-05-09T06:38:28.584 INFO:tasks.workunit.client.1.smithi193.stdout:2/596: write db/d37/d6d/d80/f14 [3061960,71372] 0 2024-05-09T06:38:28.595 INFO:tasks.workunit.client.1.smithi193.stdout:4/651: dwrite d1a/d67/d82/dab/fbc [0,4194304] 0 2024-05-09T06:38:28.596 INFO:tasks.workunit.client.0.smithi063.stdout:7/939: rename d0/d15/d89/d10c/de7/d12a/c140 to d0/d15/d49/d54/d9f/d97/d9b/c142 0 2024-05-09T06:38:28.601 INFO:tasks.workunit.client.0.smithi063.stdout:7/940: dread d0/d15/d4a/dc8/fe9 [0,4194304] 0 2024-05-09T06:38:28.603 INFO:tasks.workunit.client.1.smithi193.stdout:1/647: mknod d4/d16/d6b/d85/cd5 0 2024-05-09T06:38:28.609 INFO:tasks.workunit.client.1.smithi193.stdout:6/554: mkdir d1/d25/d2c/d3c/db1/d58/dc9 0 2024-05-09T06:38:28.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/999: getdents dc/dd/d45/d5a/d6c/d8c/d133/d49 0 2024-05-09T06:38:28.618 INFO:tasks.workunit.client.0.smithi063.stdout:9/962: mkdir d0/d4/d144 0 2024-05-09T06:38:28.618 INFO:tasks.workunit.client.0.smithi063.stdout:9/963: chown d0/d4/d26/d47 38212 1 2024-05-09T06:38:28.619 INFO:tasks.workunit.client.1.smithi193.stdout:5/539: dread d0/d13/d2c/d37/d5b/d4e/f6b [0,4194304] 0 2024-05-09T06:38:28.619 INFO:tasks.workunit.client.1.smithi193.stdout:6/555: truncate d1/d25/d4c/f6b 547189 0 2024-05-09T06:38:28.620 INFO:tasks.workunit.client.1.smithi193.stdout:7/555: dread d6/d17/f52 [0,4194304] 0 2024-05-09T06:38:28.626 INFO:tasks.workunit.client.1.smithi193.stdout:1/648: link d4/f8c d4/d16/d17/d1a/d3a/d4a/fd6 0 2024-05-09T06:38:28.627 INFO:tasks.workunit.client.1.smithi193.stdout:5/540: rmdir d0/d13/d2c/d37/d2b/d47/d66 39 2024-05-09T06:38:28.629 INFO:tasks.workunit.client.1.smithi193.stdout:0/574: dwrite d18/d27/fa0 [0,4194304] 0 2024-05-09T06:38:28.631 INFO:tasks.workunit.client.1.smithi193.stdout:6/556: rename d1/de/d11/d18/l33 to d1/d25/lca 0 2024-05-09T06:38:28.632 INFO:tasks.workunit.client.0.smithi063.stdout:0/991: getdents dd/d1a/d2c/d51/d72/d9e/d23/d31/dda/df4 0 2024-05-09T06:38:28.632 INFO:tasks.workunit.client.1.smithi193.stdout:6/557: stat d1/d25 0 2024-05-09T06:38:28.634 INFO:tasks.workunit.client.0.smithi063.stdout:6/994: dwrite d2/d4/f92 [0,4194304] 0 2024-05-09T06:38:28.638 INFO:tasks.workunit.client.1.smithi193.stdout:2/597: rmdir db/d37/d6d/d80/daa/db4/dbd 0 2024-05-09T06:38:28.643 INFO:tasks.workunit.client.1.smithi193.stdout:1/649: mkdir d4/d16/d17/d1a/d3a/d3f/dd7 0 2024-05-09T06:38:28.645 INFO:tasks.workunit.client.1.smithi193.stdout:9/707: sync 2024-05-09T06:38:28.647 INFO:tasks.workunit.client.1.smithi193.stdout:4/652: dwrite d1a/d6d/f7b [0,4194304] 0 2024-05-09T06:38:28.649 INFO:tasks.workunit.client.1.smithi193.stdout:0/575: truncate d18/f28 2551033 0 2024-05-09T06:38:28.649 INFO:tasks.workunit.client.1.smithi193.stdout:0/576: chown d18 138593 1 2024-05-09T06:38:28.650 INFO:tasks.workunit.client.1.smithi193.stdout:0/577: dread - d18/d4c/d47/d54/f90 zero size 2024-05-09T06:38:28.650 INFO:tasks.workunit.client.0.smithi063.stdout:9/964: dwrite d0/d4/d57/f64 [0,4194304] 0 2024-05-09T06:38:28.653 INFO:tasks.workunit.client.1.smithi193.stdout:7/556: rename d6/d8/d29/d2f/f4f to d6/d8/d1a/d36/d4d/d60/d9c/fbd 0 2024-05-09T06:38:28.655 INFO:tasks.workunit.client.0.smithi063.stdout:5/969: sync 2024-05-09T06:38:28.659 INFO:tasks.workunit.client.1.smithi193.stdout:5/541: write d0/d13/d2c/d37/d2b/d47/f79 [991051,43759] 0 2024-05-09T06:38:28.660 INFO:tasks.workunit.client.1.smithi193.stdout:5/542: chown d0/d13/d2c/d7e 21037982 1 2024-05-09T06:38:28.668 INFO:tasks.workunit.client.1.smithi193.stdout:7/557: dread d6/d8/fa [0,4194304] 0 2024-05-09T06:38:28.670 INFO:tasks.workunit.client.1.smithi193.stdout:7/558: dread - d6/d17/d7d/daa/f8c zero size 2024-05-09T06:38:28.674 INFO:tasks.workunit.client.1.smithi193.stdout:9/708: creat dd/df/dc4/fec x:0 0 0 2024-05-09T06:38:28.676 INFO:tasks.workunit.client.1.smithi193.stdout:0/578: dread d18/d27/f53 [0,4194304] 0 2024-05-09T06:38:28.676 INFO:tasks.workunit.client.1.smithi193.stdout:1/650: write d4/d16/fb [3643898,57004] 0 2024-05-09T06:38:28.677 INFO:tasks.workunit.client.1.smithi193.stdout:1/651: dread - d4/d16/d60/d4f/d72/fa1 zero size 2024-05-09T06:38:28.678 INFO:tasks.workunit.client.1.smithi193.stdout:1/652: chown d4/d16/d17/d1a/f1f 53141016 1 2024-05-09T06:38:28.679 INFO:tasks.workunit.client.1.smithi193.stdout:4/653: mknod d1a/d24/d73/d94/d7c/cca 0 2024-05-09T06:38:28.679 INFO:tasks.workunit.client.0.smithi063.stdout:1/920: sync 2024-05-09T06:38:28.680 INFO:tasks.workunit.client.0.smithi063.stdout:1/921: fdatasync d9/d2a/d3c/d4c/d83/d88/dc2/f110 0 2024-05-09T06:38:28.683 INFO:tasks.workunit.client.1.smithi193.stdout:2/598: dread db/d4b/f5d [0,4194304] 0 2024-05-09T06:38:28.683 INFO:tasks.workunit.client.1.smithi193.stdout:9/709: creat dd/d1e/dc2/d49/dbd/fed x:0 0 0 2024-05-09T06:38:28.686 INFO:tasks.workunit.client.1.smithi193.stdout:0/579: write d18/d4c/f45 [4244623,8371] 0 2024-05-09T06:38:28.687 INFO:tasks.workunit.client.1.smithi193.stdout:1/653: creat d4/d16/d6b/d6e/fd8 x:0 0 0 2024-05-09T06:38:28.690 INFO:tasks.workunit.client.1.smithi193.stdout:4/654: creat d1a/d24/d73/fcb x:0 0 0 2024-05-09T06:38:28.694 INFO:tasks.workunit.client.1.smithi193.stdout:5/543: rename d0/c4c to d0/d13/d2c/d37/d2b/d47/d66/d3a/cb2 0 2024-05-09T06:38:28.696 INFO:tasks.workunit.client.1.smithi193.stdout:5/544: chown d0/d13/d2c/d37/d5b/d4e/f7a 3669 1 2024-05-09T06:38:28.699 INFO:tasks.workunit.client.1.smithi193.stdout:0/580: rmdir d18/d4c 39 2024-05-09T06:38:28.699 INFO:tasks.workunit.client.1.smithi193.stdout:7/559: dwrite d6/d8/d1a/fb3 [0,4194304] 0 2024-05-09T06:38:28.701 INFO:tasks.workunit.client.1.smithi193.stdout:9/710: dread dd/df/d31/d7e/f62 [0,4194304] 0 2024-05-09T06:38:28.702 INFO:tasks.workunit.client.1.smithi193.stdout:7/560: write d6/dd/d70/f8e [1128785,75534] 0 2024-05-09T06:38:28.703 INFO:tasks.workunit.client.1.smithi193.stdout:7/561: chown d6/d8/d1a/d36/d4d/d60/cb2 816 1 2024-05-09T06:38:28.704 INFO:tasks.workunit.client.1.smithi193.stdout:7/562: read d6/d17/f52 [1355695,119418] 0 2024-05-09T06:38:28.706 INFO:tasks.workunit.client.1.smithi193.stdout:4/655: mkdir d1a/d24/d73/dcc 0 2024-05-09T06:38:28.709 INFO:tasks.workunit.client.1.smithi193.stdout:7/563: dread d6/d8/d1a/d36/f3b [0,4194304] 0 2024-05-09T06:38:28.709 INFO:tasks.workunit.client.1.smithi193.stdout:7/564: stat d6/d8/f6e 0 2024-05-09T06:38:28.718 INFO:tasks.workunit.client.1.smithi193.stdout:2/599: dwrite db/d25/d38/d40/d8f/f93 [0,4194304] 0 2024-05-09T06:38:28.726 INFO:tasks.workunit.client.1.smithi193.stdout:9/711: truncate dd/d1e/dc2/d49/dbd/d5b/f5d 4780324 0 2024-05-09T06:38:28.729 INFO:tasks.workunit.client.1.smithi193.stdout:1/654: truncate d4/d16/d17/d1a/d3a/d3f/f65 1512575 0 2024-05-09T06:38:28.729 INFO:tasks.workunit.client.1.smithi193.stdout:4/656: dread d1a/d98/fb8 [0,4194304] 0 2024-05-09T06:38:28.735 INFO:tasks.workunit.client.1.smithi193.stdout:9/712: readlink dd/d1e/dc2/d49/dbd/d5b/l99 0 2024-05-09T06:38:28.737 INFO:tasks.workunit.client.0.smithi063.stdout:7/941: rmdir d0/dcb/d11c 0 2024-05-09T06:38:28.737 INFO:tasks.workunit.client.0.smithi063.stdout:7/942: chown d0/d15/d49/d4f/l6d 1464516672 1 2024-05-09T06:38:28.738 INFO:tasks.workunit.client.1.smithi193.stdout:7/565: dwrite d6/f16 [0,4194304] 0 2024-05-09T06:38:28.739 INFO:tasks.workunit.client.1.smithi193.stdout:5/545: rename d0/d13/d2c/d37/d2b/d47/c6c to d0/d13/d2c/d37/d5b/cb3 0 2024-05-09T06:38:28.739 INFO:tasks.workunit.client.0.smithi063.stdout:4/988: getdents d7/da3/d28/d31/d7b/d81 0 2024-05-09T06:38:28.741 INFO:tasks.workunit.client.1.smithi193.stdout:1/655: creat d4/d16/d60/d4f/d72/d7f/fd9 x:0 0 0 2024-05-09T06:38:28.741 INFO:tasks.workunit.client.0.smithi063.stdout:0/992: chown dd/c10 2141481 1 2024-05-09T06:38:28.744 INFO:tasks.workunit.client.1.smithi193.stdout:2/600: dread f8 [8388608,4194304] 0 2024-05-09T06:38:28.744 INFO:tasks.workunit.client.0.smithi063.stdout:6/995: unlink d2/df/d59/cc7 0 2024-05-09T06:38:28.748 INFO:tasks.workunit.client.0.smithi063.stdout:5/970: chown d8/d17/d18/f38 657 1 2024-05-09T06:38:28.749 INFO:tasks.workunit.client.0.smithi063.stdout:5/971: chown d8/d35/db6/dc2/d138/f42 0 1 2024-05-09T06:38:28.750 INFO:tasks.workunit.client.0.smithi063.stdout:1/922: rename d9/d2a/d35/d1e/d9d/d3f to d9/d2a/d39/d5d/d75/d139 0 2024-05-09T06:38:28.750 INFO:tasks.workunit.client.0.smithi063.stdout:9/965: dread d0/f1f [0,4194304] 0 2024-05-09T06:38:28.751 INFO:tasks.workunit.client.1.smithi193.stdout:4/657: mknod d1a/d22/ccd 0 2024-05-09T06:38:28.752 INFO:tasks.workunit.client.1.smithi193.stdout:4/658: chown d1a/d24/f40 1886 1 2024-05-09T06:38:28.754 INFO:tasks.workunit.client.0.smithi063.stdout:4/989: creat d7/df/d35/d5a/d60/f13e x:0 0 0 2024-05-09T06:38:28.755 INFO:tasks.workunit.client.0.smithi063.stdout:5/972: dread d8/d17/d75/dc9/dd5/fbe [0,4194304] 0 2024-05-09T06:38:28.756 INFO:tasks.workunit.client.0.smithi063.stdout:6/996: symlink d2/df/d91/ddf/l143 0 2024-05-09T06:38:28.758 INFO:tasks.workunit.client.0.smithi063.stdout:9/966: dwrite d0/d4/d26/d39/d3e/ff1 [0,4194304] 0 2024-05-09T06:38:28.761 INFO:tasks.workunit.client.0.smithi063.stdout:0/993: rename dd/d1a/d2c/d51/d72/d9e/feb to dd/d1a/d2c/d51/d76/f131 0 2024-05-09T06:38:28.761 INFO:tasks.workunit.client.0.smithi063.stdout:0/994: write f2 [4406,17909] 0 2024-05-09T06:38:28.762 INFO:tasks.workunit.client.0.smithi063.stdout:0/995: write dd/d60/d44/f69 [2239722,77809] 0 2024-05-09T06:38:28.766 INFO:tasks.workunit.client.0.smithi063.stdout:4/990: rmdir d7/da3/d28/d31/d71/d98/dff 39 2024-05-09T06:38:28.766 INFO:tasks.workunit.client.1.smithi193.stdout:4/659: dread d1a/f65 [0,4194304] 0 2024-05-09T06:38:28.767 INFO:tasks.workunit.client.0.smithi063.stdout:4/991: chown d7/da3/d28/d31/d32/l135 3 1 2024-05-09T06:38:28.768 INFO:tasks.workunit.client.1.smithi193.stdout:8/585: sync 2024-05-09T06:38:28.771 INFO:tasks.workunit.client.1.smithi193.stdout:1/656: unlink d4/d14/c19 0 2024-05-09T06:38:28.771 INFO:tasks.workunit.client.1.smithi193.stdout:1/657: fsync d4/f87 0 2024-05-09T06:38:28.772 INFO:tasks.workunit.client.0.smithi063.stdout:6/997: fdatasync d2/d4/d7c/d7e/f87 0 2024-05-09T06:38:28.773 INFO:tasks.workunit.client.1.smithi193.stdout:0/581: getdents d18/d4c/d64/d74 0 2024-05-09T06:38:28.777 INFO:tasks.workunit.client.1.smithi193.stdout:2/601: creat db/df/d54/d63/dca/fcc x:0 0 0 2024-05-09T06:38:28.778 INFO:tasks.workunit.client.1.smithi193.stdout:5/546: write d0/f29 [1257875,32685] 0 2024-05-09T06:38:28.780 INFO:tasks.workunit.client.1.smithi193.stdout:4/660: symlink d1a/d24/d73/d94/lce 0 2024-05-09T06:38:28.781 INFO:tasks.workunit.client.0.smithi063.stdout:1/923: rename d9/d2a/d39/d5d/d75/d139/ldb to d9/d2a/d3c/d4c/d83/d88/dc2/dd0/l13a 0 2024-05-09T06:38:28.784 INFO:tasks.workunit.client.1.smithi193.stdout:7/566: truncate d6/d8/d1a/d36/f50 303321 0 2024-05-09T06:38:28.789 INFO:tasks.workunit.client.0.smithi063.stdout:7/943: creat d0/d15/d49/d54/d9f/f143 x:0 0 0 2024-05-09T06:38:28.789 INFO:tasks.workunit.client.0.smithi063.stdout:7/944: chown d0/fc 466401030 1 2024-05-09T06:38:28.789 INFO:tasks.workunit.client.1.smithi193.stdout:8/586: dwrite d6/d1c/d18/d2f/f39 [0,4194304] 0 2024-05-09T06:38:28.791 INFO:tasks.workunit.client.1.smithi193.stdout:1/658: fdatasync d4/d16/d17/d1a/d3a/fa3 0 2024-05-09T06:38:28.795 INFO:tasks.workunit.client.1.smithi193.stdout:0/582: creat d18/d4c/d64/d66/fb1 x:0 0 0 2024-05-09T06:38:28.796 INFO:tasks.workunit.client.1.smithi193.stdout:2/602: mkdir db/d25/d38/dcd 0 2024-05-09T06:38:28.799 INFO:tasks.workunit.client.1.smithi193.stdout:8/587: dwrite d6/d1c/d18/f1d [0,4194304] 0 2024-05-09T06:38:28.802 INFO:tasks.workunit.client.1.smithi193.stdout:9/713: creat dd/d1e/d1f/d48/d55/d86/fee x:0 0 0 2024-05-09T06:38:28.803 INFO:tasks.workunit.client.1.smithi193.stdout:9/714: stat dd/df/d15/d8c/c30 0 2024-05-09T06:38:28.805 INFO:tasks.workunit.client.1.smithi193.stdout:5/547: mkdir d0/d13/d2c/d37/d5b/d4e/d9d/db4 0 2024-05-09T06:38:28.805 INFO:tasks.workunit.client.1.smithi193.stdout:9/715: write dd/df/d31/fd1 [1000677,126479] 0 2024-05-09T06:38:28.806 INFO:tasks.workunit.client.0.smithi063.stdout:0/996: write dd/d1a/f65 [4185282,38247] 0 2024-05-09T06:38:28.806 INFO:tasks.workunit.client.1.smithi193.stdout:9/716: readlink dd/d1e/d1f/d48/d55/l92 0 2024-05-09T06:38:28.807 INFO:tasks.workunit.client.1.smithi193.stdout:5/548: chown d0/d13/d2c/d37/d5b/d4e/f6b 3 1 2024-05-09T06:38:28.807 INFO:tasks.workunit.client.0.smithi063.stdout:0/997: fdatasync dd/d1a/d2c/d51/d72/d9e/f2b 0 2024-05-09T06:38:28.808 INFO:tasks.workunit.client.0.smithi063.stdout:9/967: mknod d0/d4/d26/d39/c145 0 2024-05-09T06:38:28.815 INFO:tasks.workunit.client.0.smithi063.stdout:6/998: truncate d2/d20/d26/d7f/dcf/fda 267361 0 2024-05-09T06:38:28.818 INFO:tasks.workunit.client.1.smithi193.stdout:6/558: sync 2024-05-09T06:38:28.818 INFO:tasks.workunit.client.1.smithi193.stdout:3/555: sync 2024-05-09T06:38:28.818 INFO:tasks.workunit.client.0.smithi063.stdout:1/924: mkdir d9/d2a/d39/d5d/d75/d139/d13b 0 2024-05-09T06:38:28.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/925: chown d9/d2a/d3c/dbc/d114/f125 43382096 1 2024-05-09T06:38:28.819 INFO:tasks.workunit.client.0.smithi063.stdout:0/998: mknod dd/d60/d119/d8f/c132 0 2024-05-09T06:38:28.819 INFO:tasks.workunit.client.1.smithi193.stdout:2/603: dread db/f45 [4194304,4194304] 0 2024-05-09T06:38:28.820 INFO:tasks.workunit.client.1.smithi193.stdout:2/604: dread - db/d37/d6d/d80/fbf zero size 2024-05-09T06:38:28.821 INFO:tasks.workunit.client.0.smithi063.stdout:0/999: chown dd/d1a/d2c/d51/d72/d9e/d23/l45 118 1 2024-05-09T06:38:28.821 INFO:tasks.workunit.client.0.smithi063.stdout:9/968: mkdir d0/d8/da6/db0/db5/d146 0 2024-05-09T06:38:28.822 INFO:tasks.workunit.client.0.smithi063.stdout:4/992: link d7/da3/d28/d31/d7b/d81/cc5 d7/d50/deb/c13f 0 2024-05-09T06:38:28.823 INFO:tasks.workunit.client.0.smithi063.stdout:4/993: chown d7/da3/da2/d112 1 1 2024-05-09T06:38:28.824 INFO:tasks.workunit.client.1.smithi193.stdout:0/583: fdatasync d18/d4c/d64/d8b/fa6 0 2024-05-09T06:38:28.825 INFO:tasks.workunit.client.1.smithi193.stdout:8/588: mknod d6/d1c/d18/da2/cb1 0 2024-05-09T06:38:28.825 INFO:tasks.workunit.client.0.smithi063.stdout:5/973: getdents d8/d19/d30/d46 0 2024-05-09T06:38:28.826 INFO:tasks.workunit.client.0.smithi063.stdout:5/974: chown d8/d17/c32 102810864 1 2024-05-09T06:38:28.826 INFO:tasks.workunit.client.0.smithi063.stdout:1/926: dwrite d9/d2a/d35/d1e/fd1 [0,4194304] 0 2024-05-09T06:38:28.826 INFO:tasks.workunit.client.1.smithi193.stdout:3/556: creat d1/d2d/fcc x:0 0 0 2024-05-09T06:38:28.827 INFO:tasks.workunit.client.1.smithi193.stdout:3/557: dread - d1/db/d59/d63/fb6 zero size 2024-05-09T06:38:28.827 INFO:tasks.workunit.client.0.smithi063.stdout:1/927: chown d9/d46/fc0 7 1 2024-05-09T06:38:28.829 INFO:tasks.workunit.client.0.smithi063.stdout:1/928: read d9/d2a/d3c/d4c/f120 [470514,53568] 0 2024-05-09T06:38:28.829 INFO:tasks.workunit.client.0.smithi063.stdout:9/969: mkdir d0/d3b/d63/d147 0 2024-05-09T06:38:28.831 INFO:tasks.workunit.client.0.smithi063.stdout:4/994: creat d7/da6/f140 x:0 0 0 2024-05-09T06:38:28.832 INFO:tasks.workunit.client.0.smithi063.stdout:4/995: chown d7/df/d77/f8e 7 1 2024-05-09T06:38:28.833 INFO:tasks.workunit.client.1.smithi193.stdout:4/661: rename d1a/d6d/f7b to d1a/fcf 0 2024-05-09T06:38:28.835 INFO:tasks.workunit.client.0.smithi063.stdout:6/999: creat d2/df/dd2/dfe/d132/f144 x:0 0 0 2024-05-09T06:38:28.837 INFO:tasks.workunit.client.1.smithi193.stdout:7/567: creat d6/d17/d7d/fbe x:0 0 0 2024-05-09T06:38:28.838 INFO:tasks.workunit.client.1.smithi193.stdout:7/568: dread - d6/d8/d1a/d36/d4d/d60/d9c/fb1 zero size 2024-05-09T06:38:28.839 INFO:tasks.workunit.client.1.smithi193.stdout:7/569: chown d6/f8b 0 1 2024-05-09T06:38:28.840 INFO:tasks.workunit.client.1.smithi193.stdout:7/570: write d6/d8/d1a/d36/d4d/fb0 [250406,32448] 0 2024-05-09T06:38:28.841 INFO:tasks.workunit.client.1.smithi193.stdout:7/571: dread - d6/d17/d7d/daa/f96 zero size 2024-05-09T06:38:28.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/945: getdents d0/d15/d4a/d68 0 2024-05-09T06:38:28.845 INFO:tasks.workunit.client.1.smithi193.stdout:5/549: write d0/d13/d2c/d37/d5b/d4e/f6b [1863347,127753] 0 2024-05-09T06:38:28.847 INFO:tasks.workunit.client.1.smithi193.stdout:6/559: mknod d1/d25/d2c/d3c/db1/d58/dc9/ccb 0 2024-05-09T06:38:28.848 INFO:tasks.workunit.client.0.smithi063.stdout:1/929: creat d9/d46/dfe/d94/d112/f13c x:0 0 0 2024-05-09T06:38:28.849 INFO:tasks.workunit.client.1.smithi193.stdout:9/717: dread dd/df/f10 [0,4194304] 0 2024-05-09T06:38:28.850 INFO:tasks.workunit.client.1.smithi193.stdout:3/558: creat d1/db/d2f/d84/d97/fcd x:0 0 0 2024-05-09T06:38:28.850 INFO:tasks.workunit.client.0.smithi063.stdout:5/975: dread d8/d19/d30/d74/f117 [0,4194304] 0 2024-05-09T06:38:28.851 INFO:tasks.workunit.client.1.smithi193.stdout:3/559: fsync d1/f1d 0 2024-05-09T06:38:28.852 INFO:tasks.workunit.client.1.smithi193.stdout:6/560: dread d1/de/d11/d2d/fa8 [0,4194304] 0 2024-05-09T06:38:28.852 INFO:tasks.workunit.client.0.smithi063.stdout:1/930: dread d9/d2a/d35/f11a [0,4194304] 0 2024-05-09T06:38:28.853 INFO:tasks.workunit.client.0.smithi063.stdout:4/996: dread d7/df/d57/f91 [0,4194304] 0 2024-05-09T06:38:28.854 INFO:tasks.workunit.client.1.smithi193.stdout:2/605: symlink db/df/d87/d9f/lce 0 2024-05-09T06:38:28.857 INFO:tasks.workunit.client.0.smithi063.stdout:7/946: rename d0/d15/d4a/d50/l7c to d0/d15/d89/d10c/l144 0 2024-05-09T06:38:28.858 INFO:tasks.workunit.client.1.smithi193.stdout:4/662: mkdir d1a/d24/dd0 0 2024-05-09T06:38:28.858 INFO:tasks.workunit.client.0.smithi063.stdout:1/931: dwrite d9/d2a/d35/d1e/d9d/f82 [0,4194304] 0 2024-05-09T06:38:28.861 INFO:tasks.workunit.client.1.smithi193.stdout:7/572: symlink d6/dd/d7e/lbf 0 2024-05-09T06:38:28.863 INFO:tasks.workunit.client.0.smithi063.stdout:5/976: mkdir d8/d19/d30/d46/d148 0 2024-05-09T06:38:28.863 INFO:tasks.workunit.client.1.smithi193.stdout:8/589: symlink d6/d1c/d18/d2f/d40/lb2 0 2024-05-09T06:38:28.864 INFO:tasks.workunit.client.1.smithi193.stdout:5/550: creat d0/d13/d2c/d4d/d67/fb5 x:0 0 0 2024-05-09T06:38:28.864 INFO:tasks.workunit.client.1.smithi193.stdout:8/590: fsync d6/d1c/d1b/d29/f91 0 2024-05-09T06:38:28.866 INFO:tasks.workunit.client.1.smithi193.stdout:5/551: stat d0/d13/d2c/d37/d2b/d47/d60/d75/da9 0 2024-05-09T06:38:28.866 INFO:tasks.workunit.client.0.smithi063.stdout:4/997: fdatasync d7/df/d35/f7f 0 2024-05-09T06:38:28.868 INFO:tasks.workunit.client.1.smithi193.stdout:9/718: mknod dd/df/d15/d8c/d7a/cef 0 2024-05-09T06:38:28.869 INFO:tasks.workunit.client.1.smithi193.stdout:9/719: chown dd/df/d15 7 1 2024-05-09T06:38:28.871 INFO:tasks.workunit.client.1.smithi193.stdout:3/560: dread d1/db/f6e [0,4194304] 0 2024-05-09T06:38:28.871 INFO:tasks.workunit.client.1.smithi193.stdout:6/561: mknod d1/de/d11/d18/d94/ccc 0 2024-05-09T06:38:28.872 INFO:tasks.workunit.client.0.smithi063.stdout:4/998: dwrite d7/df/d35/d5a/d60/d124/f133 [0,4194304] 0 2024-05-09T06:38:28.873 INFO:tasks.workunit.client.0.smithi063.stdout:9/970: creat d0/f148 x:0 0 0 2024-05-09T06:38:28.874 INFO:tasks.workunit.client.0.smithi063.stdout:9/971: chown d0/d43/l4a 954737857 1 2024-05-09T06:38:28.874 INFO:tasks.workunit.client.0.smithi063.stdout:4/999: fsync d7/d3d/d73/de5/f12c 0 2024-05-09T06:38:28.874 INFO:tasks.workunit.client.1.smithi193.stdout:5/552: dwrite d0/d6d/fb0 [0,4194304] 0 2024-05-09T06:38:28.875 INFO:tasks.workunit.client.1.smithi193.stdout:2/606: creat db/d37/d6d/d80/fcf x:0 0 0 2024-05-09T06:38:28.876 INFO:tasks.workunit.client.1.smithi193.stdout:5/553: fdatasync d0/d13/d2c/d4d/d67/f93 0 2024-05-09T06:38:28.879 INFO:tasks.workunit.client.1.smithi193.stdout:4/663: chown d1a/c74 11 1 2024-05-09T06:38:28.881 INFO:tasks.workunit.client.1.smithi193.stdout:1/659: fdatasync d4/d3c/f36 0 2024-05-09T06:38:28.882 INFO:tasks.workunit.client.1.smithi193.stdout:4/664: fsync d1a/d80/fa9 0 2024-05-09T06:38:28.885 INFO:tasks.workunit.client.1.smithi193.stdout:7/573: creat d6/d8/d29/d2f/fc0 x:0 0 0 2024-05-09T06:38:28.888 INFO:tasks.workunit.client.1.smithi193.stdout:7/574: dread - d6/d17/d7d/daa/fba zero size 2024-05-09T06:38:28.888 INFO:tasks.workunit.client.1.smithi193.stdout:0/584: getdents d18/d4c/d64/d74 0 2024-05-09T06:38:28.888 INFO:tasks.workunit.client.1.smithi193.stdout:2/607: dwrite db/df/d54/d63/dca/fcc [0,4194304] 0 2024-05-09T06:38:28.891 INFO:tasks.workunit.client.1.smithi193.stdout:8/591: rmdir d6/d1c/d18/d66 39 2024-05-09T06:38:28.891 INFO:tasks.workunit.client.1.smithi193.stdout:5/554: dwrite d0/d13/d2c/d4d/d67/fb5 [0,4194304] 0 2024-05-09T06:38:28.892 INFO:tasks.workunit.client.1.smithi193.stdout:8/592: stat d6/d1c/c10 0 2024-05-09T06:38:28.893 INFO:tasks.workunit.client.1.smithi193.stdout:5/555: read d0/d13/d2c/d4d/d67/fb5 [2323090,20514] 0 2024-05-09T06:38:28.895 INFO:tasks.workunit.client.0.smithi063.stdout:9/972: rmdir d0/d4/d57/dba 39 2024-05-09T06:38:28.901 INFO:tasks.workunit.client.0.smithi063.stdout:5/977: fsync d8/f8e 0 2024-05-09T06:38:28.902 INFO:tasks.workunit.client.0.smithi063.stdout:9/973: dwrite d0/d4/d26/d47/f7a [0,4194304] 0 2024-05-09T06:38:28.903 INFO:tasks.workunit.client.1.smithi193.stdout:3/561: mkdir d1/db/da1/dce 0 2024-05-09T06:38:28.904 INFO:tasks.workunit.client.1.smithi193.stdout:6/562: creat d1/de/d11/d18/d94/fcd x:0 0 0 2024-05-09T06:38:28.908 INFO:tasks.workunit.client.0.smithi063.stdout:1/932: creat d9/d2a/d39/d5d/d75/d139/d59/f13d x:0 0 0 2024-05-09T06:38:28.914 INFO:tasks.workunit.client.0.smithi063.stdout:7/947: rename d0/d15/d49/d4f/d66/fb7 to d0/d15/f145 0 2024-05-09T06:38:28.918 INFO:tasks.workunit.client.1.smithi193.stdout:4/665: creat d1a/d22/d5d/fd1 x:0 0 0 2024-05-09T06:38:28.920 INFO:tasks.workunit.client.1.smithi193.stdout:8/593: dread d6/d1c/f12 [0,4194304] 0 2024-05-09T06:38:28.921 INFO:tasks.workunit.client.0.smithi063.stdout:1/933: rename d9/d2a/d3c/d4c/d83/d88/dc2/dd0/c11b to d9/d2a/d3c/d4a/db8/c13e 0 2024-05-09T06:38:28.922 INFO:tasks.workunit.client.1.smithi193.stdout:0/585: mknod d18/d4c/d64/cb2 0 2024-05-09T06:38:28.923 INFO:tasks.workunit.client.0.smithi063.stdout:7/948: dread - d0/d15/d49/d54/fbd zero size 2024-05-09T06:38:28.925 INFO:tasks.workunit.client.0.smithi063.stdout:9/974: mkdir d0/d8/da6/db0/d113/d149 0 2024-05-09T06:38:28.928 INFO:tasks.workunit.client.0.smithi063.stdout:5/978: dwrite d8/d19/f44 [0,4194304] 0 2024-05-09T06:38:28.930 INFO:tasks.workunit.client.1.smithi193.stdout:3/562: truncate d1/d2d/d51/f6a 5309385 0 2024-05-09T06:38:28.930 INFO:tasks.workunit.client.1.smithi193.stdout:6/563: truncate d1/f86 202337 0 2024-05-09T06:38:28.932 INFO:tasks.workunit.client.1.smithi193.stdout:4/666: creat d1a/d1f/fd2 x:0 0 0 2024-05-09T06:38:28.934 INFO:tasks.workunit.client.0.smithi063.stdout:7/949: dwrite d0/d15/d49/f5b [0,4194304] 0 2024-05-09T06:38:28.937 INFO:tasks.workunit.client.0.smithi063.stdout:9/975: dwrite d0/f24 [0,4194304] 0 2024-05-09T06:38:28.939 INFO:tasks.workunit.client.1.smithi193.stdout:8/594: write d6/d1c/d18/d66/d93/d95/f9f [1157728,2266] 0 2024-05-09T06:38:28.942 INFO:tasks.workunit.client.0.smithi063.stdout:1/934: rename d9/d2a/d35/d1e/c71 to d9/d2a/d35/d1e/c13f 0 2024-05-09T06:38:28.946 INFO:tasks.workunit.client.0.smithi063.stdout:7/950: creat d0/d15/d49/d4f/d66/d67/d6b/f146 x:0 0 0 2024-05-09T06:38:28.947 INFO:tasks.workunit.client.1.smithi193.stdout:9/720: dwrite dd/f29 [0,4194304] 0 2024-05-09T06:38:28.951 INFO:tasks.workunit.client.1.smithi193.stdout:5/556: mkdir d0/d13/d2c/db6 0 2024-05-09T06:38:28.951 INFO:tasks.workunit.client.0.smithi063.stdout:5/979: write d8/d17/d75/fb5 [443510,2657] 0 2024-05-09T06:38:28.951 INFO:tasks.workunit.client.0.smithi063.stdout:9/976: fdatasync d0/d8/dea/f101 0 2024-05-09T06:38:28.954 INFO:tasks.workunit.client.0.smithi063.stdout:5/980: write d8/d17/faa [4108711,9993] 0 2024-05-09T06:38:28.959 INFO:tasks.workunit.client.1.smithi193.stdout:6/564: truncate d1/de/d11/d2d/fa8 195060 0 2024-05-09T06:38:28.961 INFO:tasks.workunit.client.1.smithi193.stdout:2/608: dwrite db/df/f10 [0,4194304] 0 2024-05-09T06:38:28.961 INFO:tasks.workunit.client.1.smithi193.stdout:0/586: readlink d18/d4c/d47/d59/d76/l84 0 2024-05-09T06:38:28.963 INFO:tasks.workunit.client.0.smithi063.stdout:5/981: symlink d8/d65/d9e/l149 0 2024-05-09T06:38:28.963 INFO:tasks.workunit.client.1.smithi193.stdout:2/609: stat db/df/d54/d63/dca/fcc 0 2024-05-09T06:38:28.966 INFO:tasks.workunit.client.1.smithi193.stdout:7/575: rename d6/dd/l95 to d6/dd/d5e/lc1 0 2024-05-09T06:38:28.967 INFO:tasks.workunit.client.1.smithi193.stdout:2/610: truncate db/d25/d38/d40/d8b/fc3 931200 0 2024-05-09T06:38:28.968 INFO:tasks.workunit.client.0.smithi063.stdout:9/977: creat d0/d8/da6/db0/db5/d146/f14a x:0 0 0 2024-05-09T06:38:28.969 INFO:tasks.workunit.client.1.smithi193.stdout:2/611: chown db/d25/d38/d5b/l5f 0 1 2024-05-09T06:38:28.969 INFO:tasks.workunit.client.0.smithi063.stdout:9/978: chown d0/d4/d26/d39/d3e/d66/fe3 143 1 2024-05-09T06:38:28.970 INFO:tasks.workunit.client.1.smithi193.stdout:5/557: mknod d0/d13/d2c/d37/d2b/d47/d66/d3a/cb7 0 2024-05-09T06:38:28.971 INFO:tasks.workunit.client.1.smithi193.stdout:5/558: chown d0/d13/d2c/d4d/d67/f93 1356539020 1 2024-05-09T06:38:28.972 INFO:tasks.workunit.client.0.smithi063.stdout:5/982: symlink d8/d19/d30/d46/d148/l14a 0 2024-05-09T06:38:28.974 INFO:tasks.workunit.client.0.smithi063.stdout:5/983: fsync d8/f1d 0 2024-05-09T06:38:28.979 INFO:tasks.workunit.client.1.smithi193.stdout:8/595: dread d6/fd [0,4194304] 0 2024-05-09T06:38:28.979 INFO:tasks.workunit.client.1.smithi193.stdout:1/660: rename d4/d16/d60/fcf to d4/d16/d60/d4f/fda 0 2024-05-09T06:38:28.981 INFO:tasks.workunit.client.0.smithi063.stdout:5/984: dwrite d8/f16 [0,4194304] 0 2024-05-09T06:38:28.982 INFO:tasks.workunit.client.1.smithi193.stdout:9/721: symlink dd/d1e/d1f/lf0 0 2024-05-09T06:38:28.983 INFO:tasks.workunit.client.1.smithi193.stdout:4/667: write d1a/d6b/f63 [941905,87861] 0 2024-05-09T06:38:28.984 INFO:tasks.workunit.client.0.smithi063.stdout:7/951: dwrite d0/d15/d4a/d50/f3b [0,4194304] 0 2024-05-09T06:38:28.984 INFO:tasks.workunit.client.1.smithi193.stdout:2/612: rmdir db/d25 39 2024-05-09T06:38:28.986 INFO:tasks.workunit.client.1.smithi193.stdout:5/559: symlink d0/d13/d2c/d37/d2b/d47/d6f/lb8 0 2024-05-09T06:38:28.989 INFO:tasks.workunit.client.1.smithi193.stdout:0/587: mknod d18/d65/cb3 0 2024-05-09T06:38:28.990 INFO:tasks.workunit.client.0.smithi063.stdout:5/985: truncate d8/d17/d18/f51 1092973 0 2024-05-09T06:38:28.991 INFO:tasks.workunit.client.1.smithi193.stdout:9/722: dwrite dd/df/d31/d7e/f60 [0,4194304] 0 2024-05-09T06:38:28.992 INFO:tasks.workunit.client.0.smithi063.stdout:1/935: dwrite d9/fe [0,4194304] 0 2024-05-09T06:38:28.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/936: chown d9/d46 910387 1 2024-05-09T06:38:29.000 INFO:tasks.workunit.client.0.smithi063.stdout:5/986: dwrite d8/f16 [0,4194304] 0 2024-05-09T06:38:29.001 INFO:tasks.workunit.client.0.smithi063.stdout:9/979: write d0/d4/d26/d39/d48/d80/d84/dd8/ddf/f103 [419525,101395] 0 2024-05-09T06:38:29.003 INFO:tasks.workunit.client.1.smithi193.stdout:7/576: write d6/dd/f89 [1732316,54937] 0 2024-05-09T06:38:29.006 INFO:tasks.workunit.client.0.smithi063.stdout:1/937: dread - d9/d2a/d3c/d4c/f55 zero size 2024-05-09T06:38:29.007 INFO:tasks.workunit.client.1.smithi193.stdout:2/613: rmdir db/d37/d6d/d80/daa 39 2024-05-09T06:38:29.009 INFO:tasks.workunit.client.1.smithi193.stdout:3/563: getdents d1/db/d2f/d78/d86/d94 0 2024-05-09T06:38:29.010 INFO:tasks.workunit.client.0.smithi063.stdout:5/987: dread d8/d35/db6/dc2/d138/f8c [0,4194304] 0 2024-05-09T06:38:29.011 INFO:tasks.workunit.client.1.smithi193.stdout:3/564: write d1/d44/f96 [4520503,69017] 0 2024-05-09T06:38:29.011 INFO:tasks.workunit.client.1.smithi193.stdout:5/560: symlink d0/d13/d2c/d37/d5b/d89/lb9 0 2024-05-09T06:38:29.011 INFO:tasks.workunit.client.1.smithi193.stdout:3/565: chown d1/db/da1 3425 1 2024-05-09T06:38:29.014 INFO:tasks.workunit.client.1.smithi193.stdout:0/588: creat d18/d4c/d47/d59/d76/fb4 x:0 0 0 2024-05-09T06:38:29.014 INFO:tasks.workunit.client.1.smithi193.stdout:9/723: rmdir dd/d1e/d1f 39 2024-05-09T06:38:29.015 INFO:tasks.workunit.client.1.smithi193.stdout:0/589: chown d18/d4c/d47/d54/f68 26965186 1 2024-05-09T06:38:29.016 INFO:tasks.workunit.client.0.smithi063.stdout:9/980: link d0/d4/d26/d39/d48/d80/d84/dc2/f124 d0/d4/d26/d39/f14b 0 2024-05-09T06:38:29.017 INFO:tasks.workunit.client.1.smithi193.stdout:7/577: symlink d6/d17/d7d/daa/lc2 0 2024-05-09T06:38:29.018 INFO:tasks.workunit.client.1.smithi193.stdout:7/578: stat d6/d8/d29/c73 0 2024-05-09T06:38:29.019 INFO:tasks.workunit.client.0.smithi063.stdout:7/952: dread d0/d15/d49/d54/d9f/d97/d9b/d9d/fa8 [0,4194304] 0 2024-05-09T06:38:29.019 INFO:tasks.workunit.client.1.smithi193.stdout:4/668: rename d1a/c1c to d1a/d24/cd3 0 2024-05-09T06:38:29.020 INFO:tasks.workunit.client.1.smithi193.stdout:4/669: truncate d1a/d22/d5d/fd1 306775 0 2024-05-09T06:38:29.021 INFO:tasks.workunit.client.0.smithi063.stdout:9/981: creat d0/d43/f14c x:0 0 0 2024-05-09T06:38:29.023 INFO:tasks.workunit.client.0.smithi063.stdout:7/953: truncate d0/d15/d49/d4f/d66/d67/d86/ff0 839988 0 2024-05-09T06:38:29.024 INFO:tasks.workunit.client.1.smithi193.stdout:5/561: creat d0/d13/d2c/d37/d5b/d4e/d9d/fba x:0 0 0 2024-05-09T06:38:29.026 INFO:tasks.workunit.client.0.smithi063.stdout:9/982: rename d0/f71 to d0/d8/da6/db0/d113/f14d 0 2024-05-09T06:38:29.030 INFO:tasks.workunit.client.0.smithi063.stdout:7/954: dwrite d0/d15/f109 [0,4194304] 0 2024-05-09T06:38:29.030 INFO:tasks.workunit.client.1.smithi193.stdout:6/565: getdents d1/de/d11/d18/d94/dac 0 2024-05-09T06:38:29.031 INFO:tasks.workunit.client.0.smithi063.stdout:7/955: chown d0/d15/d4a/d50/f29 5573374 1 2024-05-09T06:38:29.032 INFO:tasks.workunit.client.0.smithi063.stdout:7/956: chown d0/d15/d4a/d68/f6f 529653392 1 2024-05-09T06:38:29.034 INFO:tasks.workunit.client.1.smithi193.stdout:0/590: fsync d18/d1d/f1f 0 2024-05-09T06:38:29.035 INFO:tasks.workunit.client.0.smithi063.stdout:9/983: mknod d0/d4/d57/dba/dd5/c14e 0 2024-05-09T06:38:29.035 INFO:tasks.workunit.client.1.smithi193.stdout:0/591: stat d18/d4c/d64/d8b/fa6 0 2024-05-09T06:38:29.037 INFO:tasks.workunit.client.1.smithi193.stdout:4/670: creat d1a/d22/d43/fd4 x:0 0 0 2024-05-09T06:38:29.038 INFO:tasks.workunit.client.0.smithi063.stdout:7/957: creat d0/d15/d49/d54/d9f/d106/f147 x:0 0 0 2024-05-09T06:38:29.039 INFO:tasks.workunit.client.0.smithi063.stdout:1/938: dread d9/d2a/d35/d1e/fa1 [0,4194304] 0 2024-05-09T06:38:29.040 INFO:tasks.workunit.client.1.smithi193.stdout:7/579: dread d6/d8/d1a/d36/d4d/d60/d9c/faf [0,4194304] 0 2024-05-09T06:38:29.043 INFO:tasks.workunit.client.0.smithi063.stdout:5/988: truncate d8/d53/f13a 331885 0 2024-05-09T06:38:29.046 INFO:tasks.workunit.client.0.smithi063.stdout:1/939: symlink d9/d2a/d3c/d4a/db8/ddf/l140 0 2024-05-09T06:38:29.046 INFO:tasks.workunit.client.0.smithi063.stdout:1/940: stat d9/d2a/d3c/d4a/f12e 0 2024-05-09T06:38:29.047 INFO:tasks.workunit.client.0.smithi063.stdout:7/958: truncate d0/d15/d49/d4f/d66/d67/d6b/f6e 2066908 0 2024-05-09T06:38:29.048 INFO:tasks.workunit.client.1.smithi193.stdout:2/614: write db/f45 [1383175,28812] 0 2024-05-09T06:38:29.049 INFO:tasks.workunit.client.0.smithi063.stdout:9/984: dread d0/d4/d26/d39/d48/d80/d84/dd8/ddf/fe2 [4194304,4194304] 0 2024-05-09T06:38:29.050 INFO:tasks.workunit.client.0.smithi063.stdout:9/985: chown d0/d4/d26/d39/f55 1790507 1 2024-05-09T06:38:29.051 INFO:tasks.workunit.client.0.smithi063.stdout:5/989: unlink d8/f8e 0 2024-05-09T06:38:29.055 INFO:tasks.workunit.client.0.smithi063.stdout:1/941: unlink d9/d46/fdd 0 2024-05-09T06:38:29.055 INFO:tasks.workunit.client.0.smithi063.stdout:1/942: dread - d9/d2a/d35/d1e/f12c zero size 2024-05-09T06:38:29.058 INFO:tasks.workunit.client.1.smithi193.stdout:0/592: mkdir d18/d4c/d64/d66/db5 0 2024-05-09T06:38:29.060 INFO:tasks.workunit.client.1.smithi193.stdout:4/671: rmdir d1a/d1f 39 2024-05-09T06:38:29.066 INFO:tasks.workunit.client.1.smithi193.stdout:3/566: link d1/db/d2f/d78/d86/la7 d1/d44/d7d/lcf 0 2024-05-09T06:38:29.066 INFO:tasks.workunit.client.1.smithi193.stdout:4/672: dwrite fa [0,4194304] 0 2024-05-09T06:38:29.068 INFO:tasks.workunit.client.1.smithi193.stdout:4/673: write d1a/d6b/f63 [1552462,27455] 0 2024-05-09T06:38:29.071 INFO:tasks.workunit.client.1.smithi193.stdout:9/724: link dd/d1e/d1f/d48/fc8 dd/df/d15/d98/ff1 0 2024-05-09T06:38:29.073 INFO:tasks.workunit.client.0.smithi063.stdout:5/990: rmdir d8/d19/d105 39 2024-05-09T06:38:29.075 INFO:tasks.workunit.client.1.smithi193.stdout:1/661: getdents d4/d14 0 2024-05-09T06:38:29.077 INFO:tasks.workunit.client.0.smithi063.stdout:1/943: mknod d9/d2a/d35/d1e/d9d/c141 0 2024-05-09T06:38:29.078 INFO:tasks.workunit.client.1.smithi193.stdout:2/615: creat db/d37/d6d/fd0 x:0 0 0 2024-05-09T06:38:29.079 INFO:tasks.workunit.client.0.smithi063.stdout:7/959: write d0/d15/d49/f65 [4190771,109347] 0 2024-05-09T06:38:29.082 INFO:tasks.workunit.client.1.smithi193.stdout:0/593: mknod d18/d4c/d64/d66/d9d/da5/cb6 0 2024-05-09T06:38:29.083 INFO:tasks.workunit.client.0.smithi063.stdout:9/986: write d0/f97 [1249248,123373] 0 2024-05-09T06:38:29.083 INFO:tasks.workunit.client.0.smithi063.stdout:1/944: fdatasync d9/d2a/d3c/d4a/f67 0 2024-05-09T06:38:29.085 INFO:tasks.workunit.client.0.smithi063.stdout:9/987: truncate d0/d3b/da0/dcf/f125 4648915 0 2024-05-09T06:38:29.085 INFO:tasks.workunit.client.0.smithi063.stdout:5/991: mkdir d8/d17/dfe/d14b 0 2024-05-09T06:38:29.086 INFO:tasks.workunit.client.1.smithi193.stdout:7/580: mkdir d6/dc3 0 2024-05-09T06:38:29.088 INFO:tasks.workunit.client.0.smithi063.stdout:7/960: mkdir d0/d15/d4a/dc8/d148 0 2024-05-09T06:38:29.088 INFO:tasks.workunit.client.1.smithi193.stdout:7/581: read d6/d8/d29/d2f/f4c [500289,59398] 0 2024-05-09T06:38:29.089 INFO:tasks.workunit.client.1.smithi193.stdout:5/562: creat d0/d13/d2c/d37/d5b/fbb x:0 0 0 2024-05-09T06:38:29.094 INFO:tasks.workunit.client.1.smithi193.stdout:1/662: readlink d4/d16/d60/d4f/lc8 0 2024-05-09T06:38:29.095 INFO:tasks.workunit.client.1.smithi193.stdout:2/616: truncate db/df/f50 3704282 0 2024-05-09T06:38:29.096 INFO:tasks.workunit.client.1.smithi193.stdout:2/617: chown db/d37/d6d/d80/f24 366 1 2024-05-09T06:38:29.096 INFO:tasks.workunit.client.0.smithi063.stdout:5/992: symlink d8/d19/d30/d46/d148/l14c 0 2024-05-09T06:38:29.096 INFO:tasks.workunit.client.1.smithi193.stdout:6/566: link d1/d25/d2c/d3c/f49 d1/d25/d2c/d3c/db1/d58/fce 0 2024-05-09T06:38:29.097 INFO:tasks.workunit.client.0.smithi063.stdout:7/961: fsync d0/d15/d49/d54/d9f/d97/fde 0 2024-05-09T06:38:29.098 INFO:tasks.workunit.client.0.smithi063.stdout:1/945: getdents d9/d2a/d3c/d4c/d83/d88/dc2/df8 0 2024-05-09T06:38:29.099 INFO:tasks.workunit.client.0.smithi063.stdout:9/988: creat d0/d4/d7d/d105/d121/f14f x:0 0 0 2024-05-09T06:38:29.101 INFO:tasks.workunit.client.1.smithi193.stdout:7/582: rename d6/dd/d5e/d6b/f88 to d6/dd/dbc/fc4 0 2024-05-09T06:38:29.104 INFO:tasks.workunit.client.1.smithi193.stdout:4/674: mkdir d1a/da6/dd5 0 2024-05-09T06:38:29.104 INFO:tasks.workunit.client.0.smithi063.stdout:5/993: dwrite d8/d35/db6/dc2/d138/f5f [4194304,4194304] 0 2024-05-09T06:38:29.108 INFO:tasks.workunit.client.1.smithi193.stdout:2/618: mknod db/df/d54/d63/dca/cd1 0 2024-05-09T06:38:29.110 INFO:tasks.workunit.client.1.smithi193.stdout:6/567: truncate d1/f23 11077457 0 2024-05-09T06:38:29.115 INFO:tasks.workunit.client.1.smithi193.stdout:4/675: fdatasync d1a/d6d/fa1 0 2024-05-09T06:38:29.116 INFO:tasks.workunit.client.1.smithi193.stdout:1/663: write d4/d16/d6b/d6e/f71 [494132,52633] 0 2024-05-09T06:38:29.118 INFO:tasks.workunit.client.1.smithi193.stdout:0/594: link d18/d1d/f3d d18/d4c/d64/fb7 0 2024-05-09T06:38:29.120 INFO:tasks.workunit.client.1.smithi193.stdout:3/567: getdents d1/d44 0 2024-05-09T06:38:29.120 INFO:tasks.workunit.client.1.smithi193.stdout:0/595: write d18/d27/fa0 [4165847,25744] 0 2024-05-09T06:38:29.121 INFO:tasks.workunit.client.1.smithi193.stdout:4/676: mknod d1a/da6/cd6 0 2024-05-09T06:38:29.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:28 smithi193 ceph-mon[35141]: pgmap v17: 65 pgs: 65 active+clean; 3.0 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 106 MiB/s rd, 285 MiB/s wr, 598 op/s 2024-05-09T06:38:29.122 INFO:tasks.workunit.client.1.smithi193.stdout:1/664: write d4/d16/de/f28 [1501569,5038] 0 2024-05-09T06:38:29.122 INFO:tasks.workunit.client.1.smithi193.stdout:4/677: write d1a/d22/d43/fd4 [417604,61340] 0 2024-05-09T06:38:29.124 INFO:tasks.workunit.client.1.smithi193.stdout:6/568: fsync d1/de/d11/f87 0 2024-05-09T06:38:29.127 INFO:tasks.workunit.client.1.smithi193.stdout:3/568: mknod d1/d2d/cd0 0 2024-05-09T06:38:29.129 INFO:tasks.workunit.client.1.smithi193.stdout:3/569: write d1/db/d2f/fc1 [281867,109160] 0 2024-05-09T06:38:29.131 INFO:tasks.workunit.client.1.smithi193.stdout:0/596: read d18/d1d/f3d [552621,42464] 0 2024-05-09T06:38:29.132 INFO:tasks.workunit.client.1.smithi193.stdout:0/597: readlink l11 0 2024-05-09T06:38:29.132 INFO:tasks.workunit.client.1.smithi193.stdout:4/678: rmdir d1a/d1f 39 2024-05-09T06:38:29.132 INFO:tasks.workunit.client.1.smithi193.stdout:1/665: dwrite d4/d16/d60/d4f/d72/fc1 [0,4194304] 0 2024-05-09T06:38:29.133 INFO:tasks.workunit.client.1.smithi193.stdout:6/569: dread - d1/de/d11/d2d/f95 zero size 2024-05-09T06:38:29.134 INFO:tasks.workunit.client.1.smithi193.stdout:7/583: write d6/f13 [2160374,102815] 0 2024-05-09T06:38:29.137 INFO:tasks.workunit.client.1.smithi193.stdout:2/619: write db/d37/d6d/d80/d60/d7d/f85 [3615935,123992] 0 2024-05-09T06:38:29.138 INFO:tasks.workunit.client.1.smithi193.stdout:1/666: write d4/d16/d17/d1a/d3a/d3f/f55 [1555588,31743] 0 2024-05-09T06:38:29.144 INFO:tasks.workunit.client.1.smithi193.stdout:6/570: fdatasync d1/de/d11/f6c 0 2024-05-09T06:38:29.144 INFO:tasks.workunit.client.0.smithi063.stdout:9/989: getdents d0/d3b/da0 0 2024-05-09T06:38:29.145 INFO:tasks.workunit.client.0.smithi063.stdout:1/946: dread d9/f4e [0,4194304] 0 2024-05-09T06:38:29.145 INFO:tasks.workunit.client.0.smithi063.stdout:9/990: write d0/f97 [153462,126368] 0 2024-05-09T06:38:29.146 INFO:tasks.workunit.client.1.smithi193.stdout:7/584: creat d6/dd/dbc/fc5 x:0 0 0 2024-05-09T06:38:29.148 INFO:tasks.workunit.client.1.smithi193.stdout:0/598: dwrite d18/d4c/d47/d59/d76/fb4 [0,4194304] 0 2024-05-09T06:38:29.150 INFO:tasks.workunit.client.1.smithi193.stdout:1/667: readlink d4/d16/d6b/laf 0 2024-05-09T06:38:29.150 INFO:tasks.workunit.client.1.smithi193.stdout:0/599: rename d18/d1d to d18/d1d/d56/da7/db8 22 2024-05-09T06:38:29.151 INFO:tasks.workunit.client.0.smithi063.stdout:9/991: write d0/d4/d26/d39/d91/f114 [540046,14893] 0 2024-05-09T06:38:29.156 INFO:tasks.workunit.client.1.smithi193.stdout:3/570: dread d1/d2d/d51/f6a [0,4194304] 0 2024-05-09T06:38:29.157 INFO:tasks.workunit.client.1.smithi193.stdout:6/571: unlink d1/d25/d2c/d81/c6e 0 2024-05-09T06:38:29.161 INFO:tasks.workunit.client.1.smithi193.stdout:0/600: creat d18/d4c/d47/d55/fb9 x:0 0 0 2024-05-09T06:38:29.163 INFO:tasks.workunit.client.1.smithi193.stdout:4/679: dread d1a/d6b/f3c [0,4194304] 0 2024-05-09T06:38:29.164 INFO:tasks.workunit.client.1.smithi193.stdout:3/571: creat d1/db/da1/fd1 x:0 0 0 2024-05-09T06:38:29.165 INFO:tasks.workunit.client.1.smithi193.stdout:1/668: getdents d4/d16/d17/d1a/d3a/d3f/dd7 0 2024-05-09T06:38:29.166 INFO:tasks.workunit.client.1.smithi193.stdout:0/601: rmdir d18 39 2024-05-09T06:38:29.168 INFO:tasks.workunit.client.1.smithi193.stdout:4/680: read d1a/d1f/f20 [42996,45988] 0 2024-05-09T06:38:29.169 INFO:tasks.workunit.client.1.smithi193.stdout:2/620: dread db/d25/f27 [0,4194304] 0 2024-05-09T06:38:29.171 INFO:tasks.workunit.client.1.smithi193.stdout:0/602: write d18/d4c/d64/d74/dad/d70/f94 [323259,30580] 0 2024-05-09T06:38:29.172 INFO:tasks.workunit.client.1.smithi193.stdout:6/572: getdents d1/d25/d2c/d3c/db1 0 2024-05-09T06:38:29.173 INFO:tasks.workunit.client.1.smithi193.stdout:0/603: write d18/d27/fa9 [465003,77791] 0 2024-05-09T06:38:29.174 INFO:tasks.workunit.client.1.smithi193.stdout:7/585: dread d6/f13 [0,4194304] 0 2024-05-09T06:38:29.174 INFO:tasks.workunit.client.1.smithi193.stdout:4/681: creat d1a/da6/dd5/fd7 x:0 0 0 2024-05-09T06:38:29.176 INFO:tasks.workunit.client.1.smithi193.stdout:1/669: getdents d4/d16 0 2024-05-09T06:38:29.177 INFO:tasks.workunit.client.1.smithi193.stdout:3/572: rename d1/d2d/c32 to d1/cd2 0 2024-05-09T06:38:29.178 INFO:tasks.workunit.client.1.smithi193.stdout:1/670: read d4/d16/d35/f48 [165609,93023] 0 2024-05-09T06:38:29.180 INFO:tasks.workunit.client.1.smithi193.stdout:4/682: creat d1a/d24/d73/d94/fd8 x:0 0 0 2024-05-09T06:38:29.181 INFO:tasks.workunit.client.1.smithi193.stdout:2/621: rename db/d25/d38/d5b/f66 to db/d25/d38/dcd/fd2 0 2024-05-09T06:38:29.183 INFO:tasks.workunit.client.1.smithi193.stdout:4/683: chown d1a/d6b/d60/f95 0 1 2024-05-09T06:38:29.183 INFO:tasks.workunit.client.1.smithi193.stdout:7/586: dwrite d6/f8b [0,4194304] 0 2024-05-09T06:38:29.184 INFO:tasks.workunit.client.1.smithi193.stdout:3/573: mknod d1/db/d2f/d78/d86/cd3 0 2024-05-09T06:38:29.185 INFO:tasks.workunit.client.1.smithi193.stdout:3/574: fsync d1/d44/f96 0 2024-05-09T06:38:29.186 INFO:tasks.workunit.client.1.smithi193.stdout:3/575: chown d1/db/lb2 488810577 1 2024-05-09T06:38:29.190 INFO:tasks.workunit.client.1.smithi193.stdout:4/684: chown d1a/f1d 3283 1 2024-05-09T06:38:29.193 INFO:tasks.workunit.client.1.smithi193.stdout:6/573: chown d1/d25/d2c/d81/fb5 50 1 2024-05-09T06:38:29.195 INFO:tasks.workunit.client.1.smithi193.stdout:7/587: rename d6/d8/d1a/l7f to d6/d17/d7d/daa/lc6 0 2024-05-09T06:38:29.197 INFO:tasks.workunit.client.1.smithi193.stdout:1/671: mkdir d4/d16/d17/d1a/d3a/d3f/dd7/ddb 0 2024-05-09T06:38:29.197 INFO:tasks.workunit.client.1.smithi193.stdout:7/588: read d6/d8/d29/d2f/f8d [363131,55594] 0 2024-05-09T06:38:29.199 INFO:tasks.workunit.client.1.smithi193.stdout:4/685: fsync d1a/d22/d43/f4d 0 2024-05-09T06:38:29.199 INFO:tasks.workunit.client.1.smithi193.stdout:0/604: dwrite d18/d4c/d64/d74/dad/f7e [0,4194304] 0 2024-05-09T06:38:29.200 INFO:tasks.workunit.client.0.smithi063.stdout:1/947: dread - d9/d2a/d39/d5d/d75/d139/d59/d62/d7e/fea zero size 2024-05-09T06:38:29.202 INFO:tasks.workunit.client.0.smithi063.stdout:9/992: rmdir d0/d4/d26/d39/d42 39 2024-05-09T06:38:29.203 INFO:tasks.workunit.client.1.smithi193.stdout:6/574: creat d1/fcf x:0 0 0 2024-05-09T06:38:29.203 INFO:tasks.workunit.client.1.smithi193.stdout:3/576: symlink d1/ld4 0 2024-05-09T06:38:29.204 INFO:tasks.workunit.client.0.smithi063.stdout:9/993: stat d0/d8/d2d 0 2024-05-09T06:38:29.204 INFO:tasks.workunit.client.0.smithi063.stdout:5/994: dwrite d8/d19/fe2 [0,4194304] 0 2024-05-09T06:38:29.206 INFO:tasks.workunit.client.1.smithi193.stdout:2/622: dread db/d37/d6d/d80/fa6 [0,4194304] 0 2024-05-09T06:38:29.208 INFO:tasks.workunit.client.1.smithi193.stdout:1/672: creat d4/d3c/fdc x:0 0 0 2024-05-09T06:38:29.209 INFO:tasks.workunit.client.1.smithi193.stdout:7/589: unlink d6/dd/d5e/cb8 0 2024-05-09T06:38:29.210 INFO:tasks.workunit.client.1.smithi193.stdout:4/686: dwrite d1a/d1f/fd2 [0,4194304] 0 2024-05-09T06:38:29.210 INFO:tasks.workunit.client.1.smithi193.stdout:7/590: chown d6/d8/f55 136550499 1 2024-05-09T06:38:29.212 INFO:tasks.workunit.client.1.smithi193.stdout:7/591: write d6/dd/f47 [1496064,22798] 0 2024-05-09T06:38:29.216 INFO:tasks.workunit.client.1.smithi193.stdout:7/592: write d6/d8/d29/d2f/fc0 [984892,11953] 0 2024-05-09T06:38:29.218 INFO:tasks.workunit.client.1.smithi193.stdout:4/687: dread d1a/d24/f2e [0,4194304] 0 2024-05-09T06:38:29.219 INFO:tasks.workunit.client.1.smithi193.stdout:0/605: symlink d18/d4c/d64/d66/lba 0 2024-05-09T06:38:29.221 INFO:tasks.workunit.client.0.smithi063.stdout:1/948: creat d9/d2a/d39/d5d/d75/d139/d13b/f142 x:0 0 0 2024-05-09T06:38:29.224 INFO:tasks.workunit.client.0.smithi063.stdout:5/995: write d8/d65/f80 [5380574,96085] 0 2024-05-09T06:38:29.226 INFO:tasks.workunit.client.1.smithi193.stdout:0/606: dwrite d18/d27/f34 [4194304,4194304] 0 2024-05-09T06:38:29.227 INFO:tasks.workunit.client.1.smithi193.stdout:3/577: unlink d1/db/d2f/d78/d86/d94/lc0 0 2024-05-09T06:38:29.227 INFO:tasks.workunit.client.1.smithi193.stdout:0/607: chown d18/l3b 48 1 2024-05-09T06:38:29.227 INFO:tasks.workunit.client.0.smithi063.stdout:1/949: dwrite d9/d46/dfe/f132 [0,4194304] 0 2024-05-09T06:38:29.229 INFO:tasks.workunit.client.1.smithi193.stdout:0/608: fsync d18/d4c/d64/d74/dad/f7e 0 2024-05-09T06:38:29.233 INFO:tasks.workunit.client.1.smithi193.stdout:7/593: creat d6/d8/d1a/d36/d4d/fc7 x:0 0 0 2024-05-09T06:38:29.237 INFO:tasks.workunit.client.0.smithi063.stdout:1/950: rmdir d9/d46 39 2024-05-09T06:38:29.240 INFO:tasks.workunit.client.1.smithi193.stdout:4/688: unlink d1a/f64 0 2024-05-09T06:38:29.243 INFO:tasks.workunit.client.0.smithi063.stdout:1/951: mknod d9/d2a/d39/d5d/d75/d139/d59/d62/d7e/c143 0 2024-05-09T06:38:29.243 INFO:tasks.workunit.client.1.smithi193.stdout:4/689: dread d1a/d22/d43/fd4 [0,4194304] 0 2024-05-09T06:38:29.244 INFO:tasks.workunit.client.0.smithi063.stdout:9/994: dwrite d0/d4/d57/d60/f7c [0,4194304] 0 2024-05-09T06:38:29.245 INFO:tasks.workunit.client.0.smithi063.stdout:5/996: rename d8/f4c to d8/d19/d105/f14d 0 2024-05-09T06:38:29.247 INFO:tasks.workunit.client.0.smithi063.stdout:9/995: write d0/d4/d26/d39/d48/d80/d84/f10c [3367401,81655] 0 2024-05-09T06:38:29.247 INFO:tasks.workunit.client.1.smithi193.stdout:3/578: read d1/db/d2f/d78/fb4 [328138,36910] 0 2024-05-09T06:38:29.247 INFO:tasks.workunit.client.0.smithi063.stdout:1/952: read d9/d2a/d35/f29 [2944570,47005] 0 2024-05-09T06:38:29.250 INFO:tasks.workunit.client.0.smithi063.stdout:1/953: write d9/d2a/d39/d5d/d75/d139/d59/f79 [1578636,79323] 0 2024-05-09T06:38:29.252 INFO:tasks.workunit.client.0.smithi063.stdout:5/997: mkdir d8/d19/d105/d14e 0 2024-05-09T06:38:29.258 INFO:tasks.workunit.client.0.smithi063.stdout:5/998: dread d8/d17/f1c [0,4194304] 0 2024-05-09T06:38:29.262 INFO:tasks.workunit.client.0.smithi063.stdout:5/999: mkdir d8/d19/d30/d74/d14f 0 2024-05-09T06:38:29.265 INFO:tasks.workunit.client.0.smithi063.stdout:1/954: write d9/d46/fc0 [10006,125693] 0 2024-05-09T06:38:29.269 INFO:tasks.workunit.client.0.smithi063.stdout:9/996: dwrite d0/d43/f4b [4194304,4194304] 0 2024-05-09T06:38:29.273 INFO:tasks.workunit.client.0.smithi063.stdout:9/997: symlink d0/d8/da6/db0/d113/l150 0 2024-05-09T06:38:29.273 INFO:tasks.workunit.client.1.smithi193.stdout:1/673: write d4/d16/f42 [891956,26281] 0 2024-05-09T06:38:29.275 INFO:tasks.workunit.client.0.smithi063.stdout:1/955: dwrite d9/d2a/d39/d5d/d75/d139/d59/d62/fa5 [0,4194304] 0 2024-05-09T06:38:29.275 INFO:tasks.workunit.client.1.smithi193.stdout:4/690: fsync d1a/d98/fb8 0 2024-05-09T06:38:29.279 INFO:tasks.workunit.client.0.smithi063.stdout:9/998: fsync d0/d8/d2d/faf 0 2024-05-09T06:38:29.279 INFO:tasks.workunit.client.1.smithi193.stdout:6/575: readlink d1/d25/d2c/d81/d37/l45 0 2024-05-09T06:38:29.282 INFO:tasks.workunit.client.1.smithi193.stdout:2/623: dwrite db/d37/f42 [0,4194304] 0 2024-05-09T06:38:29.283 INFO:tasks.workunit.client.1.smithi193.stdout:7/594: mkdir d6/d8/d29/dc8 0 2024-05-09T06:38:29.285 INFO:tasks.workunit.client.0.smithi063.stdout:9/999: mkdir d0/d8/da6/db0/d113/d151 0 2024-05-09T06:38:29.286 INFO:tasks.workunit.client.1.smithi193.stdout:1/674: mkdir d4/d16/d6b/d85/ddd 0 2024-05-09T06:38:29.290 INFO:tasks.workunit.client.1.smithi193.stdout:0/609: link d18/d1d/d56/c89 d18/d4c/d47/cbb 0 2024-05-09T06:38:29.291 INFO:tasks.workunit.client.1.smithi193.stdout:1/675: chown d4/d16/d35/fcc 339491235 1 2024-05-09T06:38:29.291 INFO:tasks.workunit.client.1.smithi193.stdout:2/624: creat db/d4b/fd3 x:0 0 0 2024-05-09T06:38:29.292 INFO:tasks.workunit.client.1.smithi193.stdout:4/691: dwrite d1a/d24/d73/d94/fae [0,4194304] 0 2024-05-09T06:38:29.296 INFO:tasks.workunit.client.1.smithi193.stdout:6/576: rename d1/de/f13 to d1/de/d11/d2d/da4/fd0 0 2024-05-09T06:38:29.296 INFO:tasks.workunit.client.1.smithi193.stdout:4/692: chown d1a/fa5 114115218 1 2024-05-09T06:38:29.298 INFO:tasks.workunit.client.1.smithi193.stdout:3/579: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:29.305 INFO:tasks.workunit.client.1.smithi193.stdout:3/580: creat d1/d44/fd5 x:0 0 0 2024-05-09T06:38:29.306 INFO:tasks.workunit.client.1.smithi193.stdout:6/577: rmdir d1/d25/d2c/d3c/db1/d58/dc9 39 2024-05-09T06:38:29.307 INFO:tasks.workunit.client.0.smithi063.stdout:1/956: write d9/d10/f43 [3870181,49166] 0 2024-05-09T06:38:29.307 INFO:tasks.workunit.client.1.smithi193.stdout:3/581: chown d1/db/da1/dce 1738 1 2024-05-09T06:38:29.307 INFO:tasks.workunit.client.1.smithi193.stdout:1/676: dwrite d4/d3c/fdc [0,4194304] 0 2024-05-09T06:38:29.310 INFO:tasks.workunit.client.1.smithi193.stdout:2/625: rmdir db/d37/d6d/d80/d60/d7d/da8 0 2024-05-09T06:38:29.311 INFO:tasks.workunit.client.1.smithi193.stdout:6/578: fsync d1/f70 0 2024-05-09T06:38:29.313 INFO:tasks.workunit.client.1.smithi193.stdout:3/582: mknod d1/db/d2f/d84/cd6 0 2024-05-09T06:38:29.315 INFO:tasks.workunit.client.1.smithi193.stdout:6/579: readlink d1/l15 0 2024-05-09T06:38:29.319 INFO:tasks.workunit.client.1.smithi193.stdout:6/580: symlink d1/d25/d2c/d3c/db1/ld1 0 2024-05-09T06:38:29.320 INFO:tasks.workunit.client.0.smithi063.stdout:7/962: sync 2024-05-09T06:38:29.321 INFO:tasks.workunit.client.1.smithi193.stdout:7/595: dread d6/d8/f44 [0,4194304] 0 2024-05-09T06:38:29.322 INFO:tasks.workunit.client.1.smithi193.stdout:2/626: dwrite db/d25/d38/d40/f72 [12582912,4194304] 0 2024-05-09T06:38:29.325 INFO:tasks.workunit.client.1.smithi193.stdout:2/627: readlink db/l51 0 2024-05-09T06:38:29.326 INFO:tasks.workunit.client.0.smithi063.stdout:7/963: dwrite d0/d15/d49/d4f/d66/d67/d6b/f146 [0,4194304] 0 2024-05-09T06:38:29.328 INFO:tasks.workunit.client.0.smithi063.stdout:7/964: readlink d0/d15/d49/d4f/l6d 0 2024-05-09T06:38:29.328 INFO:tasks.workunit.client.1.smithi193.stdout:2/628: fsync db/d25/d38/d40/d8f/f93 0 2024-05-09T06:38:29.331 INFO:tasks.workunit.client.1.smithi193.stdout:2/629: chown db/d25/d38/d61/f70 1 1 2024-05-09T06:38:29.333 INFO:tasks.workunit.client.1.smithi193.stdout:4/693: dwrite d1a/d67/fc1 [0,4194304] 0 2024-05-09T06:38:29.339 INFO:tasks.workunit.client.1.smithi193.stdout:6/581: chown d1/d25/d2c/d3c/db1 86 1 2024-05-09T06:38:29.342 INFO:tasks.workunit.client.1.smithi193.stdout:4/694: fsync f10 0 2024-05-09T06:38:29.342 INFO:tasks.workunit.client.1.smithi193.stdout:3/583: write d1/d44/f5f [300791,128426] 0 2024-05-09T06:38:29.343 INFO:tasks.workunit.client.1.smithi193.stdout:7/596: write d6/d8/d1a/d36/d4d/d74/f7c [642577,111355] 0 2024-05-09T06:38:29.343 INFO:tasks.workunit.client.0.smithi063.stdout:7/965: dread d0/d15/d49/d4f/d66/d67/d86/fff [0,4194304] 0 2024-05-09T06:38:29.347 INFO:tasks.workunit.client.1.smithi193.stdout:2/630: dwrite db/df/d54/fb8 [0,4194304] 0 2024-05-09T06:38:29.348 INFO:tasks.workunit.client.1.smithi193.stdout:1/677: dread d4/d16/d17/d1a/f9d [0,4194304] 0 2024-05-09T06:38:29.348 INFO:tasks.workunit.client.0.smithi063.stdout:7/966: creat d0/d15/f149 x:0 0 0 2024-05-09T06:38:29.353 INFO:tasks.workunit.client.1.smithi193.stdout:3/584: mknod d1/db/d2f/cd7 0 2024-05-09T06:38:29.353 INFO:tasks.workunit.client.1.smithi193.stdout:2/631: rmdir db/d9b 39 2024-05-09T06:38:29.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:28 smithi063 ceph-mon[27171]: pgmap v17: 65 pgs: 65 active+clean; 3.0 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 106 MiB/s rd, 285 MiB/s wr, 598 op/s 2024-05-09T06:38:29.355 INFO:tasks.workunit.client.0.smithi063.stdout:7/967: dread - d0/d15/d49/d4f/d66/d67/d86/fe1 zero size 2024-05-09T06:38:29.356 INFO:tasks.workunit.client.0.smithi063.stdout:7/968: readlink d0/lea 0 2024-05-09T06:38:29.356 INFO:tasks.workunit.client.1.smithi193.stdout:1/678: dwrite d4/d14/f2e [4194304,4194304] 0 2024-05-09T06:38:29.358 INFO:tasks.workunit.client.1.smithi193.stdout:1/679: chown d4/d16/d60/d4f/lc8 149011197 1 2024-05-09T06:38:29.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/632: dread db/d4b/f5d [0,4194304] 0 2024-05-09T06:38:29.360 INFO:tasks.workunit.client.1.smithi193.stdout:1/680: chown d4/d16/d35/f48 8869 1 2024-05-09T06:38:29.362 INFO:tasks.workunit.client.1.smithi193.stdout:6/582: link d1/d25/d2c/d81/fb5 d1/d25/d2c/d76/fd2 0 2024-05-09T06:38:29.366 INFO:tasks.workunit.client.1.smithi193.stdout:7/597: creat d6/d8/fc9 x:0 0 0 2024-05-09T06:38:29.366 INFO:tasks.workunit.client.0.smithi063.stdout:7/969: dread d0/d15/d49/d4f/d66/d67/ffd [0,4194304] 0 2024-05-09T06:38:29.369 INFO:tasks.workunit.client.1.smithi193.stdout:1/681: dwrite d4/d16/de/f28 [0,4194304] 0 2024-05-09T06:38:29.373 INFO:tasks.workunit.client.1.smithi193.stdout:1/682: chown d4/d3c/d3e/f77 617439 1 2024-05-09T06:38:29.379 INFO:tasks.workunit.client.1.smithi193.stdout:7/598: dwrite d6/d17/d7d/daa/f8c [0,4194304] 0 2024-05-09T06:38:29.380 INFO:tasks.workunit.client.1.smithi193.stdout:1/683: rmdir d4/d16/d6b/d89/dc4 39 2024-05-09T06:38:29.390 INFO:tasks.workunit.client.1.smithi193.stdout:4/695: dwrite d1a/d24/d73/d94/d7c/d5b/f87 [0,4194304] 0 2024-05-09T06:38:29.390 INFO:tasks.workunit.client.0.smithi063.stdout:7/970: dwrite d0/d15/d49/d54/f60 [0,4194304] 0 2024-05-09T06:38:29.393 INFO:tasks.workunit.client.1.smithi193.stdout:3/585: dread d1/d2d/d51/f65 [0,4194304] 0 2024-05-09T06:38:29.398 INFO:tasks.workunit.client.1.smithi193.stdout:6/583: dwrite d1/f86 [0,4194304] 0 2024-05-09T06:38:29.407 INFO:tasks.workunit.client.1.smithi193.stdout:3/586: dread d1/db/d59/f69 [0,4194304] 0 2024-05-09T06:38:29.409 INFO:tasks.workunit.client.1.smithi193.stdout:6/584: dread d1/d25/d2c/fb9 [0,4194304] 0 2024-05-09T06:38:29.412 INFO:tasks.workunit.client.0.smithi063.stdout:7/971: write d0/d15/d4a/d68/f7d [101197,46818] 0 2024-05-09T06:38:29.413 INFO:tasks.workunit.client.1.smithi193.stdout:7/599: fdatasync d6/d8/d1a/f8f 0 2024-05-09T06:38:29.414 INFO:tasks.workunit.client.0.smithi063.stdout:7/972: read d0/d15/f1b [3805,584] 0 2024-05-09T06:38:29.415 INFO:tasks.workunit.client.0.smithi063.stdout:7/973: write d0/d15/d49/d54/d9f/f143 [499415,104749] 0 2024-05-09T06:38:29.415 INFO:tasks.workunit.client.1.smithi193.stdout:1/684: symlink d4/d16/d60/d4f/d72/d7f/lde 0 2024-05-09T06:38:29.417 INFO:tasks.workunit.client.1.smithi193.stdout:4/696: fdatasync d1a/d24/d73/d94/d7c/d5b/f70 0 2024-05-09T06:38:29.419 INFO:tasks.workunit.client.1.smithi193.stdout:3/587: rmdir d1/db/da1 39 2024-05-09T06:38:29.423 INFO:tasks.workunit.client.0.smithi063.stdout:7/974: dwrite d0/d15/f4e [4194304,4194304] 0 2024-05-09T06:38:29.424 INFO:tasks.workunit.client.1.smithi193.stdout:1/685: rename d4/d16/d17/d1a/d3a/d3f/f55 to d4/d16/de/fdf 0 2024-05-09T06:38:29.424 INFO:tasks.workunit.client.0.smithi063.stdout:7/975: chown d0/d15/d49/c11e 4299052 1 2024-05-09T06:38:29.424 INFO:tasks.workunit.client.1.smithi193.stdout:1/686: chown d4/d16/de/f12 109646 1 2024-05-09T06:38:29.431 INFO:tasks.workunit.client.1.smithi193.stdout:6/585: chown d1/d25/d2c/d3c/db1/d58/dc9 665591 1 2024-05-09T06:38:29.433 INFO:tasks.workunit.client.0.smithi063.stdout:7/976: creat d0/d15/d49/d54/f14a x:0 0 0 2024-05-09T06:38:29.434 INFO:tasks.workunit.client.1.smithi193.stdout:6/586: write d1/de/d11/d2d/f40 [1885134,85304] 0 2024-05-09T06:38:29.437 INFO:tasks.workunit.client.0.smithi063.stdout:7/977: truncate d0/d15/d4a/d50/f30 1311641 0 2024-05-09T06:38:29.437 INFO:tasks.workunit.client.0.smithi063.stdout:7/978: stat d0/l94 0 2024-05-09T06:38:29.438 INFO:tasks.workunit.client.0.smithi063.stdout:7/979: write d0/d15/d49/f5b [3387613,90574] 0 2024-05-09T06:38:29.440 INFO:tasks.workunit.client.0.smithi063.stdout:7/980: chown d0/d15/d49/d54/d9f/d97/d9b/c10d 151540 1 2024-05-09T06:38:29.442 INFO:tasks.workunit.client.1.smithi193.stdout:1/687: dwrite d4/d14/f2f [0,4194304] 0 2024-05-09T06:38:29.444 INFO:tasks.workunit.client.1.smithi193.stdout:6/587: rename d1/de/d11/d2d/f2e to d1/d25/d2c/d81/d37/d4f/fd3 0 2024-05-09T06:38:29.446 INFO:tasks.workunit.client.1.smithi193.stdout:3/588: dread d1/d2d/f7c [0,4194304] 0 2024-05-09T06:38:29.447 INFO:tasks.workunit.client.1.smithi193.stdout:7/600: getdents d6/dd/d7e 0 2024-05-09T06:38:29.452 INFO:tasks.workunit.client.1.smithi193.stdout:1/688: creat d4/d16/d6b/d89/dc4/fe0 x:0 0 0 2024-05-09T06:38:29.453 INFO:tasks.workunit.client.1.smithi193.stdout:4/697: write f10 [895838,29783] 0 2024-05-09T06:38:29.460 INFO:tasks.workunit.client.1.smithi193.stdout:3/589: link d1/db/d2f/d84/d97/lbe d1/db/d2f/d78/d86/d94/ld8 0 2024-05-09T06:38:29.461 INFO:tasks.workunit.client.1.smithi193.stdout:3/590: read - d1/db/d2f/f9d zero size 2024-05-09T06:38:29.462 INFO:tasks.workunit.client.1.smithi193.stdout:4/698: fdatasync f5 0 2024-05-09T06:38:29.469 INFO:tasks.workunit.client.0.smithi063.stdout:7/981: dwrite d0/d15/f1b [0,4194304] 0 2024-05-09T06:38:29.470 INFO:tasks.workunit.client.0.smithi063.stdout:7/982: read d0/fbe [11982946,46717] 0 2024-05-09T06:38:29.471 INFO:tasks.workunit.client.1.smithi193.stdout:1/689: link d4/f26 d4/d16/d17/d1a/d3a/fe1 0 2024-05-09T06:38:29.472 INFO:tasks.workunit.client.0.smithi063.stdout:1/957: sync 2024-05-09T06:38:29.474 INFO:tasks.workunit.client.1.smithi193.stdout:6/588: dwrite d1/d25/d2c/d76/f7d [0,4194304] 0 2024-05-09T06:38:29.482 INFO:tasks.workunit.client.1.smithi193.stdout:4/699: getdents d1a/d98 0 2024-05-09T06:38:29.483 INFO:tasks.workunit.client.1.smithi193.stdout:3/591: dread d1/db/d2f/d78/f8b [0,4194304] 0 2024-05-09T06:38:29.483 INFO:tasks.workunit.client.1.smithi193.stdout:4/700: truncate d1a/d24/d73/d94/f8f 5213745 0 2024-05-09T06:38:29.484 INFO:tasks.workunit.client.1.smithi193.stdout:4/701: dread - d1a/d24/fbf zero size 2024-05-09T06:38:29.485 INFO:tasks.workunit.client.1.smithi193.stdout:6/589: fdatasync d1/d25/d2c/d81/d37/d4f/f66 0 2024-05-09T06:38:29.487 INFO:tasks.workunit.client.1.smithi193.stdout:1/690: mknod d4/ce2 0 2024-05-09T06:38:29.487 INFO:tasks.workunit.client.1.smithi193.stdout:6/590: write d1/f86 [4795549,121413] 0 2024-05-09T06:38:29.488 INFO:tasks.workunit.client.0.smithi063.stdout:7/983: creat d0/d15/d49/f14b x:0 0 0 2024-05-09T06:38:29.489 INFO:tasks.workunit.client.1.smithi193.stdout:4/702: mkdir d1a/d6d/dd9 0 2024-05-09T06:38:29.490 INFO:tasks.workunit.client.0.smithi063.stdout:7/984: truncate d0/d15/fd1 898474 0 2024-05-09T06:38:29.493 INFO:tasks.workunit.client.0.smithi063.stdout:7/985: creat d0/d15/d89/d10c/dfb/d13e/f14c x:0 0 0 2024-05-09T06:38:29.495 INFO:tasks.workunit.client.1.smithi193.stdout:1/691: readlink d4/d16/d35/ld3 0 2024-05-09T06:38:29.495 INFO:tasks.workunit.client.1.smithi193.stdout:6/591: dwrite d1/d25/d4c/d9e/fb0 [0,4194304] 0 2024-05-09T06:38:29.497 INFO:tasks.workunit.client.0.smithi063.stdout:7/986: symlink d0/l14d 0 2024-05-09T06:38:29.497 INFO:tasks.workunit.client.0.smithi063.stdout:7/987: fdatasync d0/d15/d89/d113/f133 0 2024-05-09T06:38:29.498 INFO:tasks.workunit.client.0.smithi063.stdout:7/988: write d0/d15/d49/d54/d9f/d97/f124 [455065,68545] 0 2024-05-09T06:38:29.499 INFO:tasks.workunit.client.1.smithi193.stdout:4/703: dwrite d1a/da6/dd5/fd7 [0,4194304] 0 2024-05-09T06:38:29.503 INFO:tasks.workunit.client.1.smithi193.stdout:4/704: write d1a/d98/fb8 [3932930,21705] 0 2024-05-09T06:38:29.504 INFO:tasks.workunit.client.1.smithi193.stdout:4/705: write d1a/d1f/f2a [269832,53573] 0 2024-05-09T06:38:29.506 INFO:tasks.workunit.client.0.smithi063.stdout:7/989: mknod d0/d15/d49/d54/d9f/dd2/df9/c14e 0 2024-05-09T06:38:29.507 INFO:tasks.workunit.client.1.smithi193.stdout:6/592: rename d1/de/d11/f5c to d1/d25/d2c/d76/d69/fd4 0 2024-05-09T06:38:29.508 INFO:tasks.workunit.client.1.smithi193.stdout:1/692: truncate d4/f9 963912 0 2024-05-09T06:38:29.512 INFO:tasks.workunit.client.1.smithi193.stdout:6/593: dread d1/d25/d2c/fb9 [0,4194304] 0 2024-05-09T06:38:29.514 INFO:tasks.workunit.client.1.smithi193.stdout:1/693: rmdir d4/d16/d17/d1a 39 2024-05-09T06:38:29.515 INFO:tasks.workunit.client.1.smithi193.stdout:6/594: readlink d1/d25/lca 0 2024-05-09T06:38:29.518 INFO:tasks.workunit.client.1.smithi193.stdout:1/694: read d4/d3c/d3e/f5c [2680345,38431] 0 2024-05-09T06:38:29.519 INFO:tasks.workunit.client.1.smithi193.stdout:6/595: dread d1/d25/d2c/d76/f7d [0,4194304] 0 2024-05-09T06:38:29.520 INFO:tasks.workunit.client.1.smithi193.stdout:6/596: readlink d1/de/d11/l9b 0 2024-05-09T06:38:29.521 INFO:tasks.workunit.client.1.smithi193.stdout:1/695: creat d4/d16/de/dca/fe3 x:0 0 0 2024-05-09T06:38:29.522 INFO:tasks.workunit.client.1.smithi193.stdout:1/696: readlink d4/d16/de/l64 0 2024-05-09T06:38:29.523 INFO:tasks.workunit.client.1.smithi193.stdout:6/597: creat d1/d25/d4c/fd5 x:0 0 0 2024-05-09T06:38:29.525 INFO:tasks.workunit.client.1.smithi193.stdout:6/598: symlink d1/d25/d4c/d9e/dc8/ld6 0 2024-05-09T06:38:29.526 INFO:tasks.workunit.client.0.smithi063.stdout:7/990: dread d0/fbe [0,4194304] 0 2024-05-09T06:38:29.527 INFO:tasks.workunit.client.1.smithi193.stdout:1/697: rename d4/d16/d6b/d6e/cc3 to d4/d16/d60/d4f/d59/ce4 0 2024-05-09T06:38:29.528 INFO:tasks.workunit.client.1.smithi193.stdout:4/706: write d1a/f1d [959006,19575] 0 2024-05-09T06:38:29.529 INFO:tasks.workunit.client.1.smithi193.stdout:6/599: creat d1/d25/d4c/fd7 x:0 0 0 2024-05-09T06:38:29.531 INFO:tasks.workunit.client.1.smithi193.stdout:1/698: read d4/f8c [1625485,15549] 0 2024-05-09T06:38:29.532 INFO:tasks.workunit.client.1.smithi193.stdout:6/600: creat d1/de/d11/d2d/da4/dc2/fd8 x:0 0 0 2024-05-09T06:38:29.533 INFO:tasks.workunit.client.0.smithi063.stdout:7/991: dwrite d0/d15/f149 [0,4194304] 0 2024-05-09T06:38:29.533 INFO:tasks.workunit.client.0.smithi063.stdout:7/992: fsync d0/d15/fcc 0 2024-05-09T06:38:29.534 INFO:tasks.workunit.client.1.smithi193.stdout:6/601: creat d1/de/d11/d2d/da4/dc2/fd9 x:0 0 0 2024-05-09T06:38:29.536 INFO:tasks.workunit.client.0.smithi063.stdout:7/993: symlink d0/d15/d49/d4f/d66/d67/d6b/l14f 0 2024-05-09T06:38:29.539 INFO:tasks.workunit.client.0.smithi063.stdout:7/994: fdatasync d0/d15/d4a/d50/f1a 0 2024-05-09T06:38:29.540 INFO:tasks.workunit.client.0.smithi063.stdout:7/995: stat d0/d15/f4e 0 2024-05-09T06:38:29.540 INFO:tasks.workunit.client.1.smithi193.stdout:6/602: link d1/d25/d4c/l7b d1/de/d11/d2d/d3a/lda 0 2024-05-09T06:38:29.542 INFO:tasks.workunit.client.1.smithi193.stdout:6/603: creat d1/de/d11/d2d/fdb x:0 0 0 2024-05-09T06:38:29.543 INFO:tasks.workunit.client.0.smithi063.stdout:7/996: symlink d0/d15/d49/d54/d9f/d106/l150 0 2024-05-09T06:38:29.543 INFO:tasks.workunit.client.1.smithi193.stdout:6/604: write d1/d25/d2c/fc4 [4561189,78406] 0 2024-05-09T06:38:29.547 INFO:tasks.workunit.client.1.smithi193.stdout:6/605: mkdir d1/de/d11/d18/ddc 0 2024-05-09T06:38:29.549 INFO:tasks.workunit.client.1.smithi193.stdout:6/606: mknod d1/d25/d2c/d81/d37/d4f/cdd 0 2024-05-09T06:38:29.551 INFO:tasks.workunit.client.1.smithi193.stdout:6/607: truncate d1/de/d11/d2d/f34 1686952 0 2024-05-09T06:38:29.552 INFO:tasks.workunit.client.1.smithi193.stdout:6/608: readlink d1/d25/d2c/d3c/db1/d58/l59 0 2024-05-09T06:38:29.552 INFO:tasks.workunit.client.0.smithi063.stdout:7/997: truncate d0/d15/d49/fbf 7854436 0 2024-05-09T06:38:29.558 INFO:tasks.workunit.client.1.smithi193.stdout:6/609: write d1/d25/d2c/f6f [989334,10393] 0 2024-05-09T06:38:29.559 INFO:tasks.workunit.client.0.smithi063.stdout:7/998: dwrite d0/d15/d49/d54/d9f/d97/d9b/f11b [0,4194304] 0 2024-05-09T06:38:29.561 INFO:tasks.workunit.client.0.smithi063.stdout:7/999: chown d0/d15/d4a/d50/l59 0 1 2024-05-09T06:38:29.568 INFO:tasks.workunit.client.1.smithi193.stdout:6/610: dwrite d1/d25/d2c/d81/d37/d4f/f9a [0,4194304] 0 2024-05-09T06:38:29.571 INFO:tasks.workunit.client.0.smithi063.stdout:1/958: sync 2024-05-09T06:38:29.574 INFO:tasks.workunit.client.0.smithi063.stdout:1/959: creat d9/d2a/d3c/d4c/d83/d88/dc2/f144 x:0 0 0 2024-05-09T06:38:29.575 INFO:tasks.workunit.client.1.smithi193.stdout:6/611: link d1/d25/d2c/d3c/db1/ld1 d1/de/d11/d18/d94/lde 0 2024-05-09T06:38:29.577 INFO:tasks.workunit.client.1.smithi193.stdout:6/612: mkdir d1/de/d11/d2d/ddf 0 2024-05-09T06:38:29.578 INFO:tasks.workunit.client.1.smithi193.stdout:6/613: truncate d1/d25/d2c/d81/d37/d4f/fab 213199 0 2024-05-09T06:38:29.579 INFO:tasks.workunit.client.0.smithi063.stdout:1/960: dread d9/d2a/d35/d1e/d9d/f82 [0,4194304] 0 2024-05-09T06:38:29.585 INFO:tasks.workunit.client.1.smithi193.stdout:6/614: fsync d1/d25/d2c/d81/d37/d4f/fd3 0 2024-05-09T06:38:29.587 INFO:tasks.workunit.client.0.smithi063.stdout:1/961: creat d9/d2a/d39/d5d/d75/d139/d59/f145 x:0 0 0 2024-05-09T06:38:29.588 INFO:tasks.workunit.client.0.smithi063.stdout:1/962: chown d9/d46/dfe/d94/d112 31984297 1 2024-05-09T06:38:29.591 INFO:tasks.workunit.client.0.smithi063.stdout:1/963: read d9/d2a/d3c/dbc/d11c/f11d [3499958,23983] 0 2024-05-09T06:38:29.591 INFO:tasks.workunit.client.0.smithi063.stdout:1/964: readlink d9/d10/lc5 0 2024-05-09T06:38:29.591 INFO:tasks.workunit.client.1.smithi193.stdout:6/615: dwrite d1/de/d11/d2d/da4/dc2/fd9 [0,4194304] 0 2024-05-09T06:38:29.594 INFO:tasks.workunit.client.0.smithi063.stdout:1/965: mknod d9/d2a/d3c/d4c/d83/c146 0 2024-05-09T06:38:29.595 INFO:tasks.workunit.client.0.smithi063.stdout:1/966: chown d9/d2a/d3c/d4c/d83/de9 1054578 1 2024-05-09T06:38:29.607 INFO:tasks.workunit.client.0.smithi063.stdout:1/967: write d9/f4e [2173578,27163] 0 2024-05-09T06:38:29.611 INFO:tasks.workunit.client.1.smithi193.stdout:6/616: dwrite d1/de/d11/d2d/fa8 [0,4194304] 0 2024-05-09T06:38:29.614 INFO:tasks.workunit.client.1.smithi193.stdout:6/617: creat d1/de/d11/d2d/da4/dc2/fe0 x:0 0 0 2024-05-09T06:38:29.615 INFO:tasks.workunit.client.1.smithi193.stdout:6/618: readlink d1/d25/d2c/d3c/db1/ld1 0 2024-05-09T06:38:29.617 INFO:tasks.workunit.client.1.smithi193.stdout:6/619: getdents d1/d25/d2c/d3c/db1/d58/dc9 0 2024-05-09T06:38:29.621 INFO:tasks.workunit.client.0.smithi063.stdout:1/968: dwrite d9/d2a/d3c/d4c/d83/d88/dc2/fe0 [0,4194304] 0 2024-05-09T06:38:29.624 INFO:tasks.workunit.client.0.smithi063.stdout:1/969: fdatasync d9/d2a/d35/f8f 0 2024-05-09T06:38:29.634 INFO:tasks.workunit.client.0.smithi063.stdout:1/970: rename d9/d2a/d2e/df4/f12f to d9/d2a/d39/d5d/f147 0 2024-05-09T06:38:29.656 INFO:tasks.workunit.client.0.smithi063.stdout:1/971: mkdir d9/d2a/d3c/d4c/d83/d148 0 2024-05-09T06:38:29.657 INFO:tasks.workunit.client.0.smithi063.stdout:1/972: mknod d9/d2a/d3c/d4a/db8/ddf/c149 0 2024-05-09T06:38:29.663 INFO:tasks.workunit.client.0.smithi063.stdout:1/973: dwrite d9/d2a/d3c/d4c/d83/d88/fe3 [0,4194304] 0 2024-05-09T06:38:29.668 INFO:tasks.workunit.client.0.smithi063.stdout:1/974: link d9/d2a/d2e/f50 d9/d2a/d3c/d4c/d83/d88/dfb/f14a 0 2024-05-09T06:38:29.674 INFO:tasks.workunit.client.0.smithi063.stdout:1/975: dread d9/d2a/d3c/d4a/db8/f11e [0,4194304] 0 2024-05-09T06:38:29.704 INFO:tasks.workunit.client.0.smithi063.stdout:1/976: write d9/d2a/d39/d5d/d75/fa9 [840592,57943] 0 2024-05-09T06:38:29.711 INFO:tasks.workunit.client.0.smithi063.stdout:1/977: dwrite d9/d2a/d3c/dbc/d114/f125 [0,4194304] 0 2024-05-09T06:38:29.714 INFO:tasks.workunit.client.0.smithi063.stdout:1/978: mknod d9/d2a/d3c/d4c/d83/d88/dc2/df8/c14b 0 2024-05-09T06:38:29.714 INFO:tasks.workunit.client.0.smithi063.stdout:1/979: chown d9/d10 0 1 2024-05-09T06:38:29.716 INFO:tasks.workunit.client.0.smithi063.stdout:1/980: rmdir d9/d2a 39 2024-05-09T06:38:29.718 INFO:tasks.workunit.client.0.smithi063.stdout:1/981: truncate d9/d2a/d3c/d4a/f12e 250595 0 2024-05-09T06:38:29.719 INFO:tasks.workunit.client.0.smithi063.stdout:1/982: readlink d9/d2a/d39/d5d/d75/d139/d59/d62/l118 0 2024-05-09T06:38:29.722 INFO:tasks.workunit.client.0.smithi063.stdout:1/983: creat d9/d2a/d3c/d4c/d83/d88/f14c x:0 0 0 2024-05-09T06:38:29.725 INFO:tasks.workunit.client.0.smithi063.stdout:1/984: getdents d9/d2a/d3c 0 2024-05-09T06:38:29.727 INFO:tasks.workunit.client.0.smithi063.stdout:1/985: fdatasync d9/d2a/d3c/d4c/d83/d88/fcd 0 2024-05-09T06:38:29.737 INFO:tasks.workunit.client.0.smithi063.stdout:1/986: dread d9/d2a/d35/d1e/d9d/f33 [0,4194304] 0 2024-05-09T06:38:29.752 INFO:tasks.workunit.client.0.smithi063.stdout:1/987: dread d9/d2a/d3c/d4c/d83/d88/fcd [0,4194304] 0 2024-05-09T06:38:29.755 INFO:tasks.workunit.client.0.smithi063.stdout:1/988: dread d9/d2a/d3c/d4c/d83/d88/dc2/fe0 [0,4194304] 0 2024-05-09T06:38:29.756 INFO:tasks.workunit.client.0.smithi063.stdout:1/989: truncate d9/d2a/d3c/d4a/fe5 1027677 0 2024-05-09T06:38:29.761 INFO:tasks.workunit.client.0.smithi063.stdout:1/990: dwrite d9/d46/f138 [4194304,4194304] 0 2024-05-09T06:38:29.766 INFO:tasks.workunit.client.0.smithi063.stdout:1/991: rename d9/d2a/d3c/d4c/d83/d88/dc2/dd0/l13a to d9/d2a/d39/d5d/d75/d139/d59/d62/l14d 0 2024-05-09T06:38:29.768 INFO:tasks.workunit.client.0.smithi063.stdout:1/992: truncate d9/d2a/d39/d5d/d75/d139/d59/f13d 596264 0 2024-05-09T06:38:29.786 INFO:tasks.workunit.client.1.smithi193.stdout:8/596: sync 2024-05-09T06:38:29.787 INFO:tasks.workunit.client.1.smithi193.stdout:8/597: truncate d6/d1c/d53/f60 4492099 0 2024-05-09T06:38:29.789 INFO:tasks.workunit.client.1.smithi193.stdout:8/598: mkdir d6/d1c/d18/d66/db3 0 2024-05-09T06:38:29.802 INFO:tasks.workunit.client.1.smithi193.stdout:8/599: dwrite d6/d1c/d53/f81 [0,4194304] 0 2024-05-09T06:38:29.807 INFO:tasks.workunit.client.1.smithi193.stdout:8/600: rename d6/d1c/d18/d66/f47 to d6/d1c/d18/fb4 0 2024-05-09T06:38:29.810 INFO:tasks.workunit.client.1.smithi193.stdout:8/601: mknod d6/d1c/d1b/d5d/d84/d87/da5/daa/cb5 0 2024-05-09T06:38:29.813 INFO:tasks.workunit.client.1.smithi193.stdout:8/602: getdents d6/d1c/d1b/d5d/d84/d87/da5 0 2024-05-09T06:38:29.816 INFO:tasks.workunit.client.1.smithi193.stdout:8/603: rmdir d6/d1c/d1b/d29/d89 0 2024-05-09T06:38:29.820 INFO:tasks.workunit.client.1.smithi193.stdout:8/604: dwrite d6/d1c/d1b/d29/f91 [0,4194304] 0 2024-05-09T06:38:29.824 INFO:tasks.workunit.client.1.smithi193.stdout:8/605: creat d6/d1c/d1b/d29/fb6 x:0 0 0 2024-05-09T06:38:29.824 INFO:tasks.workunit.client.1.smithi193.stdout:8/606: readlink d6/d1c/d27/l48 0 2024-05-09T06:38:29.830 INFO:tasks.workunit.client.1.smithi193.stdout:8/607: dwrite d6/d1c/d18/d66/d93/fa6 [0,4194304] 0 2024-05-09T06:38:29.832 INFO:tasks.workunit.client.1.smithi193.stdout:8/608: chown d6/d1c/d18/d2f/d40/lb2 8526 1 2024-05-09T06:38:29.837 INFO:tasks.workunit.client.1.smithi193.stdout:8/609: dwrite d6/d1c/f14 [0,4194304] 0 2024-05-09T06:38:29.846 INFO:tasks.workunit.client.1.smithi193.stdout:8/610: dwrite d6/d1c/d27/f43 [0,4194304] 0 2024-05-09T06:38:29.850 INFO:tasks.workunit.client.1.smithi193.stdout:8/611: creat d6/d1c/d18/d2f/d40/d55/d7e/fb7 x:0 0 0 2024-05-09T06:38:29.859 INFO:tasks.workunit.client.1.smithi193.stdout:8/612: dwrite d6/d1c/d53/f63 [4194304,4194304] 0 2024-05-09T06:38:29.861 INFO:tasks.workunit.client.1.smithi193.stdout:8/613: chown d6/d1c/d18/d66/d93/l59 5943173 1 2024-05-09T06:38:29.863 INFO:tasks.workunit.client.1.smithi193.stdout:8/614: truncate d6/d1c/d27/f65 140336 0 2024-05-09T06:38:29.864 INFO:tasks.workunit.client.1.smithi193.stdout:8/615: readlink d6/d1c/d27/l2d 0 2024-05-09T06:38:29.865 INFO:tasks.workunit.client.1.smithi193.stdout:8/616: mknod d6/d1c/cb8 0 2024-05-09T06:38:29.872 INFO:tasks.workunit.client.1.smithi193.stdout:8/617: creat d6/d1c/d1b/d29/d57/fb9 x:0 0 0 2024-05-09T06:38:29.876 INFO:tasks.workunit.client.1.smithi193.stdout:8/618: mkdir d6/d1c/d18/dba 0 2024-05-09T06:38:29.876 INFO:tasks.workunit.client.1.smithi193.stdout:8/619: chown d6/c88 976435 1 2024-05-09T06:38:29.877 INFO:tasks.workunit.client.1.smithi193.stdout:8/620: write d6/d1c/d18/d66/fb0 [399903,35672] 0 2024-05-09T06:38:29.911 INFO:tasks.workunit.client.1.smithi193.stdout:9/725: sync 2024-05-09T06:38:29.913 INFO:tasks.workunit.client.0.smithi063.stdout:1/993: sync 2024-05-09T06:38:29.914 INFO:tasks.workunit.client.0.smithi063.stdout:1/994: read - d9/d46/f63 zero size 2024-05-09T06:38:29.914 INFO:tasks.workunit.client.1.smithi193.stdout:8/621: dwrite d6/d1c/f3d [0,4194304] 0 2024-05-09T06:38:29.916 INFO:tasks.workunit.client.1.smithi193.stdout:9/726: mknod dd/d1e/dc2/d49/dbd/d5b/d8a/dad/cf2 0 2024-05-09T06:38:29.917 INFO:tasks.workunit.client.1.smithi193.stdout:9/727: truncate dd/df/d31/d7e/d71/fe3 538202 0 2024-05-09T06:38:29.920 INFO:tasks.workunit.client.1.smithi193.stdout:9/728: write dd/d1e/d1f/f6f [1923331,99868] 0 2024-05-09T06:38:29.921 INFO:tasks.workunit.client.1.smithi193.stdout:9/729: write dd/d1e/f88 [564026,61389] 0 2024-05-09T06:38:29.925 INFO:tasks.workunit.client.0.smithi063.stdout:1/995: write d9/d2a/d3c/d4c/d83/d88/dc2/f123 [766750,78716] 0 2024-05-09T06:38:29.925 INFO:tasks.workunit.client.1.smithi193.stdout:9/730: creat dd/d1e/dc2/db9/ff3 x:0 0 0 2024-05-09T06:38:29.927 INFO:tasks.workunit.client.0.smithi063.stdout:1/996: truncate d9/d2a/d3c/d4c/dc3/f111 187925 0 2024-05-09T06:38:29.927 INFO:tasks.workunit.client.1.smithi193.stdout:9/731: dread - dd/d1e/dc2/d49/f65 zero size 2024-05-09T06:38:29.928 INFO:tasks.workunit.client.1.smithi193.stdout:9/732: write dd/d1e/d93/fa3 [1257382,84567] 0 2024-05-09T06:38:29.930 INFO:tasks.workunit.client.0.smithi063.stdout:1/997: read d9/d2a/d39/d5d/d75/d139/d59/d62/d7e/f133 [274675,37986] 0 2024-05-09T06:38:29.934 INFO:tasks.workunit.client.0.smithi063.stdout:1/998: creat d9/d46/dfe/f14e x:0 0 0 2024-05-09T06:38:29.936 INFO:tasks.workunit.client.1.smithi193.stdout:9/733: truncate dd/df/d15/d98/fb3 1623561 0 2024-05-09T06:38:29.936 INFO:tasks.workunit.client.1.smithi193.stdout:8/622: dread d6/d1c/d18/f1d [0,4194304] 0 2024-05-09T06:38:29.937 INFO:tasks.workunit.client.1.smithi193.stdout:8/623: fdatasync d6/d1c/d18/d2f/d40/d55/d7e/fb7 0 2024-05-09T06:38:29.938 INFO:tasks.workunit.client.1.smithi193.stdout:8/624: write d6/d5b/f8d [1493980,62649] 0 2024-05-09T06:38:29.939 INFO:tasks.workunit.client.1.smithi193.stdout:9/734: mkdir dd/df/d15/db7/df4 0 2024-05-09T06:38:29.939 INFO:tasks.workunit.client.0.smithi063.stdout:1/999: dwrite d9/d2a/d35/d1e/f134 [0,4194304] 0 2024-05-09T06:38:29.944 INFO:tasks.workunit.client.0.smithi063.stderr:+ rm -rf -- ./tmp.iIBGAiE8sD 2024-05-09T06:38:29.945 INFO:tasks.workunit.client.1.smithi193.stdout:9/735: rename dd/d1e/d1f/d48/d55/fb1 to dd/d1e/dc2/db9/ff5 0 2024-05-09T06:38:29.947 INFO:tasks.workunit.client.1.smithi193.stdout:8/625: dwrite d6/d1c/d1b/d5d/f6d [0,4194304] 0 2024-05-09T06:38:29.955 INFO:tasks.workunit.client.1.smithi193.stdout:8/626: dread d6/d1c/f14 [0,4194304] 0 2024-05-09T06:38:29.956 INFO:tasks.workunit.client.1.smithi193.stdout:9/736: rmdir dd/d1e/d1f/d48/d55 39 2024-05-09T06:38:29.961 INFO:tasks.workunit.client.1.smithi193.stdout:9/737: dwrite dd/df/d15/f9e [0,4194304] 0 2024-05-09T06:38:29.962 INFO:tasks.workunit.client.1.smithi193.stdout:9/738: chown dd/d1e/dc2/db9/fe1 574 1 2024-05-09T06:38:29.963 INFO:tasks.workunit.client.1.smithi193.stdout:8/627: creat d6/d1c/d18/d2f/d40/fbb x:0 0 0 2024-05-09T06:38:29.965 INFO:tasks.workunit.client.1.smithi193.stdout:9/739: creat dd/df/d15/d98/ff6 x:0 0 0 2024-05-09T06:38:29.969 INFO:tasks.workunit.client.1.smithi193.stdout:9/740: creat dd/d1e/d93/dbb/ff7 x:0 0 0 2024-05-09T06:38:29.972 INFO:tasks.workunit.client.1.smithi193.stdout:9/741: mkdir dd/d1e/dc2/d49/dbd/d5b/d8a/dad/df8 0 2024-05-09T06:38:29.973 INFO:tasks.workunit.client.1.smithi193.stdout:8/628: link d6/d1c/d1b/c77 d6/d1c/d1b/d5d/d84/d87/cbc 0 2024-05-09T06:38:29.974 INFO:tasks.workunit.client.1.smithi193.stdout:9/742: fdatasync dd/df/d31/d7e/f32 0 2024-05-09T06:38:29.976 INFO:tasks.workunit.client.1.smithi193.stdout:8/629: rmdir d6/d1c/d1b/d29 39 2024-05-09T06:38:29.977 INFO:tasks.workunit.client.1.smithi193.stdout:8/630: readlink d6/d1c/d18/d66/d93/d95/la8 0 2024-05-09T06:38:29.977 INFO:tasks.workunit.client.1.smithi193.stdout:8/631: dread - d6/d1c/d18/d66/d93/fad zero size 2024-05-09T06:38:29.978 INFO:tasks.workunit.client.1.smithi193.stdout:8/632: chown d6/d1c/d1b 4081 1 2024-05-09T06:38:29.978 INFO:tasks.workunit.client.1.smithi193.stdout:9/743: unlink dd/d1e/dc2/db9/ff5 0 2024-05-09T06:38:29.982 INFO:tasks.workunit.client.1.smithi193.stdout:9/744: mkdir dd/d1e/dc2/d49/dbd/d5b/d8a/dad/df9 0 2024-05-09T06:38:29.984 INFO:tasks.workunit.client.1.smithi193.stdout:9/745: unlink dd/df/d15/d8c/d7a/ce9 0 2024-05-09T06:38:29.984 INFO:tasks.workunit.client.1.smithi193.stdout:8/633: rmdir d6/d1c/d18/d2f/d40 39 2024-05-09T06:38:29.986 INFO:tasks.workunit.client.1.smithi193.stdout:9/746: truncate dd/d1e/dc2/db9/ff3 495236 0 2024-05-09T06:38:29.987 INFO:tasks.workunit.client.1.smithi193.stdout:9/747: chown dd/d1e/d93/dbb/lde 26 1 2024-05-09T06:38:29.992 INFO:tasks.workunit.client.1.smithi193.stdout:8/634: dwrite d6/d1c/d18/d2f/d40/f2c [0,4194304] 0 2024-05-09T06:38:29.994 INFO:tasks.workunit.client.1.smithi193.stdout:9/748: dwrite dd/d1e/f28 [0,4194304] 0 2024-05-09T06:38:29.995 INFO:tasks.workunit.client.1.smithi193.stdout:8/635: readlink d6/d1c/d27/l2d 0 2024-05-09T06:38:29.997 INFO:tasks.workunit.client.1.smithi193.stdout:9/749: symlink dd/d1e/d1f/d61/lfa 0 2024-05-09T06:38:29.999 INFO:tasks.workunit.client.1.smithi193.stdout:8/636: mkdir d6/d1c/d27/dbd 0 2024-05-09T06:38:30.001 INFO:tasks.workunit.client.1.smithi193.stdout:8/637: write d6/d1c/f1e [338084,72451] 0 2024-05-09T06:38:30.004 INFO:tasks.workunit.client.1.smithi193.stdout:8/638: mknod d6/d1c/d18/d2f/cbe 0 2024-05-09T06:38:30.008 INFO:tasks.workunit.client.1.smithi193.stdout:8/639: rename d6/d5b/c68 to d6/d1c/d1b/d5d/d84/d87/cbf 0 2024-05-09T06:38:30.031 INFO:tasks.workunit.client.1.smithi193.stdout:8/640: dwrite d6/d1c/d18/d2f/d40/d55/d7e/f8a [0,4194304] 0 2024-05-09T06:38:30.037 INFO:tasks.workunit.client.1.smithi193.stdout:8/641: creat d6/d1c/d18/dba/fc0 x:0 0 0 2024-05-09T06:38:30.040 INFO:tasks.workunit.client.1.smithi193.stdout:8/642: read - d6/d1c/d1b/d29/fb6 zero size 2024-05-09T06:38:30.040 INFO:tasks.workunit.client.1.smithi193.stdout:8/643: chown d6/d1c/d1b/fa7 1021 1 2024-05-09T06:38:30.041 INFO:tasks.workunit.client.1.smithi193.stdout:8/644: readlink d6/d1c/d18/d2f/d40/d5e/l6b 0 2024-05-09T06:38:30.048 INFO:tasks.workunit.client.1.smithi193.stdout:8/645: dwrite d6/d1c/d1b/d29/fb6 [0,4194304] 0 2024-05-09T06:38:30.053 INFO:tasks.workunit.client.1.smithi193.stdout:8/646: chown d6/d1c/d18/d66/d93/d95/l97 74897 1 2024-05-09T06:38:30.054 INFO:tasks.workunit.client.1.smithi193.stdout:0/610: sync 2024-05-09T06:38:30.054 INFO:tasks.workunit.client.1.smithi193.stdout:5/563: sync 2024-05-09T06:38:30.057 INFO:tasks.workunit.client.1.smithi193.stdout:5/564: dread d0/d13/d2c/d37/d5b/f7b [0,4194304] 0 2024-05-09T06:38:30.057 INFO:tasks.workunit.client.1.smithi193.stdout:8/647: write d6/d1c/d53/f60 [3168068,55115] 0 2024-05-09T06:38:30.058 INFO:tasks.workunit.client.1.smithi193.stdout:5/565: chown d0/d13/d2c/d37/d2b/d47/d66 15 1 2024-05-09T06:38:30.061 INFO:tasks.workunit.client.1.smithi193.stdout:0/611: creat d18/d4c/d47/d59/d76/fbc x:0 0 0 2024-05-09T06:38:30.066 INFO:tasks.workunit.client.1.smithi193.stdout:0/612: dwrite d18/f4e [0,4194304] 0 2024-05-09T06:38:30.074 INFO:tasks.workunit.client.1.smithi193.stdout:8/648: write d6/d1c/d1b/f75 [1578097,35185] 0 2024-05-09T06:38:30.076 INFO:tasks.workunit.client.1.smithi193.stdout:0/613: creat d18/d4c/d64/d66/d9d/da5/fbd x:0 0 0 2024-05-09T06:38:30.076 INFO:tasks.workunit.client.1.smithi193.stdout:5/566: getdents d0/d13/d2c/d37/d2b/d47/d60/d75/da9 0 2024-05-09T06:38:30.077 INFO:tasks.workunit.client.1.smithi193.stdout:0/614: read d18/d27/fa9 [505141,29547] 0 2024-05-09T06:38:30.079 INFO:tasks.workunit.client.1.smithi193.stdout:5/567: readlink d0/d13/l62 0 2024-05-09T06:38:30.081 INFO:tasks.workunit.client.1.smithi193.stdout:8/649: link d6/d1c/d18/fb4 d6/d1c/d27/dbd/fc1 0 2024-05-09T06:38:30.084 INFO:tasks.workunit.client.1.smithi193.stdout:0/615: dwrite d18/d27/f41 [0,4194304] 0 2024-05-09T06:38:30.095 INFO:tasks.workunit.client.1.smithi193.stdout:5/568: rename d0/d13/d2c/d37/d2b/d47/d6f/f7d to d0/d13/d2c/d37/d5b/fbc 0 2024-05-09T06:38:30.096 INFO:tasks.workunit.client.1.smithi193.stdout:5/569: write d0/d13/d2c/d37/d2b/d47/d60/d75/f8b [643786,87039] 0 2024-05-09T06:38:30.098 INFO:tasks.workunit.client.1.smithi193.stdout:5/570: write d0/d13/d2c/d37/d5b/d89/f8d [230981,78524] 0 2024-05-09T06:38:30.103 INFO:tasks.workunit.client.1.smithi193.stdout:5/571: dread d0/d13/d2c/d37/d5b/d89/f8d [0,4194304] 0 2024-05-09T06:38:30.106 INFO:tasks.workunit.client.1.smithi193.stdout:5/572: mkdir d0/d6d/dbd 0 2024-05-09T06:38:30.107 INFO:tasks.workunit.client.1.smithi193.stdout:0/616: dwrite d18/d1d/f33 [0,4194304] 0 2024-05-09T06:38:30.107 INFO:tasks.workunit.client.1.smithi193.stdout:5/573: chown d0/d13/d2c/d37/d5b/d89/lb9 5314601 1 2024-05-09T06:38:30.118 INFO:tasks.workunit.client.1.smithi193.stdout:5/574: dwrite d0/d13/d2c/f4a [0,4194304] 0 2024-05-09T06:38:30.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:29 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:38:30.122 INFO:tasks.workunit.client.1.smithi193.stdout:5/575: symlink d0/d13/d2c/d37/d2b/d47/d60/d87/lbe 0 2024-05-09T06:38:30.126 INFO:tasks.workunit.client.1.smithi193.stdout:5/576: symlink d0/d13/d2c/d37/d2b/d47/d66/d3a/lbf 0 2024-05-09T06:38:30.131 INFO:tasks.workunit.client.1.smithi193.stdout:5/577: read d0/d13/d2c/d37/d2b/d47/d66/d3a/f52 [2275812,54952] 0 2024-05-09T06:38:30.135 INFO:tasks.workunit.client.1.smithi193.stdout:5/578: dread d0/d13/d2c/f4a [0,4194304] 0 2024-05-09T06:38:30.142 INFO:tasks.workunit.client.1.smithi193.stdout:5/579: link d0/d13/d2c/da0/ca2 d0/d13/d2c/d37/d5b/d4e/d9d/db4/cc0 0 2024-05-09T06:38:30.151 INFO:tasks.workunit.client.1.smithi193.stdout:5/580: write d0/d13/d2c/d37/d5b/d4e/d5d/f69 [1145702,103820] 0 2024-05-09T06:38:30.154 INFO:tasks.workunit.client.1.smithi193.stdout:5/581: mkdir d0/d13/d2c/d37/d2b/d47/d60/d75/da9/dc1 0 2024-05-09T06:38:30.158 INFO:tasks.workunit.client.1.smithi193.stdout:5/582: unlink d0/d13/d2c/d37/d2b/d47/d6f/f3e 0 2024-05-09T06:38:30.159 INFO:tasks.workunit.client.1.smithi193.stdout:5/583: mknod d0/d7c/cc2 0 2024-05-09T06:38:30.160 INFO:tasks.workunit.client.1.smithi193.stdout:5/584: read - d0/d13/d2c/d37/d5b/d4e/d9d/fba zero size 2024-05-09T06:38:30.173 INFO:tasks.workunit.client.1.smithi193.stdout:5/585: truncate d0/d13/d2c/d37/d5b/f6 675161 0 2024-05-09T06:38:30.176 INFO:tasks.workunit.client.1.smithi193.stdout:5/586: creat d0/d13/d2c/d37/d2b/d6a/d90/fc3 x:0 0 0 2024-05-09T06:38:30.177 INFO:tasks.workunit.client.1.smithi193.stdout:5/587: dread - d0/d13/d2c/d37/d2b/d47/d60/d87/fa5 zero size 2024-05-09T06:38:30.178 INFO:tasks.workunit.client.1.smithi193.stdout:5/588: fdatasync d0/d13/d2c/d37/d2b/d47/f79 0 2024-05-09T06:38:30.179 INFO:tasks.workunit.client.1.smithi193.stdout:5/589: chown d0/d13/d2c/d37/d5b/c84 486 1 2024-05-09T06:38:30.180 INFO:tasks.workunit.client.1.smithi193.stdout:5/590: fdatasync d0/f8 0 2024-05-09T06:38:30.183 INFO:tasks.workunit.client.1.smithi193.stdout:5/591: symlink d0/d13/d2c/d4d/d67/lc4 0 2024-05-09T06:38:30.189 INFO:tasks.workunit.client.1.smithi193.stdout:5/592: link d0/d13/d2c/d37/c82 d0/d13/d2c/d37/d5b/d4e/cc5 0 2024-05-09T06:38:30.190 INFO:tasks.workunit.client.1.smithi193.stdout:5/593: chown d0/d13/d2c/d37/d2b/d47/f63 0 1 2024-05-09T06:38:30.192 INFO:tasks.workunit.client.1.smithi193.stdout:5/594: rename d0/d7c to d0/d13/d2c/db6/dc6 0 2024-05-09T06:38:30.199 INFO:tasks.workunit.client.1.smithi193.stdout:5/595: dwrite d0/d13/d2c/d37/d5b/d89/f8d [0,4194304] 0 2024-05-09T06:38:30.217 INFO:tasks.workunit.client.1.smithi193.stdout:5/596: dread d0/d13/d2c/d37/d2b/d47/d66/f23 [0,4194304] 0 2024-05-09T06:38:30.219 INFO:tasks.workunit.client.1.smithi193.stdout:5/597: creat d0/d13/d2c/d4d/d67/fc7 x:0 0 0 2024-05-09T06:38:30.220 INFO:tasks.workunit.client.1.smithi193.stdout:5/598: write d0/d13/d2c/d37/d2b/d47/f63 [2592400,89452] 0 2024-05-09T06:38:30.223 INFO:tasks.workunit.client.1.smithi193.stdout:5/599: rmdir d0/d13/d2c/d37/d2b/d47/d66/d3a 39 2024-05-09T06:38:30.227 INFO:tasks.workunit.client.1.smithi193.stdout:5/600: mkdir d0/d13/d2c/d37/d2b/d47/d60/dc8 0 2024-05-09T06:38:30.229 INFO:tasks.workunit.client.1.smithi193.stdout:5/601: truncate d0/d13/d2c/d37/d2b/d47/d60/d75/f8b 911516 0 2024-05-09T06:38:30.234 INFO:tasks.workunit.client.1.smithi193.stdout:5/602: mknod d0/d13/d2c/d37/d2b/d47/d66/d3a/cc9 0 2024-05-09T06:38:30.237 INFO:tasks.workunit.client.1.smithi193.stdout:5/603: creat d0/d13/d2c/d4d/fca x:0 0 0 2024-05-09T06:38:30.239 INFO:tasks.workunit.client.1.smithi193.stdout:5/604: mknod d0/d13/d2c/db6/ccb 0 2024-05-09T06:38:30.240 INFO:tasks.workunit.client.1.smithi193.stdout:2/633: sync 2024-05-09T06:38:30.240 INFO:tasks.workunit.client.1.smithi193.stdout:7/601: sync 2024-05-09T06:38:30.245 INFO:tasks.workunit.client.1.smithi193.stdout:5/605: dread d0/d13/d2c/d37/d5b/d4e/d5d/f69 [0,4194304] 0 2024-05-09T06:38:30.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/634: dwrite db/df/d54/fb8 [0,4194304] 0 2024-05-09T06:38:30.253 INFO:tasks.workunit.client.1.smithi193.stdout:7/602: truncate d6/d8/fa 2798120 0 2024-05-09T06:38:30.256 INFO:tasks.workunit.client.1.smithi193.stdout:5/606: creat d0/d13/d2c/d37/d2b/d47/d66/d3a/fcc x:0 0 0 2024-05-09T06:38:30.259 INFO:tasks.workunit.client.1.smithi193.stdout:7/603: mknod d6/dd/dbc/cca 0 2024-05-09T06:38:30.260 INFO:tasks.workunit.client.1.smithi193.stdout:2/635: mknod db/d25/cd4 0 2024-05-09T06:38:30.263 INFO:tasks.workunit.client.1.smithi193.stdout:2/636: symlink db/df/d54/ld5 0 2024-05-09T06:38:30.263 INFO:tasks.workunit.client.1.smithi193.stdout:2/637: readlink db/lbc 0 2024-05-09T06:38:30.264 INFO:tasks.workunit.client.1.smithi193.stdout:7/604: symlink d6/d17/d99/lcb 0 2024-05-09T06:38:30.266 INFO:tasks.workunit.client.1.smithi193.stdout:2/638: fsync db/f28 0 2024-05-09T06:38:30.267 INFO:tasks.workunit.client.1.smithi193.stdout:2/639: chown db/d25/d38/dcd/fd2 93115 1 2024-05-09T06:38:30.268 INFO:tasks.workunit.client.1.smithi193.stdout:7/605: mkdir d6/dd/d70/dcc 0 2024-05-09T06:38:30.270 INFO:tasks.workunit.client.1.smithi193.stdout:7/606: dread - d6/d8/d1a/d36/d4d/d60/d9c/fb1 zero size 2024-05-09T06:38:30.273 INFO:tasks.workunit.client.1.smithi193.stdout:7/607: creat d6/dd/d7e/fcd x:0 0 0 2024-05-09T06:38:30.276 INFO:tasks.workunit.client.1.smithi193.stdout:5/607: dwrite d0/d13/d2c/d37/d2b/d47/d6f/f54 [0,4194304] 0 2024-05-09T06:38:30.279 INFO:tasks.workunit.client.1.smithi193.stdout:2/640: write db/d25/f48 [518489,49640] 0 2024-05-09T06:38:30.290 INFO:tasks.workunit.client.1.smithi193.stdout:2/641: truncate db/d37/d6d/d80/f14 2296449 0 2024-05-09T06:38:30.293 INFO:tasks.workunit.client.1.smithi193.stdout:2/642: truncate db/f1e 1028251 0 2024-05-09T06:38:30.296 INFO:tasks.workunit.client.1.smithi193.stdout:2/643: mknod db/d37/d6d/d80/d60/d7d/cd6 0 2024-05-09T06:38:30.297 INFO:tasks.workunit.client.1.smithi193.stdout:2/644: creat db/dbb/fd7 x:0 0 0 2024-05-09T06:38:30.302 INFO:tasks.workunit.client.1.smithi193.stdout:2/645: creat db/fd8 x:0 0 0 2024-05-09T06:38:30.304 INFO:tasks.workunit.client.1.smithi193.stdout:2/646: creat db/d37/d6d/d80/d60/d7d/db3/fd9 x:0 0 0 2024-05-09T06:38:30.306 INFO:tasks.workunit.client.1.smithi193.stdout:2/647: mknod db/dbb/cda 0 2024-05-09T06:38:30.310 INFO:tasks.workunit.client.1.smithi193.stdout:2/648: dread db/d25/f49 [0,4194304] 0 2024-05-09T06:38:30.311 INFO:tasks.workunit.client.1.smithi193.stdout:2/649: truncate db/d4b/f86 759365 0 2024-05-09T06:38:30.315 INFO:tasks.workunit.client.1.smithi193.stdout:2/650: dread db/d25/d38/d40/d8b/d77/fa7 [0,4194304] 0 2024-05-09T06:38:30.327 INFO:tasks.workunit.client.1.smithi193.stdout:2/651: dread db/d25/d38/d61/f70 [0,4194304] 0 2024-05-09T06:38:30.329 INFO:tasks.workunit.client.1.smithi193.stdout:2/652: chown db/c18 127 1 2024-05-09T06:38:30.330 INFO:tasks.workunit.client.1.smithi193.stdout:2/653: fsync db/f28 0 2024-05-09T06:38:30.347 INFO:tasks.workunit.client.1.smithi193.stdout:2/654: truncate db/d25/d38/d5b/fa3 233642 0 2024-05-09T06:38:30.353 INFO:tasks.workunit.client.1.smithi193.stdout:2/655: truncate db/df/f50 967349 0 2024-05-09T06:38:30.354 INFO:tasks.workunit.client.1.smithi193.stdout:2/656: truncate db/d4b/f86 1545323 0 2024-05-09T06:38:30.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:29 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:38:30.357 INFO:tasks.workunit.client.1.smithi193.stdout:2/657: rmdir db/d25/d38/d40/d8b/d77 39 2024-05-09T06:38:30.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/658: creat db/d37/fdb x:0 0 0 2024-05-09T06:38:30.363 INFO:tasks.workunit.client.1.smithi193.stdout:2/659: creat db/d25/d38/d40/db2/dcb/fdc x:0 0 0 2024-05-09T06:38:30.368 INFO:tasks.workunit.client.1.smithi193.stdout:2/660: link db/d25/d38/d40/f9c db/d25/d38/fdd 0 2024-05-09T06:38:30.373 INFO:tasks.workunit.client.1.smithi193.stdout:2/661: dwrite db/d37/fdb [0,4194304] 0 2024-05-09T06:38:30.378 INFO:tasks.workunit.client.1.smithi193.stdout:2/662: rename db/d37/f42 to db/d4b/fde 0 2024-05-09T06:38:30.393 INFO:tasks.workunit.client.1.smithi193.stdout:2/663: truncate db/d37/d6d/d80/d60/d7d/f7e 4071371 0 2024-05-09T06:38:30.403 INFO:tasks.workunit.client.1.smithi193.stdout:2/664: symlink db/dbb/ldf 0 2024-05-09T06:38:30.409 INFO:tasks.workunit.client.1.smithi193.stdout:2/665: dwrite db/df/f69 [4194304,4194304] 0 2024-05-09T06:38:30.416 INFO:tasks.workunit.client.1.smithi193.stdout:2/666: truncate db/d25/d38/d40/f9c 771383 0 2024-05-09T06:38:30.421 INFO:tasks.workunit.client.1.smithi193.stdout:2/667: link db/d4b/fd3 db/d25/d38/fe0 0 2024-05-09T06:38:30.425 INFO:tasks.workunit.client.1.smithi193.stdout:2/668: rename db/d4b/c8a to db/d37/d6d/d80/daa/ce1 0 2024-05-09T06:38:30.428 INFO:tasks.workunit.client.1.smithi193.stdout:2/669: creat db/d25/d38/d40/db2/fe2 x:0 0 0 2024-05-09T06:38:30.433 INFO:tasks.workunit.client.1.smithi193.stdout:2/670: rename db/d25/d38/d40/d8b/d77 to db/d25/d38/dcd/de3 0 2024-05-09T06:38:30.437 INFO:tasks.workunit.client.1.smithi193.stdout:2/671: rename db/d37/d6d/d80/fbf to db/d25/d38/d40/db2/dcb/fe4 0 2024-05-09T06:38:30.442 INFO:tasks.workunit.client.1.smithi193.stdout:2/672: rename db/d37/d6d/d80/c31 to db/df/d54/dba/ce5 0 2024-05-09T06:38:30.443 INFO:tasks.workunit.client.1.smithi193.stdout:3/592: sync 2024-05-09T06:38:30.443 INFO:tasks.workunit.client.1.smithi193.stdout:4/707: sync 2024-05-09T06:38:30.446 INFO:tasks.workunit.client.1.smithi193.stdout:3/593: symlink d1/d2d/d51/ld9 0 2024-05-09T06:38:30.451 INFO:tasks.workunit.client.1.smithi193.stdout:2/673: dwrite db/df/d54/fc8 [0,4194304] 0 2024-05-09T06:38:30.452 INFO:tasks.workunit.client.1.smithi193.stdout:3/594: symlink d1/db/d1b/lda 0 2024-05-09T06:38:30.454 INFO:tasks.workunit.client.1.smithi193.stdout:4/708: dwrite d1a/d24/d73/d94/d7c/d5b/f7d [4194304,4194304] 0 2024-05-09T06:38:30.456 INFO:tasks.workunit.client.1.smithi193.stdout:2/674: mknod db/dbb/ce6 0 2024-05-09T06:38:30.459 INFO:tasks.workunit.client.1.smithi193.stdout:3/595: rmdir d1/db/d2f/d84 39 2024-05-09T06:38:30.459 INFO:tasks.workunit.client.1.smithi193.stdout:3/596: read - d1/d44/d7d/fbb zero size 2024-05-09T06:38:30.461 INFO:tasks.workunit.client.1.smithi193.stdout:2/675: mkdir db/d25/d38/d61/de7 0 2024-05-09T06:38:30.462 INFO:tasks.workunit.client.1.smithi193.stdout:2/676: write db/df/f69 [8357777,70199] 0 2024-05-09T06:38:30.463 INFO:tasks.workunit.client.1.smithi193.stdout:2/677: chown db/d37/d6d/d80/d60 224148351 1 2024-05-09T06:38:30.464 INFO:tasks.workunit.client.1.smithi193.stdout:2/678: chown db/fd8 103498 1 2024-05-09T06:38:30.469 INFO:tasks.workunit.client.1.smithi193.stdout:3/597: dwrite d1/db/da1/fd1 [0,4194304] 0 2024-05-09T06:38:30.470 INFO:tasks.workunit.client.1.smithi193.stdout:2/679: mknod db/d37/d6d/d80/d60/ce8 0 2024-05-09T06:38:30.478 INFO:tasks.workunit.client.1.smithi193.stdout:4/709: dread d1a/d24/f41 [0,4194304] 0 2024-05-09T06:38:30.479 INFO:tasks.workunit.client.1.smithi193.stdout:4/710: stat d1a/d22/d5d/fd1 0 2024-05-09T06:38:30.479 INFO:tasks.workunit.client.1.smithi193.stdout:2/680: unlink db/df/d54/dba/ce5 0 2024-05-09T06:38:30.480 INFO:tasks.workunit.client.1.smithi193.stdout:3/598: creat d1/db/d59/db8/fdb x:0 0 0 2024-05-09T06:38:30.486 INFO:tasks.workunit.client.1.smithi193.stdout:2/681: dwrite db/dbb/fd7 [0,4194304] 0 2024-05-09T06:38:30.486 INFO:tasks.workunit.client.1.smithi193.stdout:3/599: rename d1/d2d to d1/db/d2f/d78/d86/ddc 0 2024-05-09T06:38:30.488 INFO:tasks.workunit.client.1.smithi193.stdout:3/600: stat d1/db/d59/db8/fdb 0 2024-05-09T06:38:30.488 INFO:tasks.workunit.client.1.smithi193.stdout:4/711: getdents d1a/d24/d73/d94 0 2024-05-09T06:38:30.489 INFO:tasks.workunit.client.1.smithi193.stdout:4/712: readlink d1a/d24/d73/d94/d7c/lc2 0 2024-05-09T06:38:30.490 INFO:tasks.workunit.client.1.smithi193.stdout:2/682: readlink db/d25/d38/d61/laf 0 2024-05-09T06:38:30.493 INFO:tasks.workunit.client.1.smithi193.stdout:2/683: mknod db/d25/d38/d40/d8f/ce9 0 2024-05-09T06:38:30.496 INFO:tasks.workunit.client.1.smithi193.stdout:4/713: symlink d1a/d24/d73/dcc/lda 0 2024-05-09T06:38:30.501 INFO:tasks.workunit.client.1.smithi193.stdout:2/684: dwrite db/d25/f95 [0,4194304] 0 2024-05-09T06:38:30.502 INFO:tasks.workunit.client.1.smithi193.stdout:4/714: fsync d1a/d24/d73/d94/f52 0 2024-05-09T06:38:30.502 INFO:tasks.workunit.client.1.smithi193.stdout:3/601: dwrite d1/db/d2f/d84/fb7 [0,4194304] 0 2024-05-09T06:38:30.508 INFO:tasks.workunit.client.1.smithi193.stdout:2/685: creat db/df/d87/fea x:0 0 0 2024-05-09T06:38:30.524 INFO:tasks.workunit.client.1.smithi193.stdout:3/602: write d1/db/d59/f37 [3099276,28084] 0 2024-05-09T06:38:30.524 INFO:tasks.workunit.client.1.smithi193.stdout:2/686: write db/d4b/fd3 [649476,35545] 0 2024-05-09T06:38:30.528 INFO:tasks.workunit.client.1.smithi193.stdout:2/687: mknod db/d37/ceb 0 2024-05-09T06:38:30.528 INFO:tasks.workunit.client.1.smithi193.stdout:3/603: symlink d1/db/ldd 0 2024-05-09T06:38:30.529 INFO:tasks.workunit.client.1.smithi193.stdout:4/715: dread d1a/d98/fb8 [0,4194304] 0 2024-05-09T06:38:30.533 INFO:tasks.workunit.client.1.smithi193.stdout:2/688: rename db/d4b/d56/lc5 to db/dbb/lec 0 2024-05-09T06:38:30.534 INFO:tasks.workunit.client.1.smithi193.stdout:2/689: chown db/d37/d6d 963725047 1 2024-05-09T06:38:30.534 INFO:tasks.workunit.client.1.smithi193.stdout:3/604: dwrite f0 [4194304,4194304] 0 2024-05-09T06:38:30.534 INFO:tasks.workunit.client.1.smithi193.stdout:2/690: stat db/d25/f27 0 2024-05-09T06:38:30.535 INFO:tasks.workunit.client.1.smithi193.stdout:3/605: dread - d1/d44/d7d/fc3 zero size 2024-05-09T06:38:30.535 INFO:tasks.workunit.client.1.smithi193.stdout:3/606: chown d1/db/d2f/d78 1 1 2024-05-09T06:38:30.536 INFO:tasks.workunit.client.1.smithi193.stdout:2/691: write db/d25/d38/d40/db2/dcb/fdc [576699,4925] 0 2024-05-09T06:38:30.536 INFO:tasks.workunit.client.1.smithi193.stdout:6/620: sync 2024-05-09T06:38:30.536 INFO:tasks.workunit.client.1.smithi193.stdout:1/699: sync 2024-05-09T06:38:30.536 INFO:tasks.workunit.client.1.smithi193.stdout:9/750: sync 2024-05-09T06:38:30.543 INFO:tasks.workunit.client.1.smithi193.stdout:3/607: mknod d1/db/d2f/d78/d86/ddc/d51/cde 0 2024-05-09T06:38:30.543 INFO:tasks.workunit.client.1.smithi193.stdout:3/608: chown d1/d44/d7d/fc3 363 1 2024-05-09T06:38:30.546 INFO:tasks.workunit.client.1.smithi193.stdout:6/621: creat d1/d25/d2c/d3c/db1/d58/dc9/fe1 x:0 0 0 2024-05-09T06:38:30.550 INFO:tasks.workunit.client.1.smithi193.stdout:2/692: write db/d25/d38/d40/d8b/fc3 [1148793,28185] 0 2024-05-09T06:38:30.551 INFO:tasks.workunit.client.1.smithi193.stdout:9/751: mknod dd/df/cfb 0 2024-05-09T06:38:30.551 INFO:tasks.workunit.client.1.smithi193.stdout:2/693: read db/d25/f95 [3968432,15188] 0 2024-05-09T06:38:30.552 INFO:tasks.workunit.client.1.smithi193.stdout:9/752: chown dd/df/d15/d8c/d7a 56 1 2024-05-09T06:38:30.555 INFO:tasks.workunit.client.1.smithi193.stdout:1/700: creat d4/d16/d6b/fe5 x:0 0 0 2024-05-09T06:38:30.557 INFO:tasks.workunit.client.1.smithi193.stdout:2/694: symlink db/d25/d38/d61/led 0 2024-05-09T06:38:30.557 INFO:tasks.workunit.client.1.smithi193.stdout:1/701: write d4/d16/d6b/d6e/f71 [520564,71751] 0 2024-05-09T06:38:30.560 INFO:tasks.workunit.client.1.smithi193.stdout:6/622: unlink d1/d25/d2c/d3c/db1/f43 0 2024-05-09T06:38:30.561 INFO:tasks.workunit.client.1.smithi193.stdout:6/623: chown d1/d25/d2c/d81 817436 1 2024-05-09T06:38:30.570 INFO:tasks.workunit.client.1.smithi193.stdout:9/753: write dd/df/d31/d7e/fe4 [110516,43823] 0 2024-05-09T06:38:30.571 INFO:tasks.workunit.client.1.smithi193.stdout:9/754: readlink dd/d1e/dc2/lb5 0 2024-05-09T06:38:30.572 INFO:tasks.workunit.client.1.smithi193.stdout:1/702: link d4/d16/d6b/fe5 d4/d3c/d3e/fe6 0 2024-05-09T06:38:30.574 INFO:tasks.workunit.client.1.smithi193.stdout:6/624: mknod d1/de/d11/d18/ddc/ce2 0 2024-05-09T06:38:30.576 INFO:tasks.workunit.client.1.smithi193.stdout:9/755: symlink dd/df/d31/d7e/d71/lfc 0 2024-05-09T06:38:30.578 INFO:tasks.workunit.client.1.smithi193.stdout:9/756: write dd/d1e/dc2/db9/ff3 [86038,79699] 0 2024-05-09T06:38:30.579 INFO:tasks.workunit.client.1.smithi193.stdout:2/695: link db/d4b/f5d db/d9b/fee 0 2024-05-09T06:38:30.580 INFO:tasks.workunit.client.1.smithi193.stdout:2/696: write db/d4b/f90 [5991853,59005] 0 2024-05-09T06:38:30.582 INFO:tasks.workunit.client.1.smithi193.stdout:2/697: write db/df/d54/fb8 [4166417,71950] 0 2024-05-09T06:38:30.582 INFO:tasks.workunit.client.1.smithi193.stdout:1/703: unlink d4/d76/lbe 0 2024-05-09T06:38:30.584 INFO:tasks.workunit.client.1.smithi193.stdout:1/704: dread - d4/d16/d60/d4f/d72/d7f/fd9 zero size 2024-05-09T06:38:30.593 INFO:tasks.workunit.client.1.smithi193.stdout:6/625: rmdir d1/d25/d4c/dc7 0 2024-05-09T06:38:30.602 INFO:tasks.workunit.client.1.smithi193.stdout:2/698: dwrite db/df/f94 [0,4194304] 0 2024-05-09T06:38:30.602 INFO:tasks.workunit.client.1.smithi193.stdout:9/757: creat dd/d1e/d1f/d48/d55/ffd x:0 0 0 2024-05-09T06:38:30.603 INFO:tasks.workunit.client.1.smithi193.stdout:9/758: chown l8 458 1 2024-05-09T06:38:30.608 INFO:tasks.workunit.client.1.smithi193.stdout:1/705: getdents d4/d16/d60/d4f/d72/d7f 0 2024-05-09T06:38:30.609 INFO:tasks.workunit.client.1.smithi193.stdout:6/626: truncate d1/f86 516779 0 2024-05-09T06:38:30.611 INFO:tasks.workunit.client.1.smithi193.stdout:2/699: dwrite db/d9b/fc2 [0,4194304] 0 2024-05-09T06:38:30.619 INFO:tasks.workunit.client.1.smithi193.stdout:1/706: dwrite d4/d3c/fc9 [0,4194304] 0 2024-05-09T06:38:30.621 INFO:tasks.workunit.client.1.smithi193.stdout:1/707: chown d4/d16/c84 0 1 2024-05-09T06:38:30.623 INFO:tasks.workunit.client.1.smithi193.stdout:1/708: write d4/d16/f42 [1945030,108008] 0 2024-05-09T06:38:30.630 INFO:tasks.workunit.client.1.smithi193.stdout:1/709: rename d4/d3c/d3e/fe6 to d4/d76/fe7 0 2024-05-09T06:38:30.631 INFO:tasks.workunit.client.1.smithi193.stdout:2/700: truncate db/f45 4717154 0 2024-05-09T06:38:30.632 INFO:tasks.workunit.client.1.smithi193.stdout:1/710: readlink d4/d16/d6b/l82 0 2024-05-09T06:38:30.634 INFO:tasks.workunit.client.1.smithi193.stdout:2/701: fdatasync db/d37/fdb 0 2024-05-09T06:38:30.636 INFO:tasks.workunit.client.1.smithi193.stdout:6/627: dwrite d1/de/d11/f87 [0,4194304] 0 2024-05-09T06:38:30.636 INFO:tasks.workunit.client.1.smithi193.stdout:2/702: chown db/df/f10 3056216 1 2024-05-09T06:38:30.639 INFO:tasks.workunit.client.1.smithi193.stdout:1/711: write d4/d16/d60/d4f/fda [3005474,2405] 0 2024-05-09T06:38:30.643 INFO:tasks.workunit.client.1.smithi193.stdout:6/628: rmdir d1/d25/d2c/d3c 39 2024-05-09T06:38:30.643 INFO:tasks.workunit.client.1.smithi193.stdout:9/759: dread dd/d1e/d1f/f22 [0,4194304] 0 2024-05-09T06:38:30.644 INFO:tasks.workunit.client.1.smithi193.stdout:9/760: stat dd/d1e/dc2/d49/cce 0 2024-05-09T06:38:30.648 INFO:tasks.workunit.client.1.smithi193.stdout:9/761: dread dd/d1e/dc2/db9/ff3 [0,4194304] 0 2024-05-09T06:38:30.653 INFO:tasks.workunit.client.1.smithi193.stdout:6/629: mkdir d1/d25/d4c/d9e/dc8/de3 0 2024-05-09T06:38:30.653 INFO:tasks.workunit.client.1.smithi193.stdout:6/630: dread - d1/fcf zero size 2024-05-09T06:38:30.654 INFO:tasks.workunit.client.1.smithi193.stdout:1/712: dread d4/d3c/f36 [0,4194304] 0 2024-05-09T06:38:30.657 INFO:tasks.workunit.client.1.smithi193.stdout:2/703: mkdir db/def 0 2024-05-09T06:38:30.672 INFO:tasks.workunit.client.1.smithi193.stdout:6/631: creat d1/d25/d2c/fe4 x:0 0 0 2024-05-09T06:38:30.675 INFO:tasks.workunit.client.1.smithi193.stdout:2/704: rmdir db/df/d89 0 2024-05-09T06:38:30.677 INFO:tasks.workunit.client.1.smithi193.stdout:9/762: rename dd/df/fc7 to dd/d1e/d1f/d48/d55/d86/ffe 0 2024-05-09T06:38:30.678 INFO:tasks.workunit.client.1.smithi193.stdout:9/763: dread - dd/d1e/d1f/faa zero size 2024-05-09T06:38:30.680 INFO:tasks.workunit.client.1.smithi193.stdout:9/764: write dd/d1e/d93/fa3 [1362677,74578] 0 2024-05-09T06:38:30.680 INFO:tasks.workunit.client.1.smithi193.stdout:2/705: dread db/d25/d38/dcd/fd2 [0,4194304] 0 2024-05-09T06:38:30.680 INFO:tasks.workunit.client.1.smithi193.stdout:1/713: link d4/d16/d6b/d85/f86 d4/d16/d17/d1a/dc5/fe8 0 2024-05-09T06:38:30.681 INFO:tasks.workunit.client.1.smithi193.stdout:2/706: fsync db/dbb/fd7 0 2024-05-09T06:38:30.682 INFO:tasks.workunit.client.1.smithi193.stdout:6/632: dread d1/de/f10 [0,4194304] 0 2024-05-09T06:38:30.687 INFO:tasks.workunit.client.1.smithi193.stdout:9/765: read dd/df/d15/f96 [659705,98124] 0 2024-05-09T06:38:30.689 INFO:tasks.workunit.client.1.smithi193.stdout:6/633: creat d1/de/d11/d2d/da4/fe5 x:0 0 0 2024-05-09T06:38:30.690 INFO:tasks.workunit.client.1.smithi193.stdout:9/766: mkdir dd/df/d15/d98/dff 0 2024-05-09T06:38:30.691 INFO:tasks.workunit.client.1.smithi193.stdout:9/767: write dd/df/d15/fdc [127519,24398] 0 2024-05-09T06:38:30.693 INFO:tasks.workunit.client.1.smithi193.stdout:6/634: write d1/d25/d2c/d3c/fbb [1007548,2966] 0 2024-05-09T06:38:30.699 INFO:tasks.workunit.client.1.smithi193.stdout:9/768: unlink dd/df/d31/c9a 0 2024-05-09T06:38:30.702 INFO:tasks.workunit.client.1.smithi193.stdout:9/769: unlink dd/d1e/dc2/f45 0 2024-05-09T06:38:30.705 INFO:tasks.workunit.client.1.smithi193.stdout:7/608: sync 2024-05-09T06:38:30.705 INFO:tasks.workunit.client.1.smithi193.stdout:8/650: sync 2024-05-09T06:38:30.705 INFO:tasks.workunit.client.1.smithi193.stdout:0/617: sync 2024-05-09T06:38:30.705 INFO:tasks.workunit.client.1.smithi193.stdout:5/608: sync 2024-05-09T06:38:30.707 INFO:tasks.workunit.client.1.smithi193.stdout:6/635: link d1/d25/d2c/d76/fc0 d1/d25/d4c/d9e/dc5/fe6 0 2024-05-09T06:38:30.712 INFO:tasks.workunit.client.1.smithi193.stdout:7/609: creat d6/dd/dbc/fce x:0 0 0 2024-05-09T06:38:30.713 INFO:tasks.workunit.client.1.smithi193.stdout:8/651: dread - d6/d1c/d27/f7d zero size 2024-05-09T06:38:30.716 INFO:tasks.workunit.client.1.smithi193.stdout:0/618: rename d18/c98 to d18/d4c/d64/d8b/cbe 0 2024-05-09T06:38:30.716 INFO:tasks.workunit.client.1.smithi193.stdout:0/619: dread - d18/d4c/d47/f4a zero size 2024-05-09T06:38:30.718 INFO:tasks.workunit.client.1.smithi193.stdout:6/636: creat d1/d25/d2c/d76/fe7 x:0 0 0 2024-05-09T06:38:30.718 INFO:tasks.workunit.client.1.smithi193.stdout:6/637: readlink d1/d25/d4c/l63 0 2024-05-09T06:38:30.723 INFO:tasks.workunit.client.1.smithi193.stdout:8/652: rmdir d6/d1c/d1b/d5d/d84/d87/da5 39 2024-05-09T06:38:30.727 INFO:tasks.workunit.client.1.smithi193.stdout:6/638: readlink d1/l7e 0 2024-05-09T06:38:30.730 INFO:tasks.workunit.client.1.smithi193.stdout:0/620: dread d18/d4c/f2b [0,4194304] 0 2024-05-09T06:38:30.734 INFO:tasks.workunit.client.1.smithi193.stdout:6/639: dwrite d1/fcf [0,4194304] 0 2024-05-09T06:38:30.735 INFO:tasks.workunit.client.1.smithi193.stdout:6/640: chown d1/de/d11/d18/ddc/ce2 184475261 1 2024-05-09T06:38:30.739 INFO:tasks.workunit.client.1.smithi193.stdout:0/621: chown d18/d4c/d64/d66/d9d/faf 6 1 2024-05-09T06:38:30.739 INFO:tasks.workunit.client.1.smithi193.stdout:0/622: readlink d18/d27/l6c 0 2024-05-09T06:38:30.743 INFO:tasks.workunit.client.1.smithi193.stdout:5/609: write d0/f9 [2033107,91992] 0 2024-05-09T06:38:30.744 INFO:tasks.workunit.client.1.smithi193.stdout:0/623: symlink d18/d4c/d64/lbf 0 2024-05-09T06:38:30.747 INFO:tasks.workunit.client.1.smithi193.stdout:8/653: rename d6/d1c/cb8 to d6/d1c/cc2 0 2024-05-09T06:38:30.751 INFO:tasks.workunit.client.1.smithi193.stdout:5/610: symlink d0/d13/d2c/d37/d74/lcd 0 2024-05-09T06:38:30.752 INFO:tasks.workunit.client.1.smithi193.stdout:0/624: symlink d18/d4c/d47/d55/lc0 0 2024-05-09T06:38:30.759 INFO:tasks.workunit.client.1.smithi193.stdout:5/611: rename d0/d13/d2c/d37/c64 to d0/d13/d2c/d37/d2b/d47/d60/d75/da9/dc1/cce 0 2024-05-09T06:38:30.760 INFO:tasks.workunit.client.1.smithi193.stdout:5/612: write d0/d13/d2c/d4d/d67/fc7 [392636,50415] 0 2024-05-09T06:38:30.760 INFO:tasks.workunit.client.1.smithi193.stdout:6/641: link d1/d25/d2c/d3c/db1/d58/dc9/ccb d1/de/d11/d20/ce8 0 2024-05-09T06:38:30.761 INFO:tasks.workunit.client.1.smithi193.stdout:7/610: write d6/d8/d1a/d36/f50 [1093706,4826] 0 2024-05-09T06:38:30.763 INFO:tasks.workunit.client.1.smithi193.stdout:5/613: read d0/d13/d2c/d4d/d67/f93 [240229,33434] 0 2024-05-09T06:38:30.767 INFO:tasks.workunit.client.1.smithi193.stdout:6/642: truncate d1/de/d11/d18/f29 813164 0 2024-05-09T06:38:30.769 INFO:tasks.workunit.client.1.smithi193.stdout:6/643: fdatasync d1/de/d11/d2d/d3a/d47/fa1 0 2024-05-09T06:38:30.770 INFO:tasks.workunit.client.1.smithi193.stdout:6/644: stat d1/de/d11/d2d/d3a/c8c 0 2024-05-09T06:38:30.772 INFO:tasks.workunit.client.1.smithi193.stdout:8/654: write d6/d1c/d18/d2f/d40/d55/d7e/fa9 [792449,48406] 0 2024-05-09T06:38:30.777 INFO:tasks.workunit.client.1.smithi193.stdout:5/614: dread d0/f8 [0,4194304] 0 2024-05-09T06:38:30.777 INFO:tasks.workunit.client.1.smithi193.stdout:0/625: truncate d18/d4c/d64/f85 358797 0 2024-05-09T06:38:30.779 INFO:tasks.workunit.client.1.smithi193.stdout:7/611: truncate d6/dd/d53/f6a 2167031 0 2024-05-09T06:38:30.783 INFO:tasks.workunit.client.1.smithi193.stdout:8/655: rename d6/f23 to d6/d1c/d18/fc3 0 2024-05-09T06:38:30.787 INFO:tasks.workunit.client.1.smithi193.stdout:5/615: creat d0/d13/d2c/db6/dc6/fcf x:0 0 0 2024-05-09T06:38:30.789 INFO:tasks.workunit.client.1.smithi193.stdout:7/612: fsync d6/d8/d1a/d36/d4d/f64 0 2024-05-09T06:38:30.796 INFO:tasks.workunit.client.1.smithi193.stdout:0/626: symlink d18/d4c/d47/d59/lc1 0 2024-05-09T06:38:30.799 INFO:tasks.workunit.client.1.smithi193.stdout:7/613: rmdir d6/d8/d1a/d36/d4d/d60 39 2024-05-09T06:38:30.803 INFO:tasks.workunit.client.1.smithi193.stdout:8/656: truncate d6/d1c/d1b/d29/f41 255272 0 2024-05-09T06:38:30.804 INFO:tasks.workunit.client.1.smithi193.stdout:8/657: chown d6/d5b/l7f 214 1 2024-05-09T06:38:30.808 INFO:tasks.workunit.client.1.smithi193.stdout:5/616: mkdir d0/d6d/dbd/dd0 0 2024-05-09T06:38:30.811 INFO:tasks.workunit.client.1.smithi193.stdout:7/614: dread d6/d8/f44 [0,4194304] 0 2024-05-09T06:38:30.813 INFO:tasks.workunit.client.1.smithi193.stdout:5/617: creat d0/d13/d2c/d37/d2b/d47/d60/d75/da9/fd1 x:0 0 0 2024-05-09T06:38:30.814 INFO:tasks.workunit.client.1.smithi193.stdout:5/618: write d0/d13/d2c/db6/dc6/fcf [300010,74110] 0 2024-05-09T06:38:30.815 INFO:tasks.workunit.client.1.smithi193.stdout:7/615: symlink d6/d8/d1a/d36/d4d/d74/lcf 0 2024-05-09T06:38:30.817 INFO:tasks.workunit.client.1.smithi193.stdout:8/658: truncate d6/d1c/d27/f65 558029 0 2024-05-09T06:38:30.817 INFO:tasks.workunit.client.1.smithi193.stdout:8/659: readlink d6/d1c/d18/l35 0 2024-05-09T06:38:30.824 INFO:tasks.workunit.client.1.smithi193.stdout:5/619: dwrite d0/d13/d2c/d37/d5b/fbc [0,4194304] 0 2024-05-09T06:38:30.827 INFO:tasks.workunit.client.1.smithi193.stdout:7/616: dread d6/f30 [0,4194304] 0 2024-05-09T06:38:30.829 INFO:tasks.workunit.client.1.smithi193.stdout:7/617: chown d6/dd/d7e/c9f 969670 1 2024-05-09T06:38:30.833 INFO:tasks.workunit.client.1.smithi193.stdout:7/618: mkdir d6/d8/d1a/d36/dd0 0 2024-05-09T06:38:30.838 INFO:tasks.workunit.client.1.smithi193.stdout:0/627: dwrite d18/d4c/d5f/f88 [0,4194304] 0 2024-05-09T06:38:30.843 INFO:tasks.workunit.client.1.smithi193.stdout:0/628: creat d18/d4c/d5f/fc2 x:0 0 0 2024-05-09T06:38:30.844 INFO:tasks.workunit.client.1.smithi193.stdout:0/629: write d18/d27/fa0 [5208021,57699] 0 2024-05-09T06:38:30.848 INFO:tasks.workunit.client.1.smithi193.stdout:8/660: truncate d6/d1c/f1e 2089928 0 2024-05-09T06:38:30.849 INFO:tasks.workunit.client.1.smithi193.stdout:8/661: stat d6/d1c/d18/d66/d93/d95/c9b 0 2024-05-09T06:38:30.850 INFO:tasks.workunit.client.1.smithi193.stdout:8/662: chown d6/d1c/d18/d66/f78 811929 1 2024-05-09T06:38:30.851 INFO:tasks.workunit.client.1.smithi193.stdout:7/619: link d6/dd/d70/l92 d6/d8/d1a/d36/d4d/d74/ld1 0 2024-05-09T06:38:30.852 INFO:tasks.workunit.client.1.smithi193.stdout:0/630: truncate d18/f60 2253474 0 2024-05-09T06:38:30.857 INFO:tasks.workunit.client.1.smithi193.stdout:7/620: rename d6/f8b to d6/dd/dbc/fd2 0 2024-05-09T06:38:30.857 INFO:tasks.workunit.client.1.smithi193.stdout:0/631: rmdir d18/d4c 39 2024-05-09T06:38:30.862 INFO:tasks.workunit.client.1.smithi193.stdout:8/663: creat d6/d1c/d1b/d5d/fc4 x:0 0 0 2024-05-09T06:38:30.865 INFO:tasks.workunit.client.1.smithi193.stdout:7/621: dwrite d6/dd/dbc/fc4 [0,4194304] 0 2024-05-09T06:38:30.872 INFO:tasks.workunit.client.1.smithi193.stdout:0/632: getdents d18/d4c/d64 0 2024-05-09T06:38:30.874 INFO:tasks.workunit.client.1.smithi193.stdout:8/664: dwrite d6/d1c/d18/fc3 [0,4194304] 0 2024-05-09T06:38:30.877 INFO:tasks.workunit.client.1.smithi193.stdout:7/622: rename d6/l81 to d6/dd/ld3 0 2024-05-09T06:38:30.878 INFO:tasks.workunit.client.1.smithi193.stdout:7/623: readlink d6/dd/d53/l59 0 2024-05-09T06:38:30.880 INFO:tasks.workunit.client.1.smithi193.stdout:7/624: read d6/dd/dbc/f3e [1691634,72688] 0 2024-05-09T06:38:30.881 INFO:tasks.workunit.client.1.smithi193.stdout:0/633: mkdir d18/d27/dc3 0 2024-05-09T06:38:30.887 INFO:tasks.workunit.client.1.smithi193.stdout:0/634: mkdir d18/d4c/d64/d74/dad/d70/dc4 0 2024-05-09T06:38:30.887 INFO:tasks.workunit.client.1.smithi193.stdout:7/625: truncate d6/d8/d1a/f3d 655964 0 2024-05-09T06:38:30.888 INFO:tasks.workunit.client.1.smithi193.stdout:7/626: readlink d6/d17/d99/lcb 0 2024-05-09T06:38:30.888 INFO:tasks.workunit.client.1.smithi193.stdout:7/627: dread - d6/d17/d7d/fbe zero size 2024-05-09T06:38:30.891 INFO:tasks.workunit.client.1.smithi193.stdout:8/665: creat d6/d1c/d18/d2f/d40/fc5 x:0 0 0 2024-05-09T06:38:30.891 INFO:tasks.workunit.client.1.smithi193.stdout:7/628: dread d6/d8/d1a/d36/d4d/fb0 [0,4194304] 0 2024-05-09T06:38:30.896 INFO:tasks.workunit.client.1.smithi193.stdout:8/666: rmdir d6/d1c/d18/d2f/d40/d55/d7e 39 2024-05-09T06:38:30.900 INFO:tasks.workunit.client.1.smithi193.stdout:0/635: symlink d18/d1d/lc5 0 2024-05-09T06:38:30.906 INFO:tasks.workunit.client.1.smithi193.stdout:7/629: creat d6/d8/d1a/d36/d4d/fd4 x:0 0 0 2024-05-09T06:38:30.911 INFO:tasks.workunit.client.1.smithi193.stdout:8/667: rename d6/d1c/d1b/d29/l61 to d6/d1c/d1b/d5d/d84/d87/da5/lc6 0 2024-05-09T06:38:30.911 INFO:tasks.workunit.client.1.smithi193.stdout:8/668: fsync d6/d1c/d1b/d29/d57/fb9 0 2024-05-09T06:38:30.917 INFO:tasks.workunit.client.1.smithi193.stdout:8/669: write d6/d1c/d18/d66/d93/f6c [693910,75435] 0 2024-05-09T06:38:30.922 INFO:tasks.workunit.client.1.smithi193.stdout:4/716: sync 2024-05-09T06:38:30.923 INFO:tasks.workunit.client.1.smithi193.stdout:4/717: chown d1a/d6b/d60/fb1 23 1 2024-05-09T06:38:30.927 INFO:tasks.workunit.client.1.smithi193.stdout:4/718: rename d1a/d24/d73/d94/d7c/c8a to d1a/d80/dc8/cdb 0 2024-05-09T06:38:30.928 INFO:tasks.workunit.client.1.smithi193.stdout:8/670: dwrite d6/f21 [0,4194304] 0 2024-05-09T06:38:30.934 INFO:tasks.workunit.client.1.smithi193.stdout:8/671: rename d6/d1c/d18/d2f/d40/d55 to d6/d1c/d1b/d5d/d84/dc7 0 2024-05-09T06:38:30.937 INFO:tasks.workunit.client.1.smithi193.stdout:8/672: rename d6/d1c/d18/da2 to d6/d1c/d1b/d29/d57/dc8 0 2024-05-09T06:38:30.940 INFO:tasks.workunit.client.1.smithi193.stdout:8/673: mknod d6/d1c/d1b/d29/d57/cc9 0 2024-05-09T06:38:30.943 INFO:tasks.workunit.client.1.smithi193.stdout:8/674: symlink d6/d1c/d18/d2f/d6a/lca 0 2024-05-09T06:38:30.946 INFO:tasks.workunit.client.1.smithi193.stdout:4/719: dread d1a/d1f/f2a [0,4194304] 0 2024-05-09T06:38:30.947 INFO:tasks.workunit.client.1.smithi193.stdout:4/720: write d1a/d67/fc1 [4723003,115996] 0 2024-05-09T06:38:30.953 INFO:tasks.workunit.client.1.smithi193.stdout:4/721: link d1a/d22/d43/fd4 d1a/d6d/fdc 0 2024-05-09T06:38:30.954 INFO:tasks.workunit.client.1.smithi193.stdout:4/722: truncate d1a/d6b/d60/fb1 322757 0 2024-05-09T06:38:30.963 INFO:tasks.workunit.client.1.smithi193.stdout:4/723: write d1a/d24/d73/d94/d7c/d5b/f70 [618616,99798] 0 2024-05-09T06:38:30.970 INFO:tasks.workunit.client.1.smithi193.stdout:4/724: dwrite d1a/d24/d73/fcb [0,4194304] 0 2024-05-09T06:38:30.971 INFO:tasks.workunit.client.1.smithi193.stdout:4/725: write d1a/d22/d8e/fb7 [430256,85574] 0 2024-05-09T06:38:30.986 INFO:tasks.workunit.client.1.smithi193.stdout:4/726: dread d1a/d24/d73/d94/fae [0,4194304] 0 2024-05-09T06:38:30.991 INFO:tasks.workunit.client.1.smithi193.stdout:4/727: link d1a/d24/d73/d94/d7c/lad d1a/d22/d43/d9a/ldd 0 2024-05-09T06:38:30.994 INFO:tasks.workunit.client.1.smithi193.stdout:4/728: link d1a/d24/f40 d1a/d24/dd0/fde 0 2024-05-09T06:38:31.000 INFO:tasks.workunit.client.1.smithi193.stdout:4/729: dread d1a/d24/d73/d94/d7c/f86 [0,4194304] 0 2024-05-09T06:38:31.003 INFO:tasks.workunit.client.1.smithi193.stdout:4/730: creat d1a/d24/d73/d94/d7c/fdf x:0 0 0 2024-05-09T06:38:31.007 INFO:tasks.workunit.client.1.smithi193.stdout:4/731: dread d1a/d24/d73/fcb [0,4194304] 0 2024-05-09T06:38:31.021 INFO:tasks.workunit.client.1.smithi193.stdout:4/732: dread d1a/fcf [0,4194304] 0 2024-05-09T06:38:31.024 INFO:tasks.workunit.client.1.smithi193.stdout:4/733: rename d1a/d22/f56 to d1a/d24/d73/d94/fe0 0 2024-05-09T06:38:31.030 INFO:tasks.workunit.client.1.smithi193.stdout:4/734: write d1a/d22/d43/f4d [455680,92799] 0 2024-05-09T06:38:31.032 INFO:tasks.workunit.client.1.smithi193.stdout:4/735: symlink d1a/d22/d5d/le1 0 2024-05-09T06:38:31.036 INFO:tasks.workunit.client.1.smithi193.stdout:4/736: dread d1a/d1f/f26 [0,4194304] 0 2024-05-09T06:38:31.037 INFO:tasks.workunit.client.1.smithi193.stdout:4/737: chown d1a/d22/d5d/ca8 28 1 2024-05-09T06:38:31.048 INFO:tasks.workunit.client.1.smithi193.stdout:4/738: dread d1a/d1f/f2c [0,4194304] 0 2024-05-09T06:38:31.051 INFO:tasks.workunit.client.1.smithi193.stdout:4/739: symlink d1a/d24/dd0/le2 0 2024-05-09T06:38:31.052 INFO:tasks.workunit.client.1.smithi193.stdout:4/740: chown d1a/l38 54764 1 2024-05-09T06:38:31.053 INFO:tasks.workunit.client.1.smithi193.stdout:4/741: write fa [4402629,126400] 0 2024-05-09T06:38:31.059 INFO:tasks.workunit.client.1.smithi193.stdout:4/742: dwrite d1a/d22/d5d/fd1 [0,4194304] 0 2024-05-09T06:38:31.060 INFO:tasks.workunit.client.1.smithi193.stdout:4/743: dread - d1a/fc9 zero size 2024-05-09T06:38:31.060 INFO:tasks.workunit.client.1.smithi193.stdout:4/744: chown d1a/cc0 46 1 2024-05-09T06:38:31.061 INFO:tasks.workunit.client.1.smithi193.stdout:4/745: chown d1a/da6/dd5 1085531030 1 2024-05-09T06:38:31.078 INFO:tasks.workunit.client.1.smithi193.stdout:4/746: dread d1a/d22/d5d/f93 [0,4194304] 0 2024-05-09T06:38:31.080 INFO:tasks.workunit.client.1.smithi193.stdout:4/747: fdatasync d1a/d24/d73/d94/d7c/f49 0 2024-05-09T06:38:31.081 INFO:tasks.workunit.client.1.smithi193.stdout:4/748: fsync d1a/d67/fc1 0 2024-05-09T06:38:31.081 INFO:tasks.workunit.client.1.smithi193.stdout:4/749: stat d1a/d22/ccd 0 2024-05-09T06:38:31.086 INFO:tasks.workunit.client.1.smithi193.stdout:4/750: dread d1a/d22/d43/fd4 [0,4194304] 0 2024-05-09T06:38:31.089 INFO:tasks.workunit.client.1.smithi193.stdout:4/751: dread d1a/d6b/f3c [0,4194304] 0 2024-05-09T06:38:31.094 INFO:tasks.workunit.client.1.smithi193.stdout:4/752: dwrite d1a/da6/dd5/fd7 [0,4194304] 0 2024-05-09T06:38:31.146 INFO:tasks.workunit.client.1.smithi193.stdout:3/609: sync 2024-05-09T06:38:31.147 INFO:tasks.workunit.client.1.smithi193.stdout:3/610: read - d1/db/d2f/d84/d97/fcd zero size 2024-05-09T06:38:31.167 INFO:tasks.workunit.client.1.smithi193.stdout:3/611: dwrite d1/db/d2f/d78/d86/ddc/f45 [0,4194304] 0 2024-05-09T06:38:31.170 INFO:tasks.workunit.client.1.smithi193.stdout:3/612: mkdir d1/d44/ddf 0 2024-05-09T06:38:31.172 INFO:tasks.workunit.client.1.smithi193.stdout:3/613: dread - d1/db/d2f/d78/d86/ddc/d51/fbd zero size 2024-05-09T06:38:31.335 INFO:tasks.workunit.client.1.smithi193.stdout:1/714: sync 2024-05-09T06:38:31.342 INFO:tasks.workunit.client.1.smithi193.stdout:1/715: truncate d4/d16/d60/d4f/d72/fc1 2407883 0 2024-05-09T06:38:31.347 INFO:tasks.workunit.client.1.smithi193.stdout:1/716: creat d4/d16/d6b/d89/fe9 x:0 0 0 2024-05-09T06:38:31.350 INFO:tasks.workunit.client.1.smithi193.stdout:1/717: mkdir d4/d16/dea 0 2024-05-09T06:38:31.353 INFO:tasks.workunit.client.1.smithi193.stdout:1/718: creat d4/d16/feb x:0 0 0 2024-05-09T06:38:31.354 INFO:tasks.workunit.client.1.smithi193.stdout:1/719: write d4/d16/d60/d4f/fda [4692073,64033] 0 2024-05-09T06:38:31.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:30 smithi063 ceph-mon[27171]: pgmap v18: 65 pgs: 65 active+clean; 3.0 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 76 MiB/s rd, 188 MiB/s wr, 412 op/s 2024-05-09T06:38:31.359 INFO:tasks.workunit.client.1.smithi193.stdout:1/720: creat d4/fec x:0 0 0 2024-05-09T06:38:31.359 INFO:tasks.workunit.client.1.smithi193.stdout:1/721: readlink d4/d16/d6b/l6d 0 2024-05-09T06:38:31.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:30 smithi193 ceph-mon[35141]: pgmap v18: 65 pgs: 65 active+clean; 3.0 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 76 MiB/s rd, 188 MiB/s wr, 412 op/s 2024-05-09T06:38:31.375 INFO:tasks.workunit.client.1.smithi193.stdout:1/722: dread d4/d14/f75 [0,4194304] 0 2024-05-09T06:38:31.377 INFO:tasks.workunit.client.1.smithi193.stdout:1/723: mknod d4/d16/ced 0 2024-05-09T06:38:31.378 INFO:tasks.workunit.client.1.smithi193.stdout:1/724: mknod d4/d16/d60/d4f/d72/d7f/cee 0 2024-05-09T06:38:31.382 INFO:tasks.workunit.client.1.smithi193.stdout:1/725: dread d4/d16/d17/d1a/d3a/d4a/db1/fb4 [0,4194304] 0 2024-05-09T06:38:31.410 INFO:tasks.workunit.client.1.smithi193.stdout:1/726: write d4/d16/d6b/d89/f99 [173699,116352] 0 2024-05-09T06:38:31.435 INFO:tasks.workunit.client.1.smithi193.stdout:9/770: sync 2024-05-09T06:38:31.435 INFO:tasks.workunit.client.1.smithi193.stdout:2/707: sync 2024-05-09T06:38:31.452 INFO:tasks.workunit.client.1.smithi193.stdout:1/727: write d4/d16/d17/d1a/f1e [1393301,103834] 0 2024-05-09T06:38:31.456 INFO:tasks.workunit.client.1.smithi193.stdout:1/728: creat d4/d16/d60/d4f/d72/fef x:0 0 0 2024-05-09T06:38:31.459 INFO:tasks.workunit.client.1.smithi193.stdout:9/771: dwrite dd/d1e/d1f/f2f [0,4194304] 0 2024-05-09T06:38:31.459 INFO:tasks.workunit.client.1.smithi193.stdout:2/708: dwrite db/d25/d38/d61/f70 [0,4194304] 0 2024-05-09T06:38:31.461 INFO:tasks.workunit.client.1.smithi193.stdout:9/772: stat dd/df/d15/d8c/lcb 0 2024-05-09T06:38:31.462 INFO:tasks.workunit.client.1.smithi193.stdout:1/729: rename d4/d16/d60/d4f/d72/d7f/lde to d4/d16/d6b/d89/dc4/lf0 0 2024-05-09T06:38:31.470 INFO:tasks.workunit.client.1.smithi193.stdout:9/773: creat dd/d1e/d1f/f100 x:0 0 0 2024-05-09T06:38:31.472 INFO:tasks.workunit.client.1.smithi193.stdout:1/730: dwrite d4/d16/d60/d4f/fda [0,4194304] 0 2024-05-09T06:38:31.472 INFO:tasks.workunit.client.1.smithi193.stdout:9/774: readlink dd/d1e/dc2/d49/lbe 0 2024-05-09T06:38:31.473 INFO:tasks.workunit.client.1.smithi193.stdout:9/775: chown dd/d1e 0 1 2024-05-09T06:38:31.474 INFO:tasks.workunit.client.1.smithi193.stdout:1/731: write d4/d16/d6b/d89/dc4/fe0 [921525,22255] 0 2024-05-09T06:38:31.476 INFO:tasks.workunit.client.1.smithi193.stdout:9/776: mkdir dd/df/d15/d98/dea/d101 0 2024-05-09T06:38:31.479 INFO:tasks.workunit.client.1.smithi193.stdout:9/777: rename dd/d1e/c57 to dd/df/d15/d8c/d7a/c102 0 2024-05-09T06:38:31.484 INFO:tasks.workunit.client.1.smithi193.stdout:2/709: dread db/f1e [0,4194304] 0 2024-05-09T06:38:31.486 INFO:tasks.workunit.client.1.smithi193.stdout:2/710: chown db/d9b/fee 1013721 1 2024-05-09T06:38:31.516 INFO:tasks.workunit.client.1.smithi193.stdout:9/778: dwrite dd/d1e/dc2/d49/dbd/fa6 [0,4194304] 0 2024-05-09T06:38:31.518 INFO:tasks.workunit.client.1.smithi193.stdout:2/711: dwrite db/f53 [0,4194304] 0 2024-05-09T06:38:31.526 INFO:tasks.workunit.client.1.smithi193.stdout:2/712: dwrite db/d25/d38/d40/d8f/f93 [0,4194304] 0 2024-05-09T06:38:31.529 INFO:tasks.workunit.client.1.smithi193.stdout:2/713: symlink db/d25/d38/d5b/lf0 0 2024-05-09T06:38:31.535 INFO:tasks.workunit.client.1.smithi193.stdout:2/714: dread db/d25/d38/d40/f9c [0,4194304] 0 2024-05-09T06:38:31.539 INFO:tasks.workunit.client.1.smithi193.stdout:2/715: fsync db/d37/d6d/d80/f13 0 2024-05-09T06:38:31.541 INFO:tasks.workunit.client.1.smithi193.stdout:2/716: dread - db/d25/f96 zero size 2024-05-09T06:38:31.542 INFO:tasks.workunit.client.1.smithi193.stdout:2/717: write db/df/d54/d63/dca/fcc [3711697,73897] 0 2024-05-09T06:38:31.545 INFO:tasks.workunit.client.1.smithi193.stdout:2/718: creat db/d37/d6d/ff1 x:0 0 0 2024-05-09T06:38:31.554 INFO:tasks.workunit.client.1.smithi193.stdout:9/779: dread dd/f82 [0,4194304] 0 2024-05-09T06:38:31.556 INFO:tasks.workunit.client.1.smithi193.stdout:9/780: dread - dd/d1e/dc2/d49/dbd/d5b/f9c zero size 2024-05-09T06:38:31.558 INFO:tasks.workunit.client.1.smithi193.stdout:9/781: mkdir dd/d1e/d1f/d103 0 2024-05-09T06:38:31.565 INFO:tasks.workunit.client.1.smithi193.stdout:2/719: dwrite db/d25/d38/fdd [0,4194304] 0 2024-05-09T06:38:31.570 INFO:tasks.workunit.client.1.smithi193.stdout:9/782: dwrite dd/df/d31/f5f [0,4194304] 0 2024-05-09T06:38:31.573 INFO:tasks.workunit.client.1.smithi193.stdout:9/783: dread - dd/d1e/fd2 zero size 2024-05-09T06:38:31.576 INFO:tasks.workunit.client.1.smithi193.stdout:9/784: creat dd/df/d31/d7e/d71/f104 x:0 0 0 2024-05-09T06:38:31.576 INFO:tasks.workunit.client.1.smithi193.stdout:2/720: dwrite db/f53 [0,4194304] 0 2024-05-09T06:38:31.579 INFO:tasks.workunit.client.1.smithi193.stdout:2/721: rename db/d25/d38/d40/d8f/cb7 to db/df/d54/d63/dca/cf2 0 2024-05-09T06:38:31.584 INFO:tasks.workunit.client.1.smithi193.stdout:9/785: dwrite dd/df/f6e [0,4194304] 0 2024-05-09T06:38:31.588 INFO:tasks.workunit.client.1.smithi193.stdout:2/722: dwrite db/d25/d38/d40/f58 [4194304,4194304] 0 2024-05-09T06:38:31.592 INFO:tasks.workunit.client.1.smithi193.stdout:9/786: mknod dd/df/d31/c105 0 2024-05-09T06:38:31.599 INFO:tasks.workunit.client.1.smithi193.stdout:2/723: dwrite db/f44 [0,4194304] 0 2024-05-09T06:38:31.602 INFO:tasks.workunit.client.1.smithi193.stdout:9/787: dwrite dd/df/d31/d7e/d71/f84 [0,4194304] 0 2024-05-09T06:38:31.605 INFO:tasks.workunit.client.1.smithi193.stdout:9/788: chown dd/df/d31/d7e/d71/l95 12858 1 2024-05-09T06:38:31.606 INFO:tasks.workunit.client.1.smithi193.stdout:9/789: chown dd/d1e/d1f/d61/cc3 1356115429 1 2024-05-09T06:38:31.607 INFO:tasks.workunit.client.1.smithi193.stdout:9/790: stat dd/d1e/d3c 0 2024-05-09T06:38:31.609 INFO:tasks.workunit.client.1.smithi193.stdout:9/791: rmdir dd/d1e/d93/dbb 39 2024-05-09T06:38:31.613 INFO:tasks.workunit.client.1.smithi193.stdout:9/792: mkdir dd/df/d15/d8c/de6/d106 0 2024-05-09T06:38:31.617 INFO:tasks.workunit.client.1.smithi193.stdout:9/793: creat dd/d1e/d1f/f107 x:0 0 0 2024-05-09T06:38:31.622 INFO:tasks.workunit.client.1.smithi193.stdout:9/794: rename dd/d1e/dc2/d49/dbd/d5b/d8a/dad/cf2 to dd/d1e/dc2/db9/c108 0 2024-05-09T06:38:31.622 INFO:tasks.workunit.client.1.smithi193.stdout:9/795: chown dd/d1e/dc2/d49/f70 1818650 1 2024-05-09T06:38:31.625 INFO:tasks.workunit.client.1.smithi193.stdout:9/796: link dd/f13 dd/df/d15/d98/f109 0 2024-05-09T06:38:31.632 INFO:tasks.workunit.client.1.smithi193.stdout:6/645: sync 2024-05-09T06:38:31.632 INFO:tasks.workunit.client.1.smithi193.stdout:0/636: sync 2024-05-09T06:38:31.632 INFO:tasks.workunit.client.1.smithi193.stdout:7/630: sync 2024-05-09T06:38:31.632 INFO:tasks.workunit.client.1.smithi193.stdout:5/620: sync 2024-05-09T06:38:31.632 INFO:tasks.workunit.client.1.smithi193.stdout:4/753: sync 2024-05-09T06:38:31.632 INFO:tasks.workunit.client.1.smithi193.stdout:8/675: sync 2024-05-09T06:38:31.632 INFO:tasks.workunit.client.1.smithi193.stdout:3/614: sync 2024-05-09T06:38:31.634 INFO:tasks.workunit.client.1.smithi193.stdout:6/646: chown d1/de/d11/d2d/fa8 18137379 1 2024-05-09T06:38:31.636 INFO:tasks.workunit.client.1.smithi193.stdout:7/631: creat d6/dd/d7e/fd5 x:0 0 0 2024-05-09T06:38:31.643 INFO:tasks.workunit.client.1.smithi193.stdout:6/647: dwrite d1/d25/d2c/d81/d37/d4f/fb4 [0,4194304] 0 2024-05-09T06:38:31.653 INFO:tasks.workunit.client.1.smithi193.stdout:4/754: dread d1a/d24/d73/d94/d7c/d5b/fb3 [0,4194304] 0 2024-05-09T06:38:31.655 INFO:tasks.workunit.client.1.smithi193.stdout:2/724: dwrite db/d37/d6d/d80/f14 [0,4194304] 0 2024-05-09T06:38:31.658 INFO:tasks.workunit.client.1.smithi193.stdout:4/755: dread d1a/d24/d73/d94/fae [0,4194304] 0 2024-05-09T06:38:31.658 INFO:tasks.workunit.client.1.smithi193.stdout:2/725: dread - db/d37/d6d/d80/f46 zero size 2024-05-09T06:38:31.661 INFO:tasks.workunit.client.1.smithi193.stdout:2/726: chown db/d37/d6d/d80 762119068 1 2024-05-09T06:38:31.661 INFO:tasks.workunit.client.1.smithi193.stdout:2/727: stat db/d25/fa9 0 2024-05-09T06:38:31.662 INFO:tasks.workunit.client.1.smithi193.stdout:9/797: truncate dd/d1e/d1f/d48/f5a 2257595 0 2024-05-09T06:38:31.663 INFO:tasks.workunit.client.1.smithi193.stdout:9/798: chown dd/d1e/dc2/d49 1048679489 1 2024-05-09T06:38:31.663 INFO:tasks.workunit.client.1.smithi193.stdout:6/648: mknod d1/d25/d2c/d81/d37/db3/ce9 0 2024-05-09T06:38:31.665 INFO:tasks.workunit.client.1.smithi193.stdout:6/649: write d1/d25/d2c/d3c/db1/d58/f84 [4661164,99570] 0 2024-05-09T06:38:31.666 INFO:tasks.workunit.client.1.smithi193.stdout:0/637: write d18/d1d/f30 [634352,15360] 0 2024-05-09T06:38:31.666 INFO:tasks.workunit.client.1.smithi193.stdout:0/638: stat d18/d4c/d47/d59/lc1 0 2024-05-09T06:38:31.667 INFO:tasks.workunit.client.1.smithi193.stdout:5/621: symlink d0/d13/d2c/d37/d2b/d47/d60/dc8/ld2 0 2024-05-09T06:38:31.673 INFO:tasks.workunit.client.1.smithi193.stdout:3/615: dwrite d1/db/d2f/d78/d86/fa8 [0,4194304] 0 2024-05-09T06:38:31.675 INFO:tasks.workunit.client.1.smithi193.stdout:4/756: fsync d1a/d24/f78 0 2024-05-09T06:38:31.677 INFO:tasks.workunit.client.1.smithi193.stdout:5/622: dwrite d0/d13/d2c/d37/d5b/fbb [0,4194304] 0 2024-05-09T06:38:31.681 INFO:tasks.workunit.client.1.smithi193.stdout:5/623: fsync d0/d13/d2c/d37/d5b/d4e/f7a 0 2024-05-09T06:38:31.690 INFO:tasks.workunit.client.1.smithi193.stdout:9/799: dwrite dd/d1e/d1f/d48/d55/d86/fee [0,4194304] 0 2024-05-09T06:38:31.690 INFO:tasks.workunit.client.1.smithi193.stdout:6/650: fsync d1/d25/d2c/d81/d37/d4f/f66 0 2024-05-09T06:38:31.692 INFO:tasks.workunit.client.1.smithi193.stdout:6/651: chown d1/de/d11/d2d/d3a/d47/f7a 504653 1 2024-05-09T06:38:31.692 INFO:tasks.workunit.client.1.smithi193.stdout:7/632: truncate d6/dd/f89 712957 0 2024-05-09T06:38:31.694 INFO:tasks.workunit.client.1.smithi193.stdout:0/639: symlink d18/d4c/d47/d55/lc6 0 2024-05-09T06:38:31.694 INFO:tasks.workunit.client.1.smithi193.stdout:6/652: write d1/d25/d2c/d3c/db1/d58/dc9/fe1 [7968,97176] 0 2024-05-09T06:38:31.704 INFO:tasks.workunit.client.1.smithi193.stdout:3/616: write d1/db/d2f/d78/d86/ddc/d51/f7e [856158,102896] 0 2024-05-09T06:38:31.705 INFO:tasks.workunit.client.1.smithi193.stdout:2/728: creat db/d25/d38/d61/de7/ff3 x:0 0 0 2024-05-09T06:38:31.712 INFO:tasks.workunit.client.1.smithi193.stdout:5/624: rename d0/d13/d2c/d37/d5b/d89/f8d to d0/d13/d2c/d7e/fd3 0 2024-05-09T06:38:31.713 INFO:tasks.workunit.client.1.smithi193.stdout:5/625: stat d0/d13/d2c/f4f 0 2024-05-09T06:38:31.713 INFO:tasks.workunit.client.1.smithi193.stdout:5/626: chown d0/l94 1747829159 1 2024-05-09T06:38:31.719 INFO:tasks.workunit.client.1.smithi193.stdout:9/800: symlink dd/l10a 0 2024-05-09T06:38:31.721 INFO:tasks.workunit.client.1.smithi193.stdout:5/627: dwrite d0/d13/d2c/d37/d2b/d47/d60/d75/da9/fd1 [0,4194304] 0 2024-05-09T06:38:31.721 INFO:tasks.workunit.client.1.smithi193.stdout:4/757: dread d1a/d24/f62 [0,4194304] 0 2024-05-09T06:38:31.722 INFO:tasks.workunit.client.1.smithi193.stdout:7/633: mkdir d6/dd/d7e/dd6 0 2024-05-09T06:38:31.737 INFO:tasks.workunit.client.1.smithi193.stdout:2/729: truncate db/d37/d6d/d80/f46 145258 0 2024-05-09T06:38:31.743 INFO:tasks.workunit.client.1.smithi193.stdout:6/653: write d1/de/d11/d18/f65 [501854,38377] 0 2024-05-09T06:38:31.749 INFO:tasks.workunit.client.1.smithi193.stdout:4/758: creat d1a/d22/d8e/fe3 x:0 0 0 2024-05-09T06:38:31.753 INFO:tasks.workunit.client.1.smithi193.stdout:3/617: dwrite d1/db/f73 [0,4194304] 0 2024-05-09T06:38:31.753 INFO:tasks.workunit.client.1.smithi193.stdout:7/634: read d6/dd/d53/f6a [1931327,101778] 0 2024-05-09T06:38:31.754 INFO:tasks.workunit.client.1.smithi193.stdout:5/628: dwrite d0/d13/d2c/d7e/fd3 [0,4194304] 0 2024-05-09T06:38:31.755 INFO:tasks.workunit.client.1.smithi193.stdout:7/635: readlink d6/d8/d1a/d36/l45 0 2024-05-09T06:38:31.762 INFO:tasks.workunit.client.1.smithi193.stdout:2/730: rename db/d37/d6d/d80/c22 to db/d25/d38/d40/d8f/cf4 0 2024-05-09T06:38:31.763 INFO:tasks.workunit.client.1.smithi193.stdout:9/801: symlink dd/df/d15/d8c/de6/d106/l10b 0 2024-05-09T06:38:31.768 INFO:tasks.workunit.client.1.smithi193.stdout:0/640: creat d18/fc7 x:0 0 0 2024-05-09T06:38:31.769 INFO:tasks.workunit.client.1.smithi193.stdout:0/641: truncate d18/d4c/f38 4442343 0 2024-05-09T06:38:31.769 INFO:tasks.workunit.client.1.smithi193.stdout:3/618: rmdir d1/db/d2f/d84 39 2024-05-09T06:38:31.770 INFO:tasks.workunit.client.1.smithi193.stdout:5/629: mknod d0/d13/d2c/d4d/cd4 0 2024-05-09T06:38:31.771 INFO:tasks.workunit.client.1.smithi193.stdout:7/636: mkdir d6/d17/d99/dd7 0 2024-05-09T06:38:31.772 INFO:tasks.workunit.client.1.smithi193.stdout:7/637: chown d6/dd/dbc/f3e 0 1 2024-05-09T06:38:31.773 INFO:tasks.workunit.client.1.smithi193.stdout:4/759: link d1a/d22/d8e/fe3 d1a/d22/d5d/fe4 0 2024-05-09T06:38:31.774 INFO:tasks.workunit.client.1.smithi193.stdout:2/731: mknod db/d25/d38/d40/d8b/cf5 0 2024-05-09T06:38:31.776 INFO:tasks.workunit.client.1.smithi193.stdout:9/802: read dd/d1e/d1f/d48/f5a [37274,67562] 0 2024-05-09T06:38:31.776 INFO:tasks.workunit.client.1.smithi193.stdout:7/638: fsync d6/dd/ff 0 2024-05-09T06:38:31.776 INFO:tasks.workunit.client.1.smithi193.stdout:2/732: chown db/d37/d6d/d80/daa 5 1 2024-05-09T06:38:31.776 INFO:tasks.workunit.client.1.smithi193.stdout:0/642: symlink d18/d4c/d47/d59/lc8 0 2024-05-09T06:38:31.778 INFO:tasks.workunit.client.1.smithi193.stdout:5/630: symlink d0/ld5 0 2024-05-09T06:38:31.779 INFO:tasks.workunit.client.1.smithi193.stdout:7/639: fsync d6/d8/d1a/d36/d4d/fc7 0 2024-05-09T06:38:31.783 INFO:tasks.workunit.client.1.smithi193.stdout:4/760: mknod d1a/d24/d73/d94/d7c/ce5 0 2024-05-09T06:38:31.788 INFO:tasks.workunit.client.1.smithi193.stdout:9/803: dwrite dd/df/dc4/fec [0,4194304] 0 2024-05-09T06:38:31.790 INFO:tasks.workunit.client.1.smithi193.stdout:0/643: dwrite d18/d4c/d64/d74/dad/f7e [0,4194304] 0 2024-05-09T06:38:31.791 INFO:tasks.workunit.client.1.smithi193.stdout:3/619: dread d1/db/d1b/f7a [4194304,4194304] 0 2024-05-09T06:38:31.794 INFO:tasks.workunit.client.1.smithi193.stdout:3/620: write d1/db/d2f/d78/d86/fa8 [3906204,55287] 0 2024-05-09T06:38:31.802 INFO:tasks.workunit.client.1.smithi193.stdout:4/761: rename d1a/d22/f31 to d1a/d22/d43/d9a/fe6 0 2024-05-09T06:38:31.804 INFO:tasks.workunit.client.1.smithi193.stdout:9/804: stat dd/df/d15/d98/fae 0 2024-05-09T06:38:31.804 INFO:tasks.workunit.client.1.smithi193.stdout:9/805: readlink dd/ld3 0 2024-05-09T06:38:31.806 INFO:tasks.workunit.client.1.smithi193.stdout:9/806: write dd/df/d31/d7e/d71/f104 [268464,112427] 0 2024-05-09T06:38:31.807 INFO:tasks.workunit.client.1.smithi193.stdout:9/807: chown dd/df/d15/d98/dff 184003374 1 2024-05-09T06:38:31.812 INFO:tasks.workunit.client.1.smithi193.stdout:0/644: mkdir d18/dc9 0 2024-05-09T06:38:31.813 INFO:tasks.workunit.client.1.smithi193.stdout:9/808: dwrite dd/d1e/d93/fa3 [0,4194304] 0 2024-05-09T06:38:31.820 INFO:tasks.workunit.client.1.smithi193.stdout:3/621: truncate d1/f1d 1715836 0 2024-05-09T06:38:31.823 INFO:tasks.workunit.client.1.smithi193.stdout:0/645: creat d18/d4c/d64/d66/d9d/fca x:0 0 0 2024-05-09T06:38:31.825 INFO:tasks.workunit.client.1.smithi193.stdout:0/646: truncate d18/d4c/d64/d74/dad/f80 1561070 0 2024-05-09T06:38:31.827 INFO:tasks.workunit.client.1.smithi193.stdout:2/733: write db/d37/d6d/d80/f46 [1088823,68166] 0 2024-05-09T06:38:31.827 INFO:tasks.workunit.client.1.smithi193.stdout:0/647: write d18/d4c/d47/d59/d76/fbc [977740,50340] 0 2024-05-09T06:38:31.829 INFO:tasks.workunit.client.1.smithi193.stdout:9/809: dread dd/d1e/d1f/d48/f5a [0,4194304] 0 2024-05-09T06:38:31.831 INFO:tasks.workunit.client.1.smithi193.stdout:2/734: stat db/df/d54/fb8 0 2024-05-09T06:38:31.831 INFO:tasks.workunit.client.1.smithi193.stdout:0/648: fdatasync d18/d4c/d47/d59/faa 0 2024-05-09T06:38:31.832 INFO:tasks.workunit.client.1.smithi193.stdout:5/631: truncate d0/f9 328687 0 2024-05-09T06:38:31.834 INFO:tasks.workunit.client.1.smithi193.stdout:7/640: dwrite d6/f30 [0,4194304] 0 2024-05-09T06:38:31.835 INFO:tasks.workunit.client.1.smithi193.stdout:2/735: mkdir db/d37/df6 0 2024-05-09T06:38:31.841 INFO:tasks.workunit.client.1.smithi193.stdout:5/632: creat d0/d13/d2c/db6/fd6 x:0 0 0 2024-05-09T06:38:31.842 INFO:tasks.workunit.client.1.smithi193.stdout:0/649: dread d18/d4c/d5f/f88 [0,4194304] 0 2024-05-09T06:38:31.843 INFO:tasks.workunit.client.1.smithi193.stdout:7/641: dread - d6/d8/d1a/fa4 zero size 2024-05-09T06:38:31.845 INFO:tasks.workunit.client.1.smithi193.stdout:2/736: creat db/d25/d38/d40/d8f/ff7 x:0 0 0 2024-05-09T06:38:31.846 INFO:tasks.workunit.client.1.smithi193.stdout:3/622: dread d1/db/d1b/f3b [0,4194304] 0 2024-05-09T06:38:31.846 INFO:tasks.workunit.client.1.smithi193.stdout:5/633: creat d0/d13/d2c/d4d/fd7 x:0 0 0 2024-05-09T06:38:31.847 INFO:tasks.workunit.client.1.smithi193.stdout:2/737: write db/d25/d38/d40/db2/fe2 [188,54680] 0 2024-05-09T06:38:31.848 INFO:tasks.workunit.client.1.smithi193.stdout:2/738: chown db/df 15 1 2024-05-09T06:38:31.850 INFO:tasks.workunit.client.1.smithi193.stdout:3/623: readlink d1/db/d2f/d78/d86/ddc/l55 0 2024-05-09T06:38:31.850 INFO:tasks.workunit.client.1.smithi193.stdout:2/739: chown db/d25/f27 1499452070 1 2024-05-09T06:38:31.855 INFO:tasks.workunit.client.1.smithi193.stdout:3/624: rmdir d1/db/d2f/d78/d86/ddc/d51 39 2024-05-09T06:38:31.856 INFO:tasks.workunit.client.1.smithi193.stdout:0/650: dwrite d18/d4c/d47/d54/f68 [4194304,4194304] 0 2024-05-09T06:38:31.858 INFO:tasks.workunit.client.1.smithi193.stdout:0/651: chown d18/d1d 918051 1 2024-05-09T06:38:31.859 INFO:tasks.workunit.client.1.smithi193.stdout:5/634: dwrite d0/d13/d2c/d37/d2b/d47/d66/d3a/f52 [0,4194304] 0 2024-05-09T06:38:31.870 INFO:tasks.workunit.client.1.smithi193.stdout:5/635: mknod d0/d13/d2c/d7e/dac/cd8 0 2024-05-09T06:38:31.871 INFO:tasks.workunit.client.1.smithi193.stdout:2/740: getdents db/df/d54/d63/dca 0 2024-05-09T06:38:31.873 INFO:tasks.workunit.client.1.smithi193.stdout:5/636: read d0/d13/d2c/d37/d2b/d47/d60/d75/f8b [582670,37841] 0 2024-05-09T06:38:31.874 INFO:tasks.workunit.client.1.smithi193.stdout:0/652: rename d18/d4c/d5f/fc2 to d18/d4c/d47/d59/fcb 0 2024-05-09T06:38:31.875 INFO:tasks.workunit.client.1.smithi193.stdout:9/810: write dd/df/f1d [2106406,118613] 0 2024-05-09T06:38:31.880 INFO:tasks.workunit.client.1.smithi193.stdout:2/741: dwrite db/d25/d38/d40/d8b/fc3 [0,4194304] 0 2024-05-09T06:38:31.882 INFO:tasks.workunit.client.1.smithi193.stdout:5/637: creat d0/d13/d2c/d37/d2b/d6a/d90/fd9 x:0 0 0 2024-05-09T06:38:31.883 INFO:tasks.workunit.client.1.smithi193.stdout:0/653: dread d18/d4c/f4b [0,4194304] 0 2024-05-09T06:38:31.885 INFO:tasks.workunit.client.1.smithi193.stdout:9/811: rename dd/d1e/dc2/d49/dbd/fc9 to dd/d1e/d3c/f10c 0 2024-05-09T06:38:31.888 INFO:tasks.workunit.client.1.smithi193.stdout:2/742: unlink db/d25/d38/d40/f72 0 2024-05-09T06:38:31.893 INFO:tasks.workunit.client.1.smithi193.stdout:0/654: dread d18/d4c/d47/d59/d76/fbc [0,4194304] 0 2024-05-09T06:38:31.896 INFO:tasks.workunit.client.1.smithi193.stdout:2/743: unlink db/d25/d38/d40/d8f/f93 0 2024-05-09T06:38:31.897 INFO:tasks.workunit.client.1.smithi193.stdout:5/638: dwrite d0/d13/d2c/db6/dc6/fcf [0,4194304] 0 2024-05-09T06:38:31.903 INFO:tasks.workunit.client.1.smithi193.stdout:9/812: dwrite dd/df/d15/d8c/fe8 [0,4194304] 0 2024-05-09T06:38:31.906 INFO:tasks.workunit.client.1.smithi193.stdout:2/744: readlink db/df/d54/l7b 0 2024-05-09T06:38:31.909 INFO:tasks.workunit.client.1.smithi193.stdout:3/625: truncate d1/d44/f5f 193265 0 2024-05-09T06:38:31.909 INFO:tasks.workunit.client.1.smithi193.stdout:7/642: dwrite d6/d8/d1a/f8f [0,4194304] 0 2024-05-09T06:38:31.911 INFO:tasks.workunit.client.1.smithi193.stdout:9/813: creat dd/d1e/d1f/d61/f10d x:0 0 0 2024-05-09T06:38:31.914 INFO:tasks.workunit.client.1.smithi193.stdout:0/655: symlink d18/d4c/d47/d59/d76/lcc 0 2024-05-09T06:38:31.916 INFO:tasks.workunit.client.1.smithi193.stdout:9/814: write dd/df/d31/d7e/f60 [1606930,72193] 0 2024-05-09T06:38:31.925 INFO:tasks.workunit.client.1.smithi193.stdout:5/639: dwrite d0/d13/d2c/d37/d5b/d89/f95 [0,4194304] 0 2024-05-09T06:38:31.927 INFO:tasks.workunit.client.1.smithi193.stdout:5/640: readlink d0/d13/d2c/d37/d2b/d47/d66/d3a/l99 0 2024-05-09T06:38:31.928 INFO:tasks.workunit.client.1.smithi193.stdout:0/656: creat d18/d4c/d64/fcd x:0 0 0 2024-05-09T06:38:31.929 INFO:tasks.workunit.client.1.smithi193.stdout:5/641: chown d0/d13/d2c/d37/d2b/d47/d60/d87/fa5 0 1 2024-05-09T06:38:31.929 INFO:tasks.workunit.client.1.smithi193.stdout:0/657: chown d18/d4c/d64/d74/dad/d70/dc4 2635200 1 2024-05-09T06:38:31.932 INFO:tasks.workunit.client.1.smithi193.stdout:2/745: link db/dbb/fd7 db/df/d54/dba/ff8 0 2024-05-09T06:38:31.940 INFO:tasks.workunit.client.1.smithi193.stdout:5/642: dwrite d0/d13/d2c/f4b [0,4194304] 0 2024-05-09T06:38:31.940 INFO:tasks.workunit.client.1.smithi193.stdout:0/658: mknod d18/d4c/d47/d59/cce 0 2024-05-09T06:38:31.944 INFO:tasks.workunit.client.1.smithi193.stdout:0/659: write d18/d4c/d64/f99 [798813,113990] 0 2024-05-09T06:38:31.946 INFO:tasks.workunit.client.1.smithi193.stdout:5/643: rmdir d0/d13/d2c/da0 39 2024-05-09T06:38:31.947 INFO:tasks.workunit.client.1.smithi193.stdout:0/660: write d18/d27/f41 [3413126,130120] 0 2024-05-09T06:38:31.949 INFO:tasks.workunit.client.1.smithi193.stdout:0/661: chown d18/d4c/d64 1737088 1 2024-05-09T06:38:31.949 INFO:tasks.workunit.client.1.smithi193.stdout:2/746: dwrite db/d37/d6d/d80/fcf [0,4194304] 0 2024-05-09T06:38:31.955 INFO:tasks.workunit.client.1.smithi193.stdout:7/643: creat d6/d8/d1a/d36/d4d/d60/fd8 x:0 0 0 2024-05-09T06:38:31.956 INFO:tasks.workunit.client.1.smithi193.stdout:3/626: link d1/db/d2f/d78/d86/ddc/fc8 d1/db/d2f/d78/d86/ddc/d51/fe0 0 2024-05-09T06:38:31.961 INFO:tasks.workunit.client.1.smithi193.stdout:5/644: mknod d0/d13/d2c/d37/d5b/d4e/d5d/cda 0 2024-05-09T06:38:31.964 INFO:tasks.workunit.client.1.smithi193.stdout:2/747: dread db/df/d54/f8c [0,4194304] 0 2024-05-09T06:38:31.965 INFO:tasks.workunit.client.1.smithi193.stdout:3/627: stat d1/db/c1f 0 2024-05-09T06:38:31.966 INFO:tasks.workunit.client.1.smithi193.stdout:7/644: mkdir d6/d8/d1a/d36/d4d/d60/d9c/dd9 0 2024-05-09T06:38:31.967 INFO:tasks.workunit.client.1.smithi193.stdout:0/662: symlink d18/d4c/d64/d74/dad/d70/dc4/lcf 0 2024-05-09T06:38:31.979 INFO:tasks.workunit.client.1.smithi193.stdout:5/645: creat d0/d13/d2c/d37/d2b/d47/d60/d9a/fdb x:0 0 0 2024-05-09T06:38:31.988 INFO:tasks.workunit.client.1.smithi193.stdout:2/748: rmdir db/df/db6 0 2024-05-09T06:38:31.991 INFO:tasks.workunit.client.1.smithi193.stdout:2/749: dread db/d25/d38/d61/f73 [4194304,4194304] 0 2024-05-09T06:38:31.991 INFO:tasks.workunit.client.1.smithi193.stdout:3/628: link d1/db/d1b/f7a d1/db/d2f/d78/d86/ddc/fe1 0 2024-05-09T06:38:31.993 INFO:tasks.workunit.client.1.smithi193.stdout:7/645: write d6/dd/f40 [7740265,89607] 0 2024-05-09T06:38:31.995 INFO:tasks.workunit.client.1.smithi193.stdout:3/629: dread d1/db/d2f/fc1 [0,4194304] 0 2024-05-09T06:38:31.997 INFO:tasks.workunit.client.1.smithi193.stdout:3/630: dread - d1/db/d2f/d78/d86/ddc/fcc zero size 2024-05-09T06:38:31.998 INFO:tasks.workunit.client.1.smithi193.stdout:0/663: dwrite d18/d4c/d47/d59/d76/f77 [0,4194304] 0 2024-05-09T06:38:32.004 INFO:tasks.workunit.client.1.smithi193.stdout:7/646: dread d6/d8/d1a/d36/d4d/fb0 [0,4194304] 0 2024-05-09T06:38:32.005 INFO:tasks.workunit.client.1.smithi193.stdout:7/647: fdatasync d6/d8/fc9 0 2024-05-09T06:38:32.007 INFO:tasks.workunit.client.1.smithi193.stdout:3/631: unlink d1/db/d1b/c9b 0 2024-05-09T06:38:32.008 INFO:tasks.workunit.client.1.smithi193.stdout:5/646: write d0/f9 [120851,119857] 0 2024-05-09T06:38:32.015 INFO:tasks.workunit.client.1.smithi193.stdout:7/648: creat d6/d8/d29/dc8/fda x:0 0 0 2024-05-09T06:38:32.016 INFO:tasks.workunit.client.1.smithi193.stdout:2/750: dwrite db/d37/d6d/d80/d60/d7d/f7e [0,4194304] 0 2024-05-09T06:38:32.017 INFO:tasks.workunit.client.1.smithi193.stdout:5/647: dwrite d0/d13/d2c/d37/d2b/d47/d66/f26 [4194304,4194304] 0 2024-05-09T06:38:32.022 INFO:tasks.workunit.client.1.smithi193.stdout:7/649: creat d6/dd/fdb x:0 0 0 2024-05-09T06:38:32.025 INFO:tasks.workunit.client.1.smithi193.stdout:2/751: symlink db/d37/df6/lf9 0 2024-05-09T06:38:32.026 INFO:tasks.workunit.client.1.smithi193.stdout:7/650: dread - d6/d8/d29/f7a zero size 2024-05-09T06:38:32.027 INFO:tasks.workunit.client.1.smithi193.stdout:5/648: mkdir d0/d13/d2c/d37/d5b/ddc 0 2024-05-09T06:38:32.027 INFO:tasks.workunit.client.1.smithi193.stdout:7/651: stat d6/d8/c69 0 2024-05-09T06:38:32.028 INFO:tasks.workunit.client.1.smithi193.stdout:7/652: chown d6/d17/f52 355984 1 2024-05-09T06:38:32.031 INFO:tasks.workunit.client.1.smithi193.stdout:2/752: mknod db/d37/d6d/dc4/cfa 0 2024-05-09T06:38:32.033 INFO:tasks.workunit.client.1.smithi193.stdout:3/632: dread d1/f9 [0,4194304] 0 2024-05-09T06:38:32.033 INFO:tasks.workunit.client.1.smithi193.stdout:2/753: write db/d25/f48 [2940351,111978] 0 2024-05-09T06:38:32.036 INFO:tasks.workunit.client.1.smithi193.stdout:7/653: creat d6/d8/d1a/d36/d4d/d60/d9c/dd9/fdc x:0 0 0 2024-05-09T06:38:32.037 INFO:tasks.workunit.client.1.smithi193.stdout:2/754: symlink db/df/d87/lfb 0 2024-05-09T06:38:32.037 INFO:tasks.workunit.client.1.smithi193.stdout:7/654: chown d6/d8/d1a/d36/d4d/d60/d9c/fb1 8337911 1 2024-05-09T06:38:32.038 INFO:tasks.workunit.client.1.smithi193.stdout:3/633: creat d1/db/d2f/d78/fe2 x:0 0 0 2024-05-09T06:38:32.039 INFO:tasks.workunit.client.1.smithi193.stdout:7/655: write d6/dd/dbc/fc4 [4555826,48138] 0 2024-05-09T06:38:32.044 INFO:tasks.workunit.client.1.smithi193.stdout:5/649: dread d0/f29 [0,4194304] 0 2024-05-09T06:38:32.044 INFO:tasks.workunit.client.1.smithi193.stdout:3/634: truncate d1/d44/f4a 912064 0 2024-05-09T06:38:32.048 INFO:tasks.workunit.client.1.smithi193.stdout:2/755: creat db/d37/d6d/d80/ffc x:0 0 0 2024-05-09T06:38:32.051 INFO:tasks.workunit.client.1.smithi193.stdout:7/656: unlink d6/d8/d29/d2f/l9b 0 2024-05-09T06:38:32.051 INFO:tasks.workunit.client.1.smithi193.stdout:3/635: dwrite d1/db/f19 [0,4194304] 0 2024-05-09T06:38:32.052 INFO:tasks.workunit.client.1.smithi193.stdout:7/657: fsync d6/d17/d7d/daa/f96 0 2024-05-09T06:38:32.054 INFO:tasks.workunit.client.1.smithi193.stdout:7/658: write d6/d8/d1a/d36/d4d/d60/d9c/dd9/fdc [215001,67959] 0 2024-05-09T06:38:32.058 INFO:tasks.workunit.client.1.smithi193.stdout:5/650: dread d0/d13/d2c/d37/d2b/d47/d66/f15 [0,4194304] 0 2024-05-09T06:38:32.058 INFO:tasks.workunit.client.1.smithi193.stdout:5/651: stat d0/d13/d2c/c53 0 2024-05-09T06:38:32.059 INFO:tasks.workunit.client.1.smithi193.stdout:5/652: read - d0/d13/d2c/d37/d5b/d4e/d9d/fba zero size 2024-05-09T06:38:32.066 INFO:tasks.workunit.client.1.smithi193.stdout:5/653: fdatasync d0/d13/d2c/d37/d74/f91 0 2024-05-09T06:38:32.072 INFO:tasks.workunit.client.1.smithi193.stdout:1/732: sync 2024-05-09T06:38:32.072 INFO:tasks.workunit.client.1.smithi193.stdout:8/676: sync 2024-05-09T06:38:32.073 INFO:tasks.workunit.client.1.smithi193.stdout:8/677: fdatasync d6/d1c/d18/d2f/d40/f2c 0 2024-05-09T06:38:32.073 INFO:tasks.workunit.client.1.smithi193.stdout:2/756: write db/d25/f96 [747695,24168] 0 2024-05-09T06:38:32.073 INFO:tasks.workunit.client.1.smithi193.stdout:5/654: rmdir d0/d13/d2c/d37 39 2024-05-09T06:38:32.077 INFO:tasks.workunit.client.1.smithi193.stdout:1/733: symlink d4/d16/de/dca/lf1 0 2024-05-09T06:38:32.080 INFO:tasks.workunit.client.1.smithi193.stdout:1/734: read d4/d16/d6b/d89/f99 [3811230,10952] 0 2024-05-09T06:38:32.082 INFO:tasks.workunit.client.1.smithi193.stdout:3/636: dwrite d1/db/d2f/d84/f9a [0,4194304] 0 2024-05-09T06:38:32.082 INFO:tasks.workunit.client.1.smithi193.stdout:8/678: unlink d6/d1c/d1b/d29/f4f 0 2024-05-09T06:38:32.083 INFO:tasks.workunit.client.1.smithi193.stdout:3/637: chown d1/db/da1 2059238 1 2024-05-09T06:38:32.083 INFO:tasks.workunit.client.1.smithi193.stdout:7/659: dwrite d6/dd/f3f [0,4194304] 0 2024-05-09T06:38:32.084 INFO:tasks.workunit.client.1.smithi193.stdout:5/655: fsync d0/d13/d2c/f4a 0 2024-05-09T06:38:32.085 INFO:tasks.workunit.client.1.smithi193.stdout:8/679: readlink d6/d1c/d18/d2f/d40/d5e/l82 0 2024-05-09T06:38:32.094 INFO:tasks.workunit.client.1.smithi193.stdout:2/757: dwrite db/d37/d6d/d80/fa6 [4194304,4194304] 0 2024-05-09T06:38:32.095 INFO:tasks.workunit.client.1.smithi193.stdout:1/735: unlink d4/d16/ld 0 2024-05-09T06:38:32.101 INFO:tasks.workunit.client.1.smithi193.stdout:7/660: mknod d6/d8/d1a/d36/d4d/d60/d9c/cdd 0 2024-05-09T06:38:32.104 INFO:tasks.workunit.client.1.smithi193.stdout:8/680: fdatasync d6/d1c/d27/dbd/fc1 0 2024-05-09T06:38:32.108 INFO:tasks.workunit.client.1.smithi193.stdout:1/736: mkdir d4/d16/d17/d1a/d3a/d3f/dd7/df2 0 2024-05-09T06:38:32.111 INFO:tasks.workunit.client.1.smithi193.stdout:8/681: mknod d6/ccb 0 2024-05-09T06:38:32.114 INFO:tasks.workunit.client.1.smithi193.stdout:1/737: creat d4/d3c/d3e/ff3 x:0 0 0 2024-05-09T06:38:32.115 INFO:tasks.workunit.client.1.smithi193.stdout:7/661: mknod d6/d8/d29/db9/cde 0 2024-05-09T06:38:32.116 INFO:tasks.workunit.client.1.smithi193.stdout:8/682: truncate d6/d1c/d53/f81 5204530 0 2024-05-09T06:38:32.117 INFO:tasks.workunit.client.1.smithi193.stdout:5/656: link d0/d13/d2c/f4b d0/d13/d2c/d37/d5b/d4e/d5d/fdd 0 2024-05-09T06:38:32.121 INFO:tasks.workunit.client.1.smithi193.stdout:5/657: write d0/d13/d2c/d37/d5b/d4e/f7a [1356871,113754] 0 2024-05-09T06:38:32.123 INFO:tasks.workunit.client.1.smithi193.stdout:7/662: symlink d6/dd/d7e/dd6/ldf 0 2024-05-09T06:38:32.124 INFO:tasks.workunit.client.1.smithi193.stdout:8/683: dwrite d6/f21 [0,4194304] 0 2024-05-09T06:38:32.126 INFO:tasks.workunit.client.1.smithi193.stdout:1/738: dwrite d4/d14/f75 [0,4194304] 0 2024-05-09T06:38:32.127 INFO:tasks.workunit.client.1.smithi193.stdout:5/658: chown d0/d13/d2c/d37/d2b/d47/d66/d3a/f96 1707825748 1 2024-05-09T06:38:32.133 INFO:tasks.workunit.client.1.smithi193.stdout:3/638: dwrite d1/f24 [0,4194304] 0 2024-05-09T06:38:32.134 INFO:tasks.workunit.client.1.smithi193.stdout:7/663: creat d6/d8/d29/db9/fe0 x:0 0 0 2024-05-09T06:38:32.136 INFO:tasks.workunit.client.1.smithi193.stdout:8/684: dread - d6/d1c/d1b/d29/d57/f8b zero size 2024-05-09T06:38:32.140 INFO:tasks.workunit.client.1.smithi193.stdout:2/758: dwrite db/d25/f27 [0,4194304] 0 2024-05-09T06:38:32.143 INFO:tasks.workunit.client.1.smithi193.stdout:5/659: creat d0/d13/d2c/d37/d2b/d47/d60/d9a/fde x:0 0 0 2024-05-09T06:38:32.150 INFO:tasks.workunit.client.1.smithi193.stdout:2/759: rename db/d37/d6d to db/def/dfd 0 2024-05-09T06:38:32.153 INFO:tasks.workunit.client.1.smithi193.stdout:5/660: dwrite d0/d13/d2c/d37/d2b/d47/d60/d75/da9/fd1 [0,4194304] 0 2024-05-09T06:38:32.155 INFO:tasks.workunit.client.1.smithi193.stdout:5/661: rmdir d0/d13/d2c/d37 39 2024-05-09T06:38:32.156 INFO:tasks.workunit.client.1.smithi193.stdout:2/760: dread db/f53 [0,4194304] 0 2024-05-09T06:38:32.158 INFO:tasks.workunit.client.1.smithi193.stdout:5/662: truncate d0/d13/d2c/d4d/d67/f76 614381 0 2024-05-09T06:38:32.158 INFO:tasks.workunit.client.1.smithi193.stdout:2/761: rmdir db/d25/d38/d61 39 2024-05-09T06:38:32.159 INFO:tasks.workunit.client.1.smithi193.stdout:2/762: chown db/l51 3932 1 2024-05-09T06:38:32.160 INFO:tasks.workunit.client.1.smithi193.stdout:2/763: write db/def/dfd/ff1 [425358,52559] 0 2024-05-09T06:38:32.164 INFO:tasks.workunit.client.1.smithi193.stdout:2/764: rename db/d25/d38/d5b/c79 to db/d9b/cfe 0 2024-05-09T06:38:32.166 INFO:tasks.workunit.client.1.smithi193.stdout:5/663: dwrite d0/d13/d2c/d4d/d67/fb5 [0,4194304] 0 2024-05-09T06:38:32.168 INFO:tasks.workunit.client.1.smithi193.stdout:2/765: rename db/df/d54/d63/dca/cf2 to db/df/d54/cff 0 2024-05-09T06:38:32.171 INFO:tasks.workunit.client.1.smithi193.stdout:2/766: chown db/l30 1 1 2024-05-09T06:38:32.174 INFO:tasks.workunit.client.1.smithi193.stdout:5/664: stat d0/d13/d2c/d37/d5b/d4e/cc5 0 2024-05-09T06:38:32.174 INFO:tasks.workunit.client.1.smithi193.stdout:8/685: dread d6/d1c/d1b/d5d/f6d [0,4194304] 0 2024-05-09T06:38:32.174 INFO:tasks.workunit.client.1.smithi193.stdout:1/739: rmdir d4/d3c/d3e 39 2024-05-09T06:38:32.176 INFO:tasks.workunit.client.1.smithi193.stdout:5/665: rmdir d0/d6d 39 2024-05-09T06:38:32.178 INFO:tasks.workunit.client.1.smithi193.stdout:1/740: chown d4/d16/d60/d4f/d72/fc1 395 1 2024-05-09T06:38:32.180 INFO:tasks.workunit.client.1.smithi193.stdout:2/767: rename db/d25/d38/dcd/fd2 to db/d4b/f100 0 2024-05-09T06:38:32.180 INFO:tasks.workunit.client.1.smithi193.stdout:7/664: getdents d6/dd/d7e/dd6 0 2024-05-09T06:38:32.181 INFO:tasks.workunit.client.1.smithi193.stdout:5/666: symlink d0/d13/d2c/db6/dc6/ldf 0 2024-05-09T06:38:32.182 INFO:tasks.workunit.client.1.smithi193.stdout:8/686: symlink d6/d1c/d18/d2f/d40/lcc 0 2024-05-09T06:38:32.189 INFO:tasks.workunit.client.1.smithi193.stdout:8/687: dwrite d6/d1c/d53/f63 [0,4194304] 0 2024-05-09T06:38:32.196 INFO:tasks.workunit.client.1.smithi193.stdout:3/639: dwrite d1/f11 [0,4194304] 0 2024-05-09T06:38:32.198 INFO:tasks.workunit.client.1.smithi193.stdout:3/640: write d1/d44/fd5 [156728,118773] 0 2024-05-09T06:38:32.201 INFO:tasks.workunit.client.1.smithi193.stdout:1/741: getdents d4/d16/d6b/d85 0 2024-05-09T06:38:32.203 INFO:tasks.workunit.client.1.smithi193.stdout:8/688: dwrite d6/fd [0,4194304] 0 2024-05-09T06:38:32.203 INFO:tasks.workunit.client.1.smithi193.stdout:1/742: stat d4/d16/d6b/laf 0 2024-05-09T06:38:32.210 INFO:tasks.workunit.client.1.smithi193.stdout:5/667: write d0/f8 [2791204,66891] 0 2024-05-09T06:38:32.210 INFO:tasks.workunit.client.1.smithi193.stdout:1/743: mkdir d4/d16/d60/d4f/dcd/df4 0 2024-05-09T06:38:32.214 INFO:tasks.workunit.client.1.smithi193.stdout:3/641: rename d1/db/f18 to d1/db/fe3 0 2024-05-09T06:38:32.214 INFO:tasks.workunit.client.1.smithi193.stdout:7/665: dwrite d6/dd/dbc/f87 [0,4194304] 0 2024-05-09T06:38:32.218 INFO:tasks.workunit.client.1.smithi193.stdout:1/744: unlink d4/d16/d17/d1a/d3a/d4a/c5a 0 2024-05-09T06:38:32.221 INFO:tasks.workunit.client.1.smithi193.stdout:1/745: chown d4/d16/d17/d1a/d3a/fa3 52964724 1 2024-05-09T06:38:32.221 INFO:tasks.workunit.client.1.smithi193.stdout:8/689: getdents d6/d1c/d1b/d5d/d84/d87/da5/daa 0 2024-05-09T06:38:32.221 INFO:tasks.workunit.client.1.smithi193.stdout:3/642: dwrite d1/d44/f60 [0,4194304] 0 2024-05-09T06:38:32.223 INFO:tasks.workunit.client.1.smithi193.stdout:7/666: mknod d6/dd/dbc/ce1 0 2024-05-09T06:38:32.225 INFO:tasks.workunit.client.1.smithi193.stdout:1/746: creat d4/d16/d6b/d85/ff5 x:0 0 0 2024-05-09T06:38:32.228 INFO:tasks.workunit.client.1.smithi193.stdout:8/690: read d6/d1c/d18/fc3 [2983742,60558] 0 2024-05-09T06:38:32.228 INFO:tasks.workunit.client.1.smithi193.stdout:7/667: mknod d6/d8/d1a/d36/d4d/d60/d9c/ce2 0 2024-05-09T06:38:32.232 INFO:tasks.workunit.client.1.smithi193.stdout:7/668: fsync d6/dd/f6c 0 2024-05-09T06:38:32.235 INFO:tasks.workunit.client.1.smithi193.stdout:7/669: mknod d6/dd/d7e/dd6/ce3 0 2024-05-09T06:38:32.238 INFO:tasks.workunit.client.1.smithi193.stdout:7/670: rename d6/d17/d7d/fbe to d6/fe4 0 2024-05-09T06:38:32.242 INFO:tasks.workunit.client.1.smithi193.stdout:5/668: dread d0/d6d/fb0 [0,4194304] 0 2024-05-09T06:38:32.244 INFO:tasks.workunit.client.1.smithi193.stdout:7/671: dwrite d6/d8/d29/dc8/fda [0,4194304] 0 2024-05-09T06:38:32.246 INFO:tasks.workunit.client.1.smithi193.stdout:5/669: unlink d0/d13/d2c/d4d/d67/fb5 0 2024-05-09T06:38:32.248 INFO:tasks.workunit.client.1.smithi193.stdout:5/670: dread - d0/d13/d2c/d37/d2b/d6a/d90/fc3 zero size 2024-05-09T06:38:32.251 INFO:tasks.workunit.client.1.smithi193.stdout:7/672: link d6/d8/d29/l32 d6/dd/d53/le5 0 2024-05-09T06:38:32.253 INFO:tasks.workunit.client.1.smithi193.stdout:1/747: write d4/d16/d6b/d6e/f78 [670766,95413] 0 2024-05-09T06:38:32.258 INFO:tasks.workunit.client.1.smithi193.stdout:1/748: creat d4/d16/d17/d1a/ff6 x:0 0 0 2024-05-09T06:38:32.259 INFO:tasks.workunit.client.1.smithi193.stdout:1/749: dread - d4/d16/feb zero size 2024-05-09T06:38:32.261 INFO:tasks.workunit.client.1.smithi193.stdout:8/691: dwrite d6/d1c/d1b/d29/d57/f8b [0,4194304] 0 2024-05-09T06:38:32.266 INFO:tasks.workunit.client.1.smithi193.stdout:7/673: dread d6/dd/d53/f6a [0,4194304] 0 2024-05-09T06:38:32.267 INFO:tasks.workunit.client.1.smithi193.stdout:8/692: fdatasync d6/d1c/d18/fc3 0 2024-05-09T06:38:32.268 INFO:tasks.workunit.client.1.smithi193.stdout:7/674: chown d6/d8/d1a/d36/d4d/d60/d9c/fb1 0 1 2024-05-09T06:38:32.268 INFO:tasks.workunit.client.1.smithi193.stdout:8/693: write d6/d1c/d1b/d5d/fc4 [445248,18439] 0 2024-05-09T06:38:32.269 INFO:tasks.workunit.client.1.smithi193.stdout:1/750: mknod d4/d3c/cf7 0 2024-05-09T06:38:32.269 INFO:tasks.workunit.client.1.smithi193.stdout:5/671: write d0/d13/f3d [883927,49553] 0 2024-05-09T06:38:32.273 INFO:tasks.workunit.client.1.smithi193.stdout:7/675: mknod d6/dd/d70/ce6 0 2024-05-09T06:38:32.273 INFO:tasks.workunit.client.1.smithi193.stdout:8/694: creat d6/d1c/d27/fcd x:0 0 0 2024-05-09T06:38:32.274 INFO:tasks.workunit.client.1.smithi193.stdout:7/676: fsync d6/d17/d7d/daa/f96 0 2024-05-09T06:38:32.275 INFO:tasks.workunit.client.1.smithi193.stdout:5/672: unlink d0/d13/d2c/db6/ccb 0 2024-05-09T06:38:32.276 INFO:tasks.workunit.client.1.smithi193.stdout:7/677: unlink d6/d8/c9e 0 2024-05-09T06:38:32.277 INFO:tasks.workunit.client.1.smithi193.stdout:1/751: mkdir d4/d16/d17/d1a/d3a/d3f/dd7/ddb/df8 0 2024-05-09T06:38:32.278 INFO:tasks.workunit.client.1.smithi193.stdout:5/673: mkdir d0/d13/d2c/d37/d5b/d89/de0 0 2024-05-09T06:38:32.279 INFO:tasks.workunit.client.1.smithi193.stdout:5/674: chown d0/d13/d2c/d37/c41 0 1 2024-05-09T06:38:32.279 INFO:tasks.workunit.client.1.smithi193.stdout:8/695: creat d6/d1c/d1b/d5d/d84/dc7/d7e/fce x:0 0 0 2024-05-09T06:38:32.280 INFO:tasks.workunit.client.1.smithi193.stdout:5/675: read d0/d13/d2c/d37/f11 [274550,126084] 0 2024-05-09T06:38:32.281 INFO:tasks.workunit.client.1.smithi193.stdout:7/678: truncate d6/d8/d1a/d36/d4d/d60/d9c/fbd 2088764 0 2024-05-09T06:38:32.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/696: dread - d6/d1c/d1b/d29/d57/fb9 zero size 2024-05-09T06:38:32.282 INFO:tasks.workunit.client.1.smithi193.stdout:5/676: read d0/d13/d2c/d37/d2b/d47/d66/f26 [6303165,7462] 0 2024-05-09T06:38:32.284 INFO:tasks.workunit.client.1.smithi193.stdout:7/679: write d6/d8/d1a/d36/d4d/fd4 [576604,20072] 0 2024-05-09T06:38:32.284 INFO:tasks.workunit.client.1.smithi193.stdout:8/697: fsync d6/d1c/d18/d2f/f39 0 2024-05-09T06:38:32.285 INFO:tasks.workunit.client.1.smithi193.stdout:8/698: dread - d6/d1c/d18/d2f/d40/fc5 zero size 2024-05-09T06:38:32.285 INFO:tasks.workunit.client.1.smithi193.stdout:8/699: readlink d6/d1c/d1b/d5d/d84/dc7/l90 0 2024-05-09T06:38:32.288 INFO:tasks.workunit.client.1.smithi193.stdout:5/677: symlink d0/d13/d2c/d37/d5b/d4e/d9d/le1 0 2024-05-09T06:38:32.288 INFO:tasks.workunit.client.1.smithi193.stdout:7/680: fdatasync d6/dd/d7e/fa3 0 2024-05-09T06:38:32.288 INFO:tasks.workunit.client.1.smithi193.stdout:7/681: stat d6/d17/d7d/daa/lc2 0 2024-05-09T06:38:32.289 INFO:tasks.workunit.client.1.smithi193.stdout:6/654: sync 2024-05-09T06:38:32.289 INFO:tasks.workunit.client.1.smithi193.stdout:4/762: sync 2024-05-09T06:38:32.289 INFO:tasks.workunit.client.1.smithi193.stdout:9/815: sync 2024-05-09T06:38:32.292 INFO:tasks.workunit.client.1.smithi193.stdout:8/700: symlink d6/d1c/d18/dba/lcf 0 2024-05-09T06:38:32.297 INFO:tasks.workunit.client.1.smithi193.stdout:1/752: write d4/d3c/f36 [1433680,7355] 0 2024-05-09T06:38:32.299 INFO:tasks.workunit.client.1.smithi193.stdout:4/763: mkdir d1a/d24/d73/d94/de7 0 2024-05-09T06:38:32.299 INFO:tasks.workunit.client.1.smithi193.stdout:4/764: stat d1a/d24/d73/d94/c9f 0 2024-05-09T06:38:32.300 INFO:tasks.workunit.client.1.smithi193.stdout:9/816: symlink dd/d1e/d1f/d61/l10e 0 2024-05-09T06:38:32.305 INFO:tasks.workunit.client.1.smithi193.stdout:1/753: truncate d4/d16/d60/d4f/d72/fc1 2513067 0 2024-05-09T06:38:32.305 INFO:tasks.workunit.client.1.smithi193.stdout:5/678: write d0/d13/d2c/d37/d5b/f7b [171568,50986] 0 2024-05-09T06:38:32.306 INFO:tasks.workunit.client.1.smithi193.stdout:4/765: symlink d1a/d22/d43/d9a/le8 0 2024-05-09T06:38:32.310 INFO:tasks.workunit.client.1.smithi193.stdout:7/682: link d6/lae d6/dd/d53/le7 0 2024-05-09T06:38:32.312 INFO:tasks.workunit.client.1.smithi193.stdout:4/766: write d1a/d24/f2e [1823565,15739] 0 2024-05-09T06:38:32.313 INFO:tasks.workunit.client.1.smithi193.stdout:9/817: creat dd/d1e/dc2/d49/dbd/d5b/d8a/dad/df8/f10f x:0 0 0 2024-05-09T06:38:32.317 INFO:tasks.workunit.client.1.smithi193.stdout:1/754: mkdir d4/d16/d6b/df9 0 2024-05-09T06:38:32.318 INFO:tasks.workunit.client.1.smithi193.stdout:7/683: fsync d6/d8/d1a/fb3 0 2024-05-09T06:38:32.319 INFO:tasks.workunit.client.1.smithi193.stdout:6/655: write d1/d25/d4c/d9e/dc5/fe6 [2966538,41050] 0 2024-05-09T06:38:32.326 INFO:tasks.workunit.client.1.smithi193.stdout:4/767: write d1a/d6b/d60/faa [117475,94664] 0 2024-05-09T06:38:32.326 INFO:tasks.workunit.client.1.smithi193.stdout:7/684: read - d6/d8/f6e zero size 2024-05-09T06:38:32.327 INFO:tasks.workunit.client.1.smithi193.stdout:7/685: chown d6/d8/c69 442 1 2024-05-09T06:38:32.336 INFO:tasks.workunit.client.1.smithi193.stdout:4/768: write d1a/d24/f40 [4854818,9306] 0 2024-05-09T06:38:32.338 INFO:tasks.workunit.client.1.smithi193.stdout:1/755: rename d4/d16/d17/d1a/d3a/fe1 to d4/d16/d17/ffa 0 2024-05-09T06:38:32.340 INFO:tasks.workunit.client.1.smithi193.stdout:6/656: dwrite d1/de/ff [4194304,4194304] 0 2024-05-09T06:38:32.342 INFO:tasks.workunit.client.1.smithi193.stdout:9/818: rename lc to dd/d1e/d93/dbb/l110 0 2024-05-09T06:38:32.347 INFO:tasks.workunit.client.1.smithi193.stdout:4/769: write f5 [1014070,43615] 0 2024-05-09T06:38:32.349 INFO:tasks.workunit.client.1.smithi193.stdout:7/686: rename d6/d17/d7d/daa/lc2 to d6/d8/d1a/d36/d4d/d74/le8 0 2024-05-09T06:38:32.353 INFO:tasks.workunit.client.1.smithi193.stdout:9/819: mkdir dd/d1e/dc2/d49/dbd/d5b/d111 0 2024-05-09T06:38:32.354 INFO:tasks.workunit.client.1.smithi193.stdout:1/756: mkdir d4/d16/d6b/d85/ddd/dfb 0 2024-05-09T06:38:32.355 INFO:tasks.workunit.client.1.smithi193.stdout:1/757: stat d4/d16/d6b/d89/fe9 0 2024-05-09T06:38:32.355 INFO:tasks.workunit.client.1.smithi193.stdout:4/770: mkdir d1a/d24/dd0/de9 0 2024-05-09T06:38:32.359 INFO:tasks.workunit.client.1.smithi193.stdout:4/771: symlink d1a/d22/d8e/lea 0 2024-05-09T06:38:32.362 INFO:tasks.workunit.client.1.smithi193.stdout:1/758: mkdir d4/d16/d17/dfc 0 2024-05-09T06:38:32.364 INFO:tasks.workunit.client.1.smithi193.stdout:6/657: rename d1/de/d11/d2d/da4/dc2 to d1/d25/d2c/d3c/db1/dea 0 2024-05-09T06:38:32.367 INFO:tasks.workunit.client.1.smithi193.stdout:7/687: creat d6/fe9 x:0 0 0 2024-05-09T06:38:32.370 INFO:tasks.workunit.client.1.smithi193.stdout:1/759: fdatasync d4/d76/f80 0 2024-05-09T06:38:32.370 INFO:tasks.workunit.client.1.smithi193.stdout:9/820: dread dd/df/d31/f4c [0,4194304] 0 2024-05-09T06:38:32.371 INFO:tasks.workunit.client.1.smithi193.stdout:7/688: dread d6/d8/d1a/d36/d4d/fd4 [0,4194304] 0 2024-05-09T06:38:32.375 INFO:tasks.workunit.client.1.smithi193.stdout:1/760: fsync d4/d16/d35/fa9 0 2024-05-09T06:38:32.378 INFO:tasks.workunit.client.1.smithi193.stdout:7/689: rename d6/dd/dbc/fce to d6/dd/d7e/dd6/fea 0 2024-05-09T06:38:32.379 INFO:tasks.workunit.client.1.smithi193.stdout:7/690: chown d6/dd/d7e/fcd 0 1 2024-05-09T06:38:32.380 INFO:tasks.workunit.client.1.smithi193.stdout:1/761: truncate d4/d16/f9b 1166469 0 2024-05-09T06:38:32.380 INFO:tasks.workunit.client.1.smithi193.stdout:9/821: symlink dd/d1e/dc2/d49/dbd/d5b/d111/l112 0 2024-05-09T06:38:32.381 INFO:tasks.workunit.client.1.smithi193.stdout:9/822: chown dd/c9b 0 1 2024-05-09T06:38:32.381 INFO:tasks.workunit.client.1.smithi193.stdout:4/772: link d1a/l53 d1a/d24/d73/d94/d7c/leb 0 2024-05-09T06:38:32.383 INFO:tasks.workunit.client.1.smithi193.stdout:9/823: write dd/d1e/dc2/d49/f51 [3567671,31921] 0 2024-05-09T06:38:32.386 INFO:tasks.workunit.client.1.smithi193.stdout:7/691: fdatasync d6/d8/f55 0 2024-05-09T06:38:32.390 INFO:tasks.workunit.client.1.smithi193.stdout:7/692: truncate d6/d8/d1a/f3c 2085059 0 2024-05-09T06:38:32.391 INFO:tasks.workunit.client.1.smithi193.stdout:1/762: dread d4/d16/f47 [0,4194304] 0 2024-05-09T06:38:32.393 INFO:tasks.workunit.client.1.smithi193.stdout:4/773: dread d1a/d67/f84 [0,4194304] 0 2024-05-09T06:38:32.394 INFO:tasks.workunit.client.1.smithi193.stdout:7/693: rmdir d6/dd/dbc 39 2024-05-09T06:38:32.453 INFO:tasks.workunit.client.1.smithi193.stdout:0/664: sync 2024-05-09T06:38:32.461 INFO:tasks.workunit.client.1.smithi193.stdout:1/763: write d4/d16/d35/f48 [88892,43070] 0 2024-05-09T06:38:32.465 INFO:tasks.workunit.client.1.smithi193.stdout:4/774: dwrite d1a/d6b/f63 [0,4194304] 0 2024-05-09T06:38:32.467 INFO:tasks.workunit.client.1.smithi193.stdout:4/775: fdatasync f10 0 2024-05-09T06:38:32.467 INFO:tasks.workunit.client.1.smithi193.stdout:9/824: dwrite dd/d1e/dc2/d49/f65 [0,4194304] 0 2024-05-09T06:38:32.470 INFO:tasks.workunit.client.1.smithi193.stdout:0/665: dwrite d18/f57 [0,4194304] 0 2024-05-09T06:38:32.493 INFO:tasks.workunit.client.1.smithi193.stdout:7/694: creat d6/dc3/feb x:0 0 0 2024-05-09T06:38:32.495 INFO:tasks.workunit.client.1.smithi193.stdout:4/776: dread - d1a/d24/d73/d94/fb4 zero size 2024-05-09T06:38:32.496 INFO:tasks.workunit.client.1.smithi193.stdout:9/825: chown dd/d1e/d93/dbb/l110 114 1 2024-05-09T06:38:32.497 INFO:tasks.workunit.client.1.smithi193.stdout:4/777: dread d1a/d22/d43/fd4 [0,4194304] 0 2024-05-09T06:38:32.498 INFO:tasks.workunit.client.1.smithi193.stdout:7/695: rename d6/d8/f44 to d6/d8/d1a/d36/dd0/fec 0 2024-05-09T06:38:32.500 INFO:tasks.workunit.client.1.smithi193.stdout:9/826: unlink dd/d1e/d1f/d48/f5a 0 2024-05-09T06:38:32.502 INFO:tasks.workunit.client.1.smithi193.stdout:4/778: rename d1a/d6b to d1a/d67/d82/dec 0 2024-05-09T06:38:32.503 INFO:tasks.workunit.client.1.smithi193.stdout:9/827: rmdir dd/d1e/d3c 39 2024-05-09T06:38:32.509 INFO:tasks.workunit.client.1.smithi193.stdout:7/696: getdents d6/d8/d1a/d36 0 2024-05-09T06:38:32.511 INFO:tasks.workunit.client.1.smithi193.stdout:4/779: rename d1a/d22/d43/d9a to d1a/ded 0 2024-05-09T06:38:32.512 INFO:tasks.workunit.client.1.smithi193.stdout:7/697: dread - d6/dd/dbc/fc5 zero size 2024-05-09T06:38:32.514 INFO:tasks.workunit.client.1.smithi193.stdout:4/780: creat d1a/d24/d73/dcc/fee x:0 0 0 2024-05-09T06:38:32.518 INFO:tasks.workunit.client.1.smithi193.stdout:9/828: dread dd/d1e/dc2/f80 [4194304,4194304] 0 2024-05-09T06:38:32.521 INFO:tasks.workunit.client.1.smithi193.stdout:9/829: truncate dd/df/d31/d7e/d37/f4d 189441 0 2024-05-09T06:38:32.523 INFO:tasks.workunit.client.1.smithi193.stdout:9/830: truncate dd/f73 374878 0 2024-05-09T06:38:32.529 INFO:tasks.workunit.client.1.smithi193.stdout:4/781: dread d1a/d24/d73/d94/d7c/d5b/f7d [0,4194304] 0 2024-05-09T06:38:32.532 INFO:tasks.workunit.client.1.smithi193.stdout:4/782: creat d1a/d24/d73/d94/dbd/fef x:0 0 0 2024-05-09T06:38:32.534 INFO:tasks.workunit.client.1.smithi193.stdout:4/783: getdents d1a/d24/d73/d94/de7 0 2024-05-09T06:38:32.538 INFO:tasks.workunit.client.1.smithi193.stdout:4/784: getdents d1a/ded 0 2024-05-09T06:38:32.539 INFO:tasks.workunit.client.1.smithi193.stdout:9/831: dread dd/df/f1d [0,4194304] 0 2024-05-09T06:38:32.540 INFO:tasks.workunit.client.1.smithi193.stdout:4/785: mkdir d1a/d67/df0 0 2024-05-09T06:38:32.541 INFO:tasks.workunit.client.1.smithi193.stdout:4/786: read f17 [1200080,27673] 0 2024-05-09T06:38:32.542 INFO:tasks.workunit.client.1.smithi193.stdout:7/698: write d6/d8/f55 [376247,46579] 0 2024-05-09T06:38:32.543 INFO:tasks.workunit.client.1.smithi193.stdout:9/832: fdatasync dd/d1e/d3c/fda 0 2024-05-09T06:38:32.544 INFO:tasks.workunit.client.1.smithi193.stdout:7/699: dread - d6/d17/d7d/daa/f96 zero size 2024-05-09T06:38:32.545 INFO:tasks.workunit.client.1.smithi193.stdout:4/787: chown d1a/d22/c29 5474572 1 2024-05-09T06:38:32.546 INFO:tasks.workunit.client.1.smithi193.stdout:4/788: chown d1a/d80/fa9 64584334 1 2024-05-09T06:38:32.551 INFO:tasks.workunit.client.1.smithi193.stdout:7/700: creat d6/d8/d29/dc8/fed x:0 0 0 2024-05-09T06:38:32.553 INFO:tasks.workunit.client.1.smithi193.stdout:7/701: creat d6/dd/d70/fee x:0 0 0 2024-05-09T06:38:32.556 INFO:tasks.workunit.client.1.smithi193.stdout:7/702: creat d6/d17/d99/fef x:0 0 0 2024-05-09T06:38:32.556 INFO:tasks.workunit.client.1.smithi193.stdout:9/833: getdents dd/d1e/d1f/d48/d55/d41/d68/db2 0 2024-05-09T06:38:32.557 INFO:tasks.workunit.client.1.smithi193.stdout:7/703: stat d6/d8/d1a/d36/d4d/d60/d9c/dd9 0 2024-05-09T06:38:32.557 INFO:tasks.workunit.client.1.smithi193.stdout:7/704: stat d6/d8/d29/d2f/l54 0 2024-05-09T06:38:32.644 INFO:tasks.workunit.client.1.smithi193.stdout:9/834: rename dd/df/d31/d7e to dd/df/d15/d8c/de6/d113 0 2024-05-09T06:38:32.645 INFO:tasks.workunit.client.1.smithi193.stdout:7/705: truncate d6/d8/f35 1677444 0 2024-05-09T06:38:32.655 INFO:tasks.workunit.client.1.smithi193.stdout:7/706: mknod d6/cf0 0 2024-05-09T06:38:32.658 INFO:tasks.workunit.client.1.smithi193.stdout:9/835: write dd/d1e/d1f/d48/fc8 [3502260,39743] 0 2024-05-09T06:38:32.659 INFO:tasks.workunit.client.1.smithi193.stdout:9/836: chown dd/d1e/d1f/d61/cdf 94609 1 2024-05-09T06:38:32.666 INFO:tasks.workunit.client.1.smithi193.stdout:9/837: dwrite dd/df/d15/d8c/de6/d113/fe4 [0,4194304] 0 2024-05-09T06:38:32.670 INFO:tasks.workunit.client.1.smithi193.stdout:7/707: link d6/d8/d1a/d36/d4d/fd4 d6/d8/d1a/d36/d4d/d60/ff1 0 2024-05-09T06:38:32.671 INFO:tasks.workunit.client.1.smithi193.stdout:7/708: chown d6/d8/d29/d2f/fc0 145887 1 2024-05-09T06:38:32.672 INFO:tasks.workunit.client.1.smithi193.stdout:7/709: fsync d6/f30 0 2024-05-09T06:38:32.673 INFO:tasks.workunit.client.1.smithi193.stdout:7/710: truncate d6/fe9 446039 0 2024-05-09T06:38:32.680 INFO:tasks.workunit.client.1.smithi193.stdout:7/711: dwrite d6/d8/d1a/d36/d4d/d60/d9c/dd9/fdc [0,4194304] 0 2024-05-09T06:38:32.682 INFO:tasks.workunit.client.1.smithi193.stdout:7/712: truncate d6/fe4 835384 0 2024-05-09T06:38:32.682 INFO:tasks.workunit.client.1.smithi193.stdout:7/713: fdatasync d6/dd/d70/fee 0 2024-05-09T06:38:32.688 INFO:tasks.workunit.client.1.smithi193.stdout:7/714: creat d6/d8/d1a/d36/ff2 x:0 0 0 2024-05-09T06:38:32.688 INFO:tasks.workunit.client.1.smithi193.stdout:7/715: dread - d6/dc3/feb zero size 2024-05-09T06:38:32.712 INFO:tasks.workunit.client.1.smithi193.stdout:7/716: getdents d6/d8/d1a/d36 0 2024-05-09T06:38:32.714 INFO:tasks.workunit.client.1.smithi193.stdout:7/717: mknod d6/dd/d7e/dd6/cf3 0 2024-05-09T06:38:32.715 INFO:tasks.workunit.client.1.smithi193.stdout:7/718: chown d6/dd/d70/ce6 70888 1 2024-05-09T06:38:32.720 INFO:tasks.workunit.client.1.smithi193.stdout:7/719: dwrite d6/dd/f47 [0,4194304] 0 2024-05-09T06:38:32.723 INFO:tasks.workunit.client.1.smithi193.stdout:7/720: unlink d6/d8/d1a/d36/d4d/d60/d9c/cdd 0 2024-05-09T06:38:32.726 INFO:tasks.workunit.client.1.smithi193.stdout:7/721: truncate d6/dd/d53/f58 760284 0 2024-05-09T06:38:32.734 INFO:tasks.workunit.client.1.smithi193.stdout:7/722: read d6/f30 [477047,120026] 0 2024-05-09T06:38:32.739 INFO:tasks.workunit.client.1.smithi193.stdout:7/723: truncate d6/dd/f68 496141 0 2024-05-09T06:38:32.743 INFO:tasks.workunit.client.1.smithi193.stdout:7/724: symlink d6/dd/d70/dcc/lf4 0 2024-05-09T06:38:32.744 INFO:tasks.workunit.client.1.smithi193.stdout:7/725: dread - d6/dd/d7e/fd5 zero size 2024-05-09T06:38:32.746 INFO:tasks.workunit.client.1.smithi193.stdout:7/726: dread - d6/dd/d5e/f77 zero size 2024-05-09T06:38:32.747 INFO:tasks.workunit.client.1.smithi193.stdout:7/727: write d6/f13 [1339011,59039] 0 2024-05-09T06:38:32.749 INFO:tasks.workunit.client.1.smithi193.stdout:7/728: getdents d6/d17/d99/dd7 0 2024-05-09T06:38:32.750 INFO:tasks.workunit.client.1.smithi193.stdout:7/729: read - d6/dd/d70/fee zero size 2024-05-09T06:38:32.772 INFO:tasks.workunit.client.1.smithi193.stdout:7/730: dwrite d6/d8/d1a/d36/d4d/d60/d9c/fbd [0,4194304] 0 2024-05-09T06:38:32.777 INFO:tasks.workunit.client.1.smithi193.stdout:7/731: dread d6/d8/d1a/d36/d4d/d60/d9c/fbd [0,4194304] 0 2024-05-09T06:38:32.782 INFO:tasks.workunit.client.1.smithi193.stdout:7/732: mknod d6/dd/d53/cf5 0 2024-05-09T06:38:32.783 INFO:tasks.workunit.client.1.smithi193.stdout:7/733: read - d6/d8/f6e zero size 2024-05-09T06:38:32.784 INFO:tasks.workunit.client.1.smithi193.stdout:7/734: chown d6/d8/d1a/d36/d4d/d60/cb2 6129 1 2024-05-09T06:38:32.799 INFO:tasks.workunit.client.1.smithi193.stdout:2/768: sync 2024-05-09T06:38:32.805 INFO:tasks.workunit.client.1.smithi193.stdout:7/735: dwrite d6/d8/d1a/d36/f57 [0,4194304] 0 2024-05-09T06:38:32.805 INFO:tasks.workunit.client.1.smithi193.stdout:2/769: dwrite db/d25/d38/fdd [0,4194304] 0 2024-05-09T06:38:32.807 INFO:tasks.workunit.client.1.smithi193.stdout:2/770: write db/d25/d38/d40/f58 [257066,43372] 0 2024-05-09T06:38:32.810 INFO:tasks.workunit.client.1.smithi193.stdout:7/736: dread d6/dd/dbc/f87 [0,4194304] 0 2024-05-09T06:38:32.811 INFO:tasks.workunit.client.1.smithi193.stdout:2/771: creat db/d4b/f101 x:0 0 0 2024-05-09T06:38:32.814 INFO:tasks.workunit.client.1.smithi193.stdout:2/772: truncate db/d25/d38/d40/db2/dcb/fdc 984257 0 2024-05-09T06:38:32.816 INFO:tasks.workunit.client.1.smithi193.stdout:7/737: rmdir d6/d8/d29 39 2024-05-09T06:38:32.817 INFO:tasks.workunit.client.1.smithi193.stdout:7/738: chown d6/d17/d99/fef 1991778 1 2024-05-09T06:38:32.819 INFO:tasks.workunit.client.1.smithi193.stdout:7/739: symlink d6/d17/d99/lf6 0 2024-05-09T06:38:32.826 INFO:tasks.workunit.client.1.smithi193.stdout:7/740: dwrite d6/dc3/feb [0,4194304] 0 2024-05-09T06:38:32.826 INFO:tasks.workunit.client.1.smithi193.stdout:7/741: dread - d6/d17/d7d/daa/f96 zero size 2024-05-09T06:38:32.829 INFO:tasks.workunit.client.1.smithi193.stdout:7/742: write d6/d8/f55 [905844,130677] 0 2024-05-09T06:38:32.830 INFO:tasks.workunit.client.1.smithi193.stdout:7/743: stat d6/d17/d7d/daa/fba 0 2024-05-09T06:38:32.834 INFO:tasks.workunit.client.1.smithi193.stdout:7/744: mknod d6/d17/d99/dd7/cf7 0 2024-05-09T06:38:32.835 INFO:tasks.workunit.client.1.smithi193.stdout:7/745: write d6/dd/d70/fee [532426,54972] 0 2024-05-09T06:38:32.842 INFO:tasks.workunit.client.1.smithi193.stdout:7/746: fsync d6/dd/f89 0 2024-05-09T06:38:32.846 INFO:tasks.workunit.client.1.smithi193.stdout:7/747: creat d6/dd/d70/ff8 x:0 0 0 2024-05-09T06:38:32.846 INFO:tasks.workunit.client.1.smithi193.stdout:2/773: truncate db/d25/d38/d40/f9c 855085 0 2024-05-09T06:38:32.848 INFO:tasks.workunit.client.1.smithi193.stdout:7/748: read d6/d17/f37 [268864,110283] 0 2024-05-09T06:38:32.855 INFO:tasks.workunit.client.1.smithi193.stdout:7/749: dwrite d6/dd/d7e/fcd [0,4194304] 0 2024-05-09T06:38:32.866 INFO:tasks.workunit.client.1.smithi193.stdout:7/750: dread d6/d17/d7d/daa/f8c [0,4194304] 0 2024-05-09T06:38:32.881 INFO:tasks.workunit.client.1.smithi193.stdout:7/751: write d6/dd/dbc/f3e [3848093,29678] 0 2024-05-09T06:38:32.886 INFO:tasks.workunit.client.1.smithi193.stdout:7/752: mkdir d6/d8/d29/df9 0 2024-05-09T06:38:32.887 INFO:tasks.workunit.client.1.smithi193.stdout:7/753: chown d6/d8/cc 6715710 1 2024-05-09T06:38:32.887 INFO:tasks.workunit.client.1.smithi193.stdout:7/754: chown d6/d17/d7d 39390961 1 2024-05-09T06:38:32.890 INFO:tasks.workunit.client.1.smithi193.stdout:7/755: unlink d6/d8/d1a/d36/l45 0 2024-05-09T06:38:32.892 INFO:tasks.workunit.client.1.smithi193.stdout:7/756: creat d6/dd/d7e/ffa x:0 0 0 2024-05-09T06:38:32.892 INFO:tasks.workunit.client.1.smithi193.stdout:7/757: chown d6/d8/d29/db9 3026 1 2024-05-09T06:38:32.894 INFO:tasks.workunit.client.1.smithi193.stdout:7/758: unlink d6/dd/dbc/f3e 0 2024-05-09T06:38:32.915 INFO:tasks.workunit.client.1.smithi193.stdout:7/759: write d6/d8/d1a/f27 [2779131,15222] 0 2024-05-09T06:38:32.920 INFO:tasks.workunit.client.1.smithi193.stdout:7/760: creat d6/dd/d5e/d6b/ffb x:0 0 0 2024-05-09T06:38:32.929 INFO:tasks.workunit.client.1.smithi193.stdout:7/761: getdents d6/d17/d99/dd7 0 2024-05-09T06:38:32.952 INFO:tasks.workunit.client.1.smithi193.stdout:7/762: dread d6/dd/dbc/fd2 [0,4194304] 0 2024-05-09T06:38:32.957 INFO:tasks.workunit.client.1.smithi193.stdout:7/763: mkdir d6/d8/d29/d2f/dfc 0 2024-05-09T06:38:32.963 INFO:tasks.workunit.client.1.smithi193.stdout:7/764: mknod d6/d8/d1a/d36/d4d/d74/cfd 0 2024-05-09T06:38:32.964 INFO:tasks.workunit.client.1.smithi193.stdout:7/765: chown d6/dd/d7e/fd5 46268954 1 2024-05-09T06:38:32.966 INFO:tasks.workunit.client.1.smithi193.stdout:7/766: rmdir d6/dd/d5e/d6b 39 2024-05-09T06:38:32.974 INFO:tasks.workunit.client.1.smithi193.stdout:7/767: creat d6/dd/ffe x:0 0 0 2024-05-09T06:38:32.975 INFO:tasks.workunit.client.1.smithi193.stdout:7/768: chown d6/d17/d99/dd7/cf7 9 1 2024-05-09T06:38:32.981 INFO:tasks.workunit.client.1.smithi193.stdout:7/769: creat d6/d17/d7d/fff x:0 0 0 2024-05-09T06:38:32.982 INFO:tasks.workunit.client.1.smithi193.stdout:7/770: symlink d6/d8/d29/db9/l100 0 2024-05-09T06:38:32.985 INFO:tasks.workunit.client.1.smithi193.stdout:7/771: symlink d6/d17/d7d/l101 0 2024-05-09T06:38:32.985 INFO:tasks.workunit.client.1.smithi193.stdout:7/772: write d6/f16 [7950694,26537] 0 2024-05-09T06:38:32.990 INFO:tasks.workunit.client.1.smithi193.stdout:7/773: getdents d6/dd/d70/dcc 0 2024-05-09T06:38:32.992 INFO:tasks.workunit.client.1.smithi193.stdout:7/774: write d6/d8/d1a/d36/f57 [376182,83911] 0 2024-05-09T06:38:33.005 INFO:tasks.workunit.client.1.smithi193.stdout:3/643: sync 2024-05-09T06:38:33.006 INFO:tasks.workunit.client.1.smithi193.stdout:3/644: chown d1/db/d2f/fc1 4 1 2024-05-09T06:38:33.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:32 smithi063 ceph-mon[27171]: pgmap v19: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 117 MiB/s rd, 276 MiB/s wr, 619 op/s 2024-05-09T06:38:33.161 INFO:tasks.workunit.client.1.smithi193.stdout:3/645: truncate d1/db/d2f/d78/d86/ddc/d51/f6a 655726 0 2024-05-09T06:38:33.166 INFO:tasks.workunit.client.1.smithi193.stdout:3/646: dwrite d1/db/d59/fc2 [0,4194304] 0 2024-05-09T06:38:33.182 INFO:tasks.workunit.client.1.smithi193.stdout:3/647: dread d1/db/f73 [0,4194304] 0 2024-05-09T06:38:33.184 INFO:tasks.workunit.client.1.smithi193.stdout:3/648: mkdir d1/db/da1/de4 0 2024-05-09T06:38:33.185 INFO:tasks.workunit.client.1.smithi193.stdout:3/649: rmdir d1/d44/d7d 39 2024-05-09T06:38:33.268 INFO:tasks.workunit.client.1.smithi193.stdout:3/650: dwrite d1/db/d59/f9f [0,4194304] 0 2024-05-09T06:38:33.272 INFO:tasks.workunit.client.1.smithi193.stdout:3/651: symlink d1/db/d2f/d84/le5 0 2024-05-09T06:38:33.277 INFO:tasks.workunit.client.1.smithi193.stdout:5/679: sync 2024-05-09T06:38:33.277 INFO:tasks.workunit.client.1.smithi193.stdout:8/701: sync 2024-05-09T06:38:33.279 INFO:tasks.workunit.client.1.smithi193.stdout:3/652: dwrite d1/d44/d7d/f83 [0,4194304] 0 2024-05-09T06:38:33.286 INFO:tasks.workunit.client.1.smithi193.stdout:3/653: mknod d1/d44/ddf/ce6 0 2024-05-09T06:38:33.287 INFO:tasks.workunit.client.1.smithi193.stdout:3/654: chown d1/db/d2f/cd7 1418 1 2024-05-09T06:38:33.289 INFO:tasks.workunit.client.1.smithi193.stdout:8/702: creat d6/d1c/d1b/fd0 x:0 0 0 2024-05-09T06:38:33.289 INFO:tasks.workunit.client.1.smithi193.stdout:5/680: write d0/d13/d2c/d37/d2b/d47/d66/f26 [3300217,22851] 0 2024-05-09T06:38:33.290 INFO:tasks.workunit.client.1.smithi193.stdout:5/681: chown d0/d13/d2c/c97 284057 1 2024-05-09T06:38:33.295 INFO:tasks.workunit.client.1.smithi193.stdout:3/655: rename d1/db/d59/f4e to d1/db/fe7 0 2024-05-09T06:38:33.297 INFO:tasks.workunit.client.1.smithi193.stdout:3/656: readlink d1/db/d1b/l46 0 2024-05-09T06:38:33.298 INFO:tasks.workunit.client.1.smithi193.stdout:5/682: dwrite d0/d13/d2c/d37/d2b/d6a/d90/fb1 [0,4194304] 0 2024-05-09T06:38:33.299 INFO:tasks.workunit.client.1.smithi193.stdout:5/683: fsync d0/d13/d2c/db6/fd6 0 2024-05-09T06:38:33.300 INFO:tasks.workunit.client.1.smithi193.stdout:8/703: rename d6/d1c/d1b/d5d/d84/dc7/l90 to d6/d1c/d18/dba/ld1 0 2024-05-09T06:38:33.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/684: rename d0/d13/d2c/d37/d2b/d6a/d90/fd9 to d0/d6d/dbd/fe2 0 2024-05-09T06:38:33.312 INFO:tasks.workunit.client.1.smithi193.stdout:5/685: fsync d0/d13/d2c/f4f 0 2024-05-09T06:38:33.313 INFO:tasks.workunit.client.1.smithi193.stdout:8/704: creat d6/d1c/d1b/fd2 x:0 0 0 2024-05-09T06:38:33.314 INFO:tasks.workunit.client.1.smithi193.stdout:3/657: getdents d1/db/d2f/d84 0 2024-05-09T06:38:33.319 INFO:tasks.workunit.client.1.smithi193.stdout:8/705: dwrite d6/d1c/d18/dba/fc0 [0,4194304] 0 2024-05-09T06:38:33.321 INFO:tasks.workunit.client.1.smithi193.stdout:8/706: chown d6/d1c/d18/fb4 1482745 1 2024-05-09T06:38:33.323 INFO:tasks.workunit.client.1.smithi193.stdout:3/658: symlink d1/db/d1b/dbf/le8 0 2024-05-09T06:38:33.327 INFO:tasks.workunit.client.1.smithi193.stdout:8/707: dwrite d6/d1c/d27/f43 [0,4194304] 0 2024-05-09T06:38:33.327 INFO:tasks.workunit.client.1.smithi193.stdout:1/764: sync 2024-05-09T06:38:33.328 INFO:tasks.workunit.client.1.smithi193.stdout:6/658: sync 2024-05-09T06:38:33.332 INFO:tasks.workunit.client.1.smithi193.stdout:6/659: creat d1/d25/d4c/d9e/feb x:0 0 0 2024-05-09T06:38:33.333 INFO:tasks.workunit.client.1.smithi193.stdout:1/765: truncate d4/d14/f51 1345703 0 2024-05-09T06:38:33.333 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:32 smithi193 ceph-mon[35141]: pgmap v19: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 117 MiB/s rd, 276 MiB/s wr, 619 op/s 2024-05-09T06:38:33.334 INFO:tasks.workunit.client.1.smithi193.stdout:3/659: getdents d1/db/d2f 0 2024-05-09T06:38:33.337 INFO:tasks.workunit.client.1.smithi193.stdout:8/708: dwrite d6/d1c/d18/d2f/d40/f2c [0,4194304] 0 2024-05-09T06:38:33.337 INFO:tasks.workunit.client.1.smithi193.stdout:1/766: mknod d4/d16/d60/d4f/dcd/cfd 0 2024-05-09T06:38:33.339 INFO:tasks.workunit.client.1.smithi193.stdout:1/767: stat d4/d16/d17/d1a/d5b 0 2024-05-09T06:38:33.339 INFO:tasks.workunit.client.1.smithi193.stdout:8/709: fsync d6/d1c/d53/f63 0 2024-05-09T06:38:33.339 INFO:tasks.workunit.client.1.smithi193.stdout:6/660: symlink d1/d25/d4c/d9e/dc8/lec 0 2024-05-09T06:38:33.340 INFO:tasks.workunit.client.1.smithi193.stdout:1/768: chown d4/d16/lae 1892 1 2024-05-09T06:38:33.344 INFO:tasks.workunit.client.1.smithi193.stdout:8/710: truncate d6/d1c/d1b/d5d/d84/dc7/d7e/fa9 1861529 0 2024-05-09T06:38:33.348 INFO:tasks.workunit.client.1.smithi193.stdout:1/769: link d4/d3c/f36 d4/d16/d6b/d85/ddd/dfb/ffe 0 2024-05-09T06:38:33.355 INFO:tasks.workunit.client.1.smithi193.stdout:6/661: link d1/de/d11/d18/f24 d1/d25/d2c/d3c/db1/d58/fed 0 2024-05-09T06:38:33.356 INFO:tasks.workunit.client.1.smithi193.stdout:6/662: chown d1/de/d11/d20/fbf 5118 1 2024-05-09T06:38:33.357 INFO:tasks.workunit.client.1.smithi193.stdout:8/711: creat d6/d1c/d1b/d5d/d84/d87/fd3 x:0 0 0 2024-05-09T06:38:33.358 INFO:tasks.workunit.client.1.smithi193.stdout:3/660: dread d1/db/d2f/d78/d86/ddc/f7c [0,4194304] 0 2024-05-09T06:38:33.360 INFO:tasks.workunit.client.1.smithi193.stdout:3/661: read d1/db/d2f/fc1 [201098,11256] 0 2024-05-09T06:38:33.362 INFO:tasks.workunit.client.1.smithi193.stdout:1/770: dwrite d4/f13 [4194304,4194304] 0 2024-05-09T06:38:33.362 INFO:tasks.workunit.client.1.smithi193.stdout:6/663: truncate d1/de/d11/d2d/da4/fd0 4528488 0 2024-05-09T06:38:33.364 INFO:tasks.workunit.client.1.smithi193.stdout:3/662: chown d1/db/d2f/d78/d86/ddc/d51/f29 1 1 2024-05-09T06:38:33.365 INFO:tasks.workunit.client.1.smithi193.stdout:1/771: read d4/d16/f5f [626197,62741] 0 2024-05-09T06:38:33.365 INFO:tasks.workunit.client.1.smithi193.stdout:3/663: chown d1/db/da1/fd1 26889 1 2024-05-09T06:38:33.365 INFO:tasks.workunit.client.1.smithi193.stdout:3/664: fdatasync d1/f8 0 2024-05-09T06:38:33.366 INFO:tasks.workunit.client.1.smithi193.stdout:1/772: chown d4/d16/d60/fb8 140320 1 2024-05-09T06:38:33.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/773: dwrite d4/d16/fb [0,4194304] 0 2024-05-09T06:38:33.373 INFO:tasks.workunit.client.1.smithi193.stdout:1/774: dread - d4/d16/feb zero size 2024-05-09T06:38:33.374 INFO:tasks.workunit.client.1.smithi193.stdout:1/775: chown d4/d16/f47 29716650 1 2024-05-09T06:38:33.375 INFO:tasks.workunit.client.1.smithi193.stdout:3/665: creat d1/db/da1/fe9 x:0 0 0 2024-05-09T06:38:33.377 INFO:tasks.workunit.client.1.smithi193.stdout:3/666: read d1/db/d2f/d78/d86/ddc/f7c [653428,72895] 0 2024-05-09T06:38:33.378 INFO:tasks.workunit.client.1.smithi193.stdout:8/712: rename d6/d1c/d1b/d29/f7c to d6/d1c/d1b/fd4 0 2024-05-09T06:38:33.379 INFO:tasks.workunit.client.1.smithi193.stdout:8/713: chown d6/d1c/d18/d2f 406071774 1 2024-05-09T06:38:33.380 INFO:tasks.workunit.client.1.smithi193.stdout:8/714: fdatasync d6/d1c/d1b/d29/d57/fb9 0 2024-05-09T06:38:33.381 INFO:tasks.workunit.client.1.smithi193.stdout:8/715: fsync d6/d1c/d1b/f75 0 2024-05-09T06:38:33.382 INFO:tasks.workunit.client.1.smithi193.stdout:1/776: unlink d4/d16/d60/d4f/lab 0 2024-05-09T06:38:33.383 INFO:tasks.workunit.client.1.smithi193.stdout:3/667: unlink d1/db/c1f 0 2024-05-09T06:38:33.390 INFO:tasks.workunit.client.1.smithi193.stdout:3/668: getdents d1/d44 0 2024-05-09T06:38:33.392 INFO:tasks.workunit.client.1.smithi193.stdout:3/669: rmdir d1/db/d2f/d78/d86/d94 39 2024-05-09T06:38:33.398 INFO:tasks.workunit.client.1.smithi193.stdout:3/670: dwrite d1/db/d59/fc2 [4194304,4194304] 0 2024-05-09T06:38:33.401 INFO:tasks.workunit.client.1.smithi193.stdout:3/671: unlink d1/db/d2f/d84/cc5 0 2024-05-09T06:38:33.403 INFO:tasks.workunit.client.1.smithi193.stdout:3/672: truncate d1/db/d59/d63/fb6 386501 0 2024-05-09T06:38:33.406 INFO:tasks.workunit.client.1.smithi193.stdout:6/664: write d1/d25/d2c/d76/d69/fd4 [4023857,70593] 0 2024-05-09T06:38:33.410 INFO:tasks.workunit.client.1.smithi193.stdout:1/777: write d4/d16/d6b/d89/f99 [3186810,60709] 0 2024-05-09T06:38:33.411 INFO:tasks.workunit.client.1.smithi193.stdout:8/716: truncate d6/f21 852715 0 2024-05-09T06:38:33.415 INFO:tasks.workunit.client.1.smithi193.stdout:6/665: creat d1/fee x:0 0 0 2024-05-09T06:38:33.418 INFO:tasks.workunit.client.1.smithi193.stdout:3/673: truncate d1/db/d59/f9f 1238354 0 2024-05-09T06:38:33.422 INFO:tasks.workunit.client.1.smithi193.stdout:8/717: rename d6/d1c/d18/f9c to d6/d1c/d18/fd5 0 2024-05-09T06:38:33.424 INFO:tasks.workunit.client.1.smithi193.stdout:6/666: symlink d1/d25/d2c/d81/d37/d4f/lef 0 2024-05-09T06:38:33.424 INFO:tasks.workunit.client.1.smithi193.stdout:1/778: dread d4/d16/d6b/d89/f99 [0,4194304] 0 2024-05-09T06:38:33.424 INFO:tasks.workunit.client.1.smithi193.stdout:8/718: write d6/d1c/d18/d66/d93/f6c [1594187,85608] 0 2024-05-09T06:38:33.426 INFO:tasks.workunit.client.1.smithi193.stdout:8/719: dread - d6/d1c/d1b/fd0 zero size 2024-05-09T06:38:33.427 INFO:tasks.workunit.client.1.smithi193.stdout:3/674: rename d1/db/d2f/d78/d86/ddc/f7c to d1/fea 0 2024-05-09T06:38:33.428 INFO:tasks.workunit.client.1.smithi193.stdout:6/667: creat d1/d25/ff0 x:0 0 0 2024-05-09T06:38:33.430 INFO:tasks.workunit.client.1.smithi193.stdout:1/779: creat d4/d16/d60/d4f/d72/d7f/fff x:0 0 0 2024-05-09T06:38:33.433 INFO:tasks.workunit.client.1.smithi193.stdout:3/675: mknod d1/db/d2f/d78/ceb 0 2024-05-09T06:38:33.435 INFO:tasks.workunit.client.1.smithi193.stdout:6/668: readlink d1/de/d11/d2d/d3a/l8f 0 2024-05-09T06:38:33.436 INFO:tasks.workunit.client.1.smithi193.stdout:1/780: rmdir d4/d16/d6b/d89/dc4 39 2024-05-09T06:38:33.436 INFO:tasks.workunit.client.1.smithi193.stdout:8/720: dwrite d6/d1c/d1b/d5d/d84/d87/da5/daa/fac [0,4194304] 0 2024-05-09T06:38:33.438 INFO:tasks.workunit.client.1.smithi193.stdout:3/676: creat d1/db/d2f/d78/fec x:0 0 0 2024-05-09T06:38:33.438 INFO:tasks.workunit.client.1.smithi193.stdout:1/781: write d4/d16/de/dca/fe3 [593376,83740] 0 2024-05-09T06:38:33.442 INFO:tasks.workunit.client.1.smithi193.stdout:8/721: write d6/d1c/d1b/d5d/d84/dc7/d7e/f8a [1886481,128387] 0 2024-05-09T06:38:33.456 INFO:tasks.workunit.client.1.smithi193.stdout:6/669: rename d1/de/d11/d2d/d3a/f55 to d1/d25/d2c/d3c/db1/ff1 0 2024-05-09T06:38:33.465 INFO:tasks.workunit.client.1.smithi193.stdout:0/666: sync 2024-05-09T06:38:33.465 INFO:tasks.workunit.client.1.smithi193.stdout:9/838: sync 2024-05-09T06:38:33.465 INFO:tasks.workunit.client.1.smithi193.stdout:4/789: sync 2024-05-09T06:38:33.465 INFO:tasks.workunit.client.1.smithi193.stdout:7/775: sync 2024-05-09T06:38:33.465 INFO:tasks.workunit.client.1.smithi193.stdout:2/774: sync 2024-05-09T06:38:33.468 INFO:tasks.workunit.client.1.smithi193.stdout:0/667: chown d18/d27/f34 91199 1 2024-05-09T06:38:33.470 INFO:tasks.workunit.client.1.smithi193.stdout:1/782: dread d4/d16/d17/d1a/d3a/d3f/f68 [0,4194304] 0 2024-05-09T06:38:33.471 INFO:tasks.workunit.client.1.smithi193.stdout:9/839: stat dd/d1e/dc2/d49/dbd/d5b/d8a/dad/df8/f10f 0 2024-05-09T06:38:33.472 INFO:tasks.workunit.client.1.smithi193.stdout:3/677: dwrite d1/db/d2f/d78/d86/ddc/d51/fa5 [0,4194304] 0 2024-05-09T06:38:33.472 INFO:tasks.workunit.client.1.smithi193.stdout:4/790: dread d1a/d67/d82/dec/d60/faa [0,4194304] 0 2024-05-09T06:38:33.473 INFO:tasks.workunit.client.1.smithi193.stdout:4/791: truncate d1a/fc9 314167 0 2024-05-09T06:38:33.501 INFO:tasks.workunit.client.1.smithi193.stdout:5/686: sync 2024-05-09T06:38:33.505 INFO:tasks.workunit.client.1.smithi193.stdout:6/670: fdatasync d1/de/d11/f1d 0 2024-05-09T06:38:33.509 INFO:tasks.workunit.client.1.smithi193.stdout:7/776: rename d6/dd/f22 to d6/dd/d70/f102 0 2024-05-09T06:38:33.513 INFO:tasks.workunit.client.1.smithi193.stdout:9/840: unlink dd/df/c6a 0 2024-05-09T06:38:33.516 INFO:tasks.workunit.client.1.smithi193.stdout:4/792: creat d1a/d22/d5d/ff1 x:0 0 0 2024-05-09T06:38:33.516 INFO:tasks.workunit.client.1.smithi193.stdout:8/722: link d6/d1c/d18/d2f/c5c d6/d1c/d1b/d29/d57/cd6 0 2024-05-09T06:38:33.520 INFO:tasks.workunit.client.1.smithi193.stdout:2/775: dread db/d25/d38/d5b/f88 [0,4194304] 0 2024-05-09T06:38:33.522 INFO:tasks.workunit.client.1.smithi193.stdout:6/671: mknod d1/d25/d4c/d9e/dc8/cf2 0 2024-05-09T06:38:33.526 INFO:tasks.workunit.client.1.smithi193.stdout:1/783: rename d4/d16/d17/d1a/d5b/fb7 to d4/d16/d17/dfc/f100 0 2024-05-09T06:38:33.529 INFO:tasks.workunit.client.1.smithi193.stdout:8/723: truncate d6/d1c/d53/f71 873488 0 2024-05-09T06:38:33.531 INFO:tasks.workunit.client.1.smithi193.stdout:6/672: chown d1/de/d11/d18/f29 28237122 1 2024-05-09T06:38:33.533 INFO:tasks.workunit.client.1.smithi193.stdout:8/724: chown d6/d1c/d18/dba/ld1 22306319 1 2024-05-09T06:38:33.534 INFO:tasks.workunit.client.1.smithi193.stdout:6/673: mkdir d1/d25/d4c/df3 0 2024-05-09T06:38:33.536 INFO:tasks.workunit.client.1.smithi193.stdout:6/674: fsync d1/d25/d2c/d3c/db1/d58/dc9/fe1 0 2024-05-09T06:38:33.538 INFO:tasks.workunit.client.1.smithi193.stdout:6/675: creat d1/d25/d4c/d9e/ff4 x:0 0 0 2024-05-09T06:38:33.539 INFO:tasks.workunit.client.1.smithi193.stdout:6/676: rmdir d1/de/d11/d18 39 2024-05-09T06:38:33.540 INFO:tasks.workunit.client.1.smithi193.stdout:8/725: link d6/d1c/d18/c28 d6/d1c/cd7 0 2024-05-09T06:38:33.543 INFO:tasks.workunit.client.1.smithi193.stdout:8/726: creat d6/d1c/d1b/d29/d57/fd8 x:0 0 0 2024-05-09T06:38:33.544 INFO:tasks.workunit.client.1.smithi193.stdout:6/677: creat d1/d25/d2c/d3c/db1/d58/ff5 x:0 0 0 2024-05-09T06:38:33.547 INFO:tasks.workunit.client.1.smithi193.stdout:6/678: mkdir d1/d25/d2c/d3c/db1/d58/dc9/df6 0 2024-05-09T06:38:33.547 INFO:tasks.workunit.client.1.smithi193.stdout:8/727: read d6/d1c/d18/d66/d93/fa6 [4193098,78700] 0 2024-05-09T06:38:33.550 INFO:tasks.workunit.client.1.smithi193.stdout:8/728: creat d6/fd9 x:0 0 0 2024-05-09T06:38:33.550 INFO:tasks.workunit.client.1.smithi193.stdout:3/678: write d1/d44/f47 [5013914,74625] 0 2024-05-09T06:38:33.552 INFO:tasks.workunit.client.1.smithi193.stdout:0/668: dwrite d18/d4c/d47/d54/f73 [0,4194304] 0 2024-05-09T06:38:33.552 INFO:tasks.workunit.client.1.smithi193.stdout:8/729: stat d6/d1c/d18/d2f/d6a/lca 0 2024-05-09T06:38:33.553 INFO:tasks.workunit.client.1.smithi193.stdout:5/687: write d0/d13/d2c/d37/d2b/d47/d66/d3a/f96 [248142,111895] 0 2024-05-09T06:38:33.555 INFO:tasks.workunit.client.1.smithi193.stdout:6/679: rename d1/d25/d2c/f75 to d1/d25/d2c/d3c/ff7 0 2024-05-09T06:38:33.557 INFO:tasks.workunit.client.1.smithi193.stdout:3/679: unlink d1/db/d2f/cd7 0 2024-05-09T06:38:33.558 INFO:tasks.workunit.client.1.smithi193.stdout:3/680: read d1/d44/d7d/f83 [2419981,76983] 0 2024-05-09T06:38:33.559 INFO:tasks.workunit.client.1.smithi193.stdout:6/680: dread d1/d25/d2c/d81/fb5 [0,4194304] 0 2024-05-09T06:38:33.562 INFO:tasks.workunit.client.1.smithi193.stdout:8/730: rename d6/d1c/d18/f1d to d6/d1c/d1b/d5d/d84/d87/da5/daa/fda 0 2024-05-09T06:38:33.564 INFO:tasks.workunit.client.1.smithi193.stdout:5/688: creat d0/d6d/dbd/dd0/fe3 x:0 0 0 2024-05-09T06:38:33.564 INFO:tasks.workunit.client.1.smithi193.stdout:9/841: write dd/df/d15/d8c/de6/d113/d71/fac [705393,18687] 0 2024-05-09T06:38:33.567 INFO:tasks.workunit.client.1.smithi193.stdout:1/784: write d4/d16/f10 [1291222,5488] 0 2024-05-09T06:38:33.571 INFO:tasks.workunit.client.1.smithi193.stdout:6/681: fsync d1/de/d11/d2d/d3a/f3b 0 2024-05-09T06:38:33.573 INFO:tasks.workunit.client.1.smithi193.stdout:8/731: creat d6/d1c/d1b/d3f/fdb x:0 0 0 2024-05-09T06:38:33.576 INFO:tasks.workunit.client.1.smithi193.stdout:9/842: readlink dd/d1e/d1f/d48/d55/d86/la5 0 2024-05-09T06:38:33.580 INFO:tasks.workunit.client.1.smithi193.stdout:9/843: dread dd/d1e/f88 [0,4194304] 0 2024-05-09T06:38:33.585 INFO:tasks.workunit.client.1.smithi193.stdout:6/682: creat d1/d25/d2c/d3c/db1/d58/dc9/df6/ff8 x:0 0 0 2024-05-09T06:38:33.585 INFO:tasks.workunit.client.1.smithi193.stdout:6/683: fdatasync d1/d25/d2c/fc4 0 2024-05-09T06:38:33.586 INFO:tasks.workunit.client.1.smithi193.stdout:8/732: creat d6/d1c/d53/fdc x:0 0 0 2024-05-09T06:38:33.587 INFO:tasks.workunit.client.1.smithi193.stdout:9/844: fdatasync dd/d1e/fcd 0 2024-05-09T06:38:33.590 INFO:tasks.workunit.client.1.smithi193.stdout:3/681: link d1/db/d59/cc9 d1/db/d59/ced 0 2024-05-09T06:38:33.592 INFO:tasks.workunit.client.1.smithi193.stdout:5/689: write d0/d13/d2c/d37/d2b/d47/d66/f15 [1185020,128333] 0 2024-05-09T06:38:33.592 INFO:tasks.workunit.client.1.smithi193.stdout:3/682: chown d1/d44/d7d/f83 650025449 1 2024-05-09T06:38:33.593 INFO:tasks.workunit.client.1.smithi193.stdout:3/683: chown d1/db/l36 126 1 2024-05-09T06:38:33.594 INFO:tasks.workunit.client.1.smithi193.stdout:3/684: chown d1/db/d59/fa9 15564806 1 2024-05-09T06:38:33.596 INFO:tasks.workunit.client.1.smithi193.stdout:6/684: fsync d1/de/d11/d2d/d3a/f3b 0 2024-05-09T06:38:33.596 INFO:tasks.workunit.client.1.smithi193.stdout:1/785: write d4/d3c/d3e/d7c/fbd [1006135,80430] 0 2024-05-09T06:38:33.599 INFO:tasks.workunit.client.1.smithi193.stdout:8/733: symlink d6/d1c/d18/d2f/d40/d5e/ldd 0 2024-05-09T06:38:33.600 INFO:tasks.workunit.client.1.smithi193.stdout:8/734: write d6/d1c/d18/d66/d93/d95/f9f [1689473,6430] 0 2024-05-09T06:38:33.602 INFO:tasks.workunit.client.1.smithi193.stdout:8/735: write d6/d1c/d18/f26 [7391544,63493] 0 2024-05-09T06:38:33.612 INFO:tasks.workunit.client.1.smithi193.stdout:5/690: rename d0/d13/d2c/d37/d2b/d47/d66/l24 to d0/d13/d2c/d37/d2b/d47/d60/d9a/le4 0 2024-05-09T06:38:33.617 INFO:tasks.workunit.client.1.smithi193.stdout:3/685: dwrite d1/db/d2f/d84/f9a [0,4194304] 0 2024-05-09T06:38:33.621 INFO:tasks.workunit.client.1.smithi193.stdout:8/736: rename d6/ccb to d6/d1c/d1b/d5d/d84/cde 0 2024-05-09T06:38:33.623 INFO:tasks.workunit.client.1.smithi193.stdout:5/691: dwrite d0/d13/d2c/d37/f11 [0,4194304] 0 2024-05-09T06:38:33.623 INFO:tasks.workunit.client.1.smithi193.stdout:6/685: truncate d1/d25/d2c/d81/d37/d4f/fb4 2213328 0 2024-05-09T06:38:33.627 INFO:tasks.workunit.client.1.smithi193.stdout:1/786: link d4/d16/d6b/d85/ddd/dfb/ffe d4/d16/d17/d1a/d3a/d3f/dd7/f101 0 2024-05-09T06:38:33.627 INFO:tasks.workunit.client.1.smithi193.stdout:9/845: dread dd/df/d15/d8c/de6/d113/d37/f7c [0,4194304] 0 2024-05-09T06:38:33.631 INFO:tasks.workunit.client.1.smithi193.stdout:9/846: write dd/d1e/dc2/d49/f51 [4273895,91499] 0 2024-05-09T06:38:33.631 INFO:tasks.workunit.client.1.smithi193.stdout:6/686: mknod d1/d25/d2c/d76/d69/cf9 0 2024-05-09T06:38:33.632 INFO:tasks.workunit.client.1.smithi193.stdout:5/692: truncate d0/d13/d2c/d37/d2b/d6a/d90/fb1 4284934 0 2024-05-09T06:38:33.634 INFO:tasks.workunit.client.1.smithi193.stdout:8/737: dwrite d6/d1c/d18/fc3 [0,4194304] 0 2024-05-09T06:38:33.637 INFO:tasks.workunit.client.1.smithi193.stdout:9/847: rmdir dd/df/d15/db7 39 2024-05-09T06:38:33.637 INFO:tasks.workunit.client.1.smithi193.stdout:5/693: write d0/d13/d2c/d37/d2b/d47/d66/d3a/f96 [1040939,82416] 0 2024-05-09T06:38:33.640 INFO:tasks.workunit.client.1.smithi193.stdout:6/687: truncate d1/d25/f30 5094319 0 2024-05-09T06:38:33.648 INFO:tasks.workunit.client.1.smithi193.stdout:3/686: truncate d1/db/da1/fd1 1963313 0 2024-05-09T06:38:33.650 INFO:tasks.workunit.client.1.smithi193.stdout:8/738: dwrite d6/d1c/d1b/d5d/f6d [4194304,4194304] 0 2024-05-09T06:38:33.657 INFO:tasks.workunit.client.1.smithi193.stdout:3/687: mknod d1/db/da1/cee 0 2024-05-09T06:38:33.658 INFO:tasks.workunit.client.1.smithi193.stdout:1/787: dwrite d4/d16/d6b/fe5 [0,4194304] 0 2024-05-09T06:38:33.661 INFO:tasks.workunit.client.1.smithi193.stdout:9/848: rename dd/d1e/dc2/d49/dbd/f8b to dd/d1e/d1f/f114 0 2024-05-09T06:38:33.662 INFO:tasks.workunit.client.1.smithi193.stdout:8/739: symlink d6/d1c/d1b/d5d/d84/d87/da5/ldf 0 2024-05-09T06:38:33.670 INFO:tasks.workunit.client.1.smithi193.stdout:9/849: mkdir dd/d1e/dc2/d49/dbd/d115 0 2024-05-09T06:38:33.671 INFO:tasks.workunit.client.1.smithi193.stdout:5/694: getdents d0/d13/d2c/d37/d2b/d47/d60/d9a 0 2024-05-09T06:38:33.672 INFO:tasks.workunit.client.1.smithi193.stdout:5/695: dread - d0/d13/d2c/d37/d2b/d47/d60/d9a/fdb zero size 2024-05-09T06:38:33.673 INFO:tasks.workunit.client.1.smithi193.stdout:8/740: mknod d6/d1c/d1b/d5d/d84/dc7/d7e/ce0 0 2024-05-09T06:38:33.674 INFO:tasks.workunit.client.1.smithi193.stdout:8/741: dread - d6/d1c/d1b/d29/d57/fb9 zero size 2024-05-09T06:38:33.676 INFO:tasks.workunit.client.1.smithi193.stdout:3/688: unlink d1/db/d2f/d78/d86/ddc/d51/c53 0 2024-05-09T06:38:33.678 INFO:tasks.workunit.client.1.smithi193.stdout:9/850: creat dd/d1e/dc2/d49/dbd/dab/f116 x:0 0 0 2024-05-09T06:38:33.679 INFO:tasks.workunit.client.1.smithi193.stdout:5/696: creat d0/d13/d2c/db6/dc6/fe5 x:0 0 0 2024-05-09T06:38:33.688 INFO:tasks.workunit.client.1.smithi193.stdout:6/688: truncate d1/d25/f30 3784511 0 2024-05-09T06:38:33.689 INFO:tasks.workunit.client.1.smithi193.stdout:3/689: dread d1/db/f19 [4194304,4194304] 0 2024-05-09T06:38:33.693 INFO:tasks.workunit.client.1.smithi193.stdout:1/788: getdents d4/d16/d6b/d89 0 2024-05-09T06:38:33.695 INFO:tasks.workunit.client.1.smithi193.stdout:9/851: truncate dd/d1e/dc2/d49/dbd/d5b/f5d 2377347 0 2024-05-09T06:38:33.699 INFO:tasks.workunit.client.1.smithi193.stdout:8/742: creat d6/d1c/d1b/d5d/d84/d87/fe1 x:0 0 0 2024-05-09T06:38:33.700 INFO:tasks.workunit.client.1.smithi193.stdout:9/852: dwrite dd/f29 [0,4194304] 0 2024-05-09T06:38:33.701 INFO:tasks.workunit.client.1.smithi193.stdout:9/853: readlink dd/d1e/dc2/d49/l58 0 2024-05-09T06:38:33.702 INFO:tasks.workunit.client.1.smithi193.stdout:9/854: stat dd/d1e/dc2/d49/dbd/l74 0 2024-05-09T06:38:33.709 INFO:tasks.workunit.client.1.smithi193.stdout:5/697: unlink d0/d13/d2c/da0/ca2 0 2024-05-09T06:38:33.711 INFO:tasks.workunit.client.1.smithi193.stdout:6/689: mknod d1/de/dbc/cfa 0 2024-05-09T06:38:33.711 INFO:tasks.workunit.client.1.smithi193.stdout:6/690: fdatasync d1/d25/ff0 0 2024-05-09T06:38:33.717 INFO:tasks.workunit.client.1.smithi193.stdout:9/855: unlink dd/df/f1d 0 2024-05-09T06:38:33.721 INFO:tasks.workunit.client.1.smithi193.stdout:5/698: dread - d0/d13/d2c/d37/d5b/d4e/d5d/f9c zero size 2024-05-09T06:38:33.725 INFO:tasks.workunit.client.1.smithi193.stdout:3/690: link d1/d44/d7d/f9c d1/d44/ddf/fef 0 2024-05-09T06:38:33.725 INFO:tasks.workunit.client.1.smithi193.stdout:3/691: fdatasync d1/db/da1/fe9 0 2024-05-09T06:38:33.727 INFO:tasks.workunit.client.1.smithi193.stdout:3/692: write d1/d44/d7d/fbb [574357,89107] 0 2024-05-09T06:38:33.727 INFO:tasks.workunit.client.1.smithi193.stdout:6/691: dread d1/d25/d2c/d76/f7d [0,4194304] 0 2024-05-09T06:38:33.728 INFO:tasks.workunit.client.1.smithi193.stdout:9/856: rmdir dd/d1e/dc2/d49/dbd/d5b/d8a/dad/df8 39 2024-05-09T06:38:33.728 INFO:tasks.workunit.client.1.smithi193.stdout:6/692: chown d1/de/d11/d20/fbf 5 1 2024-05-09T06:38:33.730 INFO:tasks.workunit.client.1.smithi193.stdout:1/789: link d4/d16/d60/d4f/d72/fef d4/d16/d6b/f102 0 2024-05-09T06:38:33.742 INFO:tasks.workunit.client.1.smithi193.stdout:1/790: rename d4/d16/de/l4e to d4/d16/de/dca/l103 0 2024-05-09T06:38:33.745 INFO:tasks.workunit.client.1.smithi193.stdout:3/693: dread d1/db/d1b/f70 [0,4194304] 0 2024-05-09T06:38:33.746 INFO:tasks.workunit.client.1.smithi193.stdout:3/694: write d1/d44/f47 [936229,111964] 0 2024-05-09T06:38:33.747 INFO:tasks.workunit.client.1.smithi193.stdout:6/693: fdatasync d1/f2 0 2024-05-09T06:38:33.754 INFO:tasks.workunit.client.1.smithi193.stdout:1/791: rename d4/d14/f2e to d4/d16/d6b/d6e/f104 0 2024-05-09T06:38:33.755 INFO:tasks.workunit.client.1.smithi193.stdout:5/699: creat d0/d13/d2c/d37/d5b/fe6 x:0 0 0 2024-05-09T06:38:33.760 INFO:tasks.workunit.client.1.smithi193.stdout:8/743: dwrite d6/d1c/d18/d66/f78 [0,4194304] 0 2024-05-09T06:38:33.778 INFO:tasks.workunit.client.1.smithi193.stdout:5/700: dwrite d0/d13/d2c/d37/d2b/d47/d66/d3a/fab [0,4194304] 0 2024-05-09T06:38:33.780 INFO:tasks.workunit.client.1.smithi193.stdout:5/701: write d0/d13/d2c/d4d/fd7 [747242,53841] 0 2024-05-09T06:38:33.782 INFO:tasks.workunit.client.1.smithi193.stdout:9/857: getdents dd/d1e/d1f/d61 0 2024-05-09T06:38:33.787 INFO:tasks.workunit.client.1.smithi193.stdout:8/744: symlink d6/d1c/d1b/d29/le2 0 2024-05-09T06:38:33.791 INFO:tasks.workunit.client.1.smithi193.stdout:1/792: rename d4/d16/d60/d4f/d72/f7a to d4/f105 0 2024-05-09T06:38:33.792 INFO:tasks.workunit.client.1.smithi193.stdout:8/745: mknod d6/d1c/d18/d66/ce3 0 2024-05-09T06:38:33.794 INFO:tasks.workunit.client.1.smithi193.stdout:1/793: fsync d4/d16/d60/d4f/d72/d7f/fb6 0 2024-05-09T06:38:33.796 INFO:tasks.workunit.client.1.smithi193.stdout:6/694: write d1/d25/d2c/d81/d37/d4f/fd3 [2257843,54623] 0 2024-05-09T06:38:33.801 INFO:tasks.workunit.client.1.smithi193.stdout:6/695: stat d1/d25/d4c/d9e/feb 0 2024-05-09T06:38:33.802 INFO:tasks.workunit.client.1.smithi193.stdout:8/746: dwrite d6/d1c/d1b/d3f/fdb [0,4194304] 0 2024-05-09T06:38:33.804 INFO:tasks.workunit.client.1.smithi193.stdout:3/695: dwrite d1/db/d2f/d78/d86/ddc/d51/fe0 [0,4194304] 0 2024-05-09T06:38:33.805 INFO:tasks.workunit.client.1.smithi193.stdout:6/696: read - d1/de/d11/d2d/d3a/d47/fa1 zero size 2024-05-09T06:38:33.815 INFO:tasks.workunit.client.1.smithi193.stdout:8/747: creat d6/d1c/d18/d66/db3/fe4 x:0 0 0 2024-05-09T06:38:33.821 INFO:tasks.workunit.client.1.smithi193.stdout:8/748: mknod d6/d1c/d1b/d29/d57/dc8/ce5 0 2024-05-09T06:38:33.822 INFO:tasks.workunit.client.1.smithi193.stdout:5/702: dread d0/d13/d2c/d37/d2b/d47/d66/f23 [0,4194304] 0 2024-05-09T06:38:33.824 INFO:tasks.workunit.client.1.smithi193.stdout:6/697: getdents d1/d25 0 2024-05-09T06:38:33.827 INFO:tasks.workunit.client.1.smithi193.stdout:5/703: rename d0/d13/d2c/d37/d2b/d47/d66/d3a/c65 to d0/d13/d2c/d4d/d67/ce7 0 2024-05-09T06:38:33.829 INFO:tasks.workunit.client.1.smithi193.stdout:3/696: getdents d1/db/d2f/d78/d86/d94 0 2024-05-09T06:38:33.830 INFO:tasks.workunit.client.1.smithi193.stdout:6/698: dread d1/d25/d2c/d76/d69/f88 [0,4194304] 0 2024-05-09T06:38:33.830 INFO:tasks.workunit.client.1.smithi193.stdout:8/749: creat d6/d1c/d1b/fe6 x:0 0 0 2024-05-09T06:38:33.831 INFO:tasks.workunit.client.1.smithi193.stdout:5/704: unlink d0/d13/d2c/d37/d74/f80 0 2024-05-09T06:38:33.832 INFO:tasks.workunit.client.1.smithi193.stdout:3/697: symlink d1/db/d2f/d84/lf0 0 2024-05-09T06:38:33.833 INFO:tasks.workunit.client.1.smithi193.stdout:3/698: readlink d1/db/l1e 0 2024-05-09T06:38:33.833 INFO:tasks.workunit.client.1.smithi193.stdout:6/699: creat d1/d25/d2c/d3c/ffb x:0 0 0 2024-05-09T06:38:33.834 INFO:tasks.workunit.client.1.smithi193.stdout:8/750: mknod d6/d1c/d18/d66/db3/ce7 0 2024-05-09T06:38:33.836 INFO:tasks.workunit.client.1.smithi193.stdout:6/700: symlink d1/de/d11/d2d/d3a/lfc 0 2024-05-09T06:38:33.837 INFO:tasks.workunit.client.1.smithi193.stdout:5/705: mkdir d0/d13/d2c/d37/d5b/de8 0 2024-05-09T06:38:33.840 INFO:tasks.workunit.client.1.smithi193.stdout:6/701: chown d1/de/d11/d18/d94/c98 31 1 2024-05-09T06:38:33.840 INFO:tasks.workunit.client.1.smithi193.stdout:5/706: write d0/d6d/fb0 [3802678,77888] 0 2024-05-09T06:38:33.846 INFO:tasks.workunit.client.1.smithi193.stdout:8/751: dwrite d6/d1c/d1b/d5d/d84/dc7/d7e/f8a [0,4194304] 0 2024-05-09T06:38:33.848 INFO:tasks.workunit.client.1.smithi193.stdout:9/858: dread dd/d1e/dc2/d49/dbd/d5b/f5d [0,4194304] 0 2024-05-09T06:38:33.852 INFO:tasks.workunit.client.1.smithi193.stdout:8/752: rename d6/d1c/d18/d66/c86 to d6/d1c/d18/dba/ce8 0 2024-05-09T06:38:33.853 INFO:tasks.workunit.client.1.smithi193.stdout:1/794: truncate d4/d16/d17/d1a/d3a/d3f/f68 3343254 0 2024-05-09T06:38:33.858 INFO:tasks.workunit.client.1.smithi193.stdout:1/795: rmdir d4/d16/d17/dfc 39 2024-05-09T06:38:33.860 INFO:tasks.workunit.client.1.smithi193.stdout:8/753: stat d6/d1c/d1b/d5d/d84/d87/cbc 0 2024-05-09T06:38:33.861 INFO:tasks.workunit.client.1.smithi193.stdout:8/754: fdatasync d6/d1c/d18/d66/d93/fad 0 2024-05-09T06:38:33.867 INFO:tasks.workunit.client.1.smithi193.stdout:8/755: dread d6/d1c/d1b/d5d/d84/d87/da5/daa/fac [0,4194304] 0 2024-05-09T06:38:33.868 INFO:tasks.workunit.client.1.smithi193.stdout:8/756: fdatasync d6/d1c/d18/d66/d93/fad 0 2024-05-09T06:38:33.871 INFO:tasks.workunit.client.1.smithi193.stdout:8/757: stat d6/d1c/d27 0 2024-05-09T06:38:33.872 INFO:tasks.workunit.client.1.smithi193.stdout:8/758: fdatasync d6/fd9 0 2024-05-09T06:38:33.873 INFO:tasks.workunit.client.1.smithi193.stdout:8/759: dread - d6/d1c/d1b/fd2 zero size 2024-05-09T06:38:33.882 INFO:tasks.workunit.client.1.smithi193.stdout:3/699: dread d1/f24 [0,4194304] 0 2024-05-09T06:38:33.884 INFO:tasks.workunit.client.1.smithi193.stdout:8/760: dread d6/d1c/d18/d66/d93/d95/f9f [0,4194304] 0 2024-05-09T06:38:33.886 INFO:tasks.workunit.client.1.smithi193.stdout:9/859: dread dd/df/d15/f9e [0,4194304] 0 2024-05-09T06:38:33.892 INFO:tasks.workunit.client.1.smithi193.stdout:9/860: truncate dd/df/d15/d8c/de6/d113/f4b 1000231 0 2024-05-09T06:38:33.894 INFO:tasks.workunit.client.1.smithi193.stdout:9/861: getdents dd/d1e/d1f/d103 0 2024-05-09T06:38:33.898 INFO:tasks.workunit.client.1.smithi193.stdout:9/862: rename dd/d1e/d1f/d48/d55/d41/d68 to dd/d1e/dc2/d49/dbd/d115/d117 0 2024-05-09T06:38:33.903 INFO:tasks.workunit.client.1.smithi193.stdout:6/702: truncate d1/d25/d2c/d3c/db1/dea/fd9 3520819 0 2024-05-09T06:38:33.906 INFO:tasks.workunit.client.1.smithi193.stdout:6/703: chown d1/de/d11/d18/c56 231181 1 2024-05-09T06:38:33.908 INFO:tasks.workunit.client.1.smithi193.stdout:1/796: dwrite d4/d16/d17/d1a/fbb [0,4194304] 0 2024-05-09T06:38:33.911 INFO:tasks.workunit.client.1.smithi193.stdout:6/704: rename d1/de/d11/f73 to d1/d25/d2c/d81/d37/ffd 0 2024-05-09T06:38:33.912 INFO:tasks.workunit.client.1.smithi193.stdout:3/700: write d1/d44/f96 [2208959,104459] 0 2024-05-09T06:38:33.915 INFO:tasks.workunit.client.1.smithi193.stdout:6/705: fdatasync d1/f42 0 2024-05-09T06:38:33.916 INFO:tasks.workunit.client.1.smithi193.stdout:8/761: truncate d6/d1c/d1b/d29/d57/f8b 620516 0 2024-05-09T06:38:33.922 INFO:tasks.workunit.client.1.smithi193.stdout:9/863: write dd/d1e/fcd [133430,111380] 0 2024-05-09T06:38:33.928 INFO:tasks.workunit.client.1.smithi193.stdout:6/706: rename d1/d25/d2c/d76/fc0 to d1/de/d11/d18/d94/dac/ffe 0 2024-05-09T06:38:33.932 INFO:tasks.workunit.client.1.smithi193.stdout:9/864: unlink dd/df/d31/l3d 0 2024-05-09T06:38:33.933 INFO:tasks.workunit.client.1.smithi193.stdout:9/865: readlink dd/d1e/dc2/d49/dbd/d5b/l99 0 2024-05-09T06:38:33.935 INFO:tasks.workunit.client.1.smithi193.stdout:9/866: dread dd/d1e/d1f/d61/f85 [0,4194304] 0 2024-05-09T06:38:33.936 INFO:tasks.workunit.client.1.smithi193.stdout:1/797: write d4/d16/d17/d1a/d3a/d3f/f65 [1591357,88537] 0 2024-05-09T06:38:33.937 INFO:tasks.workunit.client.1.smithi193.stdout:9/867: write dd/df/d15/d8c/de6/d113/f62 [2504777,10898] 0 2024-05-09T06:38:33.938 INFO:tasks.workunit.client.1.smithi193.stdout:8/762: creat d6/d1c/d18/d2f/fe9 x:0 0 0 2024-05-09T06:38:33.938 INFO:tasks.workunit.client.1.smithi193.stdout:3/701: write d1/db/d2f/d78/d86/ddc/d51/f65 [4261407,44608] 0 2024-05-09T06:38:33.940 INFO:tasks.workunit.client.1.smithi193.stdout:6/707: rename d1/c51 to d1/d25/d4c/cff 0 2024-05-09T06:38:33.944 INFO:tasks.workunit.client.1.smithi193.stdout:3/702: rename d1/db/d2f/d84/c99 to d1/d44/ddf/cf1 0 2024-05-09T06:38:33.950 INFO:tasks.workunit.client.1.smithi193.stdout:1/798: creat d4/d16/d6b/d85/ddd/dfb/f106 x:0 0 0 2024-05-09T06:38:33.950 INFO:tasks.workunit.client.1.smithi193.stdout:1/799: stat d4/d3c 0 2024-05-09T06:38:33.953 INFO:tasks.workunit.client.1.smithi193.stdout:3/703: fsync d1/db/f6e 0 2024-05-09T06:38:33.955 INFO:tasks.workunit.client.1.smithi193.stdout:9/868: rename dd/f11 to dd/d1e/dc2/d49/dbd/d5b/d8a/dad/df8/f118 0 2024-05-09T06:38:33.963 INFO:tasks.workunit.client.1.smithi193.stdout:8/763: creat d6/d1c/d1b/d5d/d84/d87/fea x:0 0 0 2024-05-09T06:38:33.964 INFO:tasks.workunit.client.1.smithi193.stdout:1/800: read d4/f87 [645886,81517] 0 2024-05-09T06:38:33.969 INFO:tasks.workunit.client.1.smithi193.stdout:6/708: write d1/de/d11/d2d/f34 [1841218,76294] 0 2024-05-09T06:38:33.972 INFO:tasks.workunit.client.1.smithi193.stdout:8/764: rename d6/d1c/d18/d66/d93/f8c to d6/d1c/d53/feb 0 2024-05-09T06:38:33.973 INFO:tasks.workunit.client.1.smithi193.stdout:1/801: symlink d4/d16/d17/d1a/d3a/d3f/dd7/ddb/l107 0 2024-05-09T06:38:33.978 INFO:tasks.workunit.client.1.smithi193.stdout:9/869: mkdir dd/df/d15/d8c/de6/d113/d119 0 2024-05-09T06:38:33.979 INFO:tasks.workunit.client.1.smithi193.stdout:9/870: stat dd/f82 0 2024-05-09T06:38:33.979 INFO:tasks.workunit.client.1.smithi193.stdout:6/709: fdatasync d1/d25/d2c/fb9 0 2024-05-09T06:38:33.983 INFO:tasks.workunit.client.1.smithi193.stdout:1/802: creat d4/d16/d17/d1a/d3a/d3f/dd7/f108 x:0 0 0 2024-05-09T06:38:33.986 INFO:tasks.workunit.client.1.smithi193.stdout:3/704: creat d1/db/d2f/d78/d86/ddc/ff2 x:0 0 0 2024-05-09T06:38:33.988 INFO:tasks.workunit.client.1.smithi193.stdout:9/871: rmdir dd/d1e/dc2/d49/dbd/dab 39 2024-05-09T06:38:33.989 INFO:tasks.workunit.client.1.smithi193.stdout:9/872: write dd/d1e/fd2 [257133,54210] 0 2024-05-09T06:38:33.992 INFO:tasks.workunit.client.1.smithi193.stdout:3/705: dwrite d1/db/d2f/d78/d86/ddc/fe1 [0,4194304] 0 2024-05-09T06:38:33.995 INFO:tasks.workunit.client.1.smithi193.stdout:8/765: symlink d6/d1c/d18/d2f/lec 0 2024-05-09T06:38:33.999 INFO:tasks.workunit.client.1.smithi193.stdout:9/873: dwrite dd/d1e/d1f/d48/d55/ffd [0,4194304] 0 2024-05-09T06:38:34.005 INFO:tasks.workunit.client.1.smithi193.stdout:1/803: write d4/d76/f80 [791465,60896] 0 2024-05-09T06:38:34.007 INFO:tasks.workunit.client.1.smithi193.stdout:1/804: dread - d4/d16/d60/d4f/d72/fa1 zero size 2024-05-09T06:38:34.007 INFO:tasks.workunit.client.1.smithi193.stdout:8/766: rename d6/d1c/d18/d66/d93/d95/l97 to d6/d1c/d18/d66/db3/led 0 2024-05-09T06:38:34.010 INFO:tasks.workunit.client.1.smithi193.stdout:1/805: mknod d4/d16/d6b/d85/c109 0 2024-05-09T06:38:34.011 INFO:tasks.workunit.client.1.smithi193.stdout:8/767: creat d6/d1c/d53/fee x:0 0 0 2024-05-09T06:38:34.012 INFO:tasks.workunit.client.1.smithi193.stdout:8/768: write d6/d1c/d18/f74 [5005571,123615] 0 2024-05-09T06:38:34.013 INFO:tasks.workunit.client.1.smithi193.stdout:3/706: write d1/db/f58 [4750143,60449] 0 2024-05-09T06:38:34.014 INFO:tasks.workunit.client.1.smithi193.stdout:8/769: truncate d6/d1c/d18/d2f/d40/fbb 695705 0 2024-05-09T06:38:34.019 INFO:tasks.workunit.client.1.smithi193.stdout:3/707: truncate d1/db/d59/f9f 2243797 0 2024-05-09T06:38:34.021 INFO:tasks.workunit.client.1.smithi193.stdout:3/708: chown d1/db/d2f/d78/d86/ddc/d51/f65 954727320 1 2024-05-09T06:38:34.022 INFO:tasks.workunit.client.1.smithi193.stdout:8/770: dwrite d6/d1c/d18/d66/f78 [0,4194304] 0 2024-05-09T06:38:34.038 INFO:tasks.workunit.client.1.smithi193.stdout:1/806: dread d4/d3c/d3e/f5c [0,4194304] 0 2024-05-09T06:38:34.171 INFO:tasks.workunit.client.1.smithi193.stdout:3/709: write d1/db/d59/fa9 [700450,113051] 0 2024-05-09T06:38:34.171 INFO:tasks.workunit.client.1.smithi193.stdout:8/771: link d6/d1c/d18/l2a d6/d1c/d27/dbd/lef 0 2024-05-09T06:38:34.173 INFO:tasks.workunit.client.1.smithi193.stdout:8/772: write d6/d1c/d1b/d5d/d84/dc7/d7e/fb7 [82049,105968] 0 2024-05-09T06:38:34.176 INFO:tasks.workunit.client.1.smithi193.stdout:3/710: creat d1/db/d2f/d78/d86/ddc/d51/ff3 x:0 0 0 2024-05-09T06:38:34.179 INFO:tasks.workunit.client.1.smithi193.stdout:8/773: fsync d6/d1c/d27/f4d 0 2024-05-09T06:38:34.180 INFO:tasks.workunit.client.1.smithi193.stdout:3/711: dread d1/d44/d7d/fbb [0,4194304] 0 2024-05-09T06:38:34.181 INFO:tasks.workunit.client.1.smithi193.stdout:3/712: dread - d1/d44/d7d/fc3 zero size 2024-05-09T06:38:34.187 INFO:tasks.workunit.client.1.smithi193.stdout:1/807: write d4/d16/f9b [409194,4602] 0 2024-05-09T06:38:34.188 INFO:tasks.workunit.client.1.smithi193.stdout:3/713: rmdir d1/db/d59/d63 39 2024-05-09T06:38:34.190 INFO:tasks.workunit.client.1.smithi193.stdout:3/714: mkdir d1/db/d2f/d84/d97/df4 0 2024-05-09T06:38:34.195 INFO:tasks.workunit.client.1.smithi193.stdout:8/774: getdents d6/d1c/d18/d2f 0 2024-05-09T06:38:34.197 INFO:tasks.workunit.client.1.smithi193.stdout:3/715: getdents d1/db/d2f 0 2024-05-09T06:38:34.197 INFO:tasks.workunit.client.1.smithi193.stdout:8/775: fsync d6/d1c/d1b/d5d/d84/d87/da5/daa/fda 0 2024-05-09T06:38:34.200 INFO:tasks.workunit.client.1.smithi193.stdout:8/776: mknod d6/d1c/d1b/d29/d57/cf0 0 2024-05-09T06:38:34.202 INFO:tasks.workunit.client.1.smithi193.stdout:3/716: write d1/f1d [1528947,82996] 0 2024-05-09T06:38:34.202 INFO:tasks.workunit.client.1.smithi193.stdout:8/777: dread - d6/d1c/d53/feb zero size 2024-05-09T06:38:34.204 INFO:tasks.workunit.client.1.smithi193.stdout:8/778: chown d6/d1c/d18/d2f/d40/d5e/ldd 322973174 1 2024-05-09T06:38:34.208 INFO:tasks.workunit.client.1.smithi193.stdout:8/779: getdents d6/d1c/d1b 0 2024-05-09T06:38:34.213 INFO:tasks.workunit.client.1.smithi193.stdout:8/780: link d6/d1c/d18/c5a d6/d1c/d1b/d5d/cf1 0 2024-05-09T06:38:34.216 INFO:tasks.workunit.client.1.smithi193.stdout:8/781: dread d6/d1c/d18/dba/fc0 [0,4194304] 0 2024-05-09T06:38:34.217 INFO:tasks.workunit.client.1.smithi193.stdout:7/777: sync 2024-05-09T06:38:34.218 INFO:tasks.workunit.client.1.smithi193.stdout:8/782: truncate d6/d1c/d53/fee 805350 0 2024-05-09T06:38:34.218 INFO:tasks.workunit.client.1.smithi193.stdout:7/778: read - d6/d8/d1a/d36/d4d/d60/fd8 zero size 2024-05-09T06:38:34.222 INFO:tasks.workunit.client.1.smithi193.stdout:8/783: symlink d6/d1c/lf2 0 2024-05-09T06:38:34.228 INFO:tasks.workunit.client.1.smithi193.stdout:8/784: write d6/d1c/d1b/d3f/f96 [392099,119043] 0 2024-05-09T06:38:34.232 INFO:tasks.workunit.client.1.smithi193.stdout:8/785: creat d6/d1c/d18/d2f/ff3 x:0 0 0 2024-05-09T06:38:34.233 INFO:tasks.workunit.client.1.smithi193.stdout:7/779: dwrite d6/d8/d1a/f3c [0,4194304] 0 2024-05-09T06:38:34.233 INFO:tasks.workunit.client.1.smithi193.stdout:8/786: chown d6/d1c/d53/fdc 44235 1 2024-05-09T06:38:34.239 INFO:tasks.workunit.client.1.smithi193.stdout:8/787: rmdir d6/d1c/d53 39 2024-05-09T06:38:34.242 INFO:tasks.workunit.client.1.smithi193.stdout:8/788: chown d6/f21 67 1 2024-05-09T06:38:34.245 INFO:tasks.workunit.client.1.smithi193.stdout:8/789: read - d6/d1c/d18/d2f/ff3 zero size 2024-05-09T06:38:34.245 INFO:tasks.workunit.client.1.smithi193.stdout:7/780: getdents d6/dd/d7e 0 2024-05-09T06:38:34.250 INFO:tasks.workunit.client.1.smithi193.stdout:8/790: link d6/d1c/d18/d66/d93/cab d6/d1c/d1b/cf4 0 2024-05-09T06:38:34.252 INFO:tasks.workunit.client.1.smithi193.stdout:7/781: getdents d6/d8/d1a/d36/d4d/d60 0 2024-05-09T06:38:34.252 INFO:tasks.workunit.client.1.smithi193.stdout:7/782: chown d6/d8/d29/db9 539228978 1 2024-05-09T06:38:34.255 INFO:tasks.workunit.client.1.smithi193.stdout:8/791: creat d6/d1c/ff5 x:0 0 0 2024-05-09T06:38:34.258 INFO:tasks.workunit.client.1.smithi193.stdout:7/783: dwrite d6/d8/d1a/d36/f50 [0,4194304] 0 2024-05-09T06:38:34.259 INFO:tasks.workunit.client.1.smithi193.stdout:8/792: write d6/d1c/d53/f81 [6088139,94070] 0 2024-05-09T06:38:34.263 INFO:tasks.workunit.client.1.smithi193.stdout:8/793: mknod d6/d1c/d18/d66/d93/cf6 0 2024-05-09T06:38:34.264 INFO:tasks.workunit.client.1.smithi193.stdout:7/784: truncate d6/dd/d53/f58 306475 0 2024-05-09T06:38:34.272 INFO:tasks.workunit.client.1.smithi193.stdout:7/785: dwrite d6/d17/d7d/daa/fba [0,4194304] 0 2024-05-09T06:38:34.273 INFO:tasks.workunit.client.1.smithi193.stdout:8/794: rename d6/d1c/d1b/d5d/f6d to d6/d1c/d18/d2f/d6a/ff7 0 2024-05-09T06:38:34.279 INFO:tasks.workunit.client.1.smithi193.stdout:7/786: creat d6/dd/d5e/d6b/f103 x:0 0 0 2024-05-09T06:38:34.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/795: fsync d6/d1c/d18/fd5 0 2024-05-09T06:38:34.313 INFO:tasks.workunit.client.1.smithi193.stdout:4/793: sync 2024-05-09T06:38:34.313 INFO:tasks.workunit.client.1.smithi193.stdout:2/776: sync 2024-05-09T06:38:34.319 INFO:tasks.workunit.client.1.smithi193.stdout:2/777: dwrite db/def/dfd/fd0 [0,4194304] 0 2024-05-09T06:38:34.357 INFO:tasks.workunit.client.1.smithi193.stdout:4/794: fsync d1a/d6d/fdc 0 2024-05-09T06:38:34.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/778: mkdir db/df/d54/d63/dca/d102 0 2024-05-09T06:38:34.362 INFO:tasks.workunit.client.1.smithi193.stdout:8/796: symlink d6/d1c/d27/dbd/lf8 0 2024-05-09T06:38:34.366 INFO:tasks.workunit.client.1.smithi193.stdout:7/787: dwrite d6/d8/d1a/d36/d4d/f64 [0,4194304] 0 2024-05-09T06:38:34.367 INFO:tasks.workunit.client.1.smithi193.stdout:7/788: fsync d6/d8/f55 0 2024-05-09T06:38:34.369 INFO:tasks.workunit.client.1.smithi193.stdout:4/795: fdatasync d1a/d24/d73/d94/faf 0 2024-05-09T06:38:34.370 INFO:tasks.workunit.client.1.smithi193.stdout:2/779: symlink db/def/dfd/l103 0 2024-05-09T06:38:34.373 INFO:tasks.workunit.client.1.smithi193.stdout:8/797: truncate d6/d1c/d18/d2f/d40/fbb 1040990 0 2024-05-09T06:38:34.378 INFO:tasks.workunit.client.1.smithi193.stdout:2/780: unlink db/def/dfd/d80/f24 0 2024-05-09T06:38:34.379 INFO:tasks.workunit.client.1.smithi193.stdout:7/789: dwrite d6/d8/d1a/d36/dd0/fec [0,4194304] 0 2024-05-09T06:38:34.380 INFO:tasks.workunit.client.1.smithi193.stdout:8/798: creat d6/d1c/d53/ff9 x:0 0 0 2024-05-09T06:38:34.382 INFO:tasks.workunit.client.1.smithi193.stdout:8/799: fdatasync d6/fd 0 2024-05-09T06:38:34.383 INFO:tasks.workunit.client.1.smithi193.stdout:8/800: chown d6/d1c/d18/d66/l72 118961811 1 2024-05-09T06:38:34.383 INFO:tasks.workunit.client.1.smithi193.stdout:2/781: mkdir db/d25/d38/d40/db2/dcb/d104 0 2024-05-09T06:38:34.391 INFO:tasks.workunit.client.1.smithi193.stdout:4/796: write d1a/d67/d82/dec/d60/f95 [1031732,93710] 0 2024-05-09T06:38:34.392 INFO:tasks.workunit.client.1.smithi193.stdout:8/801: mkdir d6/d1c/d1b/d3f/dfa 0 2024-05-09T06:38:34.394 INFO:tasks.workunit.client.1.smithi193.stdout:2/782: symlink db/d25/d38/l105 0 2024-05-09T06:38:34.398 INFO:tasks.workunit.client.1.smithi193.stdout:8/802: mknod d6/d1c/d1b/d5d/d84/dc7/cfb 0 2024-05-09T06:38:34.399 INFO:tasks.workunit.client.1.smithi193.stdout:2/783: mkdir db/df/d106 0 2024-05-09T06:38:34.400 INFO:tasks.workunit.client.1.smithi193.stdout:4/797: creat d1a/ff2 x:0 0 0 2024-05-09T06:38:34.402 INFO:tasks.workunit.client.1.smithi193.stdout:8/803: creat d6/d1c/d1b/d29/d57/ffc x:0 0 0 2024-05-09T06:38:34.405 INFO:tasks.workunit.client.1.smithi193.stdout:7/790: dread d6/d8/d29/dc8/fda [0,4194304] 0 2024-05-09T06:38:34.406 INFO:tasks.workunit.client.1.smithi193.stdout:2/784: getdents db/def/dfd/d80/d60/d7d/db3 0 2024-05-09T06:38:34.407 INFO:tasks.workunit.client.1.smithi193.stdout:2/785: stat db/df/d54/dba/ff8 0 2024-05-09T06:38:34.409 INFO:tasks.workunit.client.1.smithi193.stdout:8/804: dwrite d6/d1c/d18/d2f/ff3 [0,4194304] 0 2024-05-09T06:38:34.411 INFO:tasks.workunit.client.1.smithi193.stdout:8/805: write d6/d1c/d18/d66/fb0 [7844,47910] 0 2024-05-09T06:38:34.414 INFO:tasks.workunit.client.1.smithi193.stdout:2/786: rmdir db/d25/d38 39 2024-05-09T06:38:34.423 INFO:tasks.workunit.client.1.smithi193.stdout:4/798: dwrite d1a/d67/d82/dec/f77 [0,4194304] 0 2024-05-09T06:38:34.424 INFO:tasks.workunit.client.1.smithi193.stdout:7/791: rmdir d6/d8/d29/d2f/dfc 0 2024-05-09T06:38:34.425 INFO:tasks.workunit.client.1.smithi193.stdout:7/792: truncate d6/d8/fc9 369737 0 2024-05-09T06:38:34.426 INFO:tasks.workunit.client.1.smithi193.stdout:8/806: mkdir d6/d1c/d1b/dfd 0 2024-05-09T06:38:34.427 INFO:tasks.workunit.client.1.smithi193.stdout:2/787: symlink db/df/d54/d63/dca/d102/l107 0 2024-05-09T06:38:34.429 INFO:tasks.workunit.client.1.smithi193.stdout:4/799: creat d1a/d24/d73/dcc/ff3 x:0 0 0 2024-05-09T06:38:34.435 INFO:tasks.workunit.client.1.smithi193.stdout:2/788: write db/d25/d38/d40/db2/dcb/fdc [1992979,115929] 0 2024-05-09T06:38:34.437 INFO:tasks.workunit.client.1.smithi193.stdout:4/800: symlink d1a/d1f/lf4 0 2024-05-09T06:38:34.437 INFO:tasks.workunit.client.1.smithi193.stdout:4/801: dread - d1a/d24/fbf zero size 2024-05-09T06:38:34.438 INFO:tasks.workunit.client.1.smithi193.stdout:4/802: readlink d1a/d24/dd0/le2 0 2024-05-09T06:38:34.439 INFO:tasks.workunit.client.1.smithi193.stdout:8/807: unlink d6/d1c/d18/d2f/f39 0 2024-05-09T06:38:34.441 INFO:tasks.workunit.client.1.smithi193.stdout:7/793: creat d6/d8/f104 x:0 0 0 2024-05-09T06:38:34.443 INFO:tasks.workunit.client.1.smithi193.stdout:2/789: dread db/d25/d38/fdd [0,4194304] 0 2024-05-09T06:38:34.446 INFO:tasks.workunit.client.1.smithi193.stdout:7/794: dread d6/d8/d1a/d36/f50 [0,4194304] 0 2024-05-09T06:38:34.448 INFO:tasks.workunit.client.1.smithi193.stdout:7/795: fdatasync d6/d8/d1a/d36/d4d/d74/f7c 0 2024-05-09T06:38:34.450 INFO:tasks.workunit.client.1.smithi193.stdout:2/790: dwrite db/def/dfd/d80/d60/d7d/f7e [0,4194304] 0 2024-05-09T06:38:34.455 INFO:tasks.workunit.client.1.smithi193.stdout:8/808: rmdir d6/d1c/d18/d2f 39 2024-05-09T06:38:34.458 INFO:tasks.workunit.client.1.smithi193.stdout:2/791: creat db/def/dfd/d80/d60/d7d/db3/f108 x:0 0 0 2024-05-09T06:38:34.459 INFO:tasks.workunit.client.1.smithi193.stdout:2/792: fsync db/d25/d38/fe0 0 2024-05-09T06:38:34.459 INFO:tasks.workunit.client.1.smithi193.stdout:7/796: mknod d6/d8/d29/df9/c105 0 2024-05-09T06:38:34.460 INFO:tasks.workunit.client.1.smithi193.stdout:2/793: write db/def/dfd/d80/f46 [353828,21342] 0 2024-05-09T06:38:34.461 INFO:tasks.workunit.client.1.smithi193.stdout:8/809: read - d6/d1c/d1b/fa7 zero size 2024-05-09T06:38:34.464 INFO:tasks.workunit.client.1.smithi193.stdout:0/669: sync 2024-05-09T06:38:34.472 INFO:tasks.workunit.client.1.smithi193.stdout:4/803: dwrite d1a/d22/d43/fd4 [0,4194304] 0 2024-05-09T06:38:34.479 INFO:tasks.workunit.client.1.smithi193.stdout:7/797: dwrite d6/d8/d1a/fa4 [0,4194304] 0 2024-05-09T06:38:34.480 INFO:tasks.workunit.client.1.smithi193.stdout:7/798: readlink d6/d8/d1a/d36/d4d/l56 0 2024-05-09T06:38:34.486 INFO:tasks.workunit.client.1.smithi193.stdout:7/799: dwrite d6/d17/d99/fef [0,4194304] 0 2024-05-09T06:38:34.499 INFO:tasks.workunit.client.1.smithi193.stdout:2/794: creat db/df/d54/d63/dca/f109 x:0 0 0 2024-05-09T06:38:34.502 INFO:tasks.workunit.client.1.smithi193.stdout:0/670: mkdir d18/d4c/d64/d66/d9d/da5/dd0 0 2024-05-09T06:38:34.505 INFO:tasks.workunit.client.1.smithi193.stdout:8/810: rename d6/d1c/d1b/d5d/d84/dc7/d7e/fb7 to d6/d1c/d1b/d5d/d84/d87/da5/daa/ffe 0 2024-05-09T06:38:34.507 INFO:tasks.workunit.client.1.smithi193.stdout:4/804: chown d1a/c7f 713 1 2024-05-09T06:38:34.507 INFO:tasks.workunit.client.1.smithi193.stdout:4/805: dread - d1a/d24/d73/d94/fb4 zero size 2024-05-09T06:38:34.507 INFO:tasks.workunit.client.1.smithi193.stdout:4/806: chown d1a/f65 24 1 2024-05-09T06:38:34.513 INFO:tasks.workunit.client.1.smithi193.stdout:8/811: rename d6/d1c/d18/d66/d93/f6c to d6/d1c/d18/d66/fff 0 2024-05-09T06:38:34.515 INFO:tasks.workunit.client.1.smithi193.stdout:8/812: truncate d6/d1c/d1b/d29/d57/fb9 9104 0 2024-05-09T06:38:34.519 INFO:tasks.workunit.client.1.smithi193.stdout:4/807: dwrite d1a/d24/d73/d94/fae [0,4194304] 0 2024-05-09T06:38:34.520 INFO:tasks.workunit.client.1.smithi193.stdout:4/808: chown d1a/l38 6749887 1 2024-05-09T06:38:34.522 INFO:tasks.workunit.client.1.smithi193.stdout:8/813: dwrite d6/d1c/d1b/d29/f91 [0,4194304] 0 2024-05-09T06:38:34.523 INFO:tasks.workunit.client.1.smithi193.stdout:8/814: chown d6/d1c/d1b/d5d/d84 25967 1 2024-05-09T06:38:34.525 INFO:tasks.workunit.client.1.smithi193.stdout:4/809: mknod d1a/d1f/cf5 0 2024-05-09T06:38:34.528 INFO:tasks.workunit.client.1.smithi193.stdout:8/815: creat d6/d1c/d1b/d29/d57/dc8/f100 x:0 0 0 2024-05-09T06:38:34.531 INFO:tasks.workunit.client.1.smithi193.stdout:8/816: symlink d6/d1c/d1b/d5d/d84/d87/da5/l101 0 2024-05-09T06:38:34.532 INFO:tasks.workunit.client.1.smithi193.stdout:8/817: chown d6/d1c/d1b/dfd 260482 1 2024-05-09T06:38:34.539 INFO:tasks.workunit.client.1.smithi193.stdout:2/795: write db/d25/d38/d5b/fc1 [377274,5868] 0 2024-05-09T06:38:34.542 INFO:tasks.workunit.client.1.smithi193.stdout:2/796: mkdir db/d25/d38/d40/db2/d10a 0 2024-05-09T06:38:34.543 INFO:tasks.workunit.client.1.smithi193.stdout:8/818: symlink d6/d1c/d18/d2f/d40/l102 0 2024-05-09T06:38:34.543 INFO:tasks.workunit.client.1.smithi193.stdout:2/797: stat db/df/c1d 0 2024-05-09T06:38:34.546 INFO:tasks.workunit.client.1.smithi193.stdout:0/671: dwrite d18/d4c/d5f/f6a [4194304,4194304] 0 2024-05-09T06:38:34.546 INFO:tasks.workunit.client.1.smithi193.stdout:0/672: stat d18/d4c/f4b 0 2024-05-09T06:38:34.547 INFO:tasks.workunit.client.1.smithi193.stdout:2/798: creat db/def/dfd/d80/daa/f10b x:0 0 0 2024-05-09T06:38:34.548 INFO:tasks.workunit.client.1.smithi193.stdout:2/799: chown db/df/d54/d63/dca 29 1 2024-05-09T06:38:34.548 INFO:tasks.workunit.client.1.smithi193.stdout:8/819: creat d6/d1c/d1b/d5d/d84/dc7/d7e/f103 x:0 0 0 2024-05-09T06:38:34.552 INFO:tasks.workunit.client.1.smithi193.stdout:0/673: creat d18/d4c/d47/d54/fd1 x:0 0 0 2024-05-09T06:38:34.553 INFO:tasks.workunit.client.1.smithi193.stdout:8/820: write d6/d1c/d1b/fe6 [841669,108161] 0 2024-05-09T06:38:34.554 INFO:tasks.workunit.client.1.smithi193.stdout:0/674: rmdir d18/d4c/d64/d74/dad 39 2024-05-09T06:38:34.557 INFO:tasks.workunit.client.1.smithi193.stdout:8/821: symlink d6/d1c/d18/dba/l104 0 2024-05-09T06:38:34.558 INFO:tasks.workunit.client.1.smithi193.stdout:0/675: rmdir d18/d4c/d47/d59 39 2024-05-09T06:38:34.605 INFO:tasks.workunit.client.1.smithi193.stdout:8/822: chown d6/d1c/d53/f71 56329215 1 2024-05-09T06:38:34.606 INFO:tasks.workunit.client.1.smithi193.stdout:0/676: chown d18/d4c/d47/d55/lc0 1481060438 1 2024-05-09T06:38:34.607 INFO:tasks.workunit.client.1.smithi193.stdout:4/810: dwrite d1a/d24/d73/d94/fd8 [0,4194304] 0 2024-05-09T06:38:34.612 INFO:tasks.workunit.client.1.smithi193.stdout:8/823: creat d6/d1c/d18/d66/db3/f105 x:0 0 0 2024-05-09T06:38:34.613 INFO:tasks.workunit.client.1.smithi193.stdout:8/824: chown d6/d1c/d18/d2f/d40/c36 443192146 1 2024-05-09T06:38:34.615 INFO:tasks.workunit.client.1.smithi193.stdout:0/677: mknod d18/d4c/d47/d59/cd2 0 2024-05-09T06:38:34.619 INFO:tasks.workunit.client.1.smithi193.stdout:8/825: creat d6/d1c/d1b/f106 x:0 0 0 2024-05-09T06:38:34.623 INFO:tasks.workunit.client.1.smithi193.stdout:8/826: getdents d6/d1c/d18/d66 0 2024-05-09T06:38:34.624 INFO:tasks.workunit.client.1.smithi193.stdout:4/811: write d1a/f65 [888633,8410] 0 2024-05-09T06:38:34.626 INFO:tasks.workunit.client.1.smithi193.stdout:0/678: write d18/d4c/d64/d66/d9d/faf [1220821,113129] 0 2024-05-09T06:38:34.628 INFO:tasks.workunit.client.1.smithi193.stdout:4/812: readlink d1a/d24/d73/d94/d7c/d5b/l99 0 2024-05-09T06:38:34.629 INFO:tasks.workunit.client.1.smithi193.stdout:0/679: fdatasync d18/d4c/d47/f58 0 2024-05-09T06:38:34.632 INFO:tasks.workunit.client.1.smithi193.stdout:0/680: dread - d18/d4c/d64/d66/fb1 zero size 2024-05-09T06:38:34.637 INFO:tasks.workunit.client.1.smithi193.stdout:4/813: getdents d1a/da6 0 2024-05-09T06:38:34.637 INFO:tasks.workunit.client.1.smithi193.stdout:8/827: dwrite d6/d1c/d18/d66/f62 [0,4194304] 0 2024-05-09T06:38:34.639 INFO:tasks.workunit.client.1.smithi193.stdout:4/814: symlink d1a/d67/lf6 0 2024-05-09T06:38:34.639 INFO:tasks.workunit.client.1.smithi193.stdout:0/681: write d18/d4c/d5f/f8c [1687233,20135] 0 2024-05-09T06:38:34.641 INFO:tasks.workunit.client.1.smithi193.stdout:4/815: write d1a/d67/d82/dec/f63 [4721245,124409] 0 2024-05-09T06:38:34.643 INFO:tasks.workunit.client.1.smithi193.stdout:8/828: creat d6/d1c/d18/d2f/d40/d5e/f107 x:0 0 0 2024-05-09T06:38:34.645 INFO:tasks.workunit.client.1.smithi193.stdout:0/682: unlink d18/d4c/d64/cb2 0 2024-05-09T06:38:34.646 INFO:tasks.workunit.client.1.smithi193.stdout:0/683: chown d18/d4c/d47/d59/d76/f77 786495271 1 2024-05-09T06:38:34.647 INFO:tasks.workunit.client.1.smithi193.stdout:0/684: stat d18/d4c/d47/d54/f97 0 2024-05-09T06:38:34.648 INFO:tasks.workunit.client.1.smithi193.stdout:8/829: dread d6/d1c/d18/dba/fc0 [0,4194304] 0 2024-05-09T06:38:34.648 INFO:tasks.workunit.client.1.smithi193.stdout:4/816: creat d1a/d24/d73/ff7 x:0 0 0 2024-05-09T06:38:34.649 INFO:tasks.workunit.client.1.smithi193.stdout:0/685: chown d18/d4c/d47/d55/lc0 4201000 1 2024-05-09T06:38:34.650 INFO:tasks.workunit.client.1.smithi193.stdout:0/686: write d18/d4c/d47/d59/d76/f77 [1042425,17771] 0 2024-05-09T06:38:34.658 INFO:tasks.workunit.client.1.smithi193.stdout:0/687: dwrite d18/d4c/d47/d54/fd1 [0,4194304] 0 2024-05-09T06:38:34.665 INFO:tasks.workunit.client.1.smithi193.stdout:4/817: dwrite d1a/d22/d5d/fe4 [0,4194304] 0 2024-05-09T06:38:34.671 INFO:tasks.workunit.client.1.smithi193.stdout:8/830: link d6/d1c/la0 d6/d1c/d1b/d5d/d84/l108 0 2024-05-09T06:38:34.672 INFO:tasks.workunit.client.1.smithi193.stdout:8/831: write d6/d1c/d53/fee [598921,21179] 0 2024-05-09T06:38:34.674 INFO:tasks.workunit.client.1.smithi193.stdout:0/688: unlink d18/d4c/d64/d74/dad/d70/ca2 0 2024-05-09T06:38:34.680 INFO:tasks.workunit.client.1.smithi193.stdout:0/689: fdatasync d18/f25 0 2024-05-09T06:38:34.680 INFO:tasks.workunit.client.1.smithi193.stdout:4/818: dwrite d1a/d24/f41 [0,4194304] 0 2024-05-09T06:38:34.686 INFO:tasks.workunit.client.1.smithi193.stdout:0/690: fdatasync d18/d4c/d47/d55/fae 0 2024-05-09T06:38:34.688 INFO:tasks.workunit.client.1.smithi193.stdout:4/819: rmdir d1a/d67 39 2024-05-09T06:38:34.695 INFO:tasks.workunit.client.1.smithi193.stdout:0/691: mknod d18/d4c/d64/d66/db5/cd3 0 2024-05-09T06:38:34.698 INFO:tasks.workunit.client.1.smithi193.stdout:0/692: read d18/d4c/d64/d66/d9d/faf [2651942,128188] 0 2024-05-09T06:38:34.701 INFO:tasks.workunit.client.1.smithi193.stdout:4/820: write f7 [9196636,34100] 0 2024-05-09T06:38:34.704 INFO:tasks.workunit.client.1.smithi193.stdout:0/693: dwrite d18/d1d/f30 [0,4194304] 0 2024-05-09T06:38:34.705 INFO:tasks.workunit.client.1.smithi193.stdout:4/821: write d1a/fcf [739354,56166] 0 2024-05-09T06:38:34.710 INFO:tasks.workunit.client.1.smithi193.stdout:0/694: dwrite d18/d27/fa0 [0,4194304] 0 2024-05-09T06:38:34.712 INFO:tasks.workunit.client.1.smithi193.stdout:4/822: mkdir d1a/d22/d43/df8 0 2024-05-09T06:38:34.717 INFO:tasks.workunit.client.1.smithi193.stdout:0/695: rmdir d18/d4c/d5f 39 2024-05-09T06:38:34.719 INFO:tasks.workunit.client.1.smithi193.stdout:3/717: sync 2024-05-09T06:38:34.719 INFO:tasks.workunit.client.1.smithi193.stdout:1/808: sync 2024-05-09T06:38:34.719 INFO:tasks.workunit.client.1.smithi193.stdout:5/707: sync 2024-05-09T06:38:34.719 INFO:tasks.workunit.client.1.smithi193.stdout:9/874: sync 2024-05-09T06:38:34.719 INFO:tasks.workunit.client.1.smithi193.stdout:6/710: sync 2024-05-09T06:38:34.721 INFO:tasks.workunit.client.1.smithi193.stdout:4/823: readlink d1a/ded/ldd 0 2024-05-09T06:38:34.721 INFO:tasks.workunit.client.1.smithi193.stdout:0/696: symlink d18/d4c/d64/d66/ld4 0 2024-05-09T06:38:34.728 INFO:tasks.workunit.client.1.smithi193.stdout:9/875: truncate dd/d1e/dc2/d49/f64 3072828 0 2024-05-09T06:38:34.730 INFO:tasks.workunit.client.1.smithi193.stdout:9/876: write dd/d1e/dc2/d49/f51 [4219832,55697] 0 2024-05-09T06:38:34.731 INFO:tasks.workunit.client.1.smithi193.stdout:3/718: symlink d1/db/lf5 0 2024-05-09T06:38:34.731 INFO:tasks.workunit.client.1.smithi193.stdout:4/824: dwrite d1a/d98/fb8 [0,4194304] 0 2024-05-09T06:38:34.733 INFO:tasks.workunit.client.1.smithi193.stdout:9/877: write dd/d1e/d1f/d48/d55/d86/fee [4910650,46337] 0 2024-05-09T06:38:34.735 INFO:tasks.workunit.client.1.smithi193.stdout:4/825: truncate d1a/ff2 244514 0 2024-05-09T06:38:34.735 INFO:tasks.workunit.client.1.smithi193.stdout:5/708: rename d0/d13/d2c/d37/d2b/d47/d6f/lb8 to d0/d6d/dbd/le9 0 2024-05-09T06:38:34.737 INFO:tasks.workunit.client.1.smithi193.stdout:4/826: chown d1a/d22/c29 0 1 2024-05-09T06:38:34.748 INFO:tasks.workunit.client.1.smithi193.stdout:6/711: write d1/d25/d2c/d76/d69/f88 [1002306,123454] 0 2024-05-09T06:38:34.753 INFO:tasks.workunit.client.1.smithi193.stdout:3/719: mkdir d1/db/d2f/d84/d97/df4/df6 0 2024-05-09T06:38:34.755 INFO:tasks.workunit.client.1.smithi193.stdout:1/809: dwrite d4/d16/d17/d1a/dc5/fe8 [4194304,4194304] 0 2024-05-09T06:38:34.759 INFO:tasks.workunit.client.1.smithi193.stdout:3/720: dread - d1/db/d1b/dbf/fc4 zero size 2024-05-09T06:38:34.766 INFO:tasks.workunit.client.1.smithi193.stdout:0/697: write d18/f28 [2861953,86056] 0 2024-05-09T06:38:34.766 INFO:tasks.workunit.client.1.smithi193.stdout:4/827: link l14 d1a/d24/d73/d94/d7c/dc6/lf9 0 2024-05-09T06:38:34.766 INFO:tasks.workunit.client.1.smithi193.stdout:0/698: chown d18/d4c/d47/d54/fa1 47 1 2024-05-09T06:38:34.766 INFO:tasks.workunit.client.1.smithi193.stdout:3/721: mknod d1/db/d59/db8/cf7 0 2024-05-09T06:38:34.766 INFO:tasks.workunit.client.1.smithi193.stdout:0/699: read d18/d4c/f38 [534583,27474] 0 2024-05-09T06:38:34.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/700: readlink d18/d4c/d47/d55/l7a 0 2024-05-09T06:38:34.769 INFO:tasks.workunit.client.1.smithi193.stdout:4/828: creat d1a/d22/d8e/ffa x:0 0 0 2024-05-09T06:38:34.773 INFO:tasks.workunit.client.1.smithi193.stdout:5/709: dwrite d0/d13/d2c/d37/d5b/d4e/f78 [0,4194304] 0 2024-05-09T06:38:34.777 INFO:tasks.workunit.client.1.smithi193.stdout:3/722: mknod d1/db/cf8 0 2024-05-09T06:38:34.779 INFO:tasks.workunit.client.1.smithi193.stdout:5/710: dread d0/d13/d2c/d37/d5b/f6 [0,4194304] 0 2024-05-09T06:38:34.786 INFO:tasks.workunit.client.1.smithi193.stdout:0/701: getdents d18/d65/d82 0 2024-05-09T06:38:34.788 INFO:tasks.workunit.client.1.smithi193.stdout:1/810: dread d4/d14/f44 [0,4194304] 0 2024-05-09T06:38:34.789 INFO:tasks.workunit.client.1.smithi193.stdout:3/723: mkdir d1/db/d59/d63/df9 0 2024-05-09T06:38:34.789 INFO:tasks.workunit.client.1.smithi193.stdout:1/811: chown d4/d16/d60/f18 719501950 1 2024-05-09T06:38:34.803 INFO:tasks.workunit.client.1.smithi193.stdout:5/711: dwrite d0/d13/d2c/d37/d5b/f21 [0,4194304] 0 2024-05-09T06:38:34.805 INFO:tasks.workunit.client.1.smithi193.stdout:5/712: dread - d0/d6d/dbd/dd0/fe3 zero size 2024-05-09T06:38:34.806 INFO:tasks.workunit.client.1.smithi193.stdout:1/812: dread d4/d14/f75 [0,4194304] 0 2024-05-09T06:38:34.809 INFO:tasks.workunit.client.1.smithi193.stdout:1/813: truncate d4/d16/d60/d4f/d72/fef 858060 0 2024-05-09T06:38:34.814 INFO:tasks.workunit.client.1.smithi193.stdout:5/713: dwrite d0/d13/d2c/d37/f11 [0,4194304] 0 2024-05-09T06:38:34.816 INFO:tasks.workunit.client.1.smithi193.stdout:3/724: dwrite d1/db/d2f/fab [0,4194304] 0 2024-05-09T06:38:34.817 INFO:tasks.workunit.client.1.smithi193.stdout:1/814: creat d4/d16/d17/d1a/d3a/d4a/db1/f10a x:0 0 0 2024-05-09T06:38:34.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/815: write d4/d16/d6b/d85/f86 [874585,11738] 0 2024-05-09T06:38:34.827 INFO:tasks.workunit.client.1.smithi193.stdout:5/714: dwrite d0/d6d/fb0 [0,4194304] 0 2024-05-09T06:38:34.828 INFO:tasks.workunit.client.1.smithi193.stdout:3/725: dwrite d1/db/d2f/d78/d86/fa8 [0,4194304] 0 2024-05-09T06:38:34.838 INFO:tasks.workunit.client.1.smithi193.stdout:5/715: creat d0/d13/d2c/d37/d2b/d6a/fea x:0 0 0 2024-05-09T06:38:34.841 INFO:tasks.workunit.client.1.smithi193.stdout:5/716: mkdir d0/d13/d2c/d37/d2b/d6a/deb 0 2024-05-09T06:38:34.845 INFO:tasks.workunit.client.1.smithi193.stdout:3/726: creat d1/db/d1b/ffa x:0 0 0 2024-05-09T06:38:34.846 INFO:tasks.workunit.client.1.smithi193.stdout:5/717: mknod d0/d13/d2c/d37/d5b/d4e/cec 0 2024-05-09T06:38:34.853 INFO:tasks.workunit.client.1.smithi193.stdout:5/718: dwrite d0/d13/d2c/d4d/fca [0,4194304] 0 2024-05-09T06:38:34.863 INFO:tasks.workunit.client.1.smithi193.stdout:5/719: getdents d0/d13/d2c/d37/d2b/d47/d60/d75 0 2024-05-09T06:38:34.863 INFO:tasks.workunit.client.1.smithi193.stdout:1/816: dwrite d4/d16/d6b/d6e/fc2 [0,4194304] 0 2024-05-09T06:38:34.867 INFO:tasks.workunit.client.1.smithi193.stdout:5/720: readlink d0/d13/d2c/d37/d2b/d47/d66/d3a/l7f 0 2024-05-09T06:38:34.867 INFO:tasks.workunit.client.1.smithi193.stdout:1/817: dread - d4/d16/d17/fbc zero size 2024-05-09T06:38:34.871 INFO:tasks.workunit.client.1.smithi193.stdout:1/818: mkdir d4/d3c/d3e/d7c/d10b 0 2024-05-09T06:38:34.874 INFO:tasks.workunit.client.1.smithi193.stdout:5/721: dread d0/d13/d2c/d37/d2b/d47/d60/d75/f8b [0,4194304] 0 2024-05-09T06:38:34.874 INFO:tasks.workunit.client.1.smithi193.stdout:1/819: unlink d4/d14/c21 0 2024-05-09T06:38:34.875 INFO:tasks.workunit.client.1.smithi193.stdout:1/820: chown d4/d16/d6b/d85 499647 1 2024-05-09T06:38:34.877 INFO:tasks.workunit.client.1.smithi193.stdout:5/722: fsync d0/d13/d2c/d37/d5b/fe6 0 2024-05-09T06:38:34.884 INFO:tasks.workunit.client.1.smithi193.stdout:5/723: dread d0/d13/d2c/d4d/fd7 [0,4194304] 0 2024-05-09T06:38:34.885 INFO:tasks.workunit.client.1.smithi193.stdout:5/724: chown d0/d13/l62 494 1 2024-05-09T06:38:34.886 INFO:tasks.workunit.client.1.smithi193.stdout:3/727: truncate d1/db/d2f/d78/d86/ddc/fe1 3490213 0 2024-05-09T06:38:34.887 INFO:tasks.workunit.client.1.smithi193.stdout:5/725: dread d0/d13/d2c/d37/d5b/f6 [0,4194304] 0 2024-05-09T06:38:34.896 INFO:tasks.workunit.client.1.smithi193.stdout:5/726: symlink d0/d13/d2c/d37/d5b/d89/de0/led 0 2024-05-09T06:38:34.898 INFO:tasks.workunit.client.1.smithi193.stdout:5/727: chown d0/d13/d2c/l8a 11349 1 2024-05-09T06:38:34.899 INFO:tasks.workunit.client.1.smithi193.stdout:3/728: dwrite d1/f24 [0,4194304] 0 2024-05-09T06:38:34.907 INFO:tasks.workunit.client.1.smithi193.stdout:3/729: mknod d1/cfb 0 2024-05-09T06:38:34.911 INFO:tasks.workunit.client.1.smithi193.stdout:5/728: rename d0/d13/d2c/d4d/d67/f93 to d0/d13/d2c/d37/d2b/d47/d60/d75/da9/fee 0 2024-05-09T06:38:34.911 INFO:tasks.workunit.client.1.smithi193.stdout:3/730: rename d1 to d1/db/d59/db8/dfc 22 2024-05-09T06:38:34.916 INFO:tasks.workunit.client.1.smithi193.stdout:1/821: truncate d4/d14/f2f 3693911 0 2024-05-09T06:38:34.918 INFO:tasks.workunit.client.1.smithi193.stdout:7/800: sync 2024-05-09T06:38:34.919 INFO:tasks.workunit.client.1.smithi193.stdout:7/801: read - d6/dd/d7e/ffa zero size 2024-05-09T06:38:34.919 INFO:tasks.workunit.client.1.smithi193.stdout:7/802: readlink d6/d8/d1a/d36/d4d/d74/lcf 0 2024-05-09T06:38:34.920 INFO:tasks.workunit.client.1.smithi193.stdout:5/729: rename d0/d13/d2c/d37/d5b/d4e/d9d to d0/d13/d2c/d37/d2b/d6a/d90/def 0 2024-05-09T06:38:34.926 INFO:tasks.workunit.client.1.smithi193.stdout:5/730: dwrite d0/d6d/fb0 [0,4194304] 0 2024-05-09T06:38:34.931 INFO:tasks.workunit.client.1.smithi193.stdout:3/731: write d1/db/d2f/d78/d86/ddc/d51/f7e [4222595,124026] 0 2024-05-09T06:38:34.931 INFO:tasks.workunit.client.1.smithi193.stdout:5/731: truncate d0/d13/d2c/d37/d2b/d6a/d90/fc3 385056 0 2024-05-09T06:38:34.934 INFO:tasks.workunit.client.1.smithi193.stdout:7/803: rename d6/d8/d29/cb6 to d6/d17/d99/c106 0 2024-05-09T06:38:34.937 INFO:tasks.workunit.client.1.smithi193.stdout:5/732: symlink d0/d13/d2c/d4d/d67/lf0 0 2024-05-09T06:38:34.938 INFO:tasks.workunit.client.1.smithi193.stdout:3/732: mknod d1/db/d59/d63/cfd 0 2024-05-09T06:38:34.939 INFO:tasks.workunit.client.1.smithi193.stdout:5/733: chown d0/d13/d2c/db6/dc6/ldf 14234878 1 2024-05-09T06:38:34.944 INFO:tasks.workunit.client.1.smithi193.stdout:5/734: dread d0/d13/d2c/d37/d5b/f7b [0,4194304] 0 2024-05-09T06:38:34.945 INFO:tasks.workunit.client.1.smithi193.stdout:1/822: dread d4/d16/d17/d1a/d3a/d4a/fd6 [0,4194304] 0 2024-05-09T06:38:34.947 INFO:tasks.workunit.client.1.smithi193.stdout:5/735: dread d0/d13/d2c/d4d/d67/fc7 [0,4194304] 0 2024-05-09T06:38:34.949 INFO:tasks.workunit.client.1.smithi193.stdout:1/823: mknod d4/d16/d6b/d85/ddd/dfb/c10c 0 2024-05-09T06:38:34.952 INFO:tasks.workunit.client.1.smithi193.stdout:5/736: symlink d0/d13/d2c/d37/d2b/d47/d60/dc8/lf1 0 2024-05-09T06:38:34.956 INFO:tasks.workunit.client.1.smithi193.stdout:1/824: truncate d4/d14/f1b 4191520 0 2024-05-09T06:38:34.957 INFO:tasks.workunit.client.1.smithi193.stdout:7/804: dwrite d6/d8/d1a/d36/f50 [0,4194304] 0 2024-05-09T06:38:34.957 INFO:tasks.workunit.client.1.smithi193.stdout:3/733: dread d1/f11 [0,4194304] 0 2024-05-09T06:38:34.957 INFO:tasks.workunit.client.1.smithi193.stdout:5/737: symlink d0/d13/d2c/d37/lf2 0 2024-05-09T06:38:34.961 INFO:tasks.workunit.client.1.smithi193.stdout:1/825: mknod d4/d16/d6b/d89/dc4/c10d 0 2024-05-09T06:38:34.961 INFO:tasks.workunit.client.1.smithi193.stdout:1/826: write d4/d16/d6b/d6e/f71 [523473,99806] 0 2024-05-09T06:38:34.970 INFO:tasks.workunit.client.1.smithi193.stdout:1/827: symlink d4/d16/d6b/d85/ddd/dfb/l10e 0 2024-05-09T06:38:34.972 INFO:tasks.workunit.client.1.smithi193.stdout:5/738: rmdir d0/d13/d2c/d37/d5b/ddc 0 2024-05-09T06:38:34.973 INFO:tasks.workunit.client.1.smithi193.stdout:3/734: creat d1/db/d2f/d78/d86/ffe x:0 0 0 2024-05-09T06:38:34.973 INFO:tasks.workunit.client.1.smithi193.stdout:5/739: fdatasync d0/d13/d2c/d37/d2b/d47/d66/d3a/f52 0 2024-05-09T06:38:34.974 INFO:tasks.workunit.client.1.smithi193.stdout:1/828: fdatasync d4/d16/d60/f18 0 2024-05-09T06:38:34.977 INFO:tasks.workunit.client.1.smithi193.stdout:3/735: mkdir d1/db/d2f/d84/d97/dff 0 2024-05-09T06:38:34.980 INFO:tasks.workunit.client.1.smithi193.stdout:7/805: truncate d6/d17/d99/fef 2841637 0 2024-05-09T06:38:34.982 INFO:tasks.workunit.client.1.smithi193.stdout:7/806: dread - d6/dd/dbc/fc5 zero size 2024-05-09T06:38:34.983 INFO:tasks.workunit.client.1.smithi193.stdout:3/736: dwrite d1/db/d59/f37 [4194304,4194304] 0 2024-05-09T06:38:34.986 INFO:tasks.workunit.client.1.smithi193.stdout:1/829: getdents d4/d16/dea 0 2024-05-09T06:38:34.988 INFO:tasks.workunit.client.1.smithi193.stdout:1/830: write d4/d16/de/dca/fe3 [870136,108865] 0 2024-05-09T06:38:34.990 INFO:tasks.workunit.client.1.smithi193.stdout:1/831: write d4/d16/feb [464769,106481] 0 2024-05-09T06:38:34.990 INFO:tasks.workunit.client.1.smithi193.stdout:7/807: dread d6/d8/f55 [0,4194304] 0 2024-05-09T06:38:34.996 INFO:tasks.workunit.client.1.smithi193.stdout:7/808: write d6/dd/dbc/f87 [2762674,107533] 0 2024-05-09T06:38:34.996 INFO:tasks.workunit.client.1.smithi193.stdout:5/740: link d0/d13/d2c/d4d/d67/l86 d0/d13/d2c/d37/d2b/d6a/deb/lf3 0 2024-05-09T06:38:34.998 INFO:tasks.workunit.client.1.smithi193.stdout:7/809: write d6/d8/d1a/fa4 [3973125,12610] 0 2024-05-09T06:38:34.999 INFO:tasks.workunit.client.1.smithi193.stdout:7/810: dread - d6/d8/d1a/d36/f65 zero size 2024-05-09T06:38:34.999 INFO:tasks.workunit.client.1.smithi193.stdout:7/811: stat d6/d8/f6e 0 2024-05-09T06:38:35.000 INFO:tasks.workunit.client.1.smithi193.stdout:1/832: rename d4/d16/d60 to d4/d3c/d3e/d7c/d10b/d10f 0 2024-05-09T06:38:35.000 INFO:tasks.workunit.client.1.smithi193.stdout:7/812: chown d6/dd/f68 131921 1 2024-05-09T06:38:35.000 INFO:tasks.workunit.client.1.smithi193.stdout:1/833: dread - d4/d16/d17/fbc zero size 2024-05-09T06:38:35.004 INFO:tasks.workunit.client.1.smithi193.stdout:7/813: dread d6/d8/d29/d2f/f4c [0,4194304] 0 2024-05-09T06:38:35.005 INFO:tasks.workunit.client.1.smithi193.stdout:5/741: mkdir d0/d13/d2c/d37/d2b/d6a/df4 0 2024-05-09T06:38:35.009 INFO:tasks.workunit.client.1.smithi193.stdout:1/834: write d4/d16/de/fdf [192562,73470] 0 2024-05-09T06:38:35.011 INFO:tasks.workunit.client.1.smithi193.stdout:7/814: dwrite d6/dd/d70/fee [0,4194304] 0 2024-05-09T06:38:35.013 INFO:tasks.workunit.client.1.smithi193.stdout:3/737: truncate f0 1546379 0 2024-05-09T06:38:35.013 INFO:tasks.workunit.client.1.smithi193.stdout:3/738: fsync d1/f1d 0 2024-05-09T06:38:35.015 INFO:tasks.workunit.client.1.smithi193.stdout:5/742: mknod d0/d13/d2c/d7e/cf5 0 2024-05-09T06:38:35.019 INFO:tasks.workunit.client.1.smithi193.stdout:7/815: symlink d6/dd/d5e/d6b/l107 0 2024-05-09T06:38:35.022 INFO:tasks.workunit.client.1.smithi193.stdout:3/739: readlink d1/db/l4f 0 2024-05-09T06:38:35.026 INFO:tasks.workunit.client.1.smithi193.stdout:3/740: write d1/db/d2f/d78/d86/ddc/d51/f7e [1529474,9750] 0 2024-05-09T06:38:35.027 INFO:tasks.workunit.client.1.smithi193.stdout:7/816: dwrite d6/d8/f55 [0,4194304] 0 2024-05-09T06:38:35.038 INFO:tasks.workunit.client.1.smithi193.stdout:3/741: dwrite d1/db/d2f/d78/d86/ddc/fc8 [0,4194304] 0 2024-05-09T06:38:35.057 INFO:tasks.workunit.client.1.smithi193.stdout:2/800: sync 2024-05-09T06:38:35.057 INFO:tasks.workunit.client.1.smithi193.stdout:8/832: sync 2024-05-09T06:38:35.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:34 smithi063 ceph-mon[27171]: pgmap v20: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 78 MiB/s rd, 175 MiB/s wr, 405 op/s 2024-05-09T06:38:35.120 INFO:tasks.workunit.client.1.smithi193.stdout:7/817: dread d6/d8/d1a/d36/d4d/d60/ff1 [0,4194304] 0 2024-05-09T06:38:35.123 INFO:tasks.workunit.client.1.smithi193.stdout:1/835: dwrite d4/d3c/d3e/d7c/d10b/d10f/f58 [4194304,4194304] 0 2024-05-09T06:38:35.123 INFO:tasks.workunit.client.1.smithi193.stdout:5/743: dread d0/d13/d2c/d37/d5b/f70 [0,4194304] 0 2024-05-09T06:38:35.125 INFO:tasks.workunit.client.1.smithi193.stdout:2/801: symlink db/df/d54/dba/l10c 0 2024-05-09T06:38:35.126 INFO:tasks.workunit.client.1.smithi193.stdout:3/742: dwrite d1/fea [4194304,4194304] 0 2024-05-09T06:38:35.128 INFO:tasks.workunit.client.1.smithi193.stdout:5/744: write d0/d13/d2c/d37/d2b/d47/d60/d75/da9/fd1 [442549,96539] 0 2024-05-09T06:38:35.133 INFO:tasks.workunit.client.1.smithi193.stdout:7/818: rename d6/d17/d7d/daa/f8c to d6/dd/d7e/f108 0 2024-05-09T06:38:35.137 INFO:tasks.workunit.client.1.smithi193.stdout:7/819: dread d6/d8/d29/dc8/fda [0,4194304] 0 2024-05-09T06:38:35.139 INFO:tasks.workunit.client.1.smithi193.stdout:1/836: creat d4/d16/d6b/d89/dc4/f110 x:0 0 0 2024-05-09T06:38:35.146 INFO:tasks.workunit.client.1.smithi193.stdout:5/745: rmdir d0/d13/d2c/d37/d2b/d47/d60/d75/da9/dc1 39 2024-05-09T06:38:35.149 INFO:tasks.workunit.client.1.smithi193.stdout:1/837: readlink d4/d16/d17/lba 0 2024-05-09T06:38:35.152 INFO:tasks.workunit.client.1.smithi193.stdout:2/802: mknod db/def/dfd/d80/d60/d7d/c10d 0 2024-05-09T06:38:35.157 INFO:tasks.workunit.client.1.smithi193.stdout:3/743: creat d1/db/d59/f100 x:0 0 0 2024-05-09T06:38:35.159 INFO:tasks.workunit.client.1.smithi193.stdout:1/838: unlink d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/d7f/cee 0 2024-05-09T06:38:35.160 INFO:tasks.workunit.client.1.smithi193.stdout:1/839: fdatasync d4/d16/d17/d1a/d3a/fa3 0 2024-05-09T06:38:35.167 INFO:tasks.workunit.client.1.smithi193.stdout:1/840: dread d4/d16/d17/d1a/d3a/d3f/f65 [0,4194304] 0 2024-05-09T06:38:35.168 INFO:tasks.workunit.client.1.smithi193.stdout:7/820: getdents d6/d8/d29/dc8 0 2024-05-09T06:38:35.174 INFO:tasks.workunit.client.1.smithi193.stdout:5/746: creat d0/d13/d2c/d37/d2b/ff6 x:0 0 0 2024-05-09T06:38:35.174 INFO:tasks.workunit.client.1.smithi193.stdout:3/744: truncate f0 2401524 0 2024-05-09T06:38:35.175 INFO:tasks.workunit.client.1.smithi193.stdout:2/803: creat db/df/f10e x:0 0 0 2024-05-09T06:38:35.177 INFO:tasks.workunit.client.1.smithi193.stdout:7/821: mkdir d6/d8/d1a/d36/d109 0 2024-05-09T06:38:35.182 INFO:tasks.workunit.client.1.smithi193.stdout:3/745: dwrite d1/d44/fd5 [0,4194304] 0 2024-05-09T06:38:35.183 INFO:tasks.workunit.client.1.smithi193.stdout:1/841: dread d4/d16/d6b/d6e/f104 [4194304,4194304] 0 2024-05-09T06:38:35.190 INFO:tasks.workunit.client.1.smithi193.stdout:2/804: write db/def/dfd/d80/f32 [4724835,10174] 0 2024-05-09T06:38:35.191 INFO:tasks.workunit.client.1.smithi193.stdout:3/746: creat d1/db/d1b/dbf/f101 x:0 0 0 2024-05-09T06:38:35.195 INFO:tasks.workunit.client.1.smithi193.stdout:5/747: dread d0/d13/d2c/d37/d2b/d47/d6f/f54 [0,4194304] 0 2024-05-09T06:38:35.195 INFO:tasks.workunit.client.1.smithi193.stdout:2/805: readlink db/d25/d38/d61/led 0 2024-05-09T06:38:35.197 INFO:tasks.workunit.client.1.smithi193.stdout:3/747: chown d1/d44/d7d/f9c 2082 1 2024-05-09T06:38:35.198 INFO:tasks.workunit.client.1.smithi193.stdout:2/806: readlink db/d25/d38/d5b/l84 0 2024-05-09T06:38:35.198 INFO:tasks.workunit.client.1.smithi193.stdout:7/822: dwrite d6/dd/d53/f6a [0,4194304] 0 2024-05-09T06:38:35.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/807: chown db/df/d54 7 1 2024-05-09T06:38:35.202 INFO:tasks.workunit.client.1.smithi193.stdout:1/842: rmdir d4/d3c/d3e/d7c/d10b/d10f/d4f/dcd/df4 0 2024-05-09T06:38:35.204 INFO:tasks.workunit.client.1.smithi193.stdout:1/843: chown d4/d16/d6b/l6d 251027 1 2024-05-09T06:38:35.205 INFO:tasks.workunit.client.1.smithi193.stdout:5/748: dwrite d0/f8 [0,4194304] 0 2024-05-09T06:38:35.206 INFO:tasks.workunit.client.1.smithi193.stdout:5/749: readlink d0/d13/d2c/d37/d2b/l5e 0 2024-05-09T06:38:35.207 INFO:tasks.workunit.client.1.smithi193.stdout:7/823: mknod d6/d8/d29/df9/c10a 0 2024-05-09T06:38:35.210 INFO:tasks.workunit.client.1.smithi193.stdout:1/844: fsync d4/d3c/f36 0 2024-05-09T06:38:35.211 INFO:tasks.workunit.client.1.smithi193.stdout:1/845: chown d4/d3c/d3e/d7c/d10b/d10f/d4f/d59/c9a 123992844 1 2024-05-09T06:38:35.212 INFO:tasks.workunit.client.1.smithi193.stdout:7/824: symlink d6/d17/d99/dd7/l10b 0 2024-05-09T06:38:35.212 INFO:tasks.workunit.client.1.smithi193.stdout:3/748: link d1/db/d2f/l9e d1/d44/d7d/l102 0 2024-05-09T06:38:35.220 INFO:tasks.workunit.client.1.smithi193.stdout:5/750: creat d0/d13/ff7 x:0 0 0 2024-05-09T06:38:35.225 INFO:tasks.workunit.client.1.smithi193.stdout:7/825: link d6/dd/f47 d6/d8/d29/f10c 0 2024-05-09T06:38:35.226 INFO:tasks.workunit.client.1.smithi193.stdout:7/826: chown d6/dd/f3f 266 1 2024-05-09T06:38:35.227 INFO:tasks.workunit.client.1.smithi193.stdout:7/827: truncate d6/d17/d7d/fff 95424 0 2024-05-09T06:38:35.227 INFO:tasks.workunit.client.1.smithi193.stdout:7/828: write d6/dd/ffe [922987,28328] 0 2024-05-09T06:38:35.228 INFO:tasks.workunit.client.1.smithi193.stdout:1/846: rename d4/d3c/d3e/d7c/d10b/d10f/d4f/f61 to d4/f111 0 2024-05-09T06:38:35.232 INFO:tasks.workunit.client.1.smithi193.stdout:7/829: mkdir d6/d8/d1a/d36/dd0/d10d 0 2024-05-09T06:38:35.237 INFO:tasks.workunit.client.1.smithi193.stdout:2/808: dwrite db/d25/d38/d61/f73 [4194304,4194304] 0 2024-05-09T06:38:35.238 INFO:tasks.workunit.client.1.smithi193.stdout:9/878: sync 2024-05-09T06:38:35.238 INFO:tasks.workunit.client.1.smithi193.stdout:1/847: dwrite d4/d16/d6b/d89/f99 [0,4194304] 0 2024-05-09T06:38:35.246 INFO:tasks.workunit.client.1.smithi193.stdout:3/749: dwrite d1/db/d1b/fca [0,4194304] 0 2024-05-09T06:38:35.247 INFO:tasks.workunit.client.1.smithi193.stdout:3/750: chown d1/db/d59/f100 16689744 1 2024-05-09T06:38:35.248 INFO:tasks.workunit.client.1.smithi193.stdout:3/751: readlink d1/db/d2f/d78/d86/ddc/d51/l30 0 2024-05-09T06:38:35.250 INFO:tasks.workunit.client.1.smithi193.stdout:3/752: chown d1/db/d2f/d78/d86/ddc/d51/l62 1850 1 2024-05-09T06:38:35.251 INFO:tasks.workunit.client.1.smithi193.stdout:3/753: stat d1/db/d2f/d84/d97/fcd 0 2024-05-09T06:38:35.252 INFO:tasks.workunit.client.1.smithi193.stdout:9/879: creat dd/df/d15/d98/f11a x:0 0 0 2024-05-09T06:38:35.255 INFO:tasks.workunit.client.1.smithi193.stdout:1/848: unlink d4/d14/c79 0 2024-05-09T06:38:35.261 INFO:tasks.workunit.client.1.smithi193.stdout:2/809: link db/f1e db/df/d87/f10f 0 2024-05-09T06:38:35.261 INFO:tasks.workunit.client.1.smithi193.stdout:7/830: dwrite d6/d8/d1a/d36/d4d/fb0 [0,4194304] 0 2024-05-09T06:38:35.262 INFO:tasks.workunit.client.1.smithi193.stdout:1/849: truncate d4/d16/d6b/d85/ff5 67180 0 2024-05-09T06:38:35.264 INFO:tasks.workunit.client.1.smithi193.stdout:7/831: write d6/dd/ffe [1467252,2243] 0 2024-05-09T06:38:35.264 INFO:tasks.workunit.client.1.smithi193.stdout:9/880: dwrite dd/d1e/d93/dbb/ff7 [0,4194304] 0 2024-05-09T06:38:35.266 INFO:tasks.workunit.client.1.smithi193.stdout:7/832: stat d6/dd/d7e/dd6 0 2024-05-09T06:38:35.270 INFO:tasks.workunit.client.1.smithi193.stdout:9/881: readlink dd/ld3 0 2024-05-09T06:38:35.272 INFO:tasks.workunit.client.1.smithi193.stdout:1/850: rmdir d4/d16/d6b/d6e 39 2024-05-09T06:38:35.278 INFO:tasks.workunit.client.1.smithi193.stdout:2/810: rename db/def/dfd/d80/d60/d7d/c10d to db/def/dfd/d80/d60/c110 0 2024-05-09T06:38:35.280 INFO:tasks.workunit.client.1.smithi193.stdout:9/882: mkdir dd/df/d15/d8c/de6/d106/d11b 0 2024-05-09T06:38:35.281 INFO:tasks.workunit.client.1.smithi193.stdout:7/833: dread d6/dd/d70/f8e [0,4194304] 0 2024-05-09T06:38:35.283 INFO:tasks.workunit.client.1.smithi193.stdout:1/851: rename d4/d16/d17/d1a/d3a/lb0 to d4/d3c/d3e/d7c/d10b/d10f/d4f/dcd/l112 0 2024-05-09T06:38:35.284 INFO:tasks.workunit.client.1.smithi193.stdout:1/852: chown d4/d16/d17/d1a/f66 212333 1 2024-05-09T06:38:35.285 INFO:tasks.workunit.client.1.smithi193.stdout:2/811: creat db/def/dfd/dc4/f111 x:0 0 0 2024-05-09T06:38:35.288 INFO:tasks.workunit.client.1.smithi193.stdout:9/883: creat dd/df/d15/d8c/d7a/f11c x:0 0 0 2024-05-09T06:38:35.289 INFO:tasks.workunit.client.1.smithi193.stdout:3/754: truncate d1/db/d2f/d78/d86/ddc/d51/f7e 3819996 0 2024-05-09T06:38:35.290 INFO:tasks.workunit.client.1.smithi193.stdout:9/884: chown dd/d1e/dc2/d49/dbd/d5b/d8a/fc5 450 1 2024-05-09T06:38:35.292 INFO:tasks.workunit.client.1.smithi193.stdout:7/834: rmdir d6/d8/d1a/d36/dd0 39 2024-05-09T06:38:35.294 INFO:tasks.workunit.client.1.smithi193.stdout:1/853: mkdir d4/d16/d6b/d85/ddd/d113 0 2024-05-09T06:38:35.295 INFO:tasks.workunit.client.1.smithi193.stdout:2/812: symlink db/def/dfd/d80/d60/d7d/db3/l112 0 2024-05-09T06:38:35.296 INFO:tasks.workunit.client.1.smithi193.stdout:3/755: creat d1/d44/f103 x:0 0 0 2024-05-09T06:38:35.298 INFO:tasks.workunit.client.1.smithi193.stdout:9/885: rename dd/dd9 to dd/d1e/dc2/d49/dc1/d11d 0 2024-05-09T06:38:35.299 INFO:tasks.workunit.client.1.smithi193.stdout:7/835: creat d6/dc3/f10e x:0 0 0 2024-05-09T06:38:35.300 INFO:tasks.workunit.client.1.smithi193.stdout:1/854: mknod d4/d16/d6b/d85/ddd/c114 0 2024-05-09T06:38:35.302 INFO:tasks.workunit.client.1.smithi193.stdout:3/756: symlink d1/db/d1b/dbf/l104 0 2024-05-09T06:38:35.302 INFO:tasks.workunit.client.1.smithi193.stdout:3/757: chown d1/db/d2f/d84/cd6 35142 1 2024-05-09T06:38:35.304 INFO:tasks.workunit.client.1.smithi193.stdout:2/813: rename db/d37/c55 to db/def/dfd/c113 0 2024-05-09T06:38:35.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:34 smithi193 ceph-mon[35141]: pgmap v20: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 78 MiB/s rd, 175 MiB/s wr, 405 op/s 2024-05-09T06:38:35.378 INFO:tasks.workunit.client.1.smithi193.stdout:1/855: fdatasync d4/d16/d35/f74 0 2024-05-09T06:38:35.378 INFO:tasks.workunit.client.1.smithi193.stdout:9/886: rename dd/df/f10 to dd/d1e/d93/dbb/f11e 0 2024-05-09T06:38:35.379 INFO:tasks.workunit.client.1.smithi193.stdout:7/836: mknod d6/dd/d5e/c10f 0 2024-05-09T06:38:35.380 INFO:tasks.workunit.client.1.smithi193.stdout:3/758: symlink d1/db/da1/dce/l105 0 2024-05-09T06:38:35.380 INFO:tasks.workunit.client.1.smithi193.stdout:3/759: chown d1/db/d1b/lda 0 1 2024-05-09T06:38:35.381 INFO:tasks.workunit.client.1.smithi193.stdout:3/760: chown d1/db/d59/c2a 25 1 2024-05-09T06:38:35.382 INFO:tasks.workunit.client.1.smithi193.stdout:9/887: mknod dd/df/dc4/c11f 0 2024-05-09T06:38:35.385 INFO:tasks.workunit.client.1.smithi193.stdout:1/856: truncate d4/d3c/d3e/d7c/d10b/d10f/f91 300929 0 2024-05-09T06:38:35.387 INFO:tasks.workunit.client.1.smithi193.stdout:2/814: link db/def/dfd/dc4/f111 db/d25/d38/d40/db2/f114 0 2024-05-09T06:38:35.388 INFO:tasks.workunit.client.1.smithi193.stdout:7/837: mknod d6/dd/d5e/c110 0 2024-05-09T06:38:35.389 INFO:tasks.workunit.client.1.smithi193.stdout:9/888: truncate dd/d1e/d1f/d61/f85 1099552 0 2024-05-09T06:38:35.391 INFO:tasks.workunit.client.1.smithi193.stdout:3/761: symlink d1/db/l106 0 2024-05-09T06:38:35.395 INFO:tasks.workunit.client.1.smithi193.stdout:1/857: creat d4/d14/f115 x:0 0 0 2024-05-09T06:38:35.396 INFO:tasks.workunit.client.1.smithi193.stdout:7/838: dwrite d6/d8/d1a/d36/d4d/d74/f7c [0,4194304] 0 2024-05-09T06:38:35.400 INFO:tasks.workunit.client.1.smithi193.stdout:3/762: dwrite d1/db/f58 [0,4194304] 0 2024-05-09T06:38:35.405 INFO:tasks.workunit.client.1.smithi193.stdout:3/763: truncate d1/db/d2f/d78/d86/ddc/ff2 575829 0 2024-05-09T06:38:35.407 INFO:tasks.workunit.client.1.smithi193.stdout:9/889: rename dd/d1e/d1f/c36 to dd/d1e/dc2/d49/dbd/dab/c120 0 2024-05-09T06:38:35.410 INFO:tasks.workunit.client.1.smithi193.stdout:2/815: link db/l34 db/d25/d38/d40/l115 0 2024-05-09T06:38:35.410 INFO:tasks.workunit.client.1.smithi193.stdout:6/712: sync 2024-05-09T06:38:35.411 INFO:tasks.workunit.client.1.smithi193.stdout:1/858: dread d4/d16/d6b/d85/ddd/dfb/ffe [0,4194304] 0 2024-05-09T06:38:35.411 INFO:tasks.workunit.client.1.smithi193.stdout:3/764: rmdir d1/db/d59/d63 39 2024-05-09T06:38:35.416 INFO:tasks.workunit.client.1.smithi193.stdout:7/839: rename d6/d8/d1a/d36/d109 to d6/dd/d7e/d111 0 2024-05-09T06:38:35.417 INFO:tasks.workunit.client.1.smithi193.stdout:7/840: chown d6/dd/d7e 432 1 2024-05-09T06:38:35.418 INFO:tasks.workunit.client.1.smithi193.stdout:1/859: dwrite d4/d16/d6b/d89/f99 [0,4194304] 0 2024-05-09T06:38:35.420 INFO:tasks.workunit.client.1.smithi193.stdout:2/816: truncate db/d25/d38/f9d 879607 0 2024-05-09T06:38:35.425 INFO:tasks.workunit.client.1.smithi193.stdout:6/713: mkdir d1/d25/d2c/d76/d100 0 2024-05-09T06:38:35.428 INFO:tasks.workunit.client.1.smithi193.stdout:2/817: dwrite db/d25/d38/d5b/fc1 [0,4194304] 0 2024-05-09T06:38:35.429 INFO:tasks.workunit.client.1.smithi193.stdout:3/765: creat d1/db/d2f/d78/d86/d94/f107 x:0 0 0 2024-05-09T06:38:35.435 INFO:tasks.workunit.client.1.smithi193.stdout:9/890: unlink dd/df/d15/d8c/de6/d113/d37/f4d 0 2024-05-09T06:38:35.437 INFO:tasks.workunit.client.1.smithi193.stdout:6/714: dread d1/de/d11/d18/f65 [0,4194304] 0 2024-05-09T06:38:35.438 INFO:tasks.workunit.client.1.smithi193.stdout:9/891: stat dd/d1e/d3c/f7d 0 2024-05-09T06:38:35.440 INFO:tasks.workunit.client.1.smithi193.stdout:3/766: dwrite d1/db/d2f/d78/d86/fa8 [0,4194304] 0 2024-05-09T06:38:35.441 INFO:tasks.workunit.client.1.smithi193.stdout:3/767: readlink d1/db/d2f/d78/d86/ddc/d51/ld9 0 2024-05-09T06:38:35.454 INFO:tasks.workunit.client.1.smithi193.stdout:3/768: creat d1/db/d2f/d78/d86/d94/f108 x:0 0 0 2024-05-09T06:38:35.456 INFO:tasks.workunit.client.1.smithi193.stdout:7/841: write d6/d8/f6e [63822,29379] 0 2024-05-09T06:38:35.457 INFO:tasks.workunit.client.1.smithi193.stdout:6/715: truncate d1/d25/d2c/d81/faf 260211 0 2024-05-09T06:38:35.457 INFO:tasks.workunit.client.1.smithi193.stdout:7/842: stat d6/d8/d29/l2a 0 2024-05-09T06:38:35.459 INFO:tasks.workunit.client.1.smithi193.stdout:2/818: dread db/def/dfd/d80/d60/d7d/f7e [0,4194304] 0 2024-05-09T06:38:35.459 INFO:tasks.workunit.client.1.smithi193.stdout:6/716: chown d1/d25/d2c/d81/fb5 0 1 2024-05-09T06:38:35.466 INFO:tasks.workunit.client.1.smithi193.stdout:6/717: unlink d1/d25/d2c/d3c/db1/d58/c72 0 2024-05-09T06:38:35.472 INFO:tasks.workunit.client.1.smithi193.stdout:1/860: dwrite d4/f37 [0,4194304] 0 2024-05-09T06:38:35.474 INFO:tasks.workunit.client.1.smithi193.stdout:9/892: dread dd/df/d15/f8e [0,4194304] 0 2024-05-09T06:38:35.479 INFO:tasks.workunit.client.1.smithi193.stdout:7/843: dread d6/d8/d1a/d36/f57 [0,4194304] 0 2024-05-09T06:38:35.481 INFO:tasks.workunit.client.1.smithi193.stdout:9/893: dwrite dd/d1e/d93/dbb/ff7 [4194304,4194304] 0 2024-05-09T06:38:35.482 INFO:tasks.workunit.client.1.smithi193.stdout:9/894: fsync dd/df/f6e 0 2024-05-09T06:38:35.482 INFO:tasks.workunit.client.1.smithi193.stdout:9/895: chown dd/df/d15/d8c/de6/d113/c6d 16926 1 2024-05-09T06:38:35.501 INFO:tasks.workunit.client.1.smithi193.stdout:3/769: mknod d1/d44/d7d/c109 0 2024-05-09T06:38:35.502 INFO:tasks.workunit.client.1.smithi193.stdout:3/770: stat d1/db/c5a 0 2024-05-09T06:38:35.504 INFO:tasks.workunit.client.1.smithi193.stdout:7/844: fsync d6/d8/d1a/d36/d4d/d60/ff1 0 2024-05-09T06:38:35.507 INFO:tasks.workunit.client.1.smithi193.stdout:9/896: creat dd/d1e/dc2/d49/f121 x:0 0 0 2024-05-09T06:38:35.508 INFO:tasks.workunit.client.1.smithi193.stdout:9/897: chown dd/c38 207 1 2024-05-09T06:38:35.510 INFO:tasks.workunit.client.1.smithi193.stdout:1/861: dwrite d4/d16/f63 [0,4194304] 0 2024-05-09T06:38:35.511 INFO:tasks.workunit.client.1.smithi193.stdout:6/718: creat d1/de/d11/f101 x:0 0 0 2024-05-09T06:38:35.514 INFO:tasks.workunit.client.1.smithi193.stdout:9/898: mkdir dd/df/d15/d98/dea/d122 0 2024-05-09T06:38:35.516 INFO:tasks.workunit.client.1.smithi193.stdout:6/719: creat d1/de/d11/d18/d94/dac/f102 x:0 0 0 2024-05-09T06:38:35.517 INFO:tasks.workunit.client.1.smithi193.stdout:3/771: rename d1/db/da1/fe9 to d1/db/d1b/f10a 0 2024-05-09T06:38:35.519 INFO:tasks.workunit.client.1.smithi193.stdout:9/899: mkdir dd/d1e/dc2/d49/dbd/d5b/d8a/d123 0 2024-05-09T06:38:35.522 INFO:tasks.workunit.client.1.smithi193.stdout:3/772: fdatasync d1/f11 0 2024-05-09T06:38:35.526 INFO:tasks.workunit.client.1.smithi193.stdout:7/845: rename d6/d8/d1a/f3d to d6/d8/d1a/d36/d4d/f112 0 2024-05-09T06:38:35.527 INFO:tasks.workunit.client.1.smithi193.stdout:3/773: creat d1/db/d2f/d84/f10b x:0 0 0 2024-05-09T06:38:35.531 INFO:tasks.workunit.client.1.smithi193.stdout:3/774: write d1/db/d1b/f10a [196333,59842] 0 2024-05-09T06:38:35.531 INFO:tasks.workunit.client.1.smithi193.stdout:9/900: link dd/df/d15/d8c/de6/d113/d37/f7c dd/d1e/d1f/d61/dc6/f124 0 2024-05-09T06:38:35.532 INFO:tasks.workunit.client.1.smithi193.stdout:3/775: readlink d1/db/lf5 0 2024-05-09T06:38:35.535 INFO:tasks.workunit.client.1.smithi193.stdout:3/776: mkdir d1/db/d2f/d78/d86/ddc/d51/d10c 0 2024-05-09T06:38:35.535 INFO:tasks.workunit.client.1.smithi193.stdout:9/901: dread dd/df/d15/f8e [0,4194304] 0 2024-05-09T06:38:35.537 INFO:tasks.workunit.client.1.smithi193.stdout:3/777: mkdir d1/db/d59/d10d 0 2024-05-09T06:38:35.538 INFO:tasks.workunit.client.1.smithi193.stdout:9/902: rmdir dd/df/d15/d8c/de6/d113/d37 39 2024-05-09T06:38:35.539 INFO:tasks.workunit.client.1.smithi193.stdout:3/778: truncate d1/d44/d7d/fc3 98429 0 2024-05-09T06:38:35.550 INFO:tasks.workunit.client.1.smithi193.stdout:1/862: write d4/d14/f44 [7753855,73119] 0 2024-05-09T06:38:35.558 INFO:tasks.workunit.client.1.smithi193.stdout:1/863: dwrite d4/f8c [0,4194304] 0 2024-05-09T06:38:35.559 INFO:tasks.workunit.client.1.smithi193.stdout:1/864: chown d4/f87 3 1 2024-05-09T06:38:35.560 INFO:tasks.workunit.client.1.smithi193.stdout:7/846: write d6/d17/f52 [3369107,85864] 0 2024-05-09T06:38:35.565 INFO:tasks.workunit.client.1.smithi193.stdout:9/903: write dd/d1e/dc2/d49/f64 [2788019,20208] 0 2024-05-09T06:38:35.566 INFO:tasks.workunit.client.1.smithi193.stdout:1/865: dwrite d4/d14/f2f [0,4194304] 0 2024-05-09T06:38:35.570 INFO:tasks.workunit.client.1.smithi193.stdout:9/904: creat dd/df/d15/d8c/de6/d113/d71/f125 x:0 0 0 2024-05-09T06:38:35.575 INFO:tasks.workunit.client.1.smithi193.stdout:3/779: dwrite d1/d44/f60 [4194304,4194304] 0 2024-05-09T06:38:35.577 INFO:tasks.workunit.client.1.smithi193.stdout:1/866: dread d4/d3c/d3e/d7c/d10b/d10f/f58 [4194304,4194304] 0 2024-05-09T06:38:35.582 INFO:tasks.workunit.client.1.smithi193.stdout:1/867: creat d4/d16/d17/d1a/dc5/f116 x:0 0 0 2024-05-09T06:38:35.585 INFO:tasks.workunit.client.1.smithi193.stdout:7/847: write d6/d8/d1a/d36/f4e [3352012,98525] 0 2024-05-09T06:38:35.589 INFO:tasks.workunit.client.1.smithi193.stdout:1/868: mkdir d4/d16/d6b/d85/ddd/d113/d117 0 2024-05-09T06:38:35.593 INFO:tasks.workunit.client.1.smithi193.stdout:4/829: sync 2024-05-09T06:38:35.599 INFO:tasks.workunit.client.1.smithi193.stdout:4/830: rename f7 to d1a/d22/d43/df8/ffb 0 2024-05-09T06:38:35.600 INFO:tasks.workunit.client.1.smithi193.stdout:7/848: link d6/dd/dbc/c94 d6/d8/d1a/d36/d4d/d60/c113 0 2024-05-09T06:38:35.604 INFO:tasks.workunit.client.1.smithi193.stdout:3/780: dread d1/f8 [0,4194304] 0 2024-05-09T06:38:35.604 INFO:tasks.workunit.client.1.smithi193.stdout:9/905: dread dd/d1e/d1f/f2f [0,4194304] 0 2024-05-09T06:38:35.605 INFO:tasks.workunit.client.1.smithi193.stdout:9/906: stat dd/le7 0 2024-05-09T06:38:35.605 INFO:tasks.workunit.client.1.smithi193.stdout:4/831: mknod d1a/d24/d73/dcc/cfc 0 2024-05-09T06:38:35.608 INFO:tasks.workunit.client.1.smithi193.stdout:3/781: mknod d1/db/d2f/d78/d86/d94/c10e 0 2024-05-09T06:38:35.609 INFO:tasks.workunit.client.1.smithi193.stdout:1/869: dread d4/d16/d17/f29 [0,4194304] 0 2024-05-09T06:38:35.609 INFO:tasks.workunit.client.1.smithi193.stdout:9/907: write dd/df/d31/f4c [25897,1343] 0 2024-05-09T06:38:35.611 INFO:tasks.workunit.client.1.smithi193.stdout:9/908: chown dd/df/d15/d8c/de6/d106/d11b 67868 1 2024-05-09T06:38:35.612 INFO:tasks.workunit.client.1.smithi193.stdout:3/782: mknod d1/db/d2f/c10f 0 2024-05-09T06:38:35.614 INFO:tasks.workunit.client.1.smithi193.stdout:9/909: truncate dd/d1e/d1f/faa 293341 0 2024-05-09T06:38:35.614 INFO:tasks.workunit.client.1.smithi193.stdout:9/910: readlink l8 0 2024-05-09T06:38:35.615 INFO:tasks.workunit.client.1.smithi193.stdout:1/870: dread d4/d16/d17/d1a/d3a/d4a/fd6 [0,4194304] 0 2024-05-09T06:38:35.621 INFO:tasks.workunit.client.1.smithi193.stdout:9/911: symlink dd/d1e/dc2/d49/dc1/l126 0 2024-05-09T06:38:35.624 INFO:tasks.workunit.client.1.smithi193.stdout:1/871: mknod d4/d16/d35/c118 0 2024-05-09T06:38:35.625 INFO:tasks.workunit.client.1.smithi193.stdout:7/849: link d6/d8/d1a/l26 d6/d17/d7d/daa/l114 0 2024-05-09T06:38:35.627 INFO:tasks.workunit.client.1.smithi193.stdout:3/783: fsync d1/db/fe3 0 2024-05-09T06:38:35.628 INFO:tasks.workunit.client.1.smithi193.stdout:4/832: write d1a/d24/d73/d94/d7c/f49 [2223465,86694] 0 2024-05-09T06:38:35.629 INFO:tasks.workunit.client.1.smithi193.stdout:4/833: read d1a/fc9 [179973,37424] 0 2024-05-09T06:38:35.630 INFO:tasks.workunit.client.1.smithi193.stdout:9/912: dread dd/d1e/dc2/db9/ff3 [0,4194304] 0 2024-05-09T06:38:35.639 INFO:tasks.workunit.client.1.smithi193.stdout:4/834: stat d1a/d67/d82/dec/f63 0 2024-05-09T06:38:35.647 INFO:tasks.workunit.client.1.smithi193.stdout:1/872: dwrite d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/fc1 [0,4194304] 0 2024-05-09T06:38:35.657 INFO:tasks.workunit.client.1.smithi193.stdout:4/835: creat d1a/d67/ffd x:0 0 0 2024-05-09T06:38:35.657 INFO:tasks.workunit.client.1.smithi193.stdout:9/913: symlink dd/df/d15/d8c/de6/d113/l127 0 2024-05-09T06:38:35.660 INFO:tasks.workunit.client.1.smithi193.stdout:1/873: creat d4/d16/d17/d1a/d3a/d4a/f119 x:0 0 0 2024-05-09T06:38:35.663 INFO:tasks.workunit.client.1.smithi193.stdout:7/850: getdents d6/d17 0 2024-05-09T06:38:35.666 INFO:tasks.workunit.client.1.smithi193.stdout:9/914: unlink dd/d1e/d93/dbb/ff7 0 2024-05-09T06:38:35.669 INFO:tasks.workunit.client.1.smithi193.stdout:1/874: unlink d4/d14/f44 0 2024-05-09T06:38:35.671 INFO:tasks.workunit.client.1.smithi193.stdout:1/875: chown d4/d16/d17/d1a/f1f 91 1 2024-05-09T06:38:35.672 INFO:tasks.workunit.client.1.smithi193.stdout:9/915: dwrite dd/df/f6e [0,4194304] 0 2024-05-09T06:38:35.672 INFO:tasks.workunit.client.1.smithi193.stdout:4/836: creat d1a/d22/dc3/ffe x:0 0 0 2024-05-09T06:38:35.680 INFO:tasks.workunit.client.1.smithi193.stdout:1/876: mknod d4/d16/d6b/d89/c11a 0 2024-05-09T06:38:35.683 INFO:tasks.workunit.client.1.smithi193.stdout:4/837: creat d1a/d80/fff x:0 0 0 2024-05-09T06:38:35.684 INFO:tasks.workunit.client.1.smithi193.stdout:4/838: write d1a/d67/d82/dec/f3c [3926924,72107] 0 2024-05-09T06:38:35.688 INFO:tasks.workunit.client.1.smithi193.stdout:3/784: getdents d1/db/d2f/d78/d86 0 2024-05-09T06:38:35.688 INFO:tasks.workunit.client.1.smithi193.stdout:3/785: chown d1/db/f58 704582245 1 2024-05-09T06:38:35.689 INFO:tasks.workunit.client.1.smithi193.stdout:3/786: read - d1/db/d2f/d78/fec zero size 2024-05-09T06:38:35.691 INFO:tasks.workunit.client.1.smithi193.stdout:7/851: creat d6/d8/d1a/f115 x:0 0 0 2024-05-09T06:38:35.695 INFO:tasks.workunit.client.1.smithi193.stdout:3/787: dread d1/db/d2f/d78/f8b [0,4194304] 0 2024-05-09T06:38:35.695 INFO:tasks.workunit.client.1.smithi193.stdout:4/839: chown d1a/d24/l61 13336810 1 2024-05-09T06:38:35.699 INFO:tasks.workunit.client.1.smithi193.stdout:1/877: truncate d4/f26 1901218 0 2024-05-09T06:38:35.702 INFO:tasks.workunit.client.1.smithi193.stdout:3/788: truncate d1/db/d2f/d78/d86/ddc/d51/f29 3799976 0 2024-05-09T06:38:35.702 INFO:tasks.workunit.client.1.smithi193.stdout:1/878: dread d4/d14/f2f [4194304,4194304] 0 2024-05-09T06:38:35.703 INFO:tasks.workunit.client.1.smithi193.stdout:8/833: sync 2024-05-09T06:38:35.703 INFO:tasks.workunit.client.1.smithi193.stdout:0/702: sync 2024-05-09T06:38:35.703 INFO:tasks.workunit.client.1.smithi193.stdout:5/751: sync 2024-05-09T06:38:35.705 INFO:tasks.workunit.client.1.smithi193.stdout:0/703: dread - d18/d4c/d47/f4a zero size 2024-05-09T06:38:35.705 INFO:tasks.workunit.client.1.smithi193.stdout:9/916: write dd/d1e/d3c/fda [970863,37622] 0 2024-05-09T06:38:35.705 INFO:tasks.workunit.client.1.smithi193.stdout:7/852: getdents d6/dd/dbc 0 2024-05-09T06:38:35.715 INFO:tasks.workunit.client.1.smithi193.stdout:3/789: write d1/db/f90 [757861,62853] 0 2024-05-09T06:38:35.716 INFO:tasks.workunit.client.1.smithi193.stdout:0/704: readlink d18/d27/l50 0 2024-05-09T06:38:35.716 INFO:tasks.workunit.client.1.smithi193.stdout:4/840: dwrite d1a/fa5 [0,4194304] 0 2024-05-09T06:38:35.716 INFO:tasks.workunit.client.1.smithi193.stdout:9/917: write dd/d1e/dc2/f8d [4108265,72588] 0 2024-05-09T06:38:35.725 INFO:tasks.workunit.client.1.smithi193.stdout:7/853: write d6/dd/d7e/fa3 [3692837,62173] 0 2024-05-09T06:38:35.728 INFO:tasks.workunit.client.1.smithi193.stdout:1/879: write d4/d16/d6b/d89/fe9 [807416,26055] 0 2024-05-09T06:38:35.731 INFO:tasks.workunit.client.1.smithi193.stdout:4/841: creat d1a/d24/d73/dcc/f100 x:0 0 0 2024-05-09T06:38:35.733 INFO:tasks.workunit.client.1.smithi193.stdout:5/752: dwrite d0/d13/d2c/d37/d2b/d47/d60/d87/fa5 [0,4194304] 0 2024-05-09T06:38:35.734 INFO:tasks.workunit.client.1.smithi193.stdout:3/790: dwrite d1/f11 [0,4194304] 0 2024-05-09T06:38:35.734 INFO:tasks.workunit.client.1.smithi193.stdout:9/918: dwrite dd/d1e/d3c/fda [0,4194304] 0 2024-05-09T06:38:35.736 INFO:tasks.workunit.client.1.smithi193.stdout:4/842: fsync d1a/d24/d73/dcc/fee 0 2024-05-09T06:38:35.736 INFO:tasks.workunit.client.1.smithi193.stdout:3/791: fsync d1/db/d59/fc2 0 2024-05-09T06:38:35.736 INFO:tasks.workunit.client.1.smithi193.stdout:7/854: creat d6/d8/d29/db9/f116 x:0 0 0 2024-05-09T06:38:35.738 INFO:tasks.workunit.client.1.smithi193.stdout:1/880: creat d4/d16/d35/f11b x:0 0 0 2024-05-09T06:38:35.739 INFO:tasks.workunit.client.1.smithi193.stdout:8/834: truncate d6/d1c/d18/dba/fc0 920868 0 2024-05-09T06:38:35.741 INFO:tasks.workunit.client.1.smithi193.stdout:8/835: write d6/fd9 [9519,55412] 0 2024-05-09T06:38:35.745 INFO:tasks.workunit.client.1.smithi193.stdout:9/919: dread dd/df/d15/f9e [0,4194304] 0 2024-05-09T06:38:35.749 INFO:tasks.workunit.client.1.smithi193.stdout:4/843: chown d1a/d22/c35 43 1 2024-05-09T06:38:35.750 INFO:tasks.workunit.client.1.smithi193.stdout:7/855: stat d6/d8/d1a/d36/dd0 0 2024-05-09T06:38:35.751 INFO:tasks.workunit.client.1.smithi193.stdout:7/856: read - d6/d8/d29/db9/f116 zero size 2024-05-09T06:38:35.754 INFO:tasks.workunit.client.1.smithi193.stdout:1/881: creat d4/d16/d17/d1a/f11c x:0 0 0 2024-05-09T06:38:35.756 INFO:tasks.workunit.client.1.smithi193.stdout:9/920: symlink dd/d1e/dc2/d49/dbd/d5b/d8a/dad/df8/l128 0 2024-05-09T06:38:35.757 INFO:tasks.workunit.client.1.smithi193.stdout:4/844: write d1a/d22/d43/df8/ffb [3699122,72902] 0 2024-05-09T06:38:35.759 INFO:tasks.workunit.client.1.smithi193.stdout:7/857: creat d6/dc3/f117 x:0 0 0 2024-05-09T06:38:35.760 INFO:tasks.workunit.client.1.smithi193.stdout:3/792: truncate d1/db/d2f/d78/d86/ddc/fe1 1343212 0 2024-05-09T06:38:35.763 INFO:tasks.workunit.client.1.smithi193.stdout:5/753: creat d0/ff8 x:0 0 0 2024-05-09T06:38:35.765 INFO:tasks.workunit.client.1.smithi193.stdout:1/882: mknod d4/d3c/d3e/d7c/c11d 0 2024-05-09T06:38:35.768 INFO:tasks.workunit.client.1.smithi193.stdout:4/845: rmdir d1a/d22/dc3 39 2024-05-09T06:38:35.773 INFO:tasks.workunit.client.1.smithi193.stdout:7/858: creat d6/dd/d70/dcc/f118 x:0 0 0 2024-05-09T06:38:35.780 INFO:tasks.workunit.client.1.smithi193.stdout:1/883: dread d4/d76/fe7 [0,4194304] 0 2024-05-09T06:38:35.793 INFO:tasks.workunit.client.1.smithi193.stdout:1/884: dwrite d4/d16/d6b/d89/f99 [0,4194304] 0 2024-05-09T06:38:35.796 INFO:tasks.workunit.client.1.smithi193.stdout:9/921: getdents dd 0 2024-05-09T06:38:35.798 INFO:tasks.workunit.client.1.smithi193.stdout:3/793: dwrite d1/db/da1/fd1 [0,4194304] 0 2024-05-09T06:38:35.800 INFO:tasks.workunit.client.1.smithi193.stdout:4/846: rmdir d1a/d24/dd0/de9 0 2024-05-09T06:38:35.808 INFO:tasks.workunit.client.1.smithi193.stdout:5/754: creat d0/d13/d2c/d37/d2b/d6a/d90/def/db4/ff9 x:0 0 0 2024-05-09T06:38:35.811 INFO:tasks.workunit.client.1.smithi193.stdout:4/847: symlink d1a/d22/d8e/l101 0 2024-05-09T06:38:35.814 INFO:tasks.workunit.client.1.smithi193.stdout:3/794: dread d1/d44/fd5 [0,4194304] 0 2024-05-09T06:38:35.814 INFO:tasks.workunit.client.1.smithi193.stdout:9/922: rename dd/df/d15/d8c to dd/df/d15/d98/dff/d129 0 2024-05-09T06:38:35.815 INFO:tasks.workunit.client.1.smithi193.stdout:7/859: getdents d6/d8/d1a/d36 0 2024-05-09T06:38:35.824 INFO:tasks.workunit.client.1.smithi193.stdout:4/848: creat d1a/d24/d73/d94/f102 x:0 0 0 2024-05-09T06:38:35.827 INFO:tasks.workunit.client.1.smithi193.stdout:9/923: creat dd/d1e/d1f/d61/f12a x:0 0 0 2024-05-09T06:38:35.831 INFO:tasks.workunit.client.1.smithi193.stdout:4/849: mknod d1a/d80/dc8/c103 0 2024-05-09T06:38:35.832 INFO:tasks.workunit.client.1.smithi193.stdout:3/795: dread d1/db/d59/fc2 [0,4194304] 0 2024-05-09T06:38:35.833 INFO:tasks.workunit.client.1.smithi193.stdout:1/885: truncate d4/d16/f63 3399761 0 2024-05-09T06:38:35.839 INFO:tasks.workunit.client.1.smithi193.stdout:4/850: dwrite d1a/d24/d73/d94/d7c/d5b/f70 [0,4194304] 0 2024-05-09T06:38:35.840 INFO:tasks.workunit.client.1.smithi193.stdout:4/851: chown d1a/d24/f40 403313276 1 2024-05-09T06:38:35.843 INFO:tasks.workunit.client.1.smithi193.stdout:1/886: creat d4/d16/d6b/d85/ddd/d113/f11e x:0 0 0 2024-05-09T06:38:35.847 INFO:tasks.workunit.client.1.smithi193.stdout:3/796: fsync d1/db/d2f/d78/d86/ddc/d51/f6a 0 2024-05-09T06:38:35.849 INFO:tasks.workunit.client.1.smithi193.stdout:4/852: dwrite d1a/d24/d73/d94/dbd/fef [0,4194304] 0 2024-05-09T06:38:35.849 INFO:tasks.workunit.client.1.smithi193.stdout:5/755: creat d0/d13/d2c/d37/d2b/d6a/d90/def/db4/ffa x:0 0 0 2024-05-09T06:38:35.850 INFO:tasks.workunit.client.1.smithi193.stdout:1/887: mknod d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/d7f/c11f 0 2024-05-09T06:38:35.856 INFO:tasks.workunit.client.1.smithi193.stdout:4/853: rmdir d1a/d24/dd0 39 2024-05-09T06:38:35.856 INFO:tasks.workunit.client.1.smithi193.stdout:4/854: chown d1a/ded/ldd 28 1 2024-05-09T06:38:35.857 INFO:tasks.workunit.client.1.smithi193.stdout:4/855: write d1a/d24/f41 [2404276,19366] 0 2024-05-09T06:38:35.859 INFO:tasks.workunit.client.1.smithi193.stdout:3/797: readlink d1/db/d2f/d84/d97/lbe 0 2024-05-09T06:38:35.863 INFO:tasks.workunit.client.1.smithi193.stdout:1/888: read d4/f111 [1290808,79578] 0 2024-05-09T06:38:35.865 INFO:tasks.workunit.client.1.smithi193.stdout:7/860: truncate d6/d8/d1a/d36/d4d/d74/f7c 2210670 0 2024-05-09T06:38:35.867 INFO:tasks.workunit.client.1.smithi193.stdout:4/856: symlink d1a/d67/d82/dab/l104 0 2024-05-09T06:38:35.868 INFO:tasks.workunit.client.1.smithi193.stdout:4/857: write d1a/d24/d73/d94/f102 [821102,126675] 0 2024-05-09T06:38:35.869 INFO:tasks.workunit.client.1.smithi193.stdout:5/756: creat d0/d13/d2c/d37/d5b/ffb x:0 0 0 2024-05-09T06:38:35.875 INFO:tasks.workunit.client.1.smithi193.stdout:9/924: truncate dd/df/d31/f5f 1711749 0 2024-05-09T06:38:35.878 INFO:tasks.workunit.client.1.smithi193.stdout:4/858: mknod d1a/d24/dd0/c105 0 2024-05-09T06:38:35.879 INFO:tasks.workunit.client.1.smithi193.stdout:4/859: write d1a/d98/fb8 [3311593,13120] 0 2024-05-09T06:38:35.881 INFO:tasks.workunit.client.1.smithi193.stdout:3/798: getdents d1/db/da1 0 2024-05-09T06:38:35.882 INFO:tasks.workunit.client.1.smithi193.stdout:9/925: dwrite dd/d1e/dc2/d49/f64 [0,4194304] 0 2024-05-09T06:38:35.884 INFO:tasks.workunit.client.1.smithi193.stdout:4/860: rmdir d1a/d24/d73/d94/d7c 39 2024-05-09T06:38:35.884 INFO:tasks.workunit.client.1.smithi193.stdout:9/926: read dd/d1e/fd2 [170114,82750] 0 2024-05-09T06:38:35.890 INFO:tasks.workunit.client.1.smithi193.stdout:4/861: read - d1a/d22/d43/fa2 zero size 2024-05-09T06:38:35.892 INFO:tasks.workunit.client.1.smithi193.stdout:3/799: rmdir d1/db/da4 0 2024-05-09T06:38:35.893 INFO:tasks.workunit.client.1.smithi193.stdout:4/862: read - d1a/d22/fba zero size 2024-05-09T06:38:35.895 INFO:tasks.workunit.client.1.smithi193.stdout:3/800: dread d1/db/d2f/d78/d86/ddc/ff2 [0,4194304] 0 2024-05-09T06:38:35.896 INFO:tasks.workunit.client.1.smithi193.stdout:3/801: stat d1/db/d2f/d84/d97/fcd 0 2024-05-09T06:38:35.900 INFO:tasks.workunit.client.1.smithi193.stdout:3/802: creat d1/db/d2f/d78/f110 x:0 0 0 2024-05-09T06:38:35.905 INFO:tasks.workunit.client.1.smithi193.stdout:3/803: rmdir d1/db/d59/d10d 0 2024-05-09T06:38:35.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/927: dread dd/d1e/d3c/f10c [0,4194304] 0 2024-05-09T06:38:35.910 INFO:tasks.workunit.client.1.smithi193.stdout:3/804: mkdir d1/d44/ddf/d111 0 2024-05-09T06:38:35.910 INFO:tasks.workunit.client.1.smithi193.stdout:3/805: stat d1/d44/ddf/d111 0 2024-05-09T06:38:35.911 INFO:tasks.workunit.client.1.smithi193.stdout:9/928: write dd/df/d15/d98/dff/d129/de6/d113/f60 [9234008,85334] 0 2024-05-09T06:38:35.913 INFO:tasks.workunit.client.1.smithi193.stdout:9/929: write dd/d1e/d1f/d48/d55/d86/fee [2891551,23517] 0 2024-05-09T06:38:35.914 INFO:tasks.workunit.client.1.smithi193.stdout:9/930: chown dd/d1e/fd2 51 1 2024-05-09T06:38:35.914 INFO:tasks.workunit.client.1.smithi193.stdout:1/889: truncate d4/d16/de/dca/fe3 256546 0 2024-05-09T06:38:35.918 INFO:tasks.workunit.client.1.smithi193.stdout:3/806: symlink d1/db/d2f/d84/d97/df4/df6/l112 0 2024-05-09T06:38:35.927 INFO:tasks.workunit.client.1.smithi193.stdout:1/890: symlink d4/d3c/d3e/d7c/l120 0 2024-05-09T06:38:35.927 INFO:tasks.workunit.client.1.smithi193.stdout:2/819: sync 2024-05-09T06:38:35.927 INFO:tasks.workunit.client.1.smithi193.stdout:6/720: sync 2024-05-09T06:38:35.928 INFO:tasks.workunit.client.1.smithi193.stdout:2/820: chown db/dbb 1 1 2024-05-09T06:38:35.928 INFO:tasks.workunit.client.1.smithi193.stdout:5/757: truncate d0/d13/d2c/d4d/fd7 441283 0 2024-05-09T06:38:35.929 INFO:tasks.workunit.client.1.smithi193.stdout:6/721: stat d1/de/d11/d2d/d3a/d47/fa1 0 2024-05-09T06:38:35.930 INFO:tasks.workunit.client.1.smithi193.stdout:3/807: chown d1/db/d59/d63/c89 431574056 1 2024-05-09T06:38:35.930 INFO:tasks.workunit.client.1.smithi193.stdout:2/821: readlink db/d4b/d56/l99 0 2024-05-09T06:38:35.931 INFO:tasks.workunit.client.1.smithi193.stdout:5/758: dread - d0/d13/d2c/d37/d2b/d47/d60/d9a/fde zero size 2024-05-09T06:38:35.933 INFO:tasks.workunit.client.1.smithi193.stdout:2/822: fsync db/d25/d38/d61/f70 0 2024-05-09T06:38:35.986 INFO:tasks.workunit.client.1.smithi193.stdout:5/759: chown d0/d13/d2c/d37/d5b/d4e/f6b 1 1 2024-05-09T06:38:35.989 INFO:tasks.workunit.client.1.smithi193.stdout:1/891: getdents d4/d16/d6b/d89 0 2024-05-09T06:38:35.993 INFO:tasks.workunit.client.1.smithi193.stdout:4/863: dwrite d1a/d24/d73/d94/f8f [0,4194304] 0 2024-05-09T06:38:35.993 INFO:tasks.workunit.client.1.smithi193.stdout:6/722: rename d1/d25/d2c/d76/d69/f9c to d1/d25/d2c/d3c/db1/d58/dc9/f103 0 2024-05-09T06:38:35.998 INFO:tasks.workunit.client.1.smithi193.stdout:4/864: dread d1a/f65 [0,4194304] 0 2024-05-09T06:38:36.001 INFO:tasks.workunit.client.1.smithi193.stdout:1/892: chown d4/d16/d6b/d6e/f104 42312271 1 2024-05-09T06:38:36.004 INFO:tasks.workunit.client.1.smithi193.stdout:4/865: dread d1a/d22/d43/f4d [0,4194304] 0 2024-05-09T06:38:36.004 INFO:tasks.workunit.client.1.smithi193.stdout:6/723: creat d1/d25/d2c/d81/d37/f104 x:0 0 0 2024-05-09T06:38:36.006 INFO:tasks.workunit.client.1.smithi193.stdout:2/823: link db/d4b/fde db/d25/d38/d40/db2/f116 0 2024-05-09T06:38:36.008 INFO:tasks.workunit.client.1.smithi193.stdout:5/760: symlink d0/d13/d2c/d37/d2b/lfc 0 2024-05-09T06:38:36.008 INFO:tasks.workunit.client.1.smithi193.stdout:2/824: truncate db/d25/d38/d40/db2/dcb/fdc 2921887 0 2024-05-09T06:38:36.009 INFO:tasks.workunit.client.1.smithi193.stdout:3/808: write d1/db/d2f/d78/d86/ddc/d51/f29 [1413085,65207] 0 2024-05-09T06:38:36.011 INFO:tasks.workunit.client.1.smithi193.stdout:1/893: mkdir d4/d16/d6b/d6e/d121 0 2024-05-09T06:38:36.016 INFO:tasks.workunit.client.1.smithi193.stdout:6/724: dread d1/de/f10 [0,4194304] 0 2024-05-09T06:38:36.018 INFO:tasks.workunit.client.1.smithi193.stdout:5/761: readlink d0/d13/l9e 0 2024-05-09T06:38:36.019 INFO:tasks.workunit.client.1.smithi193.stdout:5/762: read - d0/d13/d2c/d37/d2b/d47/d60/d9a/fde zero size 2024-05-09T06:38:36.029 INFO:tasks.workunit.client.1.smithi193.stdout:5/763: rmdir d0/d13/d2c/d37/d5b/d89/de0 39 2024-05-09T06:38:36.032 INFO:tasks.workunit.client.1.smithi193.stdout:3/809: mknod d1/db/d2f/d78/d86/ddc/c113 0 2024-05-09T06:38:36.037 INFO:tasks.workunit.client.1.smithi193.stdout:2/825: write f9 [3740079,15735] 0 2024-05-09T06:38:36.038 INFO:tasks.workunit.client.1.smithi193.stdout:2/826: truncate db/d4b/fd3 1029760 0 2024-05-09T06:38:36.039 INFO:tasks.workunit.client.1.smithi193.stdout:2/827: write db/def/dfd/d80/f14 [3755554,54926] 0 2024-05-09T06:38:36.043 INFO:tasks.workunit.client.1.smithi193.stdout:4/866: link d1a/d1f/c27 d1a/d67/c106 0 2024-05-09T06:38:36.043 INFO:tasks.workunit.client.1.smithi193.stdout:4/867: chown d1a/l68 51 1 2024-05-09T06:38:36.044 INFO:tasks.workunit.client.1.smithi193.stdout:6/725: link d1/de/fa6 d1/de/d11/d2d/d3a/f105 0 2024-05-09T06:38:36.049 INFO:tasks.workunit.client.1.smithi193.stdout:5/764: unlink d0/d13/d2c/d37/d2b/c46 0 2024-05-09T06:38:36.050 INFO:tasks.workunit.client.1.smithi193.stdout:6/726: dwrite d1/fee [0,4194304] 0 2024-05-09T06:38:36.050 INFO:tasks.workunit.client.1.smithi193.stdout:3/810: fdatasync f0 0 2024-05-09T06:38:36.051 INFO:tasks.workunit.client.1.smithi193.stdout:5/765: stat d0/d13/d2c/d37/d2b/d47/d60/d75/da9 0 2024-05-09T06:38:36.056 INFO:tasks.workunit.client.1.smithi193.stdout:1/894: getdents d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/d7f 0 2024-05-09T06:38:36.063 INFO:tasks.workunit.client.1.smithi193.stdout:6/727: creat d1/de/d11/d18/d94/f106 x:0 0 0 2024-05-09T06:38:36.064 INFO:tasks.workunit.client.1.smithi193.stdout:5/766: creat d0/d13/d2c/d7e/dac/ffd x:0 0 0 2024-05-09T06:38:36.065 INFO:tasks.workunit.client.1.smithi193.stdout:2/828: mkdir db/d25/d38/d40/d8b/d117 0 2024-05-09T06:38:36.066 INFO:tasks.workunit.client.1.smithi193.stdout:2/829: chown db/df/d54/d63/dca/f109 14 1 2024-05-09T06:38:36.071 INFO:tasks.workunit.client.1.smithi193.stdout:6/728: creat d1/d25/d2c/d3c/f107 x:0 0 0 2024-05-09T06:38:36.075 INFO:tasks.workunit.client.1.smithi193.stdout:4/868: rmdir d1a/d6d/dd9 0 2024-05-09T06:38:36.078 INFO:tasks.workunit.client.1.smithi193.stdout:1/895: rmdir d4/d16/d17/d1a/d3a/d3f/dd7/df2 0 2024-05-09T06:38:36.079 INFO:tasks.workunit.client.1.smithi193.stdout:2/830: symlink db/df/d106/l118 0 2024-05-09T06:38:36.080 INFO:tasks.workunit.client.1.smithi193.stdout:1/896: write d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/fc1 [25751,76624] 0 2024-05-09T06:38:36.081 INFO:tasks.workunit.client.1.smithi193.stdout:4/869: creat d1a/d67/d82/dab/f107 x:0 0 0 2024-05-09T06:38:36.081 INFO:tasks.workunit.client.1.smithi193.stdout:1/897: readlink d4/d16/d17/d1a/d3a/lbf 0 2024-05-09T06:38:36.083 INFO:tasks.workunit.client.1.smithi193.stdout:1/898: write d4/d16/d6b/d85/ff5 [556721,78635] 0 2024-05-09T06:38:36.084 INFO:tasks.workunit.client.1.smithi193.stdout:5/767: creat d0/d13/d2c/d37/d5b/d89/ffe x:0 0 0 2024-05-09T06:38:36.086 INFO:tasks.workunit.client.1.smithi193.stdout:3/811: link d1/db/d2f/l6c d1/db/d1b/l114 0 2024-05-09T06:38:36.087 INFO:tasks.workunit.client.1.smithi193.stdout:2/831: mkdir db/def/dfd/d80/d60/d7d/db3/d119 0 2024-05-09T06:38:36.093 INFO:tasks.workunit.client.1.smithi193.stdout:4/870: dwrite d1a/d24/d73/d94/d7c/d5b/f70 [0,4194304] 0 2024-05-09T06:38:36.094 INFO:tasks.workunit.client.1.smithi193.stdout:4/871: fdatasync d1a/d24/d73/dcc/f100 0 2024-05-09T06:38:36.096 INFO:tasks.workunit.client.1.smithi193.stdout:2/832: mknod db/d25/d38/d61/c11a 0 2024-05-09T06:38:36.096 INFO:tasks.workunit.client.1.smithi193.stdout:4/872: chown d1a/d24/d73/d94/d7c/f86 0 1 2024-05-09T06:38:36.101 INFO:tasks.workunit.client.1.smithi193.stdout:6/729: getdents d1/d25/d2c/d76/d69 0 2024-05-09T06:38:36.103 INFO:tasks.workunit.client.1.smithi193.stdout:5/768: mkdir d0/d13/d2c/d37/d5b/d89/de0/dff 0 2024-05-09T06:38:36.104 INFO:tasks.workunit.client.1.smithi193.stdout:5/769: stat d0/d6d/fb0 0 2024-05-09T06:38:36.109 INFO:tasks.workunit.client.1.smithi193.stdout:6/730: symlink d1/d25/d2c/d81/d37/d4f/l108 0 2024-05-09T06:38:36.109 INFO:tasks.workunit.client.1.smithi193.stdout:1/899: write d4/d14/f39 [384210,32748] 0 2024-05-09T06:38:36.150 INFO:tasks.workunit.client.1.smithi193.stdout:3/812: link d1/db/d59/l72 d1/db/d59/l115 0 2024-05-09T06:38:36.150 INFO:tasks.workunit.client.1.smithi193.stdout:4/873: write f11 [1336233,24230] 0 2024-05-09T06:38:36.150 INFO:tasks.workunit.client.1.smithi193.stdout:6/731: mknod d1/de/d11/d18/d94/c109 0 2024-05-09T06:38:36.153 INFO:tasks.workunit.client.1.smithi193.stdout:3/813: stat d1/db/d2f/l9e 0 2024-05-09T06:38:36.154 INFO:tasks.workunit.client.1.smithi193.stdout:5/770: link d0/d13/d2c/d37/d2b/lfc d0/d13/d2c/d37/d5b/de8/l100 0 2024-05-09T06:38:36.157 INFO:tasks.workunit.client.1.smithi193.stdout:5/771: fdatasync d0/d13/d2c/d37/d2b/ff6 0 2024-05-09T06:38:36.158 INFO:tasks.workunit.client.1.smithi193.stdout:4/874: dwrite d1a/d67/d82/dec/d60/fb1 [0,4194304] 0 2024-05-09T06:38:36.163 INFO:tasks.workunit.client.1.smithi193.stdout:1/900: dread d4/d3c/d3e/d7c/d10b/d10f/f18 [8388608,4194304] 0 2024-05-09T06:38:36.163 INFO:tasks.workunit.client.1.smithi193.stdout:1/901: chown d4/fec 15585895 1 2024-05-09T06:38:36.165 INFO:tasks.workunit.client.1.smithi193.stdout:4/875: rmdir d1a/da6 39 2024-05-09T06:38:36.169 INFO:tasks.workunit.client.1.smithi193.stdout:3/814: write d1/db/d2f/d78/d86/ddc/ff2 [49950,41069] 0 2024-05-09T06:38:36.171 INFO:tasks.workunit.client.1.smithi193.stdout:1/902: unlink d4/d3c/d3e/d7c/fbd 0 2024-05-09T06:38:36.172 INFO:tasks.workunit.client.1.smithi193.stdout:6/732: dwrite d1/f70 [0,4194304] 0 2024-05-09T06:38:36.175 INFO:tasks.workunit.client.1.smithi193.stdout:5/772: dwrite d0/d13/d2c/d37/d2b/d47/d60/d75/da9/fee [0,4194304] 0 2024-05-09T06:38:36.175 INFO:tasks.workunit.client.1.smithi193.stdout:6/733: stat d1/d25/d2c/d81/d37/d4f/lef 0 2024-05-09T06:38:36.180 INFO:tasks.workunit.client.1.smithi193.stdout:4/876: creat d1a/d24/f108 x:0 0 0 2024-05-09T06:38:36.182 INFO:tasks.workunit.client.1.smithi193.stdout:1/903: truncate d4/d16/d17/d1a/d3a/d4a/f119 710817 0 2024-05-09T06:38:36.184 INFO:tasks.workunit.client.1.smithi193.stdout:3/815: dwrite d1/db/d2f/d78/d86/ddc/f45 [0,4194304] 0 2024-05-09T06:38:36.188 INFO:tasks.workunit.client.1.smithi193.stdout:4/877: creat d1a/d67/f109 x:0 0 0 2024-05-09T06:38:36.191 INFO:tasks.workunit.client.1.smithi193.stdout:6/734: dwrite d1/de/d11/d2d/f40 [0,4194304] 0 2024-05-09T06:38:36.191 INFO:tasks.workunit.client.1.smithi193.stdout:5/773: chown d0/d13/d2c/d37/d5b/cb3 3219 1 2024-05-09T06:38:36.192 INFO:tasks.workunit.client.1.smithi193.stdout:1/904: creat d4/d3c/d3e/d7c/d10b/d10f/d4f/dcd/f122 x:0 0 0 2024-05-09T06:38:36.195 INFO:tasks.workunit.client.1.smithi193.stdout:3/816: truncate d1/db/d59/f69 1207927 0 2024-05-09T06:38:36.199 INFO:tasks.workunit.client.1.smithi193.stdout:6/735: unlink d1/d25/ff0 0 2024-05-09T06:38:36.200 INFO:tasks.workunit.client.1.smithi193.stdout:6/736: dread - d1/d25/d4c/d9e/feb zero size 2024-05-09T06:38:36.204 INFO:tasks.workunit.client.1.smithi193.stdout:4/878: creat d1a/da6/dd5/f10a x:0 0 0 2024-05-09T06:38:36.207 INFO:tasks.workunit.client.1.smithi193.stdout:5/774: write d0/d13/d2c/d37/d2b/d47/d66/d3a/fab [617063,115911] 0 2024-05-09T06:38:36.209 INFO:tasks.workunit.client.1.smithi193.stdout:1/905: link d4/d16/d35/f74 d4/d16/d6b/d85/ddd/dfb/f123 0 2024-05-09T06:38:36.211 INFO:tasks.workunit.client.1.smithi193.stdout:5/775: creat d0/d13/d2c/d4d/f101 x:0 0 0 2024-05-09T06:38:36.212 INFO:tasks.workunit.client.1.smithi193.stdout:4/879: dwrite d1a/d24/f108 [0,4194304] 0 2024-05-09T06:38:36.218 INFO:tasks.workunit.client.1.smithi193.stdout:5/776: rename d0/d6d to d0/d13/d2c/d37/d5b/d4e/d102 0 2024-05-09T06:38:36.220 INFO:tasks.workunit.client.1.smithi193.stdout:1/906: dread d4/d14/f51 [0,4194304] 0 2024-05-09T06:38:36.221 INFO:tasks.workunit.client.1.smithi193.stdout:4/880: rmdir d1a/d24/d73/d94 39 2024-05-09T06:38:36.224 INFO:tasks.workunit.client.1.smithi193.stdout:5/777: fdatasync d0/d13/d2c/d37/d2b/d47/d66/f83 0 2024-05-09T06:38:36.225 INFO:tasks.workunit.client.1.smithi193.stdout:6/737: dread d1/d25/f30 [0,4194304] 0 2024-05-09T06:38:36.232 INFO:tasks.workunit.client.1.smithi193.stdout:3/817: dwrite d1/d44/d7d/f9c [0,4194304] 0 2024-05-09T06:38:36.235 INFO:tasks.workunit.client.1.smithi193.stdout:6/738: creat d1/d25/d2c/d81/d37/f10a x:0 0 0 2024-05-09T06:38:36.237 INFO:tasks.workunit.client.1.smithi193.stdout:6/739: write d1/de/d11/d2d/da4/fe5 [408504,89188] 0 2024-05-09T06:38:36.240 INFO:tasks.workunit.client.1.smithi193.stdout:4/881: creat d1a/d24/f10b x:0 0 0 2024-05-09T06:38:36.242 INFO:tasks.workunit.client.1.smithi193.stdout:6/740: read d1/de/d11/d2d/d3a/f105 [46129,33944] 0 2024-05-09T06:38:36.243 INFO:tasks.workunit.client.1.smithi193.stdout:5/778: creat d0/d13/f103 x:0 0 0 2024-05-09T06:38:36.244 INFO:tasks.workunit.client.1.smithi193.stdout:5/779: stat d0/d13/d2c/d4d 0 2024-05-09T06:38:36.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/907: getdents d4/d16/d17/d1a/d5b 0 2024-05-09T06:38:36.246 INFO:tasks.workunit.client.1.smithi193.stdout:1/908: chown d4/d3c/d3e/d7c/d10b/d10f/d4f/d59/c9a 17 1 2024-05-09T06:38:36.247 INFO:tasks.workunit.client.1.smithi193.stdout:3/818: dread d1/db/f19 [4194304,4194304] 0 2024-05-09T06:38:36.247 INFO:tasks.workunit.client.1.smithi193.stdout:4/882: link d1a/d6d/c79 d1a/d1f/c10c 0 2024-05-09T06:38:36.249 INFO:tasks.workunit.client.1.smithi193.stdout:3/819: truncate d1/db/d2f/d78/d86/ddc/d51/fbd 552862 0 2024-05-09T06:38:36.249 INFO:tasks.workunit.client.1.smithi193.stdout:1/909: creat d4/d16/d17/d1a/dc5/f124 x:0 0 0 2024-05-09T06:38:36.250 INFO:tasks.workunit.client.1.smithi193.stdout:3/820: readlink d1/db/d2f/d84/le5 0 2024-05-09T06:38:36.250 INFO:tasks.workunit.client.1.smithi193.stdout:1/910: chown d4/d16/d6b/d89/fe9 3 1 2024-05-09T06:38:36.253 INFO:tasks.workunit.client.1.smithi193.stdout:3/821: dread d1/db/f19 [4194304,4194304] 0 2024-05-09T06:38:36.255 INFO:tasks.workunit.client.1.smithi193.stdout:5/780: getdents d0/d13/d2c/d37/d5b/d89 0 2024-05-09T06:38:36.264 INFO:tasks.workunit.client.1.smithi193.stdout:4/883: getdents d1a/d67/d82/dab 0 2024-05-09T06:38:36.267 INFO:tasks.workunit.client.1.smithi193.stdout:1/911: creat d4/d16/d6b/f125 x:0 0 0 2024-05-09T06:38:36.269 INFO:tasks.workunit.client.1.smithi193.stdout:5/781: mknod d0/d13/d2c/d37/d2b/d47/d66/d3a/c104 0 2024-05-09T06:38:36.271 INFO:tasks.workunit.client.1.smithi193.stdout:1/912: fdatasync d4/f87 0 2024-05-09T06:38:36.273 INFO:tasks.workunit.client.1.smithi193.stdout:3/822: dwrite d1/db/da1/fa2 [0,4194304] 0 2024-05-09T06:38:36.274 INFO:tasks.workunit.client.1.smithi193.stdout:1/913: fdatasync d4/d16/d6b/d85/f86 0 2024-05-09T06:38:36.281 INFO:tasks.workunit.client.1.smithi193.stdout:4/884: dread d1a/d1f/f2a [0,4194304] 0 2024-05-09T06:38:36.283 INFO:tasks.workunit.client.1.smithi193.stdout:5/782: dwrite d0/d13/d2c/d37/d2b/d47/d66/f23 [0,4194304] 0 2024-05-09T06:38:36.287 INFO:tasks.workunit.client.1.smithi193.stdout:1/914: dwrite d4/d76/fe7 [4194304,4194304] 0 2024-05-09T06:38:36.288 INFO:tasks.workunit.client.1.smithi193.stdout:4/885: dread d1a/d1f/f2a [0,4194304] 0 2024-05-09T06:38:36.294 INFO:tasks.workunit.client.1.smithi193.stdout:1/915: creat d4/d3c/d3e/d7c/d10b/d10f/f126 x:0 0 0 2024-05-09T06:38:36.300 INFO:tasks.workunit.client.1.smithi193.stdout:4/886: dwrite d1a/d24/f108 [0,4194304] 0 2024-05-09T06:38:36.302 INFO:tasks.workunit.client.1.smithi193.stdout:5/783: creat d0/d13/d2c/d37/d2b/d6a/d90/def/db4/f105 x:0 0 0 2024-05-09T06:38:36.303 INFO:tasks.workunit.client.1.smithi193.stdout:1/916: getdents d4/d16/dea 0 2024-05-09T06:38:36.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/784: dwrite d0/d13/d2c/d4d/f101 [0,4194304] 0 2024-05-09T06:38:36.312 INFO:tasks.workunit.client.1.smithi193.stdout:4/887: dwrite d1a/d24/d73/d94/fd8 [0,4194304] 0 2024-05-09T06:38:36.320 INFO:tasks.workunit.client.1.smithi193.stdout:5/785: rename d0/d13/d2c/d37/d2b/d47/c55 to d0/d13/d2c/db6/dc6/c106 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/917: dread d4/d16/d17/d1a/f1f [0,4194304] 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/918: symlink d4/d16/d17/d1a/d3a/d3f/dd7/l127 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:4/888: dread d1a/d24/f41 [0,4194304] 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/919: mkdir d4/d16/d6b/d85/d128 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:5/786: write d0/d13/d2c/f4b [2231871,91582] 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:4/889: rmdir d1a/d80 39 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/920: write d4/d16/d6b/d85/f86 [2298106,66093] 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/921: dread - d4/d16/d17/d1a/ff6 zero size 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/922: creat d4/d16/d17/d1a/d3a/d3f/dd7/ddb/f129 x:0 0 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:4/890: dwrite d1a/d67/d82/dec/d60/fb1 [0,4194304] 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:5/787: getdents d0/d13/d2c/d7e/dac 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/923: dwrite d4/d16/d6b/fe5 [0,4194304] 0 2024-05-09T06:38:36.372 INFO:tasks.workunit.client.1.smithi193.stdout:5/788: mknod d0/d13/d2c/d37/d2b/d6a/c107 0 2024-05-09T06:38:36.373 INFO:tasks.workunit.client.1.smithi193.stdout:1/924: chown d4/d16/d6b/f102 2 1 2024-05-09T06:38:36.373 INFO:tasks.workunit.client.1.smithi193.stdout:5/789: chown d0/d13/d2c/d37/d5b/d4e/d5d 259582 1 2024-05-09T06:38:36.373 INFO:tasks.workunit.client.1.smithi193.stdout:4/891: unlink d1a/d1f/c10c 0 2024-05-09T06:38:36.373 INFO:tasks.workunit.client.1.smithi193.stdout:5/790: unlink d0/l94 0 2024-05-09T06:38:36.373 INFO:tasks.workunit.client.1.smithi193.stdout:5/791: stat d0/d13/d2c/d37/d5b/d4e/f57 0 2024-05-09T06:38:36.373 INFO:tasks.workunit.client.1.smithi193.stdout:1/925: dwrite d4/d76/f80 [0,4194304] 0 2024-05-09T06:38:36.373 INFO:tasks.workunit.client.1.smithi193.stdout:4/892: creat d1a/d24/d73/f10d x:0 0 0 2024-05-09T06:38:36.373 INFO:tasks.workunit.client.1.smithi193.stdout:4/893: stat d1a/f65 0 2024-05-09T06:38:36.373 INFO:tasks.workunit.client.1.smithi193.stdout:1/926: fdatasync d4/d16/fb 0 2024-05-09T06:38:36.376 INFO:tasks.workunit.client.1.smithi193.stdout:4/894: write d1a/d24/f41 [1857961,129555] 0 2024-05-09T06:38:36.377 INFO:tasks.workunit.client.1.smithi193.stdout:4/895: dread - d1a/d67/d82/dec/f8c zero size 2024-05-09T06:38:36.396 INFO:tasks.workunit.client.1.smithi193.stdout:5/792: dwrite d0/d13/d2c/d37/d2b/d47/f63 [0,4194304] 0 2024-05-09T06:38:36.402 INFO:tasks.workunit.client.1.smithi193.stdout:4/896: dwrite d1a/d80/fa9 [0,4194304] 0 2024-05-09T06:38:36.411 INFO:tasks.workunit.client.1.smithi193.stdout:4/897: dwrite d1a/d80/fff [0,4194304] 0 2024-05-09T06:38:36.411 INFO:tasks.workunit.client.1.smithi193.stdout:5/793: write d0/d13/d2c/d37/d2b/d47/d60/d75/f8b [1910989,75036] 0 2024-05-09T06:38:36.413 INFO:tasks.workunit.client.1.smithi193.stdout:5/794: readlink d0/d13/d2c/d37/d2b/d47/l59 0 2024-05-09T06:38:36.413 INFO:tasks.workunit.client.1.smithi193.stdout:4/898: write d1a/d22/d8e/fb7 [133301,97427] 0 2024-05-09T06:38:36.415 INFO:tasks.workunit.client.1.smithi193.stdout:5/795: chown d0/d13/d2c/d37/d5b/d89/de0/dff 116228 1 2024-05-09T06:38:36.415 INFO:tasks.workunit.client.1.smithi193.stdout:4/899: fsync d1a/d67/f84 0 2024-05-09T06:38:36.417 INFO:tasks.workunit.client.1.smithi193.stdout:5/796: fsync d0/f7 0 2024-05-09T06:38:36.418 INFO:tasks.workunit.client.1.smithi193.stdout:4/900: mknod d1a/d67/d82/dec/c10e 0 2024-05-09T06:38:36.419 INFO:tasks.workunit.client.1.smithi193.stdout:4/901: write f11 [1477624,42579] 0 2024-05-09T06:38:36.420 INFO:tasks.workunit.client.1.smithi193.stdout:5/797: chown d0/d13/d2c/d37/d2b/d6a/deb/lf3 19103 1 2024-05-09T06:38:36.422 INFO:tasks.workunit.client.1.smithi193.stdout:4/902: symlink d1a/d22/l10f 0 2024-05-09T06:38:36.423 INFO:tasks.workunit.client.1.smithi193.stdout:4/903: rmdir d1a/d24/d73 39 2024-05-09T06:38:36.424 INFO:tasks.workunit.client.1.smithi193.stdout:5/798: truncate d0/d13/d2c/d37/d2b/d47/d66/d3a/f96 2012732 0 2024-05-09T06:38:36.426 INFO:tasks.workunit.client.1.smithi193.stdout:4/904: stat d1a/d24/d73/d94/fe0 0 2024-05-09T06:38:36.428 INFO:tasks.workunit.client.1.smithi193.stdout:5/799: mknod d0/d13/d2c/d37/d2b/d6a/d90/def/c108 0 2024-05-09T06:38:36.430 INFO:tasks.workunit.client.1.smithi193.stdout:5/800: readlink d0/d13/d2c/d37/d5b/d4e/d102/l77 0 2024-05-09T06:38:36.433 INFO:tasks.workunit.client.1.smithi193.stdout:4/905: getdents d1a/da6 0 2024-05-09T06:38:36.435 INFO:tasks.workunit.client.1.smithi193.stdout:4/906: symlink d1a/d6d/l110 0 2024-05-09T06:38:36.437 INFO:tasks.workunit.client.1.smithi193.stdout:5/801: dwrite d0/d13/d2c/d37/d5b/ffb [0,4194304] 0 2024-05-09T06:38:36.438 INFO:tasks.workunit.client.1.smithi193.stdout:4/907: symlink d1a/d98/l111 0 2024-05-09T06:38:36.439 INFO:tasks.workunit.client.1.smithi193.stdout:5/802: fdatasync d0/d13/d2c/d37/d2b/d6a/d90/def/fba 0 2024-05-09T06:38:36.444 INFO:tasks.workunit.client.1.smithi193.stdout:0/705: sync 2024-05-09T06:38:36.444 INFO:tasks.workunit.client.1.smithi193.stdout:8/836: sync 2024-05-09T06:38:36.447 INFO:tasks.workunit.client.1.smithi193.stdout:5/803: dwrite d0/d13/d2c/d37/d2b/d47/d66/d3a/fab [0,4194304] 0 2024-05-09T06:38:36.449 INFO:tasks.workunit.client.1.smithi193.stdout:5/804: write d0/d13/d2c/d37/d5b/f32 [4985395,80841] 0 2024-05-09T06:38:36.449 INFO:tasks.workunit.client.1.smithi193.stdout:0/706: symlink d18/d4c/d47/d59/d76/ld5 0 2024-05-09T06:38:36.451 INFO:tasks.workunit.client.1.smithi193.stdout:8/837: mkdir d6/d1c/d18/d66/d93/d95/d109 0 2024-05-09T06:38:36.452 INFO:tasks.workunit.client.1.smithi193.stdout:0/707: symlink d18/d4c/d47/d55/ld6 0 2024-05-09T06:38:36.458 INFO:tasks.workunit.client.1.smithi193.stdout:8/838: dwrite d6/d1c/d1b/fe6 [0,4194304] 0 2024-05-09T06:38:36.463 INFO:tasks.workunit.client.1.smithi193.stdout:0/708: creat d18/d4c/d64/d66/d9d/da5/dd0/fd7 x:0 0 0 2024-05-09T06:38:36.464 INFO:tasks.workunit.client.1.smithi193.stdout:8/839: dread d6/d1c/d1b/d29/f41 [0,4194304] 0 2024-05-09T06:38:36.464 INFO:tasks.workunit.client.1.smithi193.stdout:5/805: getdents d0/d13/d2c/d37 0 2024-05-09T06:38:36.465 INFO:tasks.workunit.client.1.smithi193.stdout:0/709: chown d18/d4c/d47/d55/fae 519269784 1 2024-05-09T06:38:36.466 INFO:tasks.workunit.client.1.smithi193.stdout:8/840: dread - d6/d1c/d1b/d29/d57/dc8/f100 zero size 2024-05-09T06:38:36.470 INFO:tasks.workunit.client.1.smithi193.stdout:0/710: dread d18/d4c/f4b [0,4194304] 0 2024-05-09T06:38:36.471 INFO:tasks.workunit.client.1.smithi193.stdout:5/806: creat d0/d13/d2c/d37/d5b/d4e/d102/dbd/dd0/f109 x:0 0 0 2024-05-09T06:38:36.479 INFO:tasks.workunit.client.1.smithi193.stdout:5/807: dread d0/d13/d2c/d37/d5b/f7b [0,4194304] 0 2024-05-09T06:38:36.480 INFO:tasks.workunit.client.1.smithi193.stdout:8/841: write d6/d1c/d18/d66/d93/fa6 [3145802,45378] 0 2024-05-09T06:38:36.499 INFO:tasks.workunit.client.1.smithi193.stdout:0/711: creat d18/dc9/fd8 x:0 0 0 2024-05-09T06:38:36.501 INFO:tasks.workunit.client.1.smithi193.stdout:5/808: unlink d0/d13/d2c/d37/d5b/d4e/d5d/f9c 0 2024-05-09T06:38:36.506 INFO:tasks.workunit.client.1.smithi193.stdout:5/809: truncate d0/d13/d2c/d4d/d67/fc7 960878 0 2024-05-09T06:38:36.506 INFO:tasks.workunit.client.1.smithi193.stdout:8/842: dread d6/d1c/f12 [0,4194304] 0 2024-05-09T06:38:36.507 INFO:tasks.workunit.client.1.smithi193.stdout:0/712: rmdir d18/d27/dc3 0 2024-05-09T06:38:36.508 INFO:tasks.workunit.client.1.smithi193.stdout:8/843: read d6/d1c/d53/fee [368812,90122] 0 2024-05-09T06:38:36.510 INFO:tasks.workunit.client.1.smithi193.stdout:5/810: creat d0/d13/d2c/da0/f10a x:0 0 0 2024-05-09T06:38:36.513 INFO:tasks.workunit.client.1.smithi193.stdout:8/844: fdatasync d6/d1c/d53/f71 0 2024-05-09T06:38:36.515 INFO:tasks.workunit.client.1.smithi193.stdout:5/811: mkdir d0/d13/d2c/d37/d2b/d6a/d90/def/d10b 0 2024-05-09T06:38:36.520 INFO:tasks.workunit.client.1.smithi193.stdout:0/713: dread d18/d27/f41 [0,4194304] 0 2024-05-09T06:38:36.521 INFO:tasks.workunit.client.1.smithi193.stdout:0/714: fdatasync d18/d4c/d47/d59/faa 0 2024-05-09T06:38:36.521 INFO:tasks.workunit.client.1.smithi193.stdout:8/845: link d6/d1c/d18/d66/d93/d95/f9f d6/d1c/d18/d66/d93/f10a 0 2024-05-09T06:38:36.522 INFO:tasks.workunit.client.1.smithi193.stdout:0/715: readlink d18/d4c/d47/d55/lc6 0 2024-05-09T06:38:36.522 INFO:tasks.workunit.client.1.smithi193.stdout:5/812: link d0/d13/d2c/d37/d5b/fbc d0/d13/d2c/d37/d5b/d4e/f10c 0 2024-05-09T06:38:36.524 INFO:tasks.workunit.client.1.smithi193.stdout:5/813: chown d0/d13/d2c/d37/d2b/d47/d60/d87 2502658 1 2024-05-09T06:38:36.526 INFO:tasks.workunit.client.1.smithi193.stdout:8/846: creat d6/d1c/d1b/d5d/d84/d87/da5/daa/f10b x:0 0 0 2024-05-09T06:38:36.527 INFO:tasks.workunit.client.1.smithi193.stdout:0/716: dread d18/d1d/f44 [0,4194304] 0 2024-05-09T06:38:36.528 INFO:tasks.workunit.client.1.smithi193.stdout:8/847: truncate d6/d1c/d1b/d5d/d84/d87/fea 216817 0 2024-05-09T06:38:36.530 INFO:tasks.workunit.client.1.smithi193.stdout:8/848: write d6/d1c/d18/d66/f78 [495313,56381] 0 2024-05-09T06:38:36.531 INFO:tasks.workunit.client.1.smithi193.stdout:8/849: chown d6/d1c/d1b/d5d/d84/dc7/d7e 74323 1 2024-05-09T06:38:36.533 INFO:tasks.workunit.client.1.smithi193.stdout:0/717: rename d18/d4c/d47/d59/d76/fbc to d18/d4c/d64/d66/fd9 0 2024-05-09T06:38:36.535 INFO:tasks.workunit.client.1.smithi193.stdout:5/814: getdents d0/d13/d2c/d37/d5b/d4e/d102/dbd/dd0 0 2024-05-09T06:38:36.536 INFO:tasks.workunit.client.1.smithi193.stdout:5/815: dread - d0/d13/d2c/da0/f10a zero size 2024-05-09T06:38:36.539 INFO:tasks.workunit.client.1.smithi193.stdout:8/850: rename d6/d1c/d27/l54 to d6/d1c/d18/d2f/d40/d5e/l10c 0 2024-05-09T06:38:36.539 INFO:tasks.workunit.client.1.smithi193.stdout:0/718: creat d18/d4c/d47/d59/d76/fda x:0 0 0 2024-05-09T06:38:36.543 INFO:tasks.workunit.client.1.smithi193.stdout:5/816: creat d0/d13/d2c/d37/d2b/d47/d6f/f10d x:0 0 0 2024-05-09T06:38:36.549 INFO:tasks.workunit.client.1.smithi193.stdout:8/851: dwrite d6/d1c/d1b/d5d/d84/d87/da5/daa/f10b [0,4194304] 0 2024-05-09T06:38:36.549 INFO:tasks.workunit.client.1.smithi193.stdout:0/719: dwrite d18/f57 [0,4194304] 0 2024-05-09T06:38:36.552 INFO:tasks.workunit.client.1.smithi193.stdout:0/720: truncate d18/f28 3446332 0 2024-05-09T06:38:36.554 INFO:tasks.workunit.client.1.smithi193.stdout:8/852: symlink d6/d1c/d1b/d5d/l10d 0 2024-05-09T06:38:36.555 INFO:tasks.workunit.client.1.smithi193.stdout:5/817: dwrite d0/d13/d2c/d37/d2b/d47/d60/d87/fa5 [0,4194304] 0 2024-05-09T06:38:36.555 INFO:tasks.workunit.client.1.smithi193.stdout:0/721: mkdir d18/d4c/d64/ddb 0 2024-05-09T06:38:36.558 INFO:tasks.workunit.client.1.smithi193.stdout:8/853: creat d6/d1c/d1b/d5d/d84/f10e x:0 0 0 2024-05-09T06:38:36.559 INFO:tasks.workunit.client.1.smithi193.stdout:0/722: fsync d18/d4c/f75 0 2024-05-09T06:38:36.562 INFO:tasks.workunit.client.1.smithi193.stdout:5/818: rename d0/d13/d2c/d37/d5b/f70 to d0/d13/d2c/d37/f10e 0 2024-05-09T06:38:36.566 INFO:tasks.workunit.client.1.smithi193.stdout:8/854: dwrite d6/d1c/d53/fdc [0,4194304] 0 2024-05-09T06:38:36.569 INFO:tasks.workunit.client.1.smithi193.stdout:8/855: write d6/d1c/d1b/d3f/fdb [2565644,28061] 0 2024-05-09T06:38:36.578 INFO:tasks.workunit.client.1.smithi193.stdout:5/819: mkdir d0/d10f 0 2024-05-09T06:38:36.589 INFO:tasks.workunit.client.1.smithi193.stdout:8/856: getdents d6/d1c/d1b/d5d 0 2024-05-09T06:38:36.594 INFO:tasks.workunit.client.1.smithi193.stdout:0/723: dwrite d18/d4c/d47/f4a [0,4194304] 0 2024-05-09T06:38:36.597 INFO:tasks.workunit.client.1.smithi193.stdout:0/724: rmdir d18/d4c/d64/d74 39 2024-05-09T06:38:36.597 INFO:tasks.workunit.client.1.smithi193.stdout:8/857: dwrite d6/d1c/d18/d66/fff [0,4194304] 0 2024-05-09T06:38:36.599 INFO:tasks.workunit.client.1.smithi193.stdout:8/858: dread - d6/d1c/d1b/d5d/d84/d87/fe1 zero size 2024-05-09T06:38:36.600 INFO:tasks.workunit.client.1.smithi193.stdout:8/859: fsync d6/d1c/d1b/f106 0 2024-05-09T06:38:36.606 INFO:tasks.workunit.client.1.smithi193.stdout:8/860: chown d6/d1c/d18/d66/l46 15469 1 2024-05-09T06:38:36.606 INFO:tasks.workunit.client.1.smithi193.stdout:0/725: write d18/d1d/f3d [1712495,105486] 0 2024-05-09T06:38:36.632 INFO:tasks.workunit.client.1.smithi193.stdout:9/931: sync 2024-05-09T06:38:36.632 INFO:tasks.workunit.client.1.smithi193.stdout:7/861: sync 2024-05-09T06:38:36.632 INFO:tasks.workunit.client.1.smithi193.stdout:2/833: sync 2024-05-09T06:38:36.632 INFO:tasks.workunit.client.1.smithi193.stdout:8/861: fdatasync d6/d1c/d1b/d5d/d84/dc7/d7e/f8a 0 2024-05-09T06:38:36.635 INFO:tasks.workunit.client.1.smithi193.stdout:5/820: dwrite d0/f85 [0,4194304] 0 2024-05-09T06:38:36.637 INFO:tasks.workunit.client.1.smithi193.stdout:7/862: fdatasync d6/d8/d1a/d36/f5b 0 2024-05-09T06:38:36.638 INFO:tasks.workunit.client.1.smithi193.stdout:0/726: symlink d18/d4c/d64/d8b/ldc 0 2024-05-09T06:38:36.638 INFO:tasks.workunit.client.1.smithi193.stdout:2/834: symlink db/def/dfd/d80/daa/db4/l11b 0 2024-05-09T06:38:36.639 INFO:tasks.workunit.client.1.smithi193.stdout:2/835: chown db/df/d54/d63 66 1 2024-05-09T06:38:36.642 INFO:tasks.workunit.client.1.smithi193.stdout:9/932: creat dd/d1e/dc2/d49/dbd/d5b/d111/f12b x:0 0 0 2024-05-09T06:38:36.643 INFO:tasks.workunit.client.1.smithi193.stdout:7/863: fdatasync d6/d17/f37 0 2024-05-09T06:38:36.649 INFO:tasks.workunit.client.1.smithi193.stdout:8/862: creat d6/d1c/d18/d66/d93/d95/d109/f10f x:0 0 0 2024-05-09T06:38:36.651 INFO:tasks.workunit.client.1.smithi193.stdout:9/933: creat dd/d1e/dc2/d49/dbd/d5b/d8a/f12c x:0 0 0 2024-05-09T06:38:36.653 INFO:tasks.workunit.client.1.smithi193.stdout:9/934: read dd/df/d15/d98/dff/d129/de6/d113/f32 [1727638,911] 0 2024-05-09T06:38:36.657 INFO:tasks.workunit.client.1.smithi193.stdout:8/863: mknod d6/d1c/d1b/d5d/d84/d87/da5/daa/c110 0 2024-05-09T06:38:36.659 INFO:tasks.workunit.client.1.smithi193.stdout:7/864: fdatasync d6/d8/d1a/d36/d4d/d74/f7c 0 2024-05-09T06:38:36.661 INFO:tasks.workunit.client.1.smithi193.stdout:9/935: rename dd/df/d15/d98/dff/d129/de6/d113/la2 to dd/d1e/dc2/l12d 0 2024-05-09T06:38:36.663 INFO:tasks.workunit.client.1.smithi193.stdout:0/727: symlink d18/d4c/d64/d74/ldd 0 2024-05-09T06:38:36.666 INFO:tasks.workunit.client.1.smithi193.stdout:5/821: dread d0/d13/d2c/d37/d2b/d47/d6f/f54 [0,4194304] 0 2024-05-09T06:38:36.667 INFO:tasks.workunit.client.1.smithi193.stdout:8/864: mkdir d6/d5b/d111 0 2024-05-09T06:38:36.668 INFO:tasks.workunit.client.1.smithi193.stdout:8/865: dread - d6/d1c/d1b/d5d/d84/f10e zero size 2024-05-09T06:38:36.671 INFO:tasks.workunit.client.1.smithi193.stdout:2/836: getdents db/def 0 2024-05-09T06:38:36.674 INFO:tasks.workunit.client.1.smithi193.stdout:8/866: creat d6/f112 x:0 0 0 2024-05-09T06:38:36.676 INFO:tasks.workunit.client.1.smithi193.stdout:7/865: truncate d6/d8/f35 2310832 0 2024-05-09T06:38:36.681 INFO:tasks.workunit.client.1.smithi193.stdout:9/936: dwrite dd/d1e/d3c/fbf [0,4194304] 0 2024-05-09T06:38:36.682 INFO:tasks.workunit.client.1.smithi193.stdout:7/866: dread d6/d8/d1a/d36/d4d/d60/ff1 [0,4194304] 0 2024-05-09T06:38:36.682 INFO:tasks.workunit.client.1.smithi193.stdout:8/867: creat d6/d1c/d1b/d5d/f113 x:0 0 0 2024-05-09T06:38:36.686 INFO:tasks.workunit.client.1.smithi193.stdout:7/867: read d6/d8/fc9 [46482,72143] 0 2024-05-09T06:38:36.687 INFO:tasks.workunit.client.1.smithi193.stdout:0/728: creat d18/d65/fde x:0 0 0 2024-05-09T06:38:36.689 INFO:tasks.workunit.client.1.smithi193.stdout:5/822: write d0/d13/d2c/d37/d5b/f6 [1583495,129141] 0 2024-05-09T06:38:36.691 INFO:tasks.workunit.client.1.smithi193.stdout:0/729: dread d18/d4c/f4b [0,4194304] 0 2024-05-09T06:38:36.693 INFO:tasks.workunit.client.1.smithi193.stdout:5/823: truncate d0/d13/f103 387635 0 2024-05-09T06:38:36.693 INFO:tasks.workunit.client.1.smithi193.stdout:9/937: mknod dd/df/d15/d98/dea/c12e 0 2024-05-09T06:38:36.694 INFO:tasks.workunit.client.1.smithi193.stdout:0/730: write d18/d65/fde [232812,39435] 0 2024-05-09T06:38:36.694 INFO:tasks.workunit.client.1.smithi193.stdout:7/868: dwrite d6/d8/d1a/f115 [0,4194304] 0 2024-05-09T06:38:36.697 INFO:tasks.workunit.client.1.smithi193.stdout:7/869: chown d6/d8/d29/db9 53228239 1 2024-05-09T06:38:36.697 INFO:tasks.workunit.client.1.smithi193.stdout:9/938: read dd/df/d31/f4c [3630938,90118] 0 2024-05-09T06:38:36.698 INFO:tasks.workunit.client.1.smithi193.stdout:9/939: chown dd/df/d31/fd1 94479227 1 2024-05-09T06:38:36.699 INFO:tasks.workunit.client.1.smithi193.stdout:2/837: link db/c98 db/def/dfd/c11c 0 2024-05-09T06:38:36.703 INFO:tasks.workunit.client.1.smithi193.stdout:5/824: readlink d0/d13/d2c/d37/d2b/d47/d66/d3a/l7f 0 2024-05-09T06:38:36.704 INFO:tasks.workunit.client.1.smithi193.stdout:0/731: mknod d18/d1d/d56/cdf 0 2024-05-09T06:38:36.710 INFO:tasks.workunit.client.1.smithi193.stdout:9/940: write dd/d1e/d1f/f2f [611904,14613] 0 2024-05-09T06:38:36.713 INFO:tasks.workunit.client.1.smithi193.stdout:2/838: symlink db/def/dfd/d80/d60/d7d/db3/l11d 0 2024-05-09T06:38:36.714 INFO:tasks.workunit.client.1.smithi193.stdout:8/868: truncate d6/d1c/d1b/d3f/f96 231230 0 2024-05-09T06:38:36.720 INFO:tasks.workunit.client.1.smithi193.stdout:7/870: creat d6/dd/f119 x:0 0 0 2024-05-09T06:38:36.720 INFO:tasks.workunit.client.1.smithi193.stdout:5/825: dwrite d0/d13/d2c/d4d/d67/fc7 [0,4194304] 0 2024-05-09T06:38:36.720 INFO:tasks.workunit.client.1.smithi193.stdout:7/871: chown d6/dd 11 1 2024-05-09T06:38:36.720 INFO:tasks.workunit.client.1.smithi193.stdout:0/732: mknod d18/d4c/d47/ce0 0 2024-05-09T06:38:36.725 INFO:tasks.workunit.client.1.smithi193.stdout:8/869: rename d6/d1c/d53/fee to d6/d1c/d18/d2f/d40/d5e/f114 0 2024-05-09T06:38:36.726 INFO:tasks.workunit.client.1.smithi193.stdout:9/941: write dd/d1e/dc2/d49/dbd/d5b/f9c [1006779,72650] 0 2024-05-09T06:38:36.730 INFO:tasks.workunit.client.1.smithi193.stdout:7/872: mkdir d6/dd/d70/dcc/d11a 0 2024-05-09T06:38:36.732 INFO:tasks.workunit.client.1.smithi193.stdout:8/870: symlink d6/d1c/d18/d66/d93/d95/d109/l115 0 2024-05-09T06:38:36.732 INFO:tasks.workunit.client.1.smithi193.stdout:5/826: dwrite d0/d13/d2c/d37/d5b/d4e/d102/dbd/dd0/fe3 [0,4194304] 0 2024-05-09T06:38:36.734 INFO:tasks.workunit.client.1.smithi193.stdout:0/733: symlink d18/d27/le1 0 2024-05-09T06:38:36.734 INFO:tasks.workunit.client.1.smithi193.stdout:7/873: write d6/d8/d1a/d36/f3b [1663007,112552] 0 2024-05-09T06:38:36.740 INFO:tasks.workunit.client.1.smithi193.stdout:9/942: dwrite dd/d1e/dc2/d49/f65 [0,4194304] 0 2024-05-09T06:38:36.742 INFO:tasks.workunit.client.1.smithi193.stdout:8/871: dread - d6/d1c/d1b/d5d/f113 zero size 2024-05-09T06:38:36.742 INFO:tasks.workunit.client.1.smithi193.stdout:5/827: creat d0/d13/d2c/da0/f110 x:0 0 0 2024-05-09T06:38:36.745 INFO:tasks.workunit.client.1.smithi193.stdout:7/874: mknod d6/d8/d29/df9/c11b 0 2024-05-09T06:38:36.746 INFO:tasks.workunit.client.1.smithi193.stdout:7/875: write d6/dd/f119 [692664,56912] 0 2024-05-09T06:38:36.748 INFO:tasks.workunit.client.1.smithi193.stdout:5/828: dread d0/d13/d2c/d37/d2b/d47/d66/f23 [0,4194304] 0 2024-05-09T06:38:36.751 INFO:tasks.workunit.client.1.smithi193.stdout:8/872: creat d6/d1c/d27/f116 x:0 0 0 2024-05-09T06:38:36.755 INFO:tasks.workunit.client.1.smithi193.stdout:9/943: mknod dd/d1e/c12f 0 2024-05-09T06:38:36.755 INFO:tasks.workunit.client.1.smithi193.stdout:0/734: write d18/d4c/d64/d66/d9d/faf [2721819,26020] 0 2024-05-09T06:38:36.756 INFO:tasks.workunit.client.1.smithi193.stdout:7/876: mknod d6/d8/d29/df9/c11c 0 2024-05-09T06:38:36.759 INFO:tasks.workunit.client.1.smithi193.stdout:9/944: dread dd/d1e/dc2/d49/f65 [0,4194304] 0 2024-05-09T06:38:36.760 INFO:tasks.workunit.client.1.smithi193.stdout:0/735: fdatasync d18/d4c/d5f/f6a 0 2024-05-09T06:38:36.764 INFO:tasks.workunit.client.1.smithi193.stdout:5/829: creat d0/d13/d2c/d37/d2b/d47/d60/d75/da9/dc1/f111 x:0 0 0 2024-05-09T06:38:36.765 INFO:tasks.workunit.client.1.smithi193.stdout:5/830: read d0/f7 [2413846,46657] 0 2024-05-09T06:38:36.771 INFO:tasks.workunit.client.1.smithi193.stdout:0/736: unlink d18/d4c/d47/d59/d76/l86 0 2024-05-09T06:38:36.773 INFO:tasks.workunit.client.1.smithi193.stdout:8/873: truncate d6/d1c/d53/f81 4868052 0 2024-05-09T06:38:36.774 INFO:tasks.workunit.client.1.smithi193.stdout:5/831: rmdir d0/d13/d2c/d37/d2b/d47/d60/dc8 39 2024-05-09T06:38:36.775 INFO:tasks.workunit.client.1.smithi193.stdout:7/877: dread d6/dd/dbc/f87 [0,4194304] 0 2024-05-09T06:38:36.776 INFO:tasks.workunit.client.1.smithi193.stdout:9/945: dwrite dd/df/d15/d98/dff/d129/de6/d113/f32 [0,4194304] 0 2024-05-09T06:38:36.776 INFO:tasks.workunit.client.1.smithi193.stdout:0/737: creat d18/d4c/d64/d66/d9d/da5/fe2 x:0 0 0 2024-05-09T06:38:36.776 INFO:tasks.workunit.client.1.smithi193.stdout:8/874: write d6/d1c/d18/d2f/d40/fbb [1952092,66665] 0 2024-05-09T06:38:36.779 INFO:tasks.workunit.client.1.smithi193.stdout:6/741: sync 2024-05-09T06:38:36.785 INFO:tasks.workunit.client.1.smithi193.stdout:7/878: symlink d6/d17/d99/dd7/l11d 0 2024-05-09T06:38:36.788 INFO:tasks.workunit.client.1.smithi193.stdout:9/946: dwrite dd/d1e/dc2/d49/f65 [0,4194304] 0 2024-05-09T06:38:36.791 INFO:tasks.workunit.client.1.smithi193.stdout:6/742: rmdir d1/de/d11/d2d 39 2024-05-09T06:38:36.792 INFO:tasks.workunit.client.1.smithi193.stdout:7/879: creat d6/dc3/f11e x:0 0 0 2024-05-09T06:38:36.792 INFO:tasks.workunit.client.1.smithi193.stdout:8/875: rmdir d6/d1c/d1b/d5d/d84/d87 39 2024-05-09T06:38:36.794 INFO:tasks.workunit.client.1.smithi193.stdout:8/876: read - d6/d1c/d1b/fa7 zero size 2024-05-09T06:38:36.796 INFO:tasks.workunit.client.1.smithi193.stdout:0/738: dwrite d18/d4c/d47/d54/fa1 [0,4194304] 0 2024-05-09T06:38:36.800 INFO:tasks.workunit.client.1.smithi193.stdout:7/880: dread d6/dd/d53/f58 [0,4194304] 0 2024-05-09T06:38:36.801 INFO:tasks.workunit.client.1.smithi193.stdout:7/881: fsync d6/d8/d1a/fb3 0 2024-05-09T06:38:36.802 INFO:tasks.workunit.client.1.smithi193.stdout:7/882: chown d6/d8/d1a/d36/d4d/d74 1483644 1 2024-05-09T06:38:36.803 INFO:tasks.workunit.client.1.smithi193.stdout:7/883: read - d6/dd/d5e/f77 zero size 2024-05-09T06:38:36.806 INFO:tasks.workunit.client.1.smithi193.stdout:8/877: read d6/d1c/d1b/d5d/d84/d87/da5/daa/f10b [3270508,91450] 0 2024-05-09T06:38:36.807 INFO:tasks.workunit.client.1.smithi193.stdout:8/878: chown d6/d1c/d18/d2f/ff3 58266752 1 2024-05-09T06:38:36.818 INFO:tasks.workunit.client.1.smithi193.stdout:7/884: creat d6/d8/d29/df9/f11f x:0 0 0 2024-05-09T06:38:36.821 INFO:tasks.workunit.client.1.smithi193.stdout:7/885: dread d6/dd/d53/f58 [0,4194304] 0 2024-05-09T06:38:36.821 INFO:tasks.workunit.client.1.smithi193.stdout:9/947: dwrite dd/df/d15/d98/dff/d129/de6/d113/f35 [0,4194304] 0 2024-05-09T06:38:36.822 INFO:tasks.workunit.client.1.smithi193.stdout:7/886: chown d6/fe9 433895804 1 2024-05-09T06:38:36.823 INFO:tasks.workunit.client.1.smithi193.stdout:0/739: truncate d18/d4c/d5f/f6a 8179701 0 2024-05-09T06:38:36.824 INFO:tasks.workunit.client.1.smithi193.stdout:6/743: creat d1/f10b x:0 0 0 2024-05-09T06:38:36.826 INFO:tasks.workunit.client.1.smithi193.stdout:8/879: chown d6/d1c/d1b/d5d/cf1 255 1 2024-05-09T06:38:36.833 INFO:tasks.workunit.client.1.smithi193.stdout:7/887: creat d6/dd/d70/dcc/f120 x:0 0 0 2024-05-09T06:38:36.838 INFO:tasks.workunit.client.1.smithi193.stdout:0/740: link d18/d4c/d47/d54/fd1 d18/d4c/d64/d8b/fe3 0 2024-05-09T06:38:36.846 INFO:tasks.workunit.client.1.smithi193.stdout:7/888: link d6/d8/d1a/d36/dd0/fec d6/dd/d70/dcc/d11a/f121 0 2024-05-09T06:38:36.847 INFO:tasks.workunit.client.1.smithi193.stdout:7/889: read - d6/d8/d1a/d36/d4d/fc7 zero size 2024-05-09T06:38:36.847 INFO:tasks.workunit.client.1.smithi193.stdout:8/880: rename d6/d1c/d1b/d3f to d6/d1c/d18/d117 0 2024-05-09T06:38:36.851 INFO:tasks.workunit.client.1.smithi193.stdout:7/890: creat d6/d8/d29/db9/f122 x:0 0 0 2024-05-09T06:38:36.852 INFO:tasks.workunit.client.1.smithi193.stdout:9/948: write dd/df/d15/f96 [1353859,20781] 0 2024-05-09T06:38:36.853 INFO:tasks.workunit.client.1.smithi193.stdout:8/881: symlink d6/d1c/d1b/d29/l118 0 2024-05-09T06:38:36.857 INFO:tasks.workunit.client.1.smithi193.stdout:7/891: dread d6/dd/f68 [0,4194304] 0 2024-05-09T06:38:36.927 INFO:tasks.workunit.client.1.smithi193.stdout:8/882: symlink d6/d1c/d1b/d29/d57/dc8/l119 0 2024-05-09T06:38:36.927 INFO:tasks.workunit.client.1.smithi193.stdout:7/892: readlink d6/d8/l76 0 2024-05-09T06:38:36.927 INFO:tasks.workunit.client.1.smithi193.stdout:9/949: getdents dd/df/d15/d98/dff/d129/de6/d106/d11b 0 2024-05-09T06:38:36.929 INFO:tasks.workunit.client.1.smithi193.stdout:7/893: chown d6/dd/d7e/d111 850297774 1 2024-05-09T06:38:36.932 INFO:tasks.workunit.client.1.smithi193.stdout:9/950: read dd/d1e/d1f/f2f [740562,124649] 0 2024-05-09T06:38:36.935 INFO:tasks.workunit.client.1.smithi193.stdout:0/741: dwrite d18/f26 [0,4194304] 0 2024-05-09T06:38:36.935 INFO:tasks.workunit.client.1.smithi193.stdout:6/744: dwrite d1/de/d11/d18/f24 [0,4194304] 0 2024-05-09T06:38:36.936 INFO:tasks.workunit.client.1.smithi193.stdout:0/742: stat d18/d4c/d5f/f8c 0 2024-05-09T06:38:36.942 INFO:tasks.workunit.client.1.smithi193.stdout:7/894: rename d6/dd/d70/dcc/f118 to d6/dd/f123 0 2024-05-09T06:38:36.942 INFO:tasks.workunit.client.1.smithi193.stdout:0/743: mkdir d18/d4c/d64/d66/db5/de4 0 2024-05-09T06:38:36.946 INFO:tasks.workunit.client.1.smithi193.stdout:6/745: rename d1/d25/d2c/d3c/db1 to d1/d25/d2c/d3c/d10c 0 2024-05-09T06:38:36.946 INFO:tasks.workunit.client.1.smithi193.stdout:8/883: link d6/d1c/d18/d2f/d40/l4e d6/d1c/d1b/d29/l11a 0 2024-05-09T06:38:36.947 INFO:tasks.workunit.client.1.smithi193.stdout:6/746: write d1/d25/d4c/d9e/feb [953203,37317] 0 2024-05-09T06:38:36.950 INFO:tasks.workunit.client.1.smithi193.stdout:7/895: creat d6/d8/d1a/d36/d4d/d60/f124 x:0 0 0 2024-05-09T06:38:36.952 INFO:tasks.workunit.client.1.smithi193.stdout:8/884: stat d6/d1c/d18/d2f/fe9 0 2024-05-09T06:38:36.957 INFO:tasks.workunit.client.1.smithi193.stdout:9/951: truncate dd/d1e/d1f/f6f 602936 0 2024-05-09T06:38:36.958 INFO:tasks.workunit.client.1.smithi193.stdout:9/952: dread - dd/d1e/dc2/d49/dbd/dab/f116 zero size 2024-05-09T06:38:36.960 INFO:tasks.workunit.client.1.smithi193.stdout:7/896: getdents d6/d17/d7d/daa 0 2024-05-09T06:38:36.961 INFO:tasks.workunit.client.1.smithi193.stdout:9/953: creat dd/d1e/dc2/d49/dbd/d5b/d8a/f130 x:0 0 0 2024-05-09T06:38:36.966 INFO:tasks.workunit.client.1.smithi193.stdout:7/897: dwrite d6/dc3/f11e [0,4194304] 0 2024-05-09T06:38:36.970 INFO:tasks.workunit.client.1.smithi193.stdout:7/898: truncate d6/d8/d29/db9/fe0 520290 0 2024-05-09T06:38:36.973 INFO:tasks.workunit.client.1.smithi193.stdout:8/885: write d6/d1c/d27/f65 [662533,96864] 0 2024-05-09T06:38:36.974 INFO:tasks.workunit.client.1.smithi193.stdout:7/899: truncate d6/d8/d29/d2f/fc0 1536019 0 2024-05-09T06:38:36.974 INFO:tasks.workunit.client.1.smithi193.stdout:3/823: sync 2024-05-09T06:38:36.977 INFO:tasks.workunit.client.1.smithi193.stdout:6/747: dwrite d1/de/d11/d2d/f95 [0,4194304] 0 2024-05-09T06:38:36.979 INFO:tasks.workunit.client.1.smithi193.stdout:3/824: write d1/db/d1b/fca [1194958,116093] 0 2024-05-09T06:38:36.983 INFO:tasks.workunit.client.1.smithi193.stdout:8/886: stat d6/d1c/d1b/c77 0 2024-05-09T06:38:36.985 INFO:tasks.workunit.client.1.smithi193.stdout:6/748: creat d1/d25/d2c/d3c/d10c/dea/f10d x:0 0 0 2024-05-09T06:38:36.987 INFO:tasks.workunit.client.1.smithi193.stdout:9/954: dwrite dd/d1e/d1f/f114 [0,4194304] 0 2024-05-09T06:38:36.988 INFO:tasks.workunit.client.1.smithi193.stdout:9/955: chown dd/d1e/dc2/d49/f70 101 1 2024-05-09T06:38:36.989 INFO:tasks.workunit.client.1.smithi193.stdout:8/887: read d6/d1c/d53/f63 [6440759,32670] 0 2024-05-09T06:38:36.990 INFO:tasks.workunit.client.1.smithi193.stdout:7/900: link d6/dd/d7e/f108 d6/d8/d29/db9/f125 0 2024-05-09T06:38:36.992 INFO:tasks.workunit.client.1.smithi193.stdout:0/744: dread d18/d1d/f33 [0,4194304] 0 2024-05-09T06:38:36.992 INFO:tasks.workunit.client.1.smithi193.stdout:9/956: read - dd/df/d15/d98/dff/d129/de6/d113/d71/f125 zero size 2024-05-09T06:38:36.994 INFO:tasks.workunit.client.1.smithi193.stdout:8/888: chown d6/d1c/d1b/d5d/d84/dc7/d7e/fce 3877829 1 2024-05-09T06:38:36.995 INFO:tasks.workunit.client.1.smithi193.stdout:3/825: mknod d1/db/da1/de4/c116 0 2024-05-09T06:38:36.995 INFO:tasks.workunit.client.1.smithi193.stdout:9/957: chown dd/df/d15/d98/dff/d129/de6/d106/d11b 25855899 1 2024-05-09T06:38:36.996 INFO:tasks.workunit.client.1.smithi193.stdout:6/749: mknod d1/de/d11/d18/d94/c10e 0 2024-05-09T06:38:37.000 INFO:tasks.workunit.client.1.smithi193.stdout:8/889: symlink d6/d1c/d18/d2f/d40/d5e/l11b 0 2024-05-09T06:38:37.001 INFO:tasks.workunit.client.1.smithi193.stdout:3/826: dread - d1/db/d2f/d84/d97/fcd zero size 2024-05-09T06:38:37.002 INFO:tasks.workunit.client.1.smithi193.stdout:8/890: write d6/d1c/d1b/d29/d57/fd8 [1025481,130445] 0 2024-05-09T06:38:37.004 INFO:tasks.workunit.client.1.smithi193.stdout:9/958: truncate dd/d1e/d1f/d48/d55/fb8 993580 0 2024-05-09T06:38:37.008 INFO:tasks.workunit.client.1.smithi193.stdout:9/959: dread dd/d1e/dc2/d49/f65 [0,4194304] 0 2024-05-09T06:38:37.009 INFO:tasks.workunit.client.1.smithi193.stdout:9/960: dread - dd/d1e/dc2/d49/dbd/d5b/d111/f12b zero size 2024-05-09T06:38:37.010 INFO:tasks.workunit.client.1.smithi193.stdout:9/961: chown dd/d1e/dc2/d49/dc1/d11d/leb 4775680 1 2024-05-09T06:38:37.015 INFO:tasks.workunit.client.1.smithi193.stdout:0/745: write d18/d4c/d47/f58 [1364040,130830] 0 2024-05-09T06:38:37.017 INFO:tasks.workunit.client.1.smithi193.stdout:8/891: mknod d6/d1c/d1b/c11c 0 2024-05-09T06:38:37.018 INFO:tasks.workunit.client.1.smithi193.stdout:0/746: chown d18/d4c/d64/d66/d9d/faf 6108 1 2024-05-09T06:38:37.018 INFO:tasks.workunit.client.1.smithi193.stdout:6/750: write d1/de/fa6 [1795791,3761] 0 2024-05-09T06:38:37.018 INFO:tasks.workunit.client.1.smithi193.stdout:9/962: dwrite dd/d1e/dc2/d49/dbd/d5b/d8a/f130 [0,4194304] 0 2024-05-09T06:38:37.022 INFO:tasks.workunit.client.1.smithi193.stdout:7/901: truncate d6/d8/d1a/f27 726025 0 2024-05-09T06:38:37.022 INFO:tasks.workunit.client.1.smithi193.stdout:0/747: rmdir d18 39 2024-05-09T06:38:37.023 INFO:tasks.workunit.client.1.smithi193.stdout:3/827: write d1/d44/f96 [921935,54821] 0 2024-05-09T06:38:37.023 INFO:tasks.workunit.client.1.smithi193.stdout:0/748: chown l11 117281 1 2024-05-09T06:38:37.027 INFO:tasks.workunit.client.1.smithi193.stdout:9/963: creat dd/d1e/dc2/d49/f131 x:0 0 0 2024-05-09T06:38:37.028 INFO:tasks.workunit.client.1.smithi193.stdout:9/964: chown fa 780 1 2024-05-09T06:38:37.034 INFO:tasks.workunit.client.1.smithi193.stdout:9/965: creat dd/df/d15/d98/dff/d129/de6/d106/d11b/f132 x:0 0 0 2024-05-09T06:38:37.035 INFO:tasks.workunit.client.1.smithi193.stdout:8/892: link d6/d1c/d1b/c77 d6/d1c/d18/d2f/c11d 0 2024-05-09T06:38:37.038 INFO:tasks.workunit.client.1.smithi193.stdout:9/966: mknod dd/d1e/dc2/d49/dbd/d5b/d8a/dad/c133 0 2024-05-09T06:38:37.039 INFO:tasks.workunit.client.1.smithi193.stdout:8/893: dread d6/d1c/d1b/fe6 [0,4194304] 0 2024-05-09T06:38:37.040 INFO:tasks.workunit.client.1.smithi193.stdout:8/894: fdatasync d6/d1c/d18/d66/db3/fe4 0 2024-05-09T06:38:37.048 INFO:tasks.workunit.client.1.smithi193.stdout:8/895: dwrite d6/d1c/d18/f74 [4194304,4194304] 0 2024-05-09T06:38:37.050 INFO:tasks.workunit.client.1.smithi193.stdout:0/749: dread d18/f25 [4194304,4194304] 0 2024-05-09T06:38:37.052 INFO:tasks.workunit.client.1.smithi193.stdout:3/828: dread d1/f24 [0,4194304] 0 2024-05-09T06:38:37.053 INFO:tasks.workunit.client.1.smithi193.stdout:3/829: fdatasync d1/db/d2f/d78/fe2 0 2024-05-09T06:38:37.054 INFO:tasks.workunit.client.1.smithi193.stdout:0/750: creat d18/d1d/d56/fe5 x:0 0 0 2024-05-09T06:38:37.055 INFO:tasks.workunit.client.1.smithi193.stdout:8/896: dwrite d6/d1c/d27/fcd [0,4194304] 0 2024-05-09T06:38:37.056 INFO:tasks.workunit.client.1.smithi193.stdout:9/967: write dd/d1e/dc2/db9/ff3 [1418192,65645] 0 2024-05-09T06:38:37.058 INFO:tasks.workunit.client.1.smithi193.stdout:8/897: fdatasync d6/d1c/d1b/d29/d57/ffc 0 2024-05-09T06:38:37.061 INFO:tasks.workunit.client.1.smithi193.stdout:0/751: unlink d18/d4c/d64/d66/l9a 0 2024-05-09T06:38:37.064 INFO:tasks.workunit.client.1.smithi193.stdout:3/830: dwrite d1/db/d2f/d78/f110 [0,4194304] 0 2024-05-09T06:38:37.067 INFO:tasks.workunit.client.1.smithi193.stdout:9/968: mkdir dd/d1e/d1f/d48/d55/d134 0 2024-05-09T06:38:37.067 INFO:tasks.workunit.client.1.smithi193.stdout:8/898: mknod d6/d1c/d1b/dfd/c11e 0 2024-05-09T06:38:37.071 INFO:tasks.workunit.client.1.smithi193.stdout:8/899: mknod d6/d1c/d18/d2f/d40/c11f 0 2024-05-09T06:38:37.074 INFO:tasks.workunit.client.1.smithi193.stdout:9/969: rename dd/d1e/dc2/d49/l58 to dd/df/d15/d98/dff/l135 0 2024-05-09T06:38:37.074 INFO:tasks.workunit.client.1.smithi193.stdout:9/970: readlink dd/d1e/dc2/d49/dbd/l74 0 2024-05-09T06:38:37.077 INFO:tasks.workunit.client.1.smithi193.stdout:9/971: chown dd/d1e/dc2/d49/dbd/dab/f116 1509 1 2024-05-09T06:38:37.079 INFO:tasks.workunit.client.1.smithi193.stdout:9/972: truncate dd/d1e/dc2/db9/ff3 2500096 0 2024-05-09T06:38:37.086 INFO:tasks.workunit.client.1.smithi193.stdout:9/973: dwrite dd/d1e/d1f/d48/d55/d86/fee [4194304,4194304] 0 2024-05-09T06:38:37.095 INFO:tasks.workunit.client.1.smithi193.stdout:8/900: dwrite d6/d1c/d53/f63 [8388608,4194304] 0 2024-05-09T06:38:37.095 INFO:tasks.workunit.client.1.smithi193.stdout:0/752: dread d18/d27/f53 [4194304,4194304] 0 2024-05-09T06:38:37.097 INFO:tasks.workunit.client.1.smithi193.stdout:0/753: chown d18/d1d/d56/da7 52569877 1 2024-05-09T06:38:37.098 INFO:tasks.workunit.client.1.smithi193.stdout:8/901: mknod d6/d1c/d27/dbd/c120 0 2024-05-09T06:38:37.100 INFO:tasks.workunit.client.1.smithi193.stdout:3/831: dread d1/db/d2f/d78/d86/ddc/d51/f7e [0,4194304] 0 2024-05-09T06:38:37.106 INFO:tasks.workunit.client.1.smithi193.stdout:8/902: unlink d6/d1c/d18/d2f/d6a/lca 0 2024-05-09T06:38:37.106 INFO:tasks.workunit.client.1.smithi193.stdout:9/974: write dd/d1e/dc2/db9/fe1 [946555,12948] 0 2024-05-09T06:38:37.109 INFO:tasks.workunit.client.1.smithi193.stdout:3/832: rename d1/db/d1b/l1c to d1/db/d2f/l117 0 2024-05-09T06:38:37.110 INFO:tasks.workunit.client.1.smithi193.stdout:3/833: truncate d1/d44/f96 2093683 0 2024-05-09T06:38:37.110 INFO:tasks.workunit.client.1.smithi193.stdout:8/903: creat d6/d1c/d27/dbd/f121 x:0 0 0 2024-05-09T06:38:37.113 INFO:tasks.workunit.client.1.smithi193.stdout:0/754: rename d18/d4c/d64/d74/dad/c71 to d18/d4c/d64/d66/d9d/da5/ce6 0 2024-05-09T06:38:37.116 INFO:tasks.workunit.client.1.smithi193.stdout:8/904: unlink d6/d1c/d1b/d29/d57/dc8/l119 0 2024-05-09T06:38:37.117 INFO:tasks.workunit.client.1.smithi193.stdout:9/975: creat dd/d1e/dc2/d49/dbd/d5b/d8a/dad/f136 x:0 0 0 2024-05-09T06:38:37.119 INFO:tasks.workunit.client.1.smithi193.stdout:9/976: chown dd/d1e/dc2/d49/dbd/d5b/d8a/lcc 0 1 2024-05-09T06:38:37.119 INFO:tasks.workunit.client.1.smithi193.stdout:0/755: dwrite d18/d4c/d64/d66/d9d/da5/dd0/fd7 [0,4194304] 0 2024-05-09T06:38:37.121 INFO:tasks.workunit.client.1.smithi193.stdout:0/756: chown d18/d4c/d64/l6d 974108 1 2024-05-09T06:38:37.121 INFO:tasks.workunit.client.1.smithi193.stdout:3/834: rename d1/db/d2f/d84/cd6 to d1/db/d59/d63/c118 0 2024-05-09T06:38:37.125 INFO:tasks.workunit.client.1.smithi193.stdout:9/977: unlink dd/df/d15/d98/dff/d129/de6/d113/d71/f125 0 2024-05-09T06:38:37.126 INFO:tasks.workunit.client.1.smithi193.stdout:9/978: readlink dd/d1e/d93/dbb/lde 0 2024-05-09T06:38:37.127 INFO:tasks.workunit.client.1.smithi193.stdout:9/979: chown dd/df/f6e 31 1 2024-05-09T06:38:37.133 INFO:tasks.workunit.client.1.smithi193.stdout:3/835: dread d1/f9 [0,4194304] 0 2024-05-09T06:38:37.134 INFO:tasks.workunit.client.1.smithi193.stdout:0/757: mknod d18/d1d/ce7 0 2024-05-09T06:38:37.138 INFO:tasks.workunit.client.1.smithi193.stdout:3/836: symlink d1/d44/ddf/l119 0 2024-05-09T06:38:37.139 INFO:tasks.workunit.client.1.smithi193.stdout:0/758: chown d18/d4c/d64/d8b/cbe 0 1 2024-05-09T06:38:37.142 INFO:tasks.workunit.client.1.smithi193.stdout:8/905: rename d6/d1c/d1b/d5d/d84/dc7/d7e/fa9 to d6/d1c/f122 0 2024-05-09T06:38:37.142 INFO:tasks.workunit.client.1.smithi193.stdout:8/906: dread - d6/d1c/d18/d66/db3/f105 zero size 2024-05-09T06:38:37.146 INFO:tasks.workunit.client.1.smithi193.stdout:3/837: symlink d1/db/d1b/l11a 0 2024-05-09T06:38:37.147 INFO:tasks.workunit.client.1.smithi193.stdout:8/907: write d6/d1c/d1b/d5d/fc4 [1406595,117774] 0 2024-05-09T06:38:37.150 INFO:tasks.workunit.client.1.smithi193.stdout:9/980: getdents dd/df/d15/d98/dea 0 2024-05-09T06:38:37.151 INFO:tasks.workunit.client.1.smithi193.stdout:0/759: write d18/d4c/d47/d55/fb9 [165547,126347] 0 2024-05-09T06:38:37.154 INFO:tasks.workunit.client.1.smithi193.stdout:3/838: chown d1/d44/f4a 5088 1 2024-05-09T06:38:37.154 INFO:tasks.workunit.client.1.smithi193.stdout:8/908: dwrite d6/d1c/d18/d2f/d40/f2c [0,4194304] 0 2024-05-09T06:38:37.165 INFO:tasks.workunit.client.1.smithi193.stdout:8/909: unlink d6/d1c/d1b/c99 0 2024-05-09T06:38:37.166 INFO:tasks.workunit.client.1.smithi193.stdout:0/760: rename d18/d4c/d64/d66/ld4 to d18/d1d/le8 0 2024-05-09T06:38:37.168 INFO:tasks.workunit.client.1.smithi193.stdout:3/839: creat d1/db/d1b/f11b x:0 0 0 2024-05-09T06:38:37.168 INFO:tasks.workunit.client.1.smithi193.stdout:8/910: truncate d6/d1c/d53/ff9 735281 0 2024-05-09T06:38:37.168 INFO:tasks.workunit.client.1.smithi193.stdout:9/981: dwrite dd/df/d15/d98/dff/d129/f50 [0,4194304] 0 2024-05-09T06:38:37.170 INFO:tasks.workunit.client.1.smithi193.stdout:3/840: chown d1/db/d1b/dbf/f101 188781226 1 2024-05-09T06:38:37.172 INFO:tasks.workunit.client.1.smithi193.stdout:0/761: mknod d18/d1d/ce9 0 2024-05-09T06:38:37.174 INFO:tasks.workunit.client.1.smithi193.stdout:3/841: mknod d1/db/d2f/d78/c11c 0 2024-05-09T06:38:37.177 INFO:tasks.workunit.client.1.smithi193.stdout:8/911: mknod d6/d1c/d18/d2f/c123 0 2024-05-09T06:38:37.179 INFO:tasks.workunit.client.1.smithi193.stdout:9/982: truncate dd/d1e/dc2/d49/dbd/d5b/d8a/dad/df8/f118 3696156 0 2024-05-09T06:38:37.182 INFO:tasks.workunit.client.1.smithi193.stdout:9/983: read dd/d1e/d3c/fda [378650,33676] 0 2024-05-09T06:38:37.184 INFO:tasks.workunit.client.1.smithi193.stdout:8/912: dread d6/d1c/f12 [0,4194304] 0 2024-05-09T06:38:37.184 INFO:tasks.workunit.client.1.smithi193.stdout:9/984: write dd/d1e/dc2/d49/dbd/d5b/f9c [552336,19822] 0 2024-05-09T06:38:37.186 INFO:tasks.workunit.client.1.smithi193.stdout:3/842: rename d1/db/d2f/d84/d97/dff to d1/db/d2f/d78/d86/ddc/d51/d11d 0 2024-05-09T06:38:37.187 INFO:tasks.workunit.client.1.smithi193.stdout:0/762: creat d18/d4c/d47/fea x:0 0 0 2024-05-09T06:38:37.190 INFO:tasks.workunit.client.1.smithi193.stdout:9/985: unlink dd/d1e/d3c/le0 0 2024-05-09T06:38:37.194 INFO:tasks.workunit.client.1.smithi193.stdout:0/763: dwrite d18/dc9/fd8 [0,4194304] 0 2024-05-09T06:38:37.201 INFO:tasks.workunit.client.1.smithi193.stdout:9/986: symlink dd/df/d15/d98/dff/d129/l137 0 2024-05-09T06:38:37.205 INFO:tasks.workunit.client.1.smithi193.stdout:0/764: creat d18/d4c/d64/d66/feb x:0 0 0 2024-05-09T06:38:37.208 INFO:tasks.workunit.client.1.smithi193.stdout:0/765: unlink d18/d1d/f3d 0 2024-05-09T06:38:37.210 INFO:tasks.workunit.client.1.smithi193.stdout:0/766: mknod d18/d4c/d64/d66/d9d/da5/cec 0 2024-05-09T06:38:37.210 INFO:tasks.workunit.client.1.smithi193.stdout:0/767: dread - d18/fc7 zero size 2024-05-09T06:38:37.213 INFO:tasks.workunit.client.1.smithi193.stdout:3/843: write d1/d44/d7d/f83 [2861236,97118] 0 2024-05-09T06:38:37.216 INFO:tasks.workunit.client.1.smithi193.stdout:8/913: dwrite d6/d1c/d1b/d5d/d84/d87/da5/daa/f10b [0,4194304] 0 2024-05-09T06:38:37.216 INFO:tasks.workunit.client.1.smithi193.stdout:0/768: dread d18/d27/fa9 [0,4194304] 0 2024-05-09T06:38:37.237 INFO:tasks.workunit.client.1.smithi193.stdout:3/844: fdatasync d1/db/d2f/f9d 0 2024-05-09T06:38:37.242 INFO:tasks.workunit.client.1.smithi193.stdout:8/914: truncate d6/d1c/d1b/f75 2055831 0 2024-05-09T06:38:37.247 INFO:tasks.workunit.client.1.smithi193.stdout:8/915: stat d6/d5b/f64 0 2024-05-09T06:38:37.248 INFO:tasks.workunit.client.1.smithi193.stdout:3/845: mknod d1/db/d2f/d78/d86/ddc/d51/d10c/c11e 0 2024-05-09T06:38:37.250 INFO:tasks.workunit.client.1.smithi193.stdout:8/916: creat d6/d1c/d18/d66/d93/f124 x:0 0 0 2024-05-09T06:38:37.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:36 smithi063 ceph-mon[27171]: pgmap v21: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 78 MiB/s rd, 175 MiB/s wr, 405 op/s 2024-05-09T06:38:37.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:36 smithi193 ceph-mon[35141]: pgmap v21: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 78 MiB/s rd, 175 MiB/s wr, 405 op/s 2024-05-09T06:38:37.415 INFO:tasks.workunit.client.1.smithi193.stdout:1/927: sync 2024-05-09T06:38:37.415 INFO:tasks.workunit.client.1.smithi193.stdout:4/908: sync 2024-05-09T06:38:37.418 INFO:tasks.workunit.client.1.smithi193.stdout:3/846: creat d1/d44/ddf/d111/f11f x:0 0 0 2024-05-09T06:38:37.419 INFO:tasks.workunit.client.1.smithi193.stdout:3/847: truncate d1/db/d1b/f10a 1253976 0 2024-05-09T06:38:37.423 INFO:tasks.workunit.client.1.smithi193.stdout:4/909: rename d1a/d24/d73/d94/cb0 to d1a/d24/c112 0 2024-05-09T06:38:37.425 INFO:tasks.workunit.client.1.smithi193.stdout:4/910: read - d1a/d67/d82/dab/f107 zero size 2024-05-09T06:38:37.428 INFO:tasks.workunit.client.1.smithi193.stdout:3/848: dread d1/db/d2f/d78/d86/ddc/d51/f6a [0,4194304] 0 2024-05-09T06:38:37.428 INFO:tasks.workunit.client.1.smithi193.stdout:1/928: rename d4/d16/d17/d1a/d3a/d4a/db1/f10a to d4/d16/d17/d1a/d3a/d3f/dd7/ddb/df8/f12a 0 2024-05-09T06:38:37.432 INFO:tasks.workunit.client.1.smithi193.stdout:8/917: dread d6/d1c/f34 [0,4194304] 0 2024-05-09T06:38:37.433 INFO:tasks.workunit.client.1.smithi193.stdout:3/849: dread d1/f9 [0,4194304] 0 2024-05-09T06:38:37.433 INFO:tasks.workunit.client.1.smithi193.stdout:8/918: chown d6/d1c/d53/fdc 16 1 2024-05-09T06:38:37.434 INFO:tasks.workunit.client.1.smithi193.stdout:8/919: write d6/d1c/d1b/f106 [278724,108418] 0 2024-05-09T06:38:37.436 INFO:tasks.workunit.client.1.smithi193.stdout:1/929: creat d4/d16/dea/f12b x:0 0 0 2024-05-09T06:38:37.437 INFO:tasks.workunit.client.1.smithi193.stdout:3/850: rmdir d1/db/d1b 39 2024-05-09T06:38:37.437 INFO:tasks.workunit.client.1.smithi193.stdout:8/920: truncate d6/d1c/d1b/d5d/d84/d87/fe1 94500 0 2024-05-09T06:38:37.438 INFO:tasks.workunit.client.1.smithi193.stdout:1/930: chown d4/d16/d17/d1a/dc5/ld0 208 1 2024-05-09T06:38:37.440 INFO:tasks.workunit.client.1.smithi193.stdout:0/769: rename d18/d4c/d64/d74/dad/d70/f94 to d18/d4c/d47/fed 0 2024-05-09T06:38:37.440 INFO:tasks.workunit.client.1.smithi193.stdout:4/911: dread d1a/d24/f2e [0,4194304] 0 2024-05-09T06:38:37.446 INFO:tasks.workunit.client.1.smithi193.stdout:0/770: rmdir d18/d4c/d47/d59 39 2024-05-09T06:38:37.451 INFO:tasks.workunit.client.1.smithi193.stdout:3/851: dread d1/db/d2f/d78/d86/ddc/f45 [0,4194304] 0 2024-05-09T06:38:37.452 INFO:tasks.workunit.client.1.smithi193.stdout:1/931: mkdir d4/d16/d6b/d12c 0 2024-05-09T06:38:37.453 INFO:tasks.workunit.client.1.smithi193.stdout:3/852: chown d1/db/l106 1333 1 2024-05-09T06:38:37.460 INFO:tasks.workunit.client.1.smithi193.stdout:8/921: link d6/d1c/d1b/d29/d57/dc8/f100 d6/d1c/f125 0 2024-05-09T06:38:37.460 INFO:tasks.workunit.client.1.smithi193.stdout:4/912: dread d1a/d24/f3d [0,4194304] 0 2024-05-09T06:38:37.463 INFO:tasks.workunit.client.1.smithi193.stdout:1/932: creat d4/d16/d6b/df9/f12d x:0 0 0 2024-05-09T06:38:37.471 INFO:tasks.workunit.client.1.smithi193.stdout:1/933: creat d4/d16/d17/d1a/d3a/d4a/db1/f12e x:0 0 0 2024-05-09T06:38:37.475 INFO:tasks.workunit.client.1.smithi193.stdout:4/913: symlink d1a/d22/l113 0 2024-05-09T06:38:37.475 INFO:tasks.workunit.client.1.smithi193.stdout:0/771: link d18/d1d/f1f d18/d65/fee 0 2024-05-09T06:38:37.476 INFO:tasks.workunit.client.1.smithi193.stdout:4/914: readlink d1a/d1f/lf4 0 2024-05-09T06:38:37.476 INFO:tasks.workunit.client.1.smithi193.stdout:3/853: dwrite d1/db/d2f/d78/f8b [0,4194304] 0 2024-05-09T06:38:37.483 INFO:tasks.workunit.client.1.smithi193.stdout:8/922: truncate d6/d1c/d1b/d5d/d84/d87/da5/daa/fda 511319 0 2024-05-09T06:38:37.485 INFO:tasks.workunit.client.1.smithi193.stdout:8/923: stat d6/d1c/d1b/d5d/d84/d87 0 2024-05-09T06:38:37.548 INFO:tasks.workunit.client.1.smithi193.stdout:0/772: readlink d18/d4c/d64/d74/ldd 0 2024-05-09T06:38:37.549 INFO:tasks.workunit.client.1.smithi193.stdout:3/854: dread - d1/db/d1b/f11b zero size 2024-05-09T06:38:37.550 INFO:tasks.workunit.client.1.smithi193.stdout:4/915: fdatasync d1a/d24/d73/d94/d7c/f34 0 2024-05-09T06:38:37.552 INFO:tasks.workunit.client.1.smithi193.stdout:4/916: stat d1a/d24/d73/d94/fae 0 2024-05-09T06:38:37.556 INFO:tasks.workunit.client.1.smithi193.stdout:7/902: sync 2024-05-09T06:38:37.556 INFO:tasks.workunit.client.1.smithi193.stdout:5/832: sync 2024-05-09T06:38:37.556 INFO:tasks.workunit.client.1.smithi193.stdout:0/773: mknod d18/d4c/d64/d74/dad/d70/cef 0 2024-05-09T06:38:37.556 INFO:tasks.workunit.client.1.smithi193.stdout:6/751: sync 2024-05-09T06:38:37.556 INFO:tasks.workunit.client.1.smithi193.stdout:2/839: sync 2024-05-09T06:38:37.558 INFO:tasks.workunit.client.1.smithi193.stdout:3/855: rename d1/f4 to d1/db/da1/de4/f120 0 2024-05-09T06:38:37.563 INFO:tasks.workunit.client.1.smithi193.stdout:4/917: symlink d1a/d67/l114 0 2024-05-09T06:38:37.565 INFO:tasks.workunit.client.1.smithi193.stdout:8/924: dread d6/d1c/d1b/d29/fb6 [0,4194304] 0 2024-05-09T06:38:37.571 INFO:tasks.workunit.client.1.smithi193.stdout:8/925: dwrite d6/d1c/d18/d66/d93/f124 [0,4194304] 0 2024-05-09T06:38:37.572 INFO:tasks.workunit.client.1.smithi193.stdout:2/840: truncate db/df/d54/fb8 1399339 0 2024-05-09T06:38:37.576 INFO:tasks.workunit.client.1.smithi193.stdout:5/833: symlink d0/d13/d2c/d37/d2b/d6a/d90/def/l112 0 2024-05-09T06:38:37.577 INFO:tasks.workunit.client.1.smithi193.stdout:3/856: truncate d1/db/d59/f52 4149259 0 2024-05-09T06:38:37.580 INFO:tasks.workunit.client.1.smithi193.stdout:1/934: getdents d4/d16/d6b/d6e 0 2024-05-09T06:38:37.581 INFO:tasks.workunit.client.1.smithi193.stdout:4/918: mknod d1a/d22/d5d/c115 0 2024-05-09T06:38:37.582 INFO:tasks.workunit.client.1.smithi193.stdout:9/987: sync 2024-05-09T06:38:37.582 INFO:tasks.workunit.client.1.smithi193.stdout:1/935: dread d4/d16/d6b/f102 [0,4194304] 0 2024-05-09T06:38:37.583 INFO:tasks.workunit.client.1.smithi193.stdout:0/774: fsync d18/d4c/d64/d8b/fa6 0 2024-05-09T06:38:37.590 INFO:tasks.workunit.client.1.smithi193.stdout:2/841: chown db/l91 116236278 1 2024-05-09T06:38:37.600 INFO:tasks.workunit.client.1.smithi193.stdout:4/919: dread d1a/d24/d73/d94/dbd/fef [0,4194304] 0 2024-05-09T06:38:37.602 INFO:tasks.workunit.client.1.smithi193.stdout:7/903: dwrite d6/d8/d29/f7a [0,4194304] 0 2024-05-09T06:38:37.603 INFO:tasks.workunit.client.1.smithi193.stdout:4/920: stat d1a/d67/d82/dab/f107 0 2024-05-09T06:38:37.604 INFO:tasks.workunit.client.1.smithi193.stdout:9/988: mknod dd/c138 0 2024-05-09T06:38:37.609 INFO:tasks.workunit.client.1.smithi193.stdout:7/904: dread - d6/d8/d29/dc8/fed zero size 2024-05-09T06:38:37.609 INFO:tasks.workunit.client.1.smithi193.stdout:6/752: dwrite d1/d25/d2c/d81/d37/d4f/fab [0,4194304] 0 2024-05-09T06:38:37.612 INFO:tasks.workunit.client.1.smithi193.stdout:1/936: fdatasync d4/d16/d35/fa9 0 2024-05-09T06:38:37.615 INFO:tasks.workunit.client.1.smithi193.stdout:4/921: dwrite d1a/d22/d43/fd4 [4194304,4194304] 0 2024-05-09T06:38:37.616 INFO:tasks.workunit.client.1.smithi193.stdout:4/922: readlink d1a/d22/d5d/l69 0 2024-05-09T06:38:37.617 INFO:tasks.workunit.client.1.smithi193.stdout:2/842: rmdir db/df/d54/d63/dca/d102 39 2024-05-09T06:38:37.617 INFO:tasks.workunit.client.1.smithi193.stdout:2/843: dread - db/def/dfd/d80/daa/f10b zero size 2024-05-09T06:38:37.621 INFO:tasks.workunit.client.1.smithi193.stdout:5/834: symlink d0/d13/d2c/d37/d2b/d47/d60/d75/da9/l113 0 2024-05-09T06:38:37.625 INFO:tasks.workunit.client.1.smithi193.stdout:2/844: dwrite db/def/dfd/d80/d60/d7d/db3/f108 [0,4194304] 0 2024-05-09T06:38:37.626 INFO:tasks.workunit.client.1.smithi193.stdout:2/845: chown db/def/dfd/dc4/f111 3357 1 2024-05-09T06:38:37.627 INFO:tasks.workunit.client.1.smithi193.stdout:2/846: readlink db/df/d106/l118 0 2024-05-09T06:38:37.633 INFO:tasks.workunit.client.1.smithi193.stdout:6/753: truncate d1/de/f53 5679265 0 2024-05-09T06:38:37.637 INFO:tasks.workunit.client.1.smithi193.stdout:4/923: truncate d1a/da6/dd5/fd7 3867301 0 2024-05-09T06:38:37.637 INFO:tasks.workunit.client.1.smithi193.stdout:8/926: link d6/d1c/d1b/d5d/d84/d87/da5/l101 d6/d1c/d1b/dfd/l126 0 2024-05-09T06:38:37.639 INFO:tasks.workunit.client.1.smithi193.stdout:7/905: dwrite d6/dd/dbc/f87 [0,4194304] 0 2024-05-09T06:38:37.641 INFO:tasks.workunit.client.1.smithi193.stdout:5/835: chown d0/d13/d2c/d37/d5b/l3c 5615553 1 2024-05-09T06:38:37.642 INFO:tasks.workunit.client.1.smithi193.stdout:3/857: write d1/db/d59/f7b [3545353,40209] 0 2024-05-09T06:38:37.642 INFO:tasks.workunit.client.1.smithi193.stdout:5/836: chown d0/d13/d2c/d37/d2b/d6a/d90/fb1 6 1 2024-05-09T06:38:37.644 INFO:tasks.workunit.client.1.smithi193.stdout:2/847: creat db/d4b/d56/f11e x:0 0 0 2024-05-09T06:38:37.644 INFO:tasks.workunit.client.1.smithi193.stdout:9/989: mkdir dd/d1e/d1f/d48/d55/d134/d139 0 2024-05-09T06:38:37.645 INFO:tasks.workunit.client.1.smithi193.stdout:6/754: chown d1/fd 231961366 1 2024-05-09T06:38:37.647 INFO:tasks.workunit.client.1.smithi193.stdout:4/924: creat d1a/d80/f116 x:0 0 0 2024-05-09T06:38:37.649 INFO:tasks.workunit.client.1.smithi193.stdout:8/927: creat d6/d1c/d1b/dfd/f127 x:0 0 0 2024-05-09T06:38:37.650 INFO:tasks.workunit.client.1.smithi193.stdout:7/906: mkdir d6/d8/d29/d2f/d126 0 2024-05-09T06:38:37.654 INFO:tasks.workunit.client.1.smithi193.stdout:3/858: read d1/db/d2f/fab [1663511,23225] 0 2024-05-09T06:38:37.655 INFO:tasks.workunit.client.1.smithi193.stdout:7/907: read d6/d8/d1a/d36/f50 [3153708,23321] 0 2024-05-09T06:38:37.656 INFO:tasks.workunit.client.1.smithi193.stdout:1/937: dread d4/d16/f42 [0,4194304] 0 2024-05-09T06:38:37.657 INFO:tasks.workunit.client.1.smithi193.stdout:3/859: chown d1/f24 524580 1 2024-05-09T06:38:37.657 INFO:tasks.workunit.client.1.smithi193.stdout:6/755: truncate d1/d25/d4c/fd5 1009648 0 2024-05-09T06:38:37.657 INFO:tasks.workunit.client.1.smithi193.stdout:5/837: dread d0/d13/d2c/d4d/fd7 [0,4194304] 0 2024-05-09T06:38:37.658 INFO:tasks.workunit.client.1.smithi193.stdout:4/925: write d1a/d22/d5d/f93 [4048257,27760] 0 2024-05-09T06:38:37.658 INFO:tasks.workunit.client.1.smithi193.stdout:8/928: mkdir d6/d1c/d27/d128 0 2024-05-09T06:38:37.661 INFO:tasks.workunit.client.1.smithi193.stdout:7/908: chown d6/d8/d1a/d36/d4d/d60/c67 13 1 2024-05-09T06:38:37.662 INFO:tasks.workunit.client.1.smithi193.stdout:7/909: write d6/dd/ffe [456447,92498] 0 2024-05-09T06:38:37.668 INFO:tasks.workunit.client.1.smithi193.stdout:3/860: unlink d1/d44/d7d/c109 0 2024-05-09T06:38:37.668 INFO:tasks.workunit.client.1.smithi193.stdout:4/926: dwrite d1a/da6/dd5/f10a [0,4194304] 0 2024-05-09T06:38:37.687 INFO:tasks.workunit.client.1.smithi193.stdout:7/910: dwrite d6/d8/d29/f7a [4194304,4194304] 0 2024-05-09T06:38:37.688 INFO:tasks.workunit.client.1.smithi193.stdout:5/838: mknod d0/c114 0 2024-05-09T06:38:37.691 INFO:tasks.workunit.client.1.smithi193.stdout:0/775: truncate d18/d1d/f30 2777739 0 2024-05-09T06:38:37.694 INFO:tasks.workunit.client.1.smithi193.stdout:9/990: link dd/d1e/dc2/l69 dd/d1e/dc2/d49/dbd/d115/d117/db2/l13a 0 2024-05-09T06:38:37.699 INFO:tasks.workunit.client.1.smithi193.stdout:3/861: rmdir d1/d44/ddf 39 2024-05-09T06:38:37.700 INFO:tasks.workunit.client.1.smithi193.stdout:3/862: write d1/db/d2f/d78/fe2 [889822,96616] 0 2024-05-09T06:38:37.700 INFO:tasks.workunit.client.1.smithi193.stdout:4/927: mkdir d1a/d6d/d117 0 2024-05-09T06:38:37.707 INFO:tasks.workunit.client.1.smithi193.stdout:2/848: dwrite db/def/dfd/d80/fa6 [4194304,4194304] 0 2024-05-09T06:38:37.709 INFO:tasks.workunit.client.1.smithi193.stdout:7/911: rmdir d6/d8/d1a/d36/d4d/d60/d9c 39 2024-05-09T06:38:37.711 INFO:tasks.workunit.client.1.smithi193.stdout:7/912: chown d6/d8/d1a/d36/d4d/d60/f124 220305062 1 2024-05-09T06:38:37.714 INFO:tasks.workunit.client.1.smithi193.stdout:1/938: write d4/d16/d6b/fc0 [439817,90512] 0 2024-05-09T06:38:37.722 INFO:tasks.workunit.client.1.smithi193.stdout:6/756: rmdir d1/d25/d2c/d76/d100 0 2024-05-09T06:38:37.726 INFO:tasks.workunit.client.1.smithi193.stdout:8/929: rename d6/d1c/f122 to d6/d1c/d1b/d5d/d84/dc7/f129 0 2024-05-09T06:38:37.727 INFO:tasks.workunit.client.1.smithi193.stdout:8/930: dread - d6/d1c/d27/f7d zero size 2024-05-09T06:38:37.727 INFO:tasks.workunit.client.1.smithi193.stdout:6/757: read d1/de/d11/d18/d94/dac/ffe [4083044,93628] 0 2024-05-09T06:38:37.728 INFO:tasks.workunit.client.1.smithi193.stdout:6/758: write d1/f42 [1166647,2972] 0 2024-05-09T06:38:37.731 INFO:tasks.workunit.client.1.smithi193.stdout:7/913: creat d6/d17/d99/dd7/f127 x:0 0 0 2024-05-09T06:38:37.732 INFO:tasks.workunit.client.1.smithi193.stdout:0/776: fsync d18/d4c/d47/d59/d76/f87 0 2024-05-09T06:38:37.735 INFO:tasks.workunit.client.1.smithi193.stdout:1/939: mkdir d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/d12f 0 2024-05-09T06:38:37.737 INFO:tasks.workunit.client.1.smithi193.stdout:3/863: mknod d1/db/d59/d63/df9/c121 0 2024-05-09T06:38:37.739 INFO:tasks.workunit.client.1.smithi193.stdout:4/928: rename d1a/d24/d73/d94/dbd/fef to d1a/d67/d82/dab/f118 0 2024-05-09T06:38:37.740 INFO:tasks.workunit.client.1.smithi193.stdout:4/929: fdatasync d1a/d80/fff 0 2024-05-09T06:38:37.743 INFO:tasks.workunit.client.1.smithi193.stdout:9/991: write dd/d1e/dc2/d49/f65 [675875,4435] 0 2024-05-09T06:38:37.748 INFO:tasks.workunit.client.1.smithi193.stdout:0/777: write d18/d4c/d64/d66/d9d/faf [5333430,56115] 0 2024-05-09T06:38:37.750 INFO:tasks.workunit.client.1.smithi193.stdout:0/778: chown d18/d4c/d47/d59/d76 34685 1 2024-05-09T06:38:37.750 INFO:tasks.workunit.client.1.smithi193.stdout:5/839: link d0/d13/d2c/d37/d2b/d47/d60/d75/cae d0/d13/d2c/d37/d2b/d47/d60/d75/da9/dc1/c115 0 2024-05-09T06:38:37.751 INFO:tasks.workunit.client.1.smithi193.stdout:0/779: chown d18/d4c/d64/d66/feb 370 1 2024-05-09T06:38:37.751 INFO:tasks.workunit.client.1.smithi193.stdout:2/849: write db/d25/f27 [2629914,83041] 0 2024-05-09T06:38:37.757 INFO:tasks.workunit.client.1.smithi193.stdout:8/931: rename d6/d1c/d53/feb to d6/d1c/d1b/d29/d57/f12a 0 2024-05-09T06:38:37.771 INFO:tasks.workunit.client.1.smithi193.stdout:4/930: write d1a/d24/d73/d94/faf [452039,128724] 0 2024-05-09T06:38:37.777 INFO:tasks.workunit.client.1.smithi193.stdout:9/992: dwrite dd/d1e/dc2/d49/f70 [0,4194304] 0 2024-05-09T06:38:37.777 INFO:tasks.workunit.client.1.smithi193.stdout:2/850: mkdir db/def/dfd/d80/daa/db4/d11f 0 2024-05-09T06:38:37.779 INFO:tasks.workunit.client.1.smithi193.stdout:9/993: readlink dd/d1e/dc2/d49/dbd/d5b/l99 0 2024-05-09T06:38:37.781 INFO:tasks.workunit.client.1.smithi193.stdout:7/914: rename d6/dd/d7e/fd5 to d6/d8/d29/df9/f128 0 2024-05-09T06:38:37.781 INFO:tasks.workunit.client.1.smithi193.stdout:7/915: readlink d6/d17/d7d/l101 0 2024-05-09T06:38:37.783 INFO:tasks.workunit.client.1.smithi193.stdout:8/932: mkdir d6/d1c/d18/d2f/d40/d5e/d12b 0 2024-05-09T06:38:37.785 INFO:tasks.workunit.client.1.smithi193.stdout:6/759: rmdir d1/d25/d4c/df3 0 2024-05-09T06:38:37.788 INFO:tasks.workunit.client.1.smithi193.stdout:5/840: mknod d0/d13/d2c/d37/c116 0 2024-05-09T06:38:37.790 INFO:tasks.workunit.client.1.smithi193.stdout:5/841: chown d0/d13/d2c/d4d/fd7 1 1 2024-05-09T06:38:37.791 INFO:tasks.workunit.client.1.smithi193.stdout:3/864: dwrite f0 [0,4194304] 0 2024-05-09T06:38:37.794 INFO:tasks.workunit.client.1.smithi193.stdout:5/842: write d0/d13/d2c/d37/d2b/d6a/d90/fc3 [430401,29779] 0 2024-05-09T06:38:37.794 INFO:tasks.workunit.client.1.smithi193.stdout:1/940: rmdir d4/d16/d6b/d6e/d121 0 2024-05-09T06:38:37.801 INFO:tasks.workunit.client.1.smithi193.stdout:9/994: rename dd/df/dc4/c11f to dd/df/d15/d98/dff/d129/de6/d106/d11b/c13b 0 2024-05-09T06:38:37.802 INFO:tasks.workunit.client.1.smithi193.stdout:4/931: write d1a/d67/d82/dec/d60/fbe [196350,126961] 0 2024-05-09T06:38:37.803 INFO:tasks.workunit.client.1.smithi193.stdout:7/916: creat d6/d8/d1a/d36/d4d/d74/f129 x:0 0 0 2024-05-09T06:38:37.804 INFO:tasks.workunit.client.1.smithi193.stdout:7/917: fsync d6/dd/d53/f6a 0 2024-05-09T06:38:37.805 INFO:tasks.workunit.client.1.smithi193.stdout:6/760: fdatasync d1/de/d11/f8b 0 2024-05-09T06:38:37.809 INFO:tasks.workunit.client.1.smithi193.stdout:8/933: dread d6/d1c/d18/d66/f78 [0,4194304] 0 2024-05-09T06:38:37.813 INFO:tasks.workunit.client.1.smithi193.stdout:3/865: mknod d1/db/d59/d63/df9/c122 0 2024-05-09T06:38:37.818 INFO:tasks.workunit.client.1.smithi193.stdout:7/918: dread d6/dd/d70/f102 [0,4194304] 0 2024-05-09T06:38:37.819 INFO:tasks.workunit.client.1.smithi193.stdout:7/919: write d6/d8/d1a/d36/d4d/fb0 [1100531,83773] 0 2024-05-09T06:38:37.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/941: mknod d4/d16/d17/d1a/d3a/d3f/dd7/c130 0 2024-05-09T06:38:37.823 INFO:tasks.workunit.client.1.smithi193.stdout:2/851: truncate db/dbb/fd7 1196447 0 2024-05-09T06:38:37.828 INFO:tasks.workunit.client.1.smithi193.stdout:6/761: dread - d1/d25/d2c/d3c/d10c/dea/fd8 zero size 2024-05-09T06:38:37.829 INFO:tasks.workunit.client.1.smithi193.stdout:6/762: chown d1/fee 7373 1 2024-05-09T06:38:37.830 INFO:tasks.workunit.client.1.smithi193.stdout:0/780: getdents d18/d4c/d64/d74/dad/d70 0 2024-05-09T06:38:37.830 INFO:tasks.workunit.client.1.smithi193.stdout:6/763: stat d1/d25/d4c/d9e/dc8/ld6 0 2024-05-09T06:38:37.831 INFO:tasks.workunit.client.1.smithi193.stdout:8/934: creat d6/d1c/d1b/d29/d57/dc8/f12c x:0 0 0 2024-05-09T06:38:37.837 INFO:tasks.workunit.client.1.smithi193.stdout:3/866: creat d1/db/da1/f123 x:0 0 0 2024-05-09T06:38:37.837 INFO:tasks.workunit.client.1.smithi193.stdout:2/852: dread db/def/dfd/fd0 [0,4194304] 0 2024-05-09T06:38:37.840 INFO:tasks.workunit.client.1.smithi193.stdout:2/853: fdatasync db/d25/d38/d40/db2/dcb/fdc 0 2024-05-09T06:38:37.846 INFO:tasks.workunit.client.1.smithi193.stdout:3/867: dwrite d1/db/d2f/d78/f110 [4194304,4194304] 0 2024-05-09T06:38:37.848 INFO:tasks.workunit.client.1.smithi193.stdout:4/932: dwrite d1a/d6d/fa3 [4194304,4194304] 0 2024-05-09T06:38:37.849 INFO:tasks.workunit.client.1.smithi193.stdout:8/935: symlink d6/l12d 0 2024-05-09T06:38:37.852 INFO:tasks.workunit.client.1.smithi193.stdout:8/936: truncate d6/d1c/d1b/d29/d57/ffc 303184 0 2024-05-09T06:38:37.856 INFO:tasks.workunit.client.1.smithi193.stdout:7/920: mkdir d6/d8/d1a/d36/d4d/d12a 0 2024-05-09T06:38:37.856 INFO:tasks.workunit.client.1.smithi193.stdout:8/937: write d6/d1c/d1b/d5d/d84/d87/fea [1078785,42995] 0 2024-05-09T06:38:37.859 INFO:tasks.workunit.client.1.smithi193.stdout:2/854: read db/d37/fdb [2471299,43695] 0 2024-05-09T06:38:37.860 INFO:tasks.workunit.client.1.smithi193.stdout:2/855: readlink db/df/d54/d63/l6f 0 2024-05-09T06:38:37.865 INFO:tasks.workunit.client.1.smithi193.stdout:8/938: dwrite d6/d1c/d27/dbd/f121 [0,4194304] 0 2024-05-09T06:38:37.866 INFO:tasks.workunit.client.1.smithi193.stdout:6/764: symlink d1/de/d11/d18/l10f 0 2024-05-09T06:38:37.867 INFO:tasks.workunit.client.1.smithi193.stdout:8/939: write d6/d1c/d1b/d29/d57/dc8/f12c [646082,22758] 0 2024-05-09T06:38:37.873 INFO:tasks.workunit.client.1.smithi193.stdout:3/868: creat d1/db/d2f/d84/d97/df4/df6/f124 x:0 0 0 2024-05-09T06:38:37.876 INFO:tasks.workunit.client.1.smithi193.stdout:8/940: dwrite d6/d1c/d18/d117/fdb [0,4194304] 0 2024-05-09T06:38:37.876 INFO:tasks.workunit.client.1.smithi193.stdout:7/921: symlink d6/dc3/l12b 0 2024-05-09T06:38:37.878 INFO:tasks.workunit.client.1.smithi193.stdout:5/843: getdents d0/d13/d2c/d37/d2b/d47/d6f 0 2024-05-09T06:38:37.883 INFO:tasks.workunit.client.1.smithi193.stdout:9/995: dwrite dd/df/d15/d98/dff/d129/de6/d113/d37/f83 [0,4194304] 0 2024-05-09T06:38:37.891 INFO:tasks.workunit.client.1.smithi193.stdout:6/765: read d1/f86 [289221,54073] 0 2024-05-09T06:38:37.892 INFO:tasks.workunit.client.1.smithi193.stdout:5/844: dwrite d0/d13/d2c/d7e/dac/ffd [0,4194304] 0 2024-05-09T06:38:37.902 INFO:tasks.workunit.client.1.smithi193.stdout:3/869: mkdir d1/db/d2f/d78/d86/ddc/d51/d11d/d125 0 2024-05-09T06:38:37.902 INFO:tasks.workunit.client.1.smithi193.stdout:3/870: fsync d1/db/d2f/d78/fe2 0 2024-05-09T06:38:37.903 INFO:tasks.workunit.client.1.smithi193.stdout:7/922: dread d6/dd/d7e/fcd [0,4194304] 0 2024-05-09T06:38:37.905 INFO:tasks.workunit.client.1.smithi193.stdout:8/941: symlink d6/d1c/d1b/d29/d57/l12e 0 2024-05-09T06:38:37.908 INFO:tasks.workunit.client.1.smithi193.stdout:9/996: creat dd/d1e/dc2/d49/f13c x:0 0 0 2024-05-09T06:38:37.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/997: stat dd/d1e/d1f/d61/cdf 0 2024-05-09T06:38:37.911 INFO:tasks.workunit.client.1.smithi193.stdout:6/766: dread - d1/de/d11/d20/fbf zero size 2024-05-09T06:38:37.911 INFO:tasks.workunit.client.1.smithi193.stdout:6/767: chown d1/de/c12 42 1 2024-05-09T06:38:37.914 INFO:tasks.workunit.client.1.smithi193.stdout:5/845: mkdir d0/d13/d2c/d37/d2b/d6a/d90/def/d117 0 2024-05-09T06:38:37.915 INFO:tasks.workunit.client.1.smithi193.stdout:0/781: getdents d18/d4c/d64/d66/d9d/da5 0 2024-05-09T06:38:37.917 INFO:tasks.workunit.client.1.smithi193.stdout:4/933: link d1a/d22/d43/c47 d1a/d98/c119 0 2024-05-09T06:38:37.920 INFO:tasks.workunit.client.1.smithi193.stdout:1/942: write d4/d16/d35/f74 [295949,61035] 0 2024-05-09T06:38:37.922 INFO:tasks.workunit.client.1.smithi193.stdout:8/942: dread - d6/d1c/d1b/d5d/d84/dc7/d7e/fce zero size 2024-05-09T06:38:37.928 INFO:tasks.workunit.client.1.smithi193.stdout:6/768: mknod d1/d25/d4c/c110 0 2024-05-09T06:38:37.931 INFO:tasks.workunit.client.1.smithi193.stdout:8/943: dread d6/d1c/d18/d117/fdb [0,4194304] 0 2024-05-09T06:38:37.941 INFO:tasks.workunit.client.1.smithi193.stdout:3/871: symlink d1/db/d2f/d78/d86/ddc/l126 0 2024-05-09T06:38:37.942 INFO:tasks.workunit.client.1.smithi193.stdout:7/923: symlink d6/d8/d1a/d36/d4d/d12a/l12c 0 2024-05-09T06:38:37.945 INFO:tasks.workunit.client.1.smithi193.stdout:1/943: mknod d4/d16/de/dca/c131 0 2024-05-09T06:38:37.946 INFO:tasks.workunit.client.1.smithi193.stdout:7/924: read d6/d8/d1a/fa4 [3175627,47992] 0 2024-05-09T06:38:37.946 INFO:tasks.workunit.client.1.smithi193.stdout:1/944: chown d4/d16/d6b/d85/ddd/d113/f11e 53 1 2024-05-09T06:38:37.955 INFO:tasks.workunit.client.1.smithi193.stdout:7/925: dwrite d6/dd/d5e/d6b/ffb [0,4194304] 0 2024-05-09T06:38:38.019 INFO:tasks.workunit.client.1.smithi193.stdout:7/926: dread d6/d8/d29/db9/f125 [0,4194304] 0 2024-05-09T06:38:38.021 INFO:tasks.workunit.client.1.smithi193.stdout:6/769: dread d1/fee [0,4194304] 0 2024-05-09T06:38:38.023 INFO:tasks.workunit.client.1.smithi193.stdout:6/770: dread - d1/d25/d2c/d81/d37/f104 zero size 2024-05-09T06:38:38.108 INFO:tasks.workunit.client.1.smithi193.stdout:1/945: truncate d4/d3c/d3e/ff3 680315 0 2024-05-09T06:38:38.121 INFO:tasks.workunit.client.1.smithi193.stdout:2/856: write db/f45 [212915,72672] 0 2024-05-09T06:38:38.122 INFO:tasks.workunit.client.1.smithi193.stdout:7/927: stat d6/d17/d99/c106 0 2024-05-09T06:38:38.123 INFO:tasks.workunit.client.1.smithi193.stdout:7/928: stat d6/d8/c69 0 2024-05-09T06:38:38.125 INFO:tasks.workunit.client.1.smithi193.stdout:7/929: dread d6/d17/d7d/fff [0,4194304] 0 2024-05-09T06:38:38.126 INFO:tasks.workunit.client.1.smithi193.stdout:4/934: getdents d1a/d67/d82/dec 0 2024-05-09T06:38:38.140 INFO:tasks.workunit.client.1.smithi193.stdout:9/998: write dd/d1e/dc2/d49/dbd/d115/d117/f78 [1177042,11475] 0 2024-05-09T06:38:38.142 INFO:tasks.workunit.client.1.smithi193.stdout:6/771: getdents d1/de/d11/d2d/d3a/d47 0 2024-05-09T06:38:38.143 INFO:tasks.workunit.client.1.smithi193.stdout:5/846: write d0/d13/d2c/d37/d5b/d89/f95 [1001099,60858] 0 2024-05-09T06:38:38.144 INFO:tasks.workunit.client.1.smithi193.stdout:9/999: unlink dd/df/c75 0 2024-05-09T06:38:38.145 INFO:tasks.workunit.client.1.smithi193.stdout:6/772: fdatasync d1/de/ff 0 2024-05-09T06:38:38.157 INFO:tasks.workunit.client.1.smithi193.stdout:0/782: dwrite d18/d4c/d47/fed [0,4194304] 0 2024-05-09T06:38:38.161 INFO:tasks.workunit.client.1.smithi193.stdout:3/872: dwrite d1/db/d2f/d78/fb4 [0,4194304] 0 2024-05-09T06:38:38.172 INFO:tasks.workunit.client.1.smithi193.stdout:3/873: dread d1/db/d1b/f70 [0,4194304] 0 2024-05-09T06:38:38.173 INFO:tasks.workunit.client.1.smithi193.stdout:3/874: dread - d1/db/d2f/f9d zero size 2024-05-09T06:38:38.177 INFO:tasks.workunit.client.1.smithi193.stdout:3/875: read d1/db/d59/f43 [1749265,96739] 0 2024-05-09T06:38:38.192 INFO:tasks.workunit.client.1.smithi193.stdout:7/930: mknod d6/d8/d29/c12d 0 2024-05-09T06:38:38.246 INFO:tasks.workunit.client.1.smithi193.stdout:2/857: dwrite db/d25/f48 [0,4194304] 0 2024-05-09T06:38:38.247 INFO:tasks.workunit.client.1.smithi193.stdout:4/935: dwrite f10 [0,4194304] 0 2024-05-09T06:38:38.249 INFO:tasks.workunit.client.1.smithi193.stdout:4/936: chown d1a/d67/l114 0 1 2024-05-09T06:38:38.254 INFO:tasks.workunit.client.1.smithi193.stdout:4/937: truncate d1a/d24/d73/d94/f52 301801 0 2024-05-09T06:38:38.257 INFO:tasks.workunit.client.1.smithi193.stdout:4/938: read d1a/fa5 [2441605,101780] 0 2024-05-09T06:38:38.258 INFO:tasks.workunit.client.1.smithi193.stdout:4/939: write d1a/d24/d73/dcc/ff3 [437044,16935] 0 2024-05-09T06:38:38.265 INFO:tasks.workunit.client.1.smithi193.stdout:4/940: dread d1a/d22/d43/fb2 [0,4194304] 0 2024-05-09T06:38:38.266 INFO:tasks.workunit.client.1.smithi193.stdout:4/941: stat d1a/d22/d5d/ff1 0 2024-05-09T06:38:38.267 INFO:tasks.workunit.client.1.smithi193.stdout:4/942: chown d1a/d24/d73/d94/d7c/f86 85 1 2024-05-09T06:38:38.270 INFO:tasks.workunit.client.1.smithi193.stdout:4/943: truncate d1a/ded/fe6 1316633 0 2024-05-09T06:38:38.271 INFO:tasks.workunit.client.1.smithi193.stdout:3/876: write d1/db/d59/f9f [1146624,19746] 0 2024-05-09T06:38:38.272 INFO:tasks.workunit.client.1.smithi193.stdout:4/944: symlink d1a/d67/d82/dab/l11a 0 2024-05-09T06:38:38.276 INFO:tasks.workunit.client.1.smithi193.stdout:3/877: read d1/d44/d7d/f83 [198325,15645] 0 2024-05-09T06:38:38.277 INFO:tasks.workunit.client.1.smithi193.stdout:4/945: truncate d1a/d80/f9b 4349285 0 2024-05-09T06:38:38.282 INFO:tasks.workunit.client.1.smithi193.stdout:3/878: dwrite d1/db/da1/fd1 [0,4194304] 0 2024-05-09T06:38:38.290 INFO:tasks.workunit.client.1.smithi193.stdout:3/879: getdents d1/d44/ddf 0 2024-05-09T06:38:38.296 INFO:tasks.workunit.client.1.smithi193.stdout:5/847: creat d0/d13/d2c/d37/d2b/d47/d60/d75/f118 x:0 0 0 2024-05-09T06:38:38.296 INFO:tasks.workunit.client.1.smithi193.stdout:3/880: dwrite d1/db/d2f/d78/d86/ddc/d51/fa5 [0,4194304] 0 2024-05-09T06:38:38.303 INFO:tasks.workunit.client.1.smithi193.stdout:4/946: rmdir d1a/d24/dd0 39 2024-05-09T06:38:38.303 INFO:tasks.workunit.client.1.smithi193.stdout:4/947: stat d1a/d22/d5d/f93 0 2024-05-09T06:38:38.308 INFO:tasks.workunit.client.1.smithi193.stdout:4/948: rmdir d1a/da6 39 2024-05-09T06:38:38.311 INFO:tasks.workunit.client.1.smithi193.stdout:4/949: creat d1a/d98/f11b x:0 0 0 2024-05-09T06:38:38.311 INFO:tasks.workunit.client.1.smithi193.stdout:4/950: dread - d1a/d22/d43/fa2 zero size 2024-05-09T06:38:38.314 INFO:tasks.workunit.client.1.smithi193.stdout:1/946: rename d4/d16/ced to d4/d16/d17/d1a/d3a/c132 0 2024-05-09T06:38:38.315 INFO:tasks.workunit.client.1.smithi193.stdout:1/947: chown d4/d16/d35/fa9 4731 1 2024-05-09T06:38:38.319 INFO:tasks.workunit.client.1.smithi193.stdout:7/931: rmdir d6/dd/d7e/d111 0 2024-05-09T06:38:38.322 INFO:tasks.workunit.client.1.smithi193.stdout:6/773: rename d1/d25/d2c/d3c/d10c/d58/c85 to d1/d25/d2c/d81/d37/c111 0 2024-05-09T06:38:38.322 INFO:tasks.workunit.client.1.smithi193.stdout:1/948: dwrite d4/d16/d6b/fc0 [0,4194304] 0 2024-05-09T06:38:38.326 INFO:tasks.workunit.client.1.smithi193.stdout:5/848: dread d0/d13/d2c/d37/d2b/d47/d66/f83 [0,4194304] 0 2024-05-09T06:38:38.327 INFO:tasks.workunit.client.1.smithi193.stdout:2/858: write db/df/d54/fb8 [12918,131007] 0 2024-05-09T06:38:38.328 INFO:tasks.workunit.client.1.smithi193.stdout:1/949: write d4/d16/d17/d1a/d3a/d3f/dd7/f108 [174585,76690] 0 2024-05-09T06:38:38.329 INFO:tasks.workunit.client.1.smithi193.stdout:4/951: symlink d1a/d24/d73/d94/l11c 0 2024-05-09T06:38:38.330 INFO:tasks.workunit.client.1.smithi193.stdout:1/950: readlink d4/d16/de/l4d 0 2024-05-09T06:38:38.331 INFO:tasks.workunit.client.1.smithi193.stdout:2/859: write db/def/dfd/d80/f46 [1430618,78854] 0 2024-05-09T06:38:38.338 INFO:tasks.workunit.client.1.smithi193.stdout:0/783: rename d18/f61 to d18/d4c/d64/d66/d9d/ff0 0 2024-05-09T06:38:38.342 INFO:tasks.workunit.client.1.smithi193.stdout:1/951: rmdir d4/d3c/d3e/d7c/d10b/d10f/d4f/dcd 39 2024-05-09T06:38:38.345 INFO:tasks.workunit.client.1.smithi193.stdout:4/952: fsync d1a/d24/dd0/fde 0 2024-05-09T06:38:38.349 INFO:tasks.workunit.client.1.smithi193.stdout:7/932: link d6/d8/d1a/d36/d4d/d60/c67 d6/d17/d99/dd7/c12e 0 2024-05-09T06:38:38.349 INFO:tasks.workunit.client.1.smithi193.stdout:5/849: dread d0/d13/d2c/f4f [0,4194304] 0 2024-05-09T06:38:38.351 INFO:tasks.workunit.client.1.smithi193.stdout:1/952: creat d4/d16/d6b/d85/f133 x:0 0 0 2024-05-09T06:38:38.353 INFO:tasks.workunit.client.1.smithi193.stdout:4/953: write d1a/d1f/f2a [1567075,118479] 0 2024-05-09T06:38:38.356 INFO:tasks.workunit.client.1.smithi193.stdout:2/860: rename db/d25/d38/dcd/de3/l9a to db/d25/d38/d40/db2/dcb/d104/l120 0 2024-05-09T06:38:38.357 INFO:tasks.workunit.client.1.smithi193.stdout:0/784: mkdir d18/d27/df1 0 2024-05-09T06:38:38.358 INFO:tasks.workunit.client.1.smithi193.stdout:3/881: write d1/db/d2f/d78/d86/ddc/d51/f6a [1258770,130121] 0 2024-05-09T06:38:38.360 INFO:tasks.workunit.client.1.smithi193.stdout:3/882: chown d1/db/d2f/d84/f10b 78902681 1 2024-05-09T06:38:38.363 INFO:tasks.workunit.client.1.smithi193.stdout:4/954: mkdir d1a/d67/d11d 0 2024-05-09T06:38:38.366 INFO:tasks.workunit.client.1.smithi193.stdout:2/861: rmdir db/d25/d38/dcd 39 2024-05-09T06:38:38.369 INFO:tasks.workunit.client.1.smithi193.stdout:0/785: mknod d18/dc9/cf2 0 2024-05-09T06:38:38.371 INFO:tasks.workunit.client.1.smithi193.stdout:7/933: symlink d6/d8/l12f 0 2024-05-09T06:38:38.373 INFO:tasks.workunit.client.1.smithi193.stdout:7/934: dread d6/fe9 [0,4194304] 0 2024-05-09T06:38:38.375 INFO:tasks.workunit.client.1.smithi193.stdout:7/935: chown d6/dc3/f11e 0 1 2024-05-09T06:38:38.376 INFO:tasks.workunit.client.1.smithi193.stdout:6/774: dwrite d1/d25/d2c/d3c/d10c/dea/fd8 [0,4194304] 0 2024-05-09T06:38:38.377 INFO:tasks.workunit.client.1.smithi193.stdout:1/953: mknod d4/d16/d6b/d85/d128/c134 0 2024-05-09T06:38:38.379 INFO:tasks.workunit.client.1.smithi193.stdout:1/954: chown d4/d16/de/fdf 0 1 2024-05-09T06:38:38.379 INFO:tasks.workunit.client.1.smithi193.stdout:1/955: chown d4/f13 55832 1 2024-05-09T06:38:38.380 INFO:tasks.workunit.client.1.smithi193.stdout:1/956: readlink d4/d3c/d3e/l50 0 2024-05-09T06:38:38.385 INFO:tasks.workunit.client.1.smithi193.stdout:3/883: mknod d1/db/d59/c127 0 2024-05-09T06:38:38.390 INFO:tasks.workunit.client.1.smithi193.stdout:5/850: truncate d0/d13/d2c/d37/d5b/d4e/f78 1760714 0 2024-05-09T06:38:38.392 INFO:tasks.workunit.client.1.smithi193.stdout:6/775: creat d1/de/f112 x:0 0 0 2024-05-09T06:38:38.395 INFO:tasks.workunit.client.1.smithi193.stdout:1/957: creat d4/d3c/d3e/d7c/d10b/d10f/d4f/d59/f135 x:0 0 0 2024-05-09T06:38:38.395 INFO:tasks.workunit.client.1.smithi193.stdout:7/936: dwrite d6/d8/d1a/d36/d4d/d60/d9c/dd9/fdc [0,4194304] 0 2024-05-09T06:38:38.400 INFO:tasks.workunit.client.1.smithi193.stdout:0/786: symlink d18/d4c/d64/d74/lf3 0 2024-05-09T06:38:38.400 INFO:tasks.workunit.client.1.smithi193.stdout:2/862: write db/def/dfd/ff1 [71683,115240] 0 2024-05-09T06:38:38.403 INFO:tasks.workunit.client.1.smithi193.stdout:2/863: truncate db/df/d54/d63/dca/f109 108813 0 2024-05-09T06:38:38.403 INFO:tasks.workunit.client.1.smithi193.stdout:5/851: rmdir d0/d13/d2c/d4d 39 2024-05-09T06:38:38.403 INFO:tasks.workunit.client.1.smithi193.stdout:3/884: dread d1/db/d1b/fca [0,4194304] 0 2024-05-09T06:38:38.406 INFO:tasks.workunit.client.1.smithi193.stdout:1/958: unlink d4/d3c/d3e/f5c 0 2024-05-09T06:38:38.418 INFO:tasks.workunit.client.1.smithi193.stdout:3/885: symlink d1/db/d2f/d84/l128 0 2024-05-09T06:38:38.419 INFO:tasks.workunit.client.1.smithi193.stdout:3/886: chown d1/db/d2f/d78/d86/la7 71091 1 2024-05-09T06:38:38.419 INFO:tasks.workunit.client.1.smithi193.stdout:7/937: write d6/dd/f47 [333672,27973] 0 2024-05-09T06:38:38.422 INFO:tasks.workunit.client.1.smithi193.stdout:7/938: read d6/d8/d1a/d36/d4d/d60/d9c/faf [1771602,11634] 0 2024-05-09T06:38:38.424 INFO:tasks.workunit.client.1.smithi193.stdout:3/887: dread d1/db/d59/fa9 [0,4194304] 0 2024-05-09T06:38:38.429 INFO:tasks.workunit.client.1.smithi193.stdout:0/787: dwrite d18/d4c/d64/d66/d9d/fca [0,4194304] 0 2024-05-09T06:38:38.431 INFO:tasks.workunit.client.1.smithi193.stdout:0/788: write d18/d4c/d5f/f8c [485459,81537] 0 2024-05-09T06:38:38.433 INFO:tasks.workunit.client.1.smithi193.stdout:4/955: link d1a/l53 d1a/d22/dc3/l11e 0 2024-05-09T06:38:38.433 INFO:tasks.workunit.client.1.smithi193.stdout:4/956: stat l2 0 2024-05-09T06:38:38.437 INFO:tasks.workunit.client.1.smithi193.stdout:2/864: write db/f39 [3085649,12337] 0 2024-05-09T06:38:38.439 INFO:tasks.workunit.client.1.smithi193.stdout:2/865: write db/d4b/d56/f11e [22402,13423] 0 2024-05-09T06:38:38.439 INFO:tasks.workunit.client.1.smithi193.stdout:6/776: creat d1/d25/d2c/d3c/d10c/f113 x:0 0 0 2024-05-09T06:38:38.439 INFO:tasks.workunit.client.1.smithi193.stdout:7/939: read d6/d8/d1a/f27 [93896,95731] 0 2024-05-09T06:38:38.442 INFO:tasks.workunit.client.1.smithi193.stdout:5/852: creat d0/d13/f119 x:0 0 0 2024-05-09T06:38:38.450 INFO:tasks.workunit.client.1.smithi193.stdout:7/940: truncate d6/f2e 2635779 0 2024-05-09T06:38:38.450 INFO:tasks.workunit.client.1.smithi193.stdout:8/944: sync 2024-05-09T06:38:38.451 INFO:tasks.workunit.client.1.smithi193.stdout:2/866: creat db/d4b/d56/f121 x:0 0 0 2024-05-09T06:38:38.452 INFO:tasks.workunit.client.1.smithi193.stdout:3/888: mkdir d1/db/d59/d129 0 2024-05-09T06:38:38.457 INFO:tasks.workunit.client.1.smithi193.stdout:4/957: write d1a/d24/dd0/fde [8510472,14593] 0 2024-05-09T06:38:38.457 INFO:tasks.workunit.client.1.smithi193.stdout:0/789: truncate d18/d4c/d64/fcd 60347 0 2024-05-09T06:38:38.458 INFO:tasks.workunit.client.1.smithi193.stdout:4/958: write d1a/d6d/fdc [6897759,117293] 0 2024-05-09T06:38:38.459 INFO:tasks.workunit.client.1.smithi193.stdout:7/941: symlink d6/dd/d5e/d6b/l130 0 2024-05-09T06:38:38.461 INFO:tasks.workunit.client.1.smithi193.stdout:5/853: dwrite d0/d13/d2c/d37/d2b/d47/d6f/f54 [4194304,4194304] 0 2024-05-09T06:38:38.462 INFO:tasks.workunit.client.1.smithi193.stdout:2/867: read db/d25/d38/d40/db2/fe2 [37135,59367] 0 2024-05-09T06:38:38.463 INFO:tasks.workunit.client.1.smithi193.stdout:3/889: rmdir d1/db/d2f/d78/d86/ddc/d51 39 2024-05-09T06:38:38.469 INFO:tasks.workunit.client.1.smithi193.stdout:1/959: link d4/d16/d6b/d6e/cce d4/d16/d6b/d89/dc4/c136 0 2024-05-09T06:38:38.478 INFO:tasks.workunit.client.1.smithi193.stdout:8/945: symlink d6/d1c/d18/d2f/l12f 0 2024-05-09T06:38:38.482 INFO:tasks.workunit.client.1.smithi193.stdout:8/946: dread d6/d1c/d18/d66/d93/f124 [0,4194304] 0 2024-05-09T06:38:38.491 INFO:tasks.workunit.client.1.smithi193.stdout:1/960: symlink d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/l137 0 2024-05-09T06:38:38.491 INFO:tasks.workunit.client.1.smithi193.stdout:1/961: chown d4/d16/d6b 164364 1 2024-05-09T06:38:38.492 INFO:tasks.workunit.client.1.smithi193.stdout:6/777: link d1/d25/d2c/d81/laa d1/d25/d2c/d81/d37/d4f/l114 0 2024-05-09T06:38:38.503 INFO:tasks.workunit.client.1.smithi193.stdout:0/790: truncate d18/d4c/d5f/f8c 651239 0 2024-05-09T06:38:38.503 INFO:tasks.workunit.client.1.smithi193.stdout:4/959: dwrite d1a/d80/f9b [0,4194304] 0 2024-05-09T06:38:38.525 INFO:tasks.workunit.client.1.smithi193.stdout:5/854: rmdir d0/d13/d2c/d4d 39 2024-05-09T06:38:38.527 INFO:tasks.workunit.client.1.smithi193.stdout:3/890: symlink d1/db/d2f/d78/d86/ddc/d51/l12a 0 2024-05-09T06:38:38.528 INFO:tasks.workunit.client.1.smithi193.stdout:0/791: dread d18/d4c/d47/d59/d76/f77 [0,4194304] 0 2024-05-09T06:38:38.529 INFO:tasks.workunit.client.1.smithi193.stdout:2/868: getdents db/def/dfd/d80/daa/db4/d11f 0 2024-05-09T06:38:38.530 INFO:tasks.workunit.client.1.smithi193.stdout:0/792: fsync d18/d4c/d47/fed 0 2024-05-09T06:38:38.530 INFO:tasks.workunit.client.1.smithi193.stdout:1/962: symlink d4/d16/d6b/d89/dc4/l138 0 2024-05-09T06:38:38.532 INFO:tasks.workunit.client.1.smithi193.stdout:6/778: fsync d1/d25/d2c/d3c/d10c/d58/f84 0 2024-05-09T06:38:38.536 INFO:tasks.workunit.client.1.smithi193.stdout:2/869: dwrite db/d4b/f5d [4194304,4194304] 0 2024-05-09T06:38:38.538 INFO:tasks.workunit.client.1.smithi193.stdout:4/960: symlink d1a/d80/dc8/l11f 0 2024-05-09T06:38:38.540 INFO:tasks.workunit.client.1.smithi193.stdout:5/855: creat d0/d13/d2c/d37/d5b/d4e/d102/f11a x:0 0 0 2024-05-09T06:38:38.544 INFO:tasks.workunit.client.1.smithi193.stdout:1/963: unlink d4/ce2 0 2024-05-09T06:38:38.545 INFO:tasks.workunit.client.1.smithi193.stdout:0/793: rmdir d18/d4c/d64/d66/d9d 39 2024-05-09T06:38:38.546 INFO:tasks.workunit.client.1.smithi193.stdout:7/942: link d6/d8/f35 d6/dd/d70/dcc/f131 0 2024-05-09T06:38:38.548 INFO:tasks.workunit.client.1.smithi193.stdout:6/779: read d1/d25/d2c/d3c/fbb [882224,5668] 0 2024-05-09T06:38:38.555 INFO:tasks.workunit.client.1.smithi193.stdout:1/964: creat d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/f139 x:0 0 0 2024-05-09T06:38:38.557 INFO:tasks.workunit.client.1.smithi193.stdout:0/794: rename d18/d4c/d47/f4a to d18/dc9/ff4 0 2024-05-09T06:38:38.558 INFO:tasks.workunit.client.1.smithi193.stdout:0/795: write d18/fc7 [919396,54463] 0 2024-05-09T06:38:38.560 INFO:tasks.workunit.client.1.smithi193.stdout:8/947: write d6/d5b/f8d [1341703,30329] 0 2024-05-09T06:38:38.561 INFO:tasks.workunit.client.1.smithi193.stdout:8/948: dread - d6/d1c/d1b/d29/d57/dc8/f100 zero size 2024-05-09T06:38:38.563 INFO:tasks.workunit.client.1.smithi193.stdout:2/870: fsync db/df/d54/dba/ff8 0 2024-05-09T06:38:38.565 INFO:tasks.workunit.client.1.smithi193.stdout:5/856: mkdir d0/d13/d2c/d37/d5b/d89/de0/dff/d11b 0 2024-05-09T06:38:38.568 INFO:tasks.workunit.client.1.smithi193.stdout:7/943: creat d6/d8/d1a/d36/dd0/d10d/f132 x:0 0 0 2024-05-09T06:38:38.570 INFO:tasks.workunit.client.1.smithi193.stdout:1/965: rename d4/d16/cc7 to d4/d76/c13a 0 2024-05-09T06:38:38.571 INFO:tasks.workunit.client.1.smithi193.stdout:0/796: fdatasync d18/d4c/d47/f8e 0 2024-05-09T06:38:38.574 INFO:tasks.workunit.client.1.smithi193.stdout:8/949: symlink d6/d1c/d27/dbd/l130 0 2024-05-09T06:38:38.577 INFO:tasks.workunit.client.1.smithi193.stdout:3/891: truncate d1/db/d2f/d78/f8b 2550664 0 2024-05-09T06:38:38.578 INFO:tasks.workunit.client.1.smithi193.stdout:0/797: dwrite d18/d27/fa0 [0,4194304] 0 2024-05-09T06:38:38.581 INFO:tasks.workunit.client.1.smithi193.stdout:4/961: dwrite d1a/d67/f6c [0,4194304] 0 2024-05-09T06:38:38.585 INFO:tasks.workunit.client.1.smithi193.stdout:7/944: creat d6/d8/d29/df9/f133 x:0 0 0 2024-05-09T06:38:38.585 INFO:tasks.workunit.client.1.smithi193.stdout:1/966: symlink d4/d16/d17/d1a/d3a/d4a/db1/l13b 0 2024-05-09T06:38:38.593 INFO:tasks.workunit.client.1.smithi193.stdout:6/780: dwrite d1/de/d11/d2d/d3a/d47/f5f [0,4194304] 0 2024-05-09T06:38:38.595 INFO:tasks.workunit.client.1.smithi193.stdout:6/781: chown d1/de/d11/d18/d94/c98 798607663 1 2024-05-09T06:38:38.597 INFO:tasks.workunit.client.1.smithi193.stdout:2/871: mkdir db/d25/d38/d40/db2/d10a/d122 0 2024-05-09T06:38:38.597 INFO:tasks.workunit.client.1.smithi193.stdout:6/782: read d1/d25/f30 [3142376,120943] 0 2024-05-09T06:38:38.603 INFO:tasks.workunit.client.1.smithi193.stdout:4/962: creat d1a/d22/d5d/f120 x:0 0 0 2024-05-09T06:38:38.604 INFO:tasks.workunit.client.1.smithi193.stdout:5/857: dread d0/d13/d2c/d37/d5b/d4e/f6b [0,4194304] 0 2024-05-09T06:38:38.608 INFO:tasks.workunit.client.1.smithi193.stdout:8/950: mkdir d6/d1c/d1b/d5d/d131 0 2024-05-09T06:38:38.611 INFO:tasks.workunit.client.1.smithi193.stdout:6/783: creat d1/d25/d2c/d3c/d10c/d58/dc9/f115 x:0 0 0 2024-05-09T06:38:38.612 INFO:tasks.workunit.client.1.smithi193.stdout:7/945: read d6/d8/d1a/d36/d4d/d74/f7c [322147,118981] 0 2024-05-09T06:38:38.614 INFO:tasks.workunit.client.1.smithi193.stdout:4/963: symlink d1a/d98/l121 0 2024-05-09T06:38:38.618 INFO:tasks.workunit.client.1.smithi193.stdout:2/872: mknod db/d25/d38/dcd/c123 0 2024-05-09T06:38:38.618 INFO:tasks.workunit.client.1.smithi193.stdout:2/873: stat db/df/d54/dba/l10c 0 2024-05-09T06:38:38.620 INFO:tasks.workunit.client.1.smithi193.stdout:6/784: unlink d1/f10b 0 2024-05-09T06:38:38.621 INFO:tasks.workunit.client.1.smithi193.stdout:3/892: link d1/c5d d1/db/da1/c12b 0 2024-05-09T06:38:38.622 INFO:tasks.workunit.client.1.smithi193.stdout:7/946: write d6/d8/f55 [4514635,20704] 0 2024-05-09T06:38:38.623 INFO:tasks.workunit.client.1.smithi193.stdout:5/858: mknod d0/d13/d2c/d37/d5b/c11c 0 2024-05-09T06:38:38.625 INFO:tasks.workunit.client.1.smithi193.stdout:4/964: unlink d1a/d24/dd0/le2 0 2024-05-09T06:38:38.628 INFO:tasks.workunit.client.1.smithi193.stdout:2/874: mkdir db/d25/d38/dcd/de3/d124 0 2024-05-09T06:38:38.818 INFO:tasks.workunit.client.1.smithi193.stdout:8/951: rename d6/d1c/d18/d2f/d40/d5e to d6/d1c/d18/d132 0 2024-05-09T06:38:38.818 INFO:tasks.workunit.client.1.smithi193.stdout:3/893: symlink d1/db/d2f/d78/d86/ddc/l12c 0 2024-05-09T06:38:38.818 INFO:tasks.workunit.client.1.smithi193.stdout:7/947: mknod d6/d8/d1a/c134 0 2024-05-09T06:38:38.821 INFO:tasks.workunit.client.1.smithi193.stdout:2/875: unlink db/def/dfd/d80/f14 0 2024-05-09T06:38:38.822 INFO:tasks.workunit.client.1.smithi193.stdout:1/967: dwrite d4/fec [0,4194304] 0 2024-05-09T06:38:38.824 INFO:tasks.workunit.client.1.smithi193.stdout:5/859: write d0/d13/d2c/d37/d5b/d4e/f6b [1470637,72425] 0 2024-05-09T06:38:38.829 INFO:tasks.workunit.client.1.smithi193.stdout:0/798: dwrite fe [0,4194304] 0 2024-05-09T06:38:38.830 INFO:tasks.workunit.client.1.smithi193.stdout:1/968: chown d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/fa1 42 1 2024-05-09T06:38:38.831 INFO:tasks.workunit.client.1.smithi193.stdout:4/965: mkdir d1a/d122 0 2024-05-09T06:38:38.834 INFO:tasks.workunit.client.1.smithi193.stdout:3/894: dwrite d1/db/d2f/d78/fb4 [0,4194304] 0 2024-05-09T06:38:38.834 INFO:tasks.workunit.client.1.smithi193.stdout:4/966: chown f17 11369451 1 2024-05-09T06:38:38.836 INFO:tasks.workunit.client.1.smithi193.stdout:3/895: chown d1/db/d59/db8/cf7 76104254 1 2024-05-09T06:38:38.838 INFO:tasks.workunit.client.1.smithi193.stdout:4/967: fsync d1a/d24/f10b 0 2024-05-09T06:38:38.839 INFO:tasks.workunit.client.1.smithi193.stdout:4/968: chown l2 0 1 2024-05-09T06:38:38.841 INFO:tasks.workunit.client.1.smithi193.stdout:4/969: chown d1a/l68 13599 1 2024-05-09T06:38:38.846 INFO:tasks.workunit.client.1.smithi193.stdout:1/969: dwrite d4/d16/d17/d1a/d3a/d4a/f119 [0,4194304] 0 2024-05-09T06:38:38.848 INFO:tasks.workunit.client.1.smithi193.stdout:1/970: write d4/d16/dea/f12b [68263,67374] 0 2024-05-09T06:38:38.854 INFO:tasks.workunit.client.1.smithi193.stdout:7/948: symlink d6/dd/d5e/l135 0 2024-05-09T06:38:38.856 INFO:tasks.workunit.client.1.smithi193.stdout:3/896: mkdir d1/d44/ddf/d111/d12d 0 2024-05-09T06:38:38.857 INFO:tasks.workunit.client.1.smithi193.stdout:3/897: stat d1/db/d2f/d78/d86/ddc/d51/cde 0 2024-05-09T06:38:38.859 INFO:tasks.workunit.client.1.smithi193.stdout:8/952: symlink d6/d1c/d18/d117/dfa/l133 0 2024-05-09T06:38:38.859 INFO:tasks.workunit.client.1.smithi193.stdout:3/898: dread - d1/db/d2f/d78/d86/d94/f107 zero size 2024-05-09T06:38:38.860 INFO:tasks.workunit.client.1.smithi193.stdout:3/899: stat d1/db/d2f/d78/c82 0 2024-05-09T06:38:38.861 INFO:tasks.workunit.client.1.smithi193.stdout:2/876: mkdir db/d25/d38/d40/db2/d10a/d122/d125 0 2024-05-09T06:38:38.869 INFO:tasks.workunit.client.1.smithi193.stdout:2/877: symlink db/d9b/l126 0 2024-05-09T06:38:38.870 INFO:tasks.workunit.client.1.smithi193.stdout:1/971: dwrite d4/d16/d17/d1a/d3a/d3f/dd7/ddb/df8/f12a [0,4194304] 0 2024-05-09T06:38:38.873 INFO:tasks.workunit.client.1.smithi193.stdout:5/860: truncate d0/d13/d2c/d4d/d67/f76 315760 0 2024-05-09T06:38:38.879 INFO:tasks.workunit.client.1.smithi193.stdout:1/972: dwrite d4/d76/fe7 [0,4194304] 0 2024-05-09T06:38:38.885 INFO:tasks.workunit.client.1.smithi193.stdout:3/900: creat d1/db/d2f/d78/d86/ddc/d51/d11d/d125/f12e x:0 0 0 2024-05-09T06:38:38.887 INFO:tasks.workunit.client.1.smithi193.stdout:7/949: dread d6/d8/d1a/d36/d4d/f112 [0,4194304] 0 2024-05-09T06:38:38.888 INFO:tasks.workunit.client.1.smithi193.stdout:5/861: dwrite d0/d13/d2c/d37/d2b/d6a/d90/fb1 [0,4194304] 0 2024-05-09T06:38:38.891 INFO:tasks.workunit.client.1.smithi193.stdout:5/862: chown d0/d13/d2c/db6/dc6/ldf 369127 1 2024-05-09T06:38:38.891 INFO:tasks.workunit.client.1.smithi193.stdout:7/950: truncate d6/d8/d29/df9/f128 509814 0 2024-05-09T06:38:38.899 INFO:tasks.workunit.client.1.smithi193.stdout:0/799: getdents d18/d4c/d64/d66/db5 0 2024-05-09T06:38:38.901 INFO:tasks.workunit.client.1.smithi193.stdout:8/953: rename d6/d1c/d27/f116 to d6/d1c/f134 0 2024-05-09T06:38:38.911 INFO:tasks.workunit.client.1.smithi193.stdout:5/863: truncate d0/d13/d2c/d37/d5b/fbb 142326 0 2024-05-09T06:38:38.917 INFO:tasks.workunit.client.1.smithi193.stdout:3/901: mkdir d1/d44/d12f 0 2024-05-09T06:38:38.924 INFO:tasks.workunit.client.1.smithi193.stdout:3/902: symlink d1/db/d2f/d78/d86/ddc/d51/d11d/l130 0 2024-05-09T06:38:38.925 INFO:tasks.workunit.client.1.smithi193.stdout:3/903: chown d1/db/d2f/d84 20424 1 2024-05-09T06:38:38.926 INFO:tasks.workunit.client.1.smithi193.stdout:3/904: fdatasync d1/db/d2f/d78/f110 0 2024-05-09T06:38:38.930 INFO:tasks.workunit.client.1.smithi193.stdout:8/954: dread d6/d1c/d1b/d5d/d84/dc7/d7e/f8a [0,4194304] 0 2024-05-09T06:38:38.932 INFO:tasks.workunit.client.1.smithi193.stdout:8/955: dread - d6/d1c/d1b/dfd/f127 zero size 2024-05-09T06:38:38.939 INFO:tasks.workunit.client.1.smithi193.stdout:8/956: dwrite d6/d1c/d53/ff9 [0,4194304] 0 2024-05-09T06:38:38.941 INFO:tasks.workunit.client.1.smithi193.stdout:8/957: stat d6/d1c/c15 0 2024-05-09T06:38:38.942 INFO:tasks.workunit.client.1.smithi193.stdout:8/958: chown d6/d1c/d1b/d29 782303360 1 2024-05-09T06:38:38.954 INFO:tasks.workunit.client.1.smithi193.stdout:2/878: truncate db/d4b/f71 2200663 0 2024-05-09T06:38:38.955 INFO:tasks.workunit.client.1.smithi193.stdout:2/879: write db/df/d54/fb8 [361246,91704] 0 2024-05-09T06:38:38.961 INFO:tasks.workunit.client.1.smithi193.stdout:0/800: write d18/d4c/d47/f8e [3296050,130098] 0 2024-05-09T06:38:38.964 INFO:tasks.workunit.client.1.smithi193.stdout:2/880: unlink db/d25/d38/d40/ca1 0 2024-05-09T06:38:38.964 INFO:tasks.workunit.client.1.smithi193.stdout:1/973: dwrite d4/d16/d17/fbc [0,4194304] 0 2024-05-09T06:38:38.969 INFO:tasks.workunit.client.1.smithi193.stdout:8/959: dread d6/d1c/d27/fcd [0,4194304] 0 2024-05-09T06:38:38.972 INFO:tasks.workunit.client.1.smithi193.stdout:7/951: truncate d6/d8/f55 2902773 0 2024-05-09T06:38:38.975 INFO:tasks.workunit.client.1.smithi193.stdout:1/974: dread d4/d16/d6b/f102 [0,4194304] 0 2024-05-09T06:38:38.980 INFO:tasks.workunit.client.1.smithi193.stdout:8/960: fsync d6/d1c/f34 0 2024-05-09T06:38:38.983 INFO:tasks.workunit.client.1.smithi193.stdout:7/952: fdatasync d6/dd/d53/f58 0 2024-05-09T06:38:38.983 INFO:tasks.workunit.client.1.smithi193.stdout:1/975: rmdir d4/d16/d6b 39 2024-05-09T06:38:38.994 INFO:tasks.workunit.client.1.smithi193.stdout:3/905: dwrite d1/db/f73 [4194304,4194304] 0 2024-05-09T06:38:38.995 INFO:tasks.workunit.client.1.smithi193.stdout:5/864: dwrite d0/d13/d2c/d37/d2b/d47/d66/f83 [0,4194304] 0 2024-05-09T06:38:38.997 INFO:tasks.workunit.client.1.smithi193.stdout:8/961: symlink d6/d1c/d18/d132/d12b/l135 0 2024-05-09T06:38:38.997 INFO:tasks.workunit.client.1.smithi193.stdout:3/906: stat d1/db/da1/cbc 0 2024-05-09T06:38:39.000 INFO:tasks.workunit.client.1.smithi193.stdout:1/976: dwrite d4/d16/d17/d1a/dc5/f124 [0,4194304] 0 2024-05-09T06:38:39.010 INFO:tasks.workunit.client.1.smithi193.stdout:1/977: stat d4/d16/d17/dfc 0 2024-05-09T06:38:39.017 INFO:tasks.workunit.client.1.smithi193.stdout:0/801: dwrite d18/d4c/d47/d59/d76/f87 [0,4194304] 0 2024-05-09T06:38:39.024 INFO:tasks.workunit.client.1.smithi193.stdout:5/865: symlink d0/d13/d2c/d37/d2b/d6a/d90/def/l11d 0 2024-05-09T06:38:39.026 INFO:tasks.workunit.client.1.smithi193.stdout:2/881: dwrite db/d25/f49 [0,4194304] 0 2024-05-09T06:38:39.029 INFO:tasks.workunit.client.1.smithi193.stdout:3/907: symlink d1/d44/ddf/d111/l131 0 2024-05-09T06:38:39.034 INFO:tasks.workunit.client.1.smithi193.stdout:0/802: dwrite d18/f26 [0,4194304] 0 2024-05-09T06:38:39.034 INFO:tasks.workunit.client.1.smithi193.stdout:5/866: mknod d0/d13/d2c/d37/d2b/d47/d60/d9a/c11e 0 2024-05-09T06:38:39.034 INFO:tasks.workunit.client.1.smithi193.stdout:3/908: write d1/d44/d7d/fc3 [591760,92791] 0 2024-05-09T06:38:39.035 INFO:tasks.workunit.client.1.smithi193.stdout:8/962: creat d6/d1c/f136 x:0 0 0 2024-05-09T06:38:39.036 INFO:tasks.workunit.client.1.smithi193.stdout:1/978: rmdir d4/d16/d6b/d89/db3 39 2024-05-09T06:38:39.058 INFO:tasks.workunit.client.1.smithi193.stdout:2/882: symlink db/def/dfd/d80/l127 0 2024-05-09T06:38:39.059 INFO:tasks.workunit.client.1.smithi193.stdout:7/953: dread d6/d8/d29/d2f/f8d [0,4194304] 0 2024-05-09T06:38:39.061 INFO:tasks.workunit.client.1.smithi193.stdout:7/954: chown d6/dd/d70/f102 1 1 2024-05-09T06:38:39.064 INFO:tasks.workunit.client.1.smithi193.stdout:7/955: read - d6/d8/d1a/d36/d4d/fc7 zero size 2024-05-09T06:38:39.069 INFO:tasks.workunit.client.1.smithi193.stdout:3/909: dwrite d1/db/da1/fa2 [0,4194304] 0 2024-05-09T06:38:39.075 INFO:tasks.workunit.client.1.smithi193.stdout:8/963: write d6/d1c/d1b/d5d/d84/d87/da5/daa/ffe [840522,99303] 0 2024-05-09T06:38:39.077 INFO:tasks.workunit.client.1.smithi193.stdout:0/803: symlink d18/d4c/d47/lf5 0 2024-05-09T06:38:39.078 INFO:tasks.workunit.client.1.smithi193.stdout:8/964: truncate d6/d1c/f134 131433 0 2024-05-09T06:38:39.079 INFO:tasks.workunit.client.1.smithi193.stdout:7/956: mknod d6/d8/d1a/d36/d4d/d60/d9c/dd9/c136 0 2024-05-09T06:38:39.080 INFO:tasks.workunit.client.1.smithi193.stdout:2/883: dread db/f1e [0,4194304] 0 2024-05-09T06:38:39.081 INFO:tasks.workunit.client.1.smithi193.stdout:1/979: creat d4/d3c/d3e/d7c/d10b/d10f/d4f/dcd/f13c x:0 0 0 2024-05-09T06:38:39.082 INFO:tasks.workunit.client.1.smithi193.stdout:0/804: dread fe [0,4194304] 0 2024-05-09T06:38:39.083 INFO:tasks.workunit.client.1.smithi193.stdout:1/980: dread - d4/d16/d6b/d85/f133 zero size 2024-05-09T06:38:39.086 INFO:tasks.workunit.client.1.smithi193.stdout:8/965: readlink d6/d1c/d27/dbd/lef 0 2024-05-09T06:38:39.087 INFO:tasks.workunit.client.1.smithi193.stdout:8/966: chown d6/d1c/d18/dba/lcf 6920346 1 2024-05-09T06:38:39.089 INFO:tasks.workunit.client.1.smithi193.stdout:2/884: dread - db/df/d87/fea zero size 2024-05-09T06:38:39.090 INFO:tasks.workunit.client.1.smithi193.stdout:0/805: write d18/dc9/ff4 [1454341,47540] 0 2024-05-09T06:38:39.091 INFO:tasks.workunit.client.1.smithi193.stdout:0/806: stat d18/d4c/d5f 0 2024-05-09T06:38:39.098 INFO:tasks.workunit.client.1.smithi193.stdout:1/981: write d4/d3c/d3e/d7c/d10b/d10f/d4f/fda [4512664,27317] 0 2024-05-09T06:38:39.099 INFO:tasks.workunit.client.1.smithi193.stdout:8/967: write d6/d1c/d1b/d5d/d84/dc7/f129 [1675256,13816] 0 2024-05-09T06:38:39.100 INFO:tasks.workunit.client.1.smithi193.stdout:8/968: chown d6/d5b/d111 1 1 2024-05-09T06:38:39.101 INFO:tasks.workunit.client.1.smithi193.stdout:7/957: write d6/d17/d99/fef [2689406,84642] 0 2024-05-09T06:38:39.104 INFO:tasks.workunit.client.1.smithi193.stdout:3/910: getdents d1/db/d59/db8 0 2024-05-09T06:38:39.109 INFO:tasks.workunit.client.1.smithi193.stdout:1/982: rename d4/d16/d35/ld3 to d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/l13d 0 2024-05-09T06:38:39.109 INFO:tasks.workunit.client.1.smithi193.stdout:7/958: dread d6/d8/d1a/f27 [0,4194304] 0 2024-05-09T06:38:39.112 INFO:tasks.workunit.client.1.smithi193.stdout:1/983: dread - d4/d14/f115 zero size 2024-05-09T06:38:39.114 INFO:tasks.workunit.client.1.smithi193.stdout:8/969: dwrite d6/d1c/d18/d66/f78 [0,4194304] 0 2024-05-09T06:38:39.118 INFO:tasks.workunit.client.1.smithi193.stdout:7/959: mknod d6/d17/d99/dd7/c137 0 2024-05-09T06:38:39.121 INFO:tasks.workunit.client.1.smithi193.stdout:2/885: dread db/d25/d38/d40/d8b/fc3 [0,4194304] 0 2024-05-09T06:38:39.122 INFO:tasks.workunit.client.1.smithi193.stdout:7/960: chown d6/d17/d99/lf6 49631 1 2024-05-09T06:38:39.125 INFO:tasks.workunit.client.1.smithi193.stdout:1/984: fsync d4/d14/f2f 0 2024-05-09T06:38:39.126 INFO:tasks.workunit.client.1.smithi193.stdout:3/911: truncate d1/db/da1/de4/f120 2940113 0 2024-05-09T06:38:39.130 INFO:tasks.workunit.client.1.smithi193.stdout:7/961: mkdir d6/d8/d29/db9/d138 0 2024-05-09T06:38:39.132 INFO:tasks.workunit.client.1.smithi193.stdout:0/807: dwrite d18/d4c/d5f/f8c [0,4194304] 0 2024-05-09T06:38:39.132 INFO:tasks.workunit.client.1.smithi193.stdout:2/886: dread db/d25/d38/fe0 [0,4194304] 0 2024-05-09T06:38:39.133 INFO:tasks.workunit.client.1.smithi193.stdout:7/962: write d6/dd/ffe [1795125,2254] 0 2024-05-09T06:38:39.135 INFO:tasks.workunit.client.1.smithi193.stdout:2/887: chown db/df/d87 70 1 2024-05-09T06:38:39.136 INFO:tasks.workunit.client.1.smithi193.stdout:0/808: chown d18/d4c/l5d 26 1 2024-05-09T06:38:39.136 INFO:tasks.workunit.client.1.smithi193.stdout:8/970: link d6/d1c/d1b/d29/l118 d6/d1c/d18/dba/l137 0 2024-05-09T06:38:39.142 INFO:tasks.workunit.client.1.smithi193.stdout:1/985: dwrite d4/d3c/d3e/d7c/d10b/d10f/f126 [0,4194304] 0 2024-05-09T06:38:39.143 INFO:tasks.workunit.client.1.smithi193.stdout:7/963: fsync d6/d8/d1a/d36/dd0/fec 0 2024-05-09T06:38:39.147 INFO:tasks.workunit.client.1.smithi193.stdout:3/912: dread d1/db/d2f/d78/d86/ddc/d51/f7e [0,4194304] 0 2024-05-09T06:38:39.148 INFO:tasks.workunit.client.1.smithi193.stdout:0/809: rmdir d18/d4c/d47/d59 39 2024-05-09T06:38:39.148 INFO:tasks.workunit.client.1.smithi193.stdout:1/986: chown d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/fef 701763469 1 2024-05-09T06:38:39.149 INFO:tasks.workunit.client.1.smithi193.stdout:8/971: mknod d6/d1c/d18/d2f/d6a/c138 0 2024-05-09T06:38:39.152 INFO:tasks.workunit.client.1.smithi193.stdout:8/972: write d6/d1c/ff5 [142905,24602] 0 2024-05-09T06:38:39.152 INFO:tasks.workunit.client.1.smithi193.stdout:2/888: creat db/def/dfd/d80/daa/f128 x:0 0 0 2024-05-09T06:38:39.154 INFO:tasks.workunit.client.1.smithi193.stdout:7/964: dwrite d6/d8/d1a/d36/d4d/fc7 [0,4194304] 0 2024-05-09T06:38:39.154 INFO:tasks.workunit.client.1.smithi193.stdout:8/973: chown d6/d1c/d27/d128 450 1 2024-05-09T06:38:39.156 INFO:tasks.workunit.client.1.smithi193.stdout:8/974: write d6/d1c/d1b/d5d/f113 [523874,57233] 0 2024-05-09T06:38:39.160 INFO:tasks.workunit.client.1.smithi193.stdout:0/810: mknod d18/dc9/cf6 0 2024-05-09T06:38:39.162 INFO:tasks.workunit.client.1.smithi193.stdout:3/913: stat d1/db/d2f/d78/d86/ddc/d51/la6 0 2024-05-09T06:38:39.164 INFO:tasks.workunit.client.1.smithi193.stdout:2/889: unlink db/d4b/fde 0 2024-05-09T06:38:39.165 INFO:tasks.workunit.client.1.smithi193.stdout:7/965: creat d6/d17/d7d/daa/f139 x:0 0 0 2024-05-09T06:38:39.172 INFO:tasks.workunit.client.1.smithi193.stdout:2/890: dread - db/d25/d38/d40/d8f/ff7 zero size 2024-05-09T06:38:39.173 INFO:tasks.workunit.client.1.smithi193.stdout:7/966: symlink d6/d8/d29/df9/l13a 0 2024-05-09T06:38:39.175 INFO:tasks.workunit.client.1.smithi193.stdout:7/967: fdatasync d6/d8/d1a/d36/d4d/d74/f129 0 2024-05-09T06:38:39.178 INFO:tasks.workunit.client.1.smithi193.stdout:8/975: rename d6/d1c/d27/dbd/c120 to d6/d1c/d1b/d29/c139 0 2024-05-09T06:38:39.181 INFO:tasks.workunit.client.1.smithi193.stdout:3/914: dread d1/f11 [0,4194304] 0 2024-05-09T06:38:39.181 INFO:tasks.workunit.client.1.smithi193.stdout:2/891: getdents db/d25/d38/d40/db2/d10a/d122/d125 0 2024-05-09T06:38:39.185 INFO:tasks.workunit.client.1.smithi193.stdout:8/976: symlink d6/d1c/d18/d66/l13a 0 2024-05-09T06:38:39.186 INFO:tasks.workunit.client.1.smithi193.stdout:2/892: rmdir db/d4b/d56 39 2024-05-09T06:38:39.187 INFO:tasks.workunit.client.1.smithi193.stdout:2/893: write db/f45 [235743,53800] 0 2024-05-09T06:38:39.189 INFO:tasks.workunit.client.1.smithi193.stdout:3/915: symlink d1/db/da1/dce/l132 0 2024-05-09T06:38:39.189 INFO:tasks.workunit.client.1.smithi193.stdout:0/811: link d18/d4c/d64/d74/l9b d18/d4c/d64/d66/d9d/da5/lf7 0 2024-05-09T06:38:39.192 INFO:tasks.workunit.client.1.smithi193.stdout:7/968: dread d6/d8/d1a/d36/dd0/fec [0,4194304] 0 2024-05-09T06:38:39.199 INFO:tasks.workunit.client.1.smithi193.stdout:0/812: write d18/d4c/d47/d59/d76/f87 [3674898,98154] 0 2024-05-09T06:38:39.201 INFO:tasks.workunit.client.1.smithi193.stdout:7/969: fsync d6/dd/f68 0 2024-05-09T06:38:39.202 INFO:tasks.workunit.client.1.smithi193.stdout:7/970: dread - d6/dd/d70/dcc/f120 zero size 2024-05-09T06:38:39.205 INFO:tasks.workunit.client.1.smithi193.stdout:8/977: truncate d6/d1c/f14 822711 0 2024-05-09T06:38:39.206 INFO:tasks.workunit.client.1.smithi193.stdout:8/978: dread - d6/f112 zero size 2024-05-09T06:38:39.207 INFO:tasks.workunit.client.1.smithi193.stdout:8/979: readlink d6/d1c/d1b/d5d/d84/dc7/d7e/l98 0 2024-05-09T06:38:39.209 INFO:tasks.workunit.client.1.smithi193.stdout:1/987: rename d4/d16/d6b/d89/dc4/lf0 to d4/d16/d6b/d89/l13e 0 2024-05-09T06:38:39.212 INFO:tasks.workunit.client.1.smithi193.stdout:0/813: truncate d18/d4c/f8d 684375 0 2024-05-09T06:38:39.218 INFO:tasks.workunit.client.1.smithi193.stdout:1/988: mkdir d4/d16/d17/dfc/d13f 0 2024-05-09T06:38:39.223 INFO:tasks.workunit.client.1.smithi193.stdout:7/971: getdents d6/d8/d29/db9/d138 0 2024-05-09T06:38:39.226 INFO:tasks.workunit.client.1.smithi193.stdout:2/894: rename db/def/dfd/d80/c4d to db/def/dfd/d80/d60/d7d/db3/c129 0 2024-05-09T06:38:39.229 INFO:tasks.workunit.client.1.smithi193.stdout:1/989: creat d4/d16/d17/d1a/d3a/d4a/db1/f140 x:0 0 0 2024-05-09T06:38:39.230 INFO:tasks.workunit.client.1.smithi193.stdout:3/916: getdents d1/db/d2f/d78/d86 0 2024-05-09T06:38:39.232 INFO:tasks.workunit.client.1.smithi193.stdout:8/980: dwrite d6/d1c/d1b/d5d/d84/d87/da5/daa/fac [0,4194304] 0 2024-05-09T06:38:39.233 INFO:tasks.workunit.client.1.smithi193.stdout:7/972: unlink d6/dd/dbc/fc4 0 2024-05-09T06:38:39.238 INFO:tasks.workunit.client.1.smithi193.stdout:0/814: rename d18/d4c/d47/f8e to d18/d4c/d64/d74/dad/d70/ff8 0 2024-05-09T06:38:39.240 INFO:tasks.workunit.client.1.smithi193.stdout:3/917: chown d1/db/cac 38 1 2024-05-09T06:38:39.243 INFO:tasks.workunit.client.1.smithi193.stdout:1/990: mkdir d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/d141 0 2024-05-09T06:38:39.245 INFO:tasks.workunit.client.1.smithi193.stdout:2/895: dread db/f45 [0,4194304] 0 2024-05-09T06:38:39.245 INFO:tasks.workunit.client.1.smithi193.stdout:7/973: creat d6/dd/d7e/dd6/f13b x:0 0 0 2024-05-09T06:38:39.247 INFO:tasks.workunit.client.1.smithi193.stdout:7/974: chown d6/d8/d1a/d36/f4e 7429266 1 2024-05-09T06:38:39.249 INFO:tasks.workunit.client.1.smithi193.stdout:0/815: creat d18/d4c/d64/d74/dad/d70/ff9 x:0 0 0 2024-05-09T06:38:39.251 INFO:tasks.workunit.client.1.smithi193.stdout:3/918: rename d1/db/cac to d1/db/da1/c133 0 2024-05-09T06:38:39.253 INFO:tasks.workunit.client.1.smithi193.stdout:8/981: mknod d6/d5b/d111/c13b 0 2024-05-09T06:38:39.253 INFO:tasks.workunit.client.1.smithi193.stdout:8/982: stat d6/fd 0 2024-05-09T06:38:39.257 INFO:tasks.workunit.client.1.smithi193.stdout:7/975: truncate d6/dd/d70/dcc/d11a/f121 318411 0 2024-05-09T06:38:39.263 INFO:tasks.workunit.client.1.smithi193.stdout:1/991: creat d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/d141/f142 x:0 0 0 2024-05-09T06:38:39.274 INFO:tasks.workunit.client.1.smithi193.stdout:2/896: dread db/d25/d38/d40/db2/dcb/fdc [0,4194304] 0 2024-05-09T06:38:39.276 INFO:tasks.workunit.client.1.smithi193.stdout:8/983: fsync d6/d1c/f3d 0 2024-05-09T06:38:39.286 INFO:tasks.workunit.client.1.smithi193.stdout:0/816: write d18/d4c/d5f/f88 [2386998,111508] 0 2024-05-09T06:38:39.288 INFO:tasks.workunit.client.1.smithi193.stdout:0/817: write d18/d4c/d47/d59/d76/fda [346345,126599] 0 2024-05-09T06:38:39.294 INFO:tasks.workunit.client.1.smithi193.stdout:7/976: write d6/d8/fc9 [1116456,83847] 0 2024-05-09T06:38:39.295 INFO:tasks.workunit.client.1.smithi193.stdout:3/919: getdents d1/db/da1/dce 0 2024-05-09T06:38:39.299 INFO:tasks.workunit.client.1.smithi193.stdout:7/977: write d6/d8/d1a/d36/d4d/d60/d9c/dd9/fdc [1657473,46850] 0 2024-05-09T06:38:39.300 INFO:tasks.workunit.client.1.smithi193.stdout:0/818: symlink d18/dc9/lfa 0 2024-05-09T06:38:39.302 INFO:tasks.workunit.client.1.smithi193.stdout:2/897: dread db/f21 [0,4194304] 0 2024-05-09T06:38:39.305 INFO:tasks.workunit.client.1.smithi193.stdout:3/920: dwrite d1/db/f58 [0,4194304] 0 2024-05-09T06:38:39.312 INFO:tasks.workunit.client.1.smithi193.stdout:1/992: rename d4/f105 to d4/d16/d17/d1a/d3a/f143 0 2024-05-09T06:38:39.313 INFO:tasks.workunit.client.1.smithi193.stdout:8/984: dwrite d6/d1c/d18/d2f/d40/fc5 [0,4194304] 0 2024-05-09T06:38:39.315 INFO:tasks.workunit.client.1.smithi193.stdout:7/978: dwrite d6/d8/d1a/f3c [0,4194304] 0 2024-05-09T06:38:39.315 INFO:tasks.workunit.client.1.smithi193.stdout:2/898: symlink db/df/d54/dba/l12a 0 2024-05-09T06:38:39.318 INFO:tasks.workunit.client.1.smithi193.stdout:3/921: creat d1/db/d2f/d84/d97/f134 x:0 0 0 2024-05-09T06:38:39.319 INFO:tasks.workunit.client.1.smithi193.stdout:2/899: chown db/d4b 218391480 1 2024-05-09T06:38:39.324 INFO:tasks.workunit.client.1.smithi193.stdout:8/985: mkdir d6/d1c/d1b/d13c 0 2024-05-09T06:38:39.328 INFO:tasks.workunit.client.1.smithi193.stdout:3/922: symlink d1/db/da1/de4/l135 0 2024-05-09T06:38:39.330 INFO:tasks.workunit.client.1.smithi193.stdout:2/900: rename db/df/d54/ld5 to db/def/dfd/d80/daa/db4/l12b 0 2024-05-09T06:38:39.331 INFO:tasks.workunit.client.1.smithi193.stdout:1/993: dwrite d4/fec [4194304,4194304] 0 2024-05-09T06:38:39.333 INFO:tasks.workunit.client.1.smithi193.stdout:2/901: creat db/d25/d38/d5b/f12c x:0 0 0 2024-05-09T06:38:39.335 INFO:tasks.workunit.client.1.smithi193.stdout:3/923: dread d1/d44/d7d/f83 [0,4194304] 0 2024-05-09T06:38:39.336 INFO:tasks.workunit.client.1.smithi193.stdout:7/979: link d6/dd/d7e/fa3 d6/d8/d29/db9/d138/f13c 0 2024-05-09T06:38:39.338 INFO:tasks.workunit.client.1.smithi193.stdout:8/986: rename d6/d1c/d18/l2a to d6/d1c/d18/d66/d93/d95/l13d 0 2024-05-09T06:38:39.341 INFO:tasks.workunit.client.1.smithi193.stdout:2/902: fdatasync db/def/dfd/d80/d60/d7d/f85 0 2024-05-09T06:38:39.341 INFO:tasks.workunit.client.1.smithi193.stdout:2/903: chown db/dbb 89529 1 2024-05-09T06:38:39.342 INFO:tasks.workunit.client.1.smithi193.stdout:0/819: write d18/d4c/d64/d66/f9e [738616,63617] 0 2024-05-09T06:38:39.343 INFO:tasks.workunit.client.1.smithi193.stdout:2/904: chown db/def/dfd/d80/d60/ce8 383813124 1 2024-05-09T06:38:39.345 INFO:tasks.workunit.client.1.smithi193.stdout:2/905: write db/d25/d38/d5b/fc1 [2486839,30339] 0 2024-05-09T06:38:39.345 INFO:tasks.workunit.client.1.smithi193.stdout:8/987: creat d6/d1c/d18/d132/f13e x:0 0 0 2024-05-09T06:38:39.347 INFO:tasks.workunit.client.1.smithi193.stdout:8/988: write d6/d1c/d18/d66/d93/fa6 [1463409,9814] 0 2024-05-09T06:38:39.349 INFO:tasks.workunit.client.1.smithi193.stdout:1/994: rename d4/d16/f5f to d4/d16/d17/d1a/d3a/d4a/db1/f144 0 2024-05-09T06:38:39.352 INFO:tasks.workunit.client.1.smithi193.stdout:3/924: mkdir d1/d44/d12f/d136 0 2024-05-09T06:38:39.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:38 smithi063 ceph-mon[27171]: pgmap v22: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 99 MiB/s rd, 232 MiB/s wr, 556 op/s 2024-05-09T06:38:39.360 INFO:tasks.workunit.client.1.smithi193.stdout:3/925: chown d1/db/d59/cc9 10 1 2024-05-09T06:38:39.363 INFO:tasks.workunit.client.1.smithi193.stdout:8/989: unlink d6/d1c/cc2 0 2024-05-09T06:38:39.365 INFO:tasks.workunit.client.1.smithi193.stdout:7/980: rename d6/d17/d7d/c8a to d6/dd/d7e/c13d 0 2024-05-09T06:38:39.368 INFO:tasks.workunit.client.1.smithi193.stdout:8/990: rmdir d6/d1c/d18/d117/dfa 39 2024-05-09T06:38:39.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:38 smithi193 ceph-mon[35141]: pgmap v22: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 99 MiB/s rd, 232 MiB/s wr, 556 op/s 2024-05-09T06:38:39.373 INFO:tasks.workunit.client.1.smithi193.stdout:2/906: rename db/d37/df6 to db/def/dfd/d12d 0 2024-05-09T06:38:39.373 INFO:tasks.workunit.client.1.smithi193.stdout:0/820: write d18/d4c/f75 [131684,94124] 0 2024-05-09T06:38:39.374 INFO:tasks.workunit.client.1.smithi193.stdout:7/981: symlink d6/d8/d29/d2f/l13e 0 2024-05-09T06:38:39.376 INFO:tasks.workunit.client.1.smithi193.stdout:8/991: dwrite d6/d1c/d27/f65 [0,4194304] 0 2024-05-09T06:38:39.379 INFO:tasks.workunit.client.1.smithi193.stdout:0/821: chown d18/d4c/f2b 2 1 2024-05-09T06:38:39.379 INFO:tasks.workunit.client.1.smithi193.stdout:8/992: chown d6/d1c/d1b/d29/d57 93 1 2024-05-09T06:38:39.380 INFO:tasks.workunit.client.1.smithi193.stdout:3/926: mkdir d1/db/d2f/d137 0 2024-05-09T06:38:39.382 INFO:tasks.workunit.client.1.smithi193.stdout:1/995: rename d4/d16/d6b/d85/f86 to d4/d16/d35/f145 0 2024-05-09T06:38:39.383 INFO:tasks.workunit.client.1.smithi193.stdout:3/927: write d1/d44/d7d/f9c [1726814,54205] 0 2024-05-09T06:38:39.384 INFO:tasks.workunit.client.1.smithi193.stdout:2/907: read - db/d25/d38/d61/de7/ff3 zero size 2024-05-09T06:38:39.384 INFO:tasks.workunit.client.1.smithi193.stdout:8/993: write d6/d1c/d1b/d5d/d84/f10e [278731,14335] 0 2024-05-09T06:38:39.387 INFO:tasks.workunit.client.1.smithi193.stdout:7/982: dread d6/d8/d1a/d36/d4d/d60/d9c/dd9/fdc [0,4194304] 0 2024-05-09T06:38:39.396 INFO:tasks.workunit.client.1.smithi193.stdout:1/996: dread d4/d16/d6b/d6e/f78 [0,4194304] 0 2024-05-09T06:38:39.396 INFO:tasks.workunit.client.1.smithi193.stdout:2/908: dwrite db/d25/f48 [4194304,4194304] 0 2024-05-09T06:38:39.398 INFO:tasks.workunit.client.1.smithi193.stdout:3/928: write d1/db/d2f/d78/d86/ddc/d51/fbd [607724,32049] 0 2024-05-09T06:38:39.399 INFO:tasks.workunit.client.1.smithi193.stdout:0/822: dread d18/d4c/d64/d66/fd9 [0,4194304] 0 2024-05-09T06:38:39.400 INFO:tasks.workunit.client.1.smithi193.stdout:7/983: symlink d6/d8/d1a/l13f 0 2024-05-09T06:38:39.400 INFO:tasks.workunit.client.1.smithi193.stdout:2/909: stat db/def/dfd/d80 0 2024-05-09T06:38:39.402 INFO:tasks.workunit.client.1.smithi193.stdout:3/929: read d1/db/da1/fa2 [2805559,16010] 0 2024-05-09T06:38:39.402 INFO:tasks.workunit.client.1.smithi193.stdout:8/994: dwrite d6/d1c/d1b/dfd/f127 [0,4194304] 0 2024-05-09T06:38:39.404 INFO:tasks.workunit.client.1.smithi193.stdout:0/823: chown d18/d65/d82/l92 1 1 2024-05-09T06:38:39.406 INFO:tasks.workunit.client.1.smithi193.stdout:3/930: readlink d1/db/d2f/d78/d86/ddc/d51/lba 0 2024-05-09T06:38:39.407 INFO:tasks.workunit.client.1.smithi193.stdout:1/997: symlink d4/d16/d17/d1a/d3a/d4a/db1/l146 0 2024-05-09T06:38:39.409 INFO:tasks.workunit.client.1.smithi193.stdout:8/995: chown d6/d1c/cd7 104 1 2024-05-09T06:38:39.410 INFO:tasks.workunit.client.1.smithi193.stdout:2/910: mkdir db/d25/d38/d61/de7/d12e 0 2024-05-09T06:38:39.412 INFO:tasks.workunit.client.1.smithi193.stdout:7/984: truncate d6/dd/f68 1523416 0 2024-05-09T06:38:39.414 INFO:tasks.workunit.client.1.smithi193.stdout:3/931: mkdir d1/db/d2f/d84/d138 0 2024-05-09T06:38:39.415 INFO:tasks.workunit.client.1.smithi193.stdout:3/932: stat d1/db/d2f/d78/d86/ddc/d51/f65 0 2024-05-09T06:38:39.417 INFO:tasks.workunit.client.1.smithi193.stdout:1/998: rename d4/d16/d17/d1a/f1f to d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/f147 0 2024-05-09T06:38:39.421 INFO:tasks.workunit.client.1.smithi193.stdout:0/824: fsync d18/d4c/d64/f85 0 2024-05-09T06:38:39.423 INFO:tasks.workunit.client.1.smithi193.stdout:2/911: dread db/d4b/d56/fa0 [0,4194304] 0 2024-05-09T06:38:39.432 INFO:tasks.workunit.client.1.smithi193.stdout:8/996: write d6/d1c/d1b/d29/d57/f8b [75237,107257] 0 2024-05-09T06:38:39.434 INFO:tasks.workunit.client.1.smithi193.stdout:2/912: mkdir db/d25/d38/dcd/d12f 0 2024-05-09T06:38:39.439 INFO:tasks.workunit.client.1.smithi193.stdout:7/985: link d6/d8/d1a/d36/c6d d6/dd/d7e/c140 0 2024-05-09T06:38:39.442 INFO:tasks.workunit.client.1.smithi193.stdout:3/933: link d1/db/d2f/d78/d86/ddc/cd0 d1/db/da1/dce/c139 0 2024-05-09T06:38:39.444 INFO:tasks.workunit.client.1.smithi193.stdout:1/999: link d4/d16/d6b/d89/dc4/l138 d4/d3c/d3e/d7c/d10b/d10f/d4f/d72/l148 0 2024-05-09T06:38:39.445 INFO:tasks.workunit.client.1.smithi193.stdout:0/825: creat d18/d4c/ffb x:0 0 0 2024-05-09T06:38:39.447 INFO:tasks.workunit.client.1.smithi193.stdout:7/986: dwrite d6/dc3/f117 [0,4194304] 0 2024-05-09T06:38:39.456 INFO:tasks.workunit.client.1.smithi193.stdout:3/934: unlink d1/db/d2f/d84/l128 0 2024-05-09T06:38:39.460 INFO:tasks.workunit.client.1.smithi193.stdout:0/826: symlink d18/d4c/d47/d59/d76/lfc 0 2024-05-09T06:38:39.462 INFO:tasks.workunit.client.1.smithi193.stdout:7/987: rmdir d6/d8/d1a/d36/d4d/d60/d9c 39 2024-05-09T06:38:39.464 INFO:tasks.workunit.client.1.smithi193.stdout:2/913: dwrite db/df/d87/fb5 [0,4194304] 0 2024-05-09T06:38:39.466 INFO:tasks.workunit.client.1.smithi193.stdout:7/988: readlink d6/dd/d5e/d6b/l130 0 2024-05-09T06:38:39.469 INFO:tasks.workunit.client.1.smithi193.stdout:3/935: creat d1/db/d59/f13a x:0 0 0 2024-05-09T06:38:39.470 INFO:tasks.workunit.client.1.smithi193.stdout:8/997: dwrite d6/d1c/d18/dba/fc0 [0,4194304] 0 2024-05-09T06:38:39.476 INFO:tasks.workunit.client.1.smithi193.stdout:7/989: truncate d6/dd/d70/ff8 87234 0 2024-05-09T06:38:39.478 INFO:tasks.workunit.client.1.smithi193.stdout:7/990: stat d6/d17/d99/dd7/f127 0 2024-05-09T06:38:39.482 INFO:tasks.workunit.client.1.smithi193.stdout:7/991: chown d6/dd/d5e/d6b/l91 96196865 1 2024-05-09T06:38:39.487 INFO:tasks.workunit.client.1.smithi193.stdout:2/914: dwrite db/f21 [0,4194304] 0 2024-05-09T06:38:39.494 INFO:tasks.workunit.client.1.smithi193.stdout:7/992: creat d6/dd/d5e/f141 x:0 0 0 2024-05-09T06:38:39.495 INFO:tasks.workunit.client.1.smithi193.stdout:8/998: dread d6/d1c/d18/d66/d93/f10a [0,4194304] 0 2024-05-09T06:38:39.550 INFO:tasks.workunit.client.1.smithi193.stdout:6/785: sync 2024-05-09T06:38:39.553 INFO:tasks.workunit.client.1.smithi193.stdout:6/786: stat d1/d25/d2c/f6f 0 2024-05-09T06:38:39.556 INFO:tasks.workunit.client.1.smithi193.stdout:6/787: mkdir d1/de/d11/d2d/ddf/d116 0 2024-05-09T06:38:39.558 INFO:tasks.workunit.client.1.smithi193.stdout:6/788: fdatasync d1/d25/d2c/d3c/d10c/d58/ff5 0 2024-05-09T06:38:39.561 INFO:tasks.workunit.client.1.smithi193.stdout:6/789: truncate d1/d25/d2c/d3c/d10c/f3f 1207792 0 2024-05-09T06:38:39.568 INFO:tasks.workunit.client.1.smithi193.stdout:3/936: truncate d1/db/f58 4548255 0 2024-05-09T06:38:39.571 INFO:tasks.workunit.client.1.smithi193.stdout:3/937: symlink d1/db/l13b 0 2024-05-09T06:38:39.576 INFO:tasks.workunit.client.1.smithi193.stdout:3/938: truncate d1/db/d2f/d78/fe2 159557 0 2024-05-09T06:38:39.577 INFO:tasks.workunit.client.1.smithi193.stdout:3/939: chown d1/db/d1b/f7a 317 1 2024-05-09T06:38:39.580 INFO:tasks.workunit.client.1.smithi193.stdout:3/940: chown d1/db/d1b/c49 146 1 2024-05-09T06:38:39.583 INFO:tasks.workunit.client.1.smithi193.stdout:3/941: symlink d1/db/d2f/l13c 0 2024-05-09T06:38:39.586 INFO:tasks.workunit.client.1.smithi193.stdout:3/942: mkdir d1/db/d2f/d84/d13d 0 2024-05-09T06:38:39.593 INFO:tasks.workunit.client.1.smithi193.stdout:3/943: dwrite d1/f24 [0,4194304] 0 2024-05-09T06:38:39.606 INFO:tasks.workunit.client.1.smithi193.stdout:3/944: dread d1/db/d2f/d78/f8b [0,4194304] 0 2024-05-09T06:38:39.620 INFO:tasks.workunit.client.1.smithi193.stdout:2/915: dwrite db/d37/fdb [0,4194304] 0 2024-05-09T06:38:39.623 INFO:tasks.workunit.client.1.smithi193.stdout:2/916: stat db/def/dfd/d80/d60/d7d/db3/l11d 0 2024-05-09T06:38:39.625 INFO:tasks.workunit.client.1.smithi193.stdout:2/917: mkdir db/d25/d38/d61/d130 0 2024-05-09T06:38:39.631 INFO:tasks.workunit.client.1.smithi193.stdout:3/945: write d1/db/da1/de4/f120 [2760046,81155] 0 2024-05-09T06:38:39.638 INFO:tasks.workunit.client.1.smithi193.stdout:3/946: dwrite d1/db/da1/fd1 [0,4194304] 0 2024-05-09T06:38:39.730 INFO:tasks.workunit.client.1.smithi193.stdout:4/970: sync 2024-05-09T06:38:39.731 INFO:tasks.workunit.client.1.smithi193.stdout:5/867: sync 2024-05-09T06:38:39.751 INFO:tasks.workunit.client.1.smithi193.stdout:2/918: write db/d4b/f86 [985650,20157] 0 2024-05-09T06:38:39.772 INFO:tasks.workunit.client.1.smithi193.stdout:3/947: rename d1/db/d2f/d78/d86/ddc/cd0 to d1/db/d59/db8/c13e 0 2024-05-09T06:38:39.772 INFO:tasks.workunit.client.1.smithi193.stdout:5/868: fsync d0/d13/d2c/d37/d74/f91 0 2024-05-09T06:38:39.773 INFO:tasks.workunit.client.1.smithi193.stdout:3/948: stat d1/db/d2f/d78/d86/ddc/f45 0 2024-05-09T06:38:39.776 INFO:tasks.workunit.client.1.smithi193.stdout:4/971: rename d1a/d6d/l110 to d1a/d24/d73/d94/de7/l123 0 2024-05-09T06:38:39.779 INFO:tasks.workunit.client.1.smithi193.stdout:3/949: write d1/d44/f96 [1651200,10365] 0 2024-05-09T06:38:39.780 INFO:tasks.workunit.client.1.smithi193.stdout:3/950: readlink d1/db/d1b/lda 0 2024-05-09T06:38:39.780 INFO:tasks.workunit.client.1.smithi193.stdout:2/919: rmdir db/def/dfd/d80/d60/d7d/db3/d119 0 2024-05-09T06:38:39.788 INFO:tasks.workunit.client.1.smithi193.stdout:2/920: creat db/def/dfd/d80/d60/d7d/db3/f131 x:0 0 0 2024-05-09T06:38:39.789 INFO:tasks.workunit.client.1.smithi193.stdout:2/921: stat db/def/dfd/dc4 0 2024-05-09T06:38:39.790 INFO:tasks.workunit.client.1.smithi193.stdout:3/951: unlink d1/db/d2f/d78/d86/ddc/fc8 0 2024-05-09T06:38:39.798 INFO:tasks.workunit.client.1.smithi193.stdout:5/869: dwrite d0/d13/d2c/d37/d2b/d47/f79 [0,4194304] 0 2024-05-09T06:38:39.799 INFO:tasks.workunit.client.1.smithi193.stdout:2/922: creat db/def/dfd/d80/d60/f132 x:0 0 0 2024-05-09T06:38:39.799 INFO:tasks.workunit.client.1.smithi193.stdout:4/972: link d1a/d67/d82/dec/d60/ca7 d1a/d24/d73/d94/d7c/dc6/c124 0 2024-05-09T06:38:39.808 INFO:tasks.workunit.client.1.smithi193.stdout:2/923: rename db/df/d54/dba/l10c to db/d25/d38/d40/db2/dcb/l133 0 2024-05-09T06:38:39.809 INFO:tasks.workunit.client.1.smithi193.stdout:4/973: dwrite d1a/d22/d5d/fe4 [4194304,4194304] 0 2024-05-09T06:38:39.820 INFO:tasks.workunit.client.1.smithi193.stdout:2/924: rename db/def/dfd/cc0 to db/d25/d38/dcd/de3/d124/c134 0 2024-05-09T06:38:39.822 INFO:tasks.workunit.client.1.smithi193.stdout:2/925: fdatasync db/df/d54/fb8 0 2024-05-09T06:38:39.824 INFO:tasks.workunit.client.1.smithi193.stdout:5/870: dread d0/d13/d2c/d37/d5b/d4e/d5d/f61 [0,4194304] 0 2024-05-09T06:38:39.829 INFO:tasks.workunit.client.1.smithi193.stdout:4/974: dread d1a/d24/d73/d94/fd8 [0,4194304] 0 2024-05-09T06:38:39.831 INFO:tasks.workunit.client.1.smithi193.stdout:2/926: fdatasync db/d25/f57 0 2024-05-09T06:38:39.833 INFO:tasks.workunit.client.1.smithi193.stdout:4/975: rename c18 to d1a/d98/c125 0 2024-05-09T06:38:39.850 INFO:tasks.workunit.client.1.smithi193.stdout:2/927: dwrite db/d25/f95 [0,4194304] 0 2024-05-09T06:38:39.857 INFO:tasks.workunit.client.1.smithi193.stdout:2/928: creat db/d25/d38/d40/d8f/f135 x:0 0 0 2024-05-09T06:38:39.859 INFO:tasks.workunit.client.1.smithi193.stdout:2/929: write db/df/d87/fb5 [2811744,20283] 0 2024-05-09T06:38:39.950 INFO:tasks.workunit.client.1.smithi193.stdout:2/930: dwrite db/df/d54/d63/dca/fcc [0,4194304] 0 2024-05-09T06:38:39.957 INFO:tasks.workunit.client.1.smithi193.stdout:2/931: rename db/d25/d38/d40/l115 to db/d25/d38/d40/db2/dcb/d104/l136 0 2024-05-09T06:38:39.959 INFO:tasks.workunit.client.1.smithi193.stdout:2/932: fdatasync db/d25/d38/f75 0 2024-05-09T06:38:39.960 INFO:tasks.workunit.client.1.smithi193.stdout:2/933: chown db/d37/l3f 26657 1 2024-05-09T06:38:39.966 INFO:tasks.workunit.client.1.smithi193.stdout:2/934: creat db/d25/f137 x:0 0 0 2024-05-09T06:38:39.971 INFO:tasks.workunit.client.1.smithi193.stdout:2/935: write db/dbb/fd7 [1090569,50046] 0 2024-05-09T06:38:39.977 INFO:tasks.workunit.client.1.smithi193.stdout:2/936: link db/d25/d38/d40/db2/fe2 db/d9b/f138 0 2024-05-09T06:38:39.991 INFO:tasks.workunit.client.1.smithi193.stdout:2/937: dwrite db/df/d87/f10f [0,4194304] 0 2024-05-09T06:38:40.043 INFO:tasks.workunit.client.1.smithi193.stdout:0/827: sync 2024-05-09T06:38:40.044 INFO:tasks.workunit.client.1.smithi193.stdout:0/828: write d18/dc9/ff4 [4014242,129249] 0 2024-05-09T06:38:40.061 INFO:tasks.workunit.client.1.smithi193.stdout:2/938: chown db/d4b/l9e 14 1 2024-05-09T06:38:40.063 INFO:tasks.workunit.client.1.smithi193.stdout:2/939: symlink db/def/dfd/l139 0 2024-05-09T06:38:40.065 INFO:tasks.workunit.client.1.smithi193.stdout:0/829: creat d18/d27/ffd x:0 0 0 2024-05-09T06:38:40.070 INFO:tasks.workunit.client.1.smithi193.stdout:2/940: creat db/d25/d38/d40/db2/d10a/d122/d125/f13a x:0 0 0 2024-05-09T06:38:40.072 INFO:tasks.workunit.client.1.smithi193.stdout:2/941: fsync db/d25/d38/f75 0 2024-05-09T06:38:40.079 INFO:tasks.workunit.client.1.smithi193.stdout:0/830: dwrite f16 [0,4194304] 0 2024-05-09T06:38:40.081 INFO:tasks.workunit.client.1.smithi193.stdout:2/942: write db/def/dfd/d80/d60/fc7 [445967,41237] 0 2024-05-09T06:38:40.087 INFO:tasks.workunit.client.1.smithi193.stdout:0/831: truncate d18/d1d/f1f 204218 0 2024-05-09T06:38:40.089 INFO:tasks.workunit.client.1.smithi193.stdout:2/943: dwrite db/d25/f95 [0,4194304] 0 2024-05-09T06:38:40.090 INFO:tasks.workunit.client.1.smithi193.stdout:0/832: read d18/d4c/d47/d54/fd1 [606332,79990] 0 2024-05-09T06:38:40.090 INFO:tasks.workunit.client.1.smithi193.stdout:2/944: write db/d25/f27 [1816764,51247] 0 2024-05-09T06:38:40.092 INFO:tasks.workunit.client.1.smithi193.stdout:2/945: stat db/d25/d38/dcd/c123 0 2024-05-09T06:38:40.097 INFO:tasks.workunit.client.1.smithi193.stdout:0/833: unlink d18/d1d/f33 0 2024-05-09T06:38:40.098 INFO:tasks.workunit.client.1.smithi193.stdout:2/946: mknod db/d37/c13b 0 2024-05-09T06:38:40.101 INFO:tasks.workunit.client.1.smithi193.stdout:0/834: creat d18/d4c/d64/d74/dad/d70/dc4/ffe x:0 0 0 2024-05-09T06:38:40.114 INFO:tasks.workunit.client.1.smithi193.stdout:2/947: write db/d4b/f100 [152240,113332] 0 2024-05-09T06:38:40.116 INFO:tasks.workunit.client.1.smithi193.stdout:0/835: dread d18/f60 [0,4194304] 0 2024-05-09T06:38:40.117 INFO:tasks.workunit.client.1.smithi193.stdout:0/836: chown d18/d4c/c42 95448820 1 2024-05-09T06:38:40.118 INFO:tasks.workunit.client.1.smithi193.stdout:2/948: symlink db/def/dfd/d80/daa/db4/d11f/l13c 0 2024-05-09T06:38:40.120 INFO:tasks.workunit.client.1.smithi193.stdout:0/837: mknod d18/d65/d82/cff 0 2024-05-09T06:38:40.127 INFO:tasks.workunit.client.1.smithi193.stdout:0/838: rename d18/d4c/d47/d59/d76/f87 to d18/d1d/f100 0 2024-05-09T06:38:40.128 INFO:tasks.workunit.client.1.smithi193.stdout:2/949: write f8 [15145045,108420] 0 2024-05-09T06:38:40.137 INFO:tasks.workunit.client.1.smithi193.stdout:0/839: link d18/d4c/d64/d66/d9d/da5/cec d18/d4c/d64/c101 0 2024-05-09T06:38:40.138 INFO:tasks.workunit.client.1.smithi193.stdout:2/950: getdents db/d37 0 2024-05-09T06:38:40.143 INFO:tasks.workunit.client.1.smithi193.stdout:0/840: dwrite d18/fc7 [0,4194304] 0 2024-05-09T06:38:40.158 INFO:tasks.workunit.client.1.smithi193.stdout:2/951: dwrite db/d4b/f101 [0,4194304] 0 2024-05-09T06:38:40.161 INFO:tasks.workunit.client.1.smithi193.stdout:0/841: dwrite d18/d4c/d47/d54/f68 [0,4194304] 0 2024-05-09T06:38:40.163 INFO:tasks.workunit.client.1.smithi193.stdout:2/952: rename db/def/dfd/d80/fb0 to db/df/d54/f13d 0 2024-05-09T06:38:40.163 INFO:tasks.workunit.client.1.smithi193.stdout:7/993: sync 2024-05-09T06:38:40.166 INFO:tasks.workunit.client.1.smithi193.stdout:2/953: unlink db/d4b/f90 0 2024-05-09T06:38:40.168 INFO:tasks.workunit.client.1.smithi193.stdout:2/954: write db/d9b/fee [4894046,67304] 0 2024-05-09T06:38:40.170 INFO:tasks.workunit.client.1.smithi193.stdout:7/994: symlink d6/d8/d1a/d36/d4d/d74/l142 0 2024-05-09T06:38:40.174 INFO:tasks.workunit.client.1.smithi193.stdout:7/995: creat d6/d8/d29/dc8/f143 x:0 0 0 2024-05-09T06:38:40.175 INFO:tasks.workunit.client.1.smithi193.stdout:7/996: chown d6/dd/d5e/d6b/l130 9209 1 2024-05-09T06:38:40.176 INFO:tasks.workunit.client.1.smithi193.stdout:0/842: rename d18/d4c/d64/c101 to d18/d4c/d64/d74/c102 0 2024-05-09T06:38:40.187 INFO:tasks.workunit.client.1.smithi193.stdout:2/955: rename db/d4b/d56/fa0 to db/d25/f13e 0 2024-05-09T06:38:40.188 INFO:tasks.workunit.client.1.smithi193.stdout:7/997: getdents d6/dd/d7e 0 2024-05-09T06:38:40.191 INFO:tasks.workunit.client.1.smithi193.stdout:2/956: rename db/d25/f95 to db/d25/d38/d61/d130/f13f 0 2024-05-09T06:38:40.197 INFO:tasks.workunit.client.1.smithi193.stdout:2/957: creat db/dbb/f140 x:0 0 0 2024-05-09T06:38:40.200 INFO:tasks.workunit.client.1.smithi193.stdout:0/843: dwrite d18/d4c/d47/d59/d76/fb4 [0,4194304] 0 2024-05-09T06:38:40.208 INFO:tasks.workunit.client.1.smithi193.stdout:7/998: dwrite d6/d8/d1a/d36/d4d/d60/d9c/dd9/fdc [0,4194304] 0 2024-05-09T06:38:40.211 INFO:tasks.workunit.client.1.smithi193.stdout:0/844: dwrite d18/d4c/fa4 [0,4194304] 0 2024-05-09T06:38:40.217 INFO:tasks.workunit.client.1.smithi193.stdout:0/845: symlink d18/d1d/d56/da7/l103 0 2024-05-09T06:38:40.218 INFO:tasks.workunit.client.1.smithi193.stdout:7/999: dwrite d6/d8/d29/df9/f11f [0,4194304] 0 2024-05-09T06:38:40.224 INFO:tasks.workunit.client.1.smithi193.stdout:0/846: rename d18/d4c/d64/lbf to d18/d4c/d5f/l104 0 2024-05-09T06:38:40.226 INFO:tasks.workunit.client.1.smithi193.stdout:2/958: dwrite db/def/dfd/d80/f17 [0,4194304] 0 2024-05-09T06:38:40.231 INFO:tasks.workunit.client.1.smithi193.stdout:2/959: creat db/df/d54/f141 x:0 0 0 2024-05-09T06:38:40.236 INFO:tasks.workunit.client.1.smithi193.stdout:2/960: symlink db/def/dfd/d80/d60/d7d/l142 0 2024-05-09T06:38:40.241 INFO:tasks.workunit.client.1.smithi193.stdout:0/847: dread d18/d4c/d47/f58 [0,4194304] 0 2024-05-09T06:38:40.257 INFO:tasks.workunit.client.1.smithi193.stdout:2/961: write db/def/dfd/d80/d60/d7d/f85 [3641619,24477] 0 2024-05-09T06:38:40.259 INFO:tasks.workunit.client.1.smithi193.stdout:0/848: write d18/d4c/d47/d54/fd1 [2605707,43351] 0 2024-05-09T06:38:40.261 INFO:tasks.workunit.client.1.smithi193.stdout:2/962: mknod db/def/dfd/d80/d60/d7d/db3/c143 0 2024-05-09T06:38:40.263 INFO:tasks.workunit.client.1.smithi193.stdout:2/963: mknod db/def/c144 0 2024-05-09T06:38:40.265 INFO:tasks.workunit.client.1.smithi193.stdout:0/849: truncate d18/d4c/d5f/f6a 301468 0 2024-05-09T06:38:40.265 INFO:tasks.workunit.client.1.smithi193.stdout:2/964: dread db/df/d54/f8c [0,4194304] 0 2024-05-09T06:38:40.265 INFO:tasks.workunit.client.1.smithi193.stdout:4/976: sync 2024-05-09T06:38:40.265 INFO:tasks.workunit.client.1.smithi193.stdout:5/871: sync 2024-05-09T06:38:40.266 INFO:tasks.workunit.client.1.smithi193.stdout:3/952: sync 2024-05-09T06:38:40.266 INFO:tasks.workunit.client.1.smithi193.stdout:6/790: sync 2024-05-09T06:38:40.266 INFO:tasks.workunit.client.1.smithi193.stdout:8/999: sync 2024-05-09T06:38:40.267 INFO:tasks.workunit.client.1.smithi193.stdout:3/953: chown d1/db/d1b/l6b 17286 1 2024-05-09T06:38:40.272 INFO:tasks.workunit.client.1.smithi193.stdout:5/872: mknod d0/d13/d2c/d37/d2b/d6a/c11f 0 2024-05-09T06:38:40.276 INFO:tasks.workunit.client.1.smithi193.stdout:3/954: symlink d1/db/d2f/d78/d86/l13f 0 2024-05-09T06:38:40.280 INFO:tasks.workunit.client.1.smithi193.stdout:6/791: mknod d1/d25/d2c/d81/c117 0 2024-05-09T06:38:40.282 INFO:tasks.workunit.client.1.smithi193.stdout:4/977: write d1a/d24/fbf [264373,21307] 0 2024-05-09T06:38:40.293 INFO:tasks.workunit.client.1.smithi193.stdout:0/850: truncate d18/d4c/d47/d54/f68 5378828 0 2024-05-09T06:38:40.294 INFO:tasks.workunit.client.1.smithi193.stdout:5/873: dread d0/d13/d2c/d37/d2b/d47/d60/d87/fa5 [0,4194304] 0 2024-05-09T06:38:40.298 INFO:tasks.workunit.client.1.smithi193.stdout:3/955: creat d1/db/d2f/d84/d138/f140 x:0 0 0 2024-05-09T06:38:40.299 INFO:tasks.workunit.client.1.smithi193.stdout:4/978: write d1a/d24/f78 [649480,62520] 0 2024-05-09T06:38:40.303 INFO:tasks.workunit.client.1.smithi193.stdout:2/965: getdents db/def/dfd/dc4 0 2024-05-09T06:38:40.308 INFO:tasks.workunit.client.1.smithi193.stdout:6/792: write d1/d25/d4c/fd5 [1127929,67941] 0 2024-05-09T06:38:40.310 INFO:tasks.workunit.client.1.smithi193.stdout:5/874: creat d0/d13/d2c/d37/d5b/d89/de0/dff/f120 x:0 0 0 2024-05-09T06:38:40.312 INFO:tasks.workunit.client.1.smithi193.stdout:0/851: write d18/f4e [3321163,124520] 0 2024-05-09T06:38:40.312 INFO:tasks.workunit.client.1.smithi193.stdout:5/875: read d0/d13/d2c/d37/d5b/f21 [1840750,45405] 0 2024-05-09T06:38:40.319 INFO:tasks.workunit.client.1.smithi193.stdout:3/956: readlink d1/db/d2f/d78/d86/ddc/d51/laf 0 2024-05-09T06:38:40.320 INFO:tasks.workunit.client.1.smithi193.stdout:2/966: dread db/f39 [0,4194304] 0 2024-05-09T06:38:40.323 INFO:tasks.workunit.client.1.smithi193.stdout:5/876: creat d0/d13/d2c/d37/d5b/d4e/d102/f121 x:0 0 0 2024-05-09T06:38:40.323 INFO:tasks.workunit.client.1.smithi193.stdout:4/979: write d1a/d22/d43/f4d [1279393,41717] 0 2024-05-09T06:38:40.329 INFO:tasks.workunit.client.1.smithi193.stdout:2/967: dwrite db/d4b/f101 [0,4194304] 0 2024-05-09T06:38:40.329 INFO:tasks.workunit.client.1.smithi193.stdout:5/877: rename d0 to d0/d13/d122 22 2024-05-09T06:38:40.333 INFO:tasks.workunit.client.1.smithi193.stdout:3/957: creat d1/db/d2f/d78/d86/ddc/f141 x:0 0 0 2024-05-09T06:38:40.334 INFO:tasks.workunit.client.1.smithi193.stdout:0/852: write d18/d4c/d64/d66/d9d/da5/fbd [585690,112104] 0 2024-05-09T06:38:40.340 INFO:tasks.workunit.client.1.smithi193.stdout:4/980: dwrite d1a/d67/d82/dec/f3c [0,4194304] 0 2024-05-09T06:38:40.340 INFO:tasks.workunit.client.1.smithi193.stdout:6/793: dread d1/d25/d2c/d81/d37/d4f/fb4 [0,4194304] 0 2024-05-09T06:38:40.341 INFO:tasks.workunit.client.1.smithi193.stdout:3/958: read d1/fea [6088406,114087] 0 2024-05-09T06:38:40.343 INFO:tasks.workunit.client.1.smithi193.stdout:5/878: symlink d0/d13/d2c/d37/d2b/d47/d60/d75/l123 0 2024-05-09T06:38:40.345 INFO:tasks.workunit.client.1.smithi193.stdout:4/981: mknod d1a/d22/d5d/c126 0 2024-05-09T06:38:40.346 INFO:tasks.workunit.client.1.smithi193.stdout:2/968: write db/d25/d38/d61/f65 [867303,66035] 0 2024-05-09T06:38:40.348 INFO:tasks.workunit.client.1.smithi193.stdout:2/969: write db/d25/f27 [3167819,43125] 0 2024-05-09T06:38:40.349 INFO:tasks.workunit.client.1.smithi193.stdout:0/853: write d18/d4c/d64/d74/dad/f7e [484289,30397] 0 2024-05-09T06:38:40.349 INFO:tasks.workunit.client.1.smithi193.stdout:2/970: dread - db/def/dfd/d80/daa/f128 zero size 2024-05-09T06:38:40.350 INFO:tasks.workunit.client.1.smithi193.stdout:3/959: truncate d1/db/d2f/d84/f9a 3784827 0 2024-05-09T06:38:40.352 INFO:tasks.workunit.client.1.smithi193.stdout:3/960: chown d1/db/d2f/d78/d86/ddc/l126 125175 1 2024-05-09T06:38:40.354 INFO:tasks.workunit.client.1.smithi193.stdout:6/794: mkdir d1/de/d11/d2d/ddf/d116/d118 0 2024-05-09T06:38:40.358 INFO:tasks.workunit.client.1.smithi193.stdout:2/971: creat db/def/dfd/d80/daa/db4/d11f/f145 x:0 0 0 2024-05-09T06:38:40.362 INFO:tasks.workunit.client.1.smithi193.stdout:0/854: rename d18/d4c/d47/l9c to d18/d4c/d64/d66/d9d/l105 0 2024-05-09T06:38:40.366 INFO:tasks.workunit.client.1.smithi193.stdout:3/961: read d1/db/d59/f69 [15197,32004] 0 2024-05-09T06:38:40.367 INFO:tasks.workunit.client.1.smithi193.stdout:6/795: rmdir d1/de/d11/d2d/d3a/d47 39 2024-05-09T06:38:40.370 INFO:tasks.workunit.client.1.smithi193.stdout:2/972: readlink db/d25/l3a 0 2024-05-09T06:38:40.371 INFO:tasks.workunit.client.1.smithi193.stdout:0/855: dwrite d18/d4c/d64/d66/d9d/da5/fbd [0,4194304] 0 2024-05-09T06:38:40.377 INFO:tasks.workunit.client.1.smithi193.stdout:6/796: dread - d1/de/d11/d2d/d3a/d47/f7a zero size 2024-05-09T06:38:40.378 INFO:tasks.workunit.client.1.smithi193.stdout:4/982: getdents d1a/d22/d43/df8 0 2024-05-09T06:38:40.380 INFO:tasks.workunit.client.1.smithi193.stdout:2/973: dread db/d37/fdb [0,4194304] 0 2024-05-09T06:38:40.381 INFO:tasks.workunit.client.1.smithi193.stdout:5/879: dwrite d0/d13/d2c/d4d/d67/f76 [0,4194304] 0 2024-05-09T06:38:40.386 INFO:tasks.workunit.client.1.smithi193.stdout:3/962: write d1/d44/d7d/fbb [865967,82755] 0 2024-05-09T06:38:40.393 INFO:tasks.workunit.client.1.smithi193.stdout:2/974: dread - db/def/dfd/d80/d60/d7d/db3/fd9 zero size 2024-05-09T06:38:40.395 INFO:tasks.workunit.client.1.smithi193.stdout:3/963: dwrite d1/db/d2f/d84/d138/f140 [0,4194304] 0 2024-05-09T06:38:40.398 INFO:tasks.workunit.client.1.smithi193.stdout:6/797: truncate d1/d25/d2c/d76/d69/fd4 1427346 0 2024-05-09T06:38:40.399 INFO:tasks.workunit.client.1.smithi193.stdout:4/983: mkdir d1a/d22/d127 0 2024-05-09T06:38:40.401 INFO:tasks.workunit.client.1.smithi193.stdout:0/856: creat d18/d1d/f106 x:0 0 0 2024-05-09T06:38:40.404 INFO:tasks.workunit.client.1.smithi193.stdout:2/975: rmdir db/df/d87/d9f 39 2024-05-09T06:38:40.405 INFO:tasks.workunit.client.1.smithi193.stdout:2/976: chown db/def/dfd/d80/d60/d7d/f85 334455 1 2024-05-09T06:38:40.406 INFO:tasks.workunit.client.1.smithi193.stdout:3/964: rmdir d1/db/d59 39 2024-05-09T06:38:40.408 INFO:tasks.workunit.client.1.smithi193.stdout:2/977: fdatasync db/d25/d38/d40/d8f/f135 0 2024-05-09T06:38:40.409 INFO:tasks.workunit.client.1.smithi193.stdout:4/984: creat d1a/d67/d82/f128 x:0 0 0 2024-05-09T06:38:40.413 INFO:tasks.workunit.client.1.smithi193.stdout:5/880: creat d0/d13/d2c/d37/d2b/d6a/d90/def/db4/f124 x:0 0 0 2024-05-09T06:38:40.414 INFO:tasks.workunit.client.1.smithi193.stdout:3/965: rmdir d1/db/da1/de4 39 2024-05-09T06:38:40.421 INFO:tasks.workunit.client.1.smithi193.stdout:6/798: dread d1/d25/d2c/d3c/d10c/d58/fed [0,4194304] 0 2024-05-09T06:38:40.423 INFO:tasks.workunit.client.1.smithi193.stdout:0/857: rename d18/f4e to d18/d65/f107 0 2024-05-09T06:38:40.424 INFO:tasks.workunit.client.1.smithi193.stdout:3/966: fdatasync d1/db/d2f/d78/d86/ddc/fcc 0 2024-05-09T06:38:40.425 INFO:tasks.workunit.client.1.smithi193.stdout:0/858: write d18/d1d/d56/fe5 [503502,70276] 0 2024-05-09T06:38:40.427 INFO:tasks.workunit.client.1.smithi193.stdout:4/985: symlink d1a/d122/l129 0 2024-05-09T06:38:40.427 INFO:tasks.workunit.client.1.smithi193.stdout:4/986: fdatasync d1a/d22/dc3/ffe 0 2024-05-09T06:38:40.431 INFO:tasks.workunit.client.1.smithi193.stdout:6/799: creat d1/d25/d4c/d9e/dc8/f119 x:0 0 0 2024-05-09T06:38:40.435 INFO:tasks.workunit.client.1.smithi193.stdout:5/881: dwrite d0/d13/d2c/d37/d5b/d4e/d5d/f61 [0,4194304] 0 2024-05-09T06:38:40.437 INFO:tasks.workunit.client.1.smithi193.stdout:5/882: chown d0/d13/d2c/d37/d2b/d6a/fea 53164 1 2024-05-09T06:38:40.439 INFO:tasks.workunit.client.1.smithi193.stdout:0/859: creat d18/d4c/d64/d74/dad/d70/f108 x:0 0 0 2024-05-09T06:38:40.444 INFO:tasks.workunit.client.1.smithi193.stdout:6/800: mkdir d1/d25/d2c/d76/d69/d11a 0 2024-05-09T06:38:40.445 INFO:tasks.workunit.client.1.smithi193.stdout:2/978: getdents db/d25/d38/d40/db2/dcb 0 2024-05-09T06:38:40.449 INFO:tasks.workunit.client.1.smithi193.stdout:3/967: rename d1/d44/f47 to d1/db/d2f/d78/f142 0 2024-05-09T06:38:40.472 INFO:tasks.workunit.client.1.smithi193.stdout:4/987: dwrite d1a/d67/d82/dec/f66 [0,4194304] 0 2024-05-09T06:38:40.472 INFO:tasks.workunit.client.1.smithi193.stdout:0/860: unlink d18/d4c/d64/f99 0 2024-05-09T06:38:40.476 INFO:tasks.workunit.client.1.smithi193.stdout:6/801: rename d1/de/d11/f8b to d1/d25/d2c/d3c/d10c/dea/f11b 0 2024-05-09T06:38:40.478 INFO:tasks.workunit.client.1.smithi193.stdout:2/979: creat db/df/d54/d63/dca/d102/f146 x:0 0 0 2024-05-09T06:38:40.478 INFO:tasks.workunit.client.1.smithi193.stdout:2/980: chown db/d9b/fee 408999 1 2024-05-09T06:38:40.479 INFO:tasks.workunit.client.1.smithi193.stdout:2/981: chown db/df/f10 9 1 2024-05-09T06:38:40.481 INFO:tasks.workunit.client.1.smithi193.stdout:2/982: truncate db/d25/d38/d61/f70 4512324 0 2024-05-09T06:38:40.484 INFO:tasks.workunit.client.1.smithi193.stdout:6/802: symlink d1/de/d11/d20/l11c 0 2024-05-09T06:38:40.487 INFO:tasks.workunit.client.1.smithi193.stdout:2/983: rename db/def/dfd/d80/d60/d7d/f7e to db/def/dfd/d80/d60/f147 0 2024-05-09T06:38:40.488 INFO:tasks.workunit.client.1.smithi193.stdout:4/988: symlink d1a/d24/d73/d94/d7c/l12a 0 2024-05-09T06:38:40.488 INFO:tasks.workunit.client.1.smithi193.stdout:2/984: chown db/d25/d38/c62 12321167 1 2024-05-09T06:38:40.489 INFO:tasks.workunit.client.1.smithi193.stdout:4/989: stat d1a/d22/d43/df8/ffb 0 2024-05-09T06:38:40.489 INFO:tasks.workunit.client.1.smithi193.stdout:2/985: dread - db/df/d87/fea zero size 2024-05-09T06:38:40.491 INFO:tasks.workunit.client.1.smithi193.stdout:6/803: symlink d1/d25/l11d 0 2024-05-09T06:38:40.499 INFO:tasks.workunit.client.1.smithi193.stdout:2/986: rename db/d4b/l9e to db/d25/d38/d61/de7/l148 0 2024-05-09T06:38:40.499 INFO:tasks.workunit.client.1.smithi193.stdout:4/990: rmdir d1a/da6 39 2024-05-09T06:38:40.503 INFO:tasks.workunit.client.1.smithi193.stdout:6/804: unlink d1/de/d11/d18/f24 0 2024-05-09T06:38:40.505 INFO:tasks.workunit.client.1.smithi193.stdout:0/861: write d18/d65/f79 [645990,7493] 0 2024-05-09T06:38:40.505 INFO:tasks.workunit.client.1.smithi193.stdout:3/968: dread d1/db/d2f/d84/f9a [0,4194304] 0 2024-05-09T06:38:40.507 INFO:tasks.workunit.client.1.smithi193.stdout:3/969: write f0 [3550437,46747] 0 2024-05-09T06:38:40.545 INFO:tasks.workunit.client.1.smithi193.stdout:4/991: mkdir d1a/d6d/d12b 0 2024-05-09T06:38:40.546 INFO:tasks.workunit.client.1.smithi193.stdout:4/992: chown d1a/d67/d82/dab/f107 711008 1 2024-05-09T06:38:40.547 INFO:tasks.workunit.client.1.smithi193.stdout:6/805: creat d1/de/d11/d2d/da4/f11e x:0 0 0 2024-05-09T06:38:40.549 INFO:tasks.workunit.client.1.smithi193.stdout:0/862: rename d18/f60 to d18/d1d/d56/f109 0 2024-05-09T06:38:40.555 INFO:tasks.workunit.client.1.smithi193.stdout:6/806: dread - d1/de/fb8 zero size 2024-05-09T06:38:40.561 INFO:tasks.workunit.client.1.smithi193.stdout:2/987: creat db/def/dfd/d80/d60/d7d/f149 x:0 0 0 2024-05-09T06:38:40.562 INFO:tasks.workunit.client.1.smithi193.stdout:4/993: unlink d1a/d24/cd3 0 2024-05-09T06:38:40.564 INFO:tasks.workunit.client.1.smithi193.stdout:4/994: chown d1a/d24/d73/d94/f102 6763 1 2024-05-09T06:38:40.566 INFO:tasks.workunit.client.1.smithi193.stdout:6/807: creat d1/d25/d2c/d76/f11f x:0 0 0 2024-05-09T06:38:40.568 INFO:tasks.workunit.client.1.smithi193.stdout:2/988: dwrite db/def/dfd/d80/fa6 [0,4194304] 0 2024-05-09T06:38:40.571 INFO:tasks.workunit.client.1.smithi193.stdout:0/863: mkdir d18/d4c/d64/d66/d10a 0 2024-05-09T06:38:40.575 INFO:tasks.workunit.client.1.smithi193.stdout:6/808: rmdir d1/d25/d2c/d76/d69 39 2024-05-09T06:38:40.577 INFO:tasks.workunit.client.1.smithi193.stdout:2/989: creat db/d25/d38/d40/db2/dcb/f14a x:0 0 0 2024-05-09T06:38:40.577 INFO:tasks.workunit.client.1.smithi193.stdout:2/990: fsync f8 0 2024-05-09T06:38:40.578 INFO:tasks.workunit.client.1.smithi193.stdout:0/864: truncate d18/d4c/d47/d54/f90 80767 0 2024-05-09T06:38:40.578 INFO:tasks.workunit.client.1.smithi193.stdout:2/991: chown db/d4b/f5d 19131863 1 2024-05-09T06:38:40.578 INFO:tasks.workunit.client.1.smithi193.stdout:0/865: readlink d18/d65/l67 0 2024-05-09T06:38:40.580 INFO:tasks.workunit.client.1.smithi193.stdout:6/809: fsync d1/de/f38 0 2024-05-09T06:38:40.581 INFO:tasks.workunit.client.1.smithi193.stdout:6/810: chown d1/de/d11/d18/d94/c10e 157 1 2024-05-09T06:38:40.587 INFO:tasks.workunit.client.1.smithi193.stdout:0/866: link d18/d27/f41 d18/d4c/d64/d66/d10a/f10b 0 2024-05-09T06:38:40.590 INFO:tasks.workunit.client.1.smithi193.stdout:6/811: creat d1/de/d11/d18/f120 x:0 0 0 2024-05-09T06:38:40.591 INFO:tasks.workunit.client.1.smithi193.stdout:6/812: chown d1/d25/d4c/fd5 21427 1 2024-05-09T06:38:40.592 INFO:tasks.workunit.client.1.smithi193.stdout:0/867: creat d18/d1d/f10c x:0 0 0 2024-05-09T06:38:40.618 INFO:tasks.workunit.client.1.smithi193.stdout:3/970: write d1/db/d2f/d78/d86/ddc/d51/ff3 [731331,51611] 0 2024-05-09T06:38:40.619 INFO:tasks.workunit.client.1.smithi193.stdout:3/971: chown d1/d44/ddf/cf1 7310 1 2024-05-09T06:38:40.622 INFO:tasks.workunit.client.1.smithi193.stdout:4/995: write d1a/f1d [1696629,26160] 0 2024-05-09T06:38:40.625 INFO:tasks.workunit.client.1.smithi193.stdout:3/972: dread d1/db/d1b/f10a [0,4194304] 0 2024-05-09T06:38:40.627 INFO:tasks.workunit.client.1.smithi193.stdout:4/996: mkdir d1a/d6d/d117/d12c 0 2024-05-09T06:38:40.628 INFO:tasks.workunit.client.1.smithi193.stdout:2/992: write db/fd8 [299719,6700] 0 2024-05-09T06:38:40.630 INFO:tasks.workunit.client.1.smithi193.stdout:2/993: stat db/d9b/fee 0 2024-05-09T06:38:40.632 INFO:tasks.workunit.client.1.smithi193.stdout:3/973: creat d1/db/d2f/d78/d86/ddc/d51/f143 x:0 0 0 2024-05-09T06:38:40.634 INFO:tasks.workunit.client.1.smithi193.stdout:6/813: dwrite d1/d25/d2c/d3c/d10c/d58/dc9/f115 [0,4194304] 0 2024-05-09T06:38:40.635 INFO:tasks.workunit.client.1.smithi193.stdout:2/994: stat db/df/d87/d9f 0 2024-05-09T06:38:40.636 INFO:tasks.workunit.client.1.smithi193.stdout:0/868: truncate d18/d4c/d64/d66/d9d/faf 6616981 0 2024-05-09T06:38:40.636 INFO:tasks.workunit.client.1.smithi193.stdout:4/997: truncate d1a/d24/d73/d94/fb4 199748 0 2024-05-09T06:38:40.637 INFO:tasks.workunit.client.1.smithi193.stdout:2/995: write db/def/dfd/d80/daa/db4/d11f/f145 [643288,73646] 0 2024-05-09T06:38:40.639 INFO:tasks.workunit.client.1.smithi193.stdout:4/998: write d1a/d67/d82/dab/f107 [684963,16691] 0 2024-05-09T06:38:40.648 INFO:tasks.workunit.client.1.smithi193.stdout:2/996: symlink db/df/d54/dba/l14b 0 2024-05-09T06:38:40.649 INFO:tasks.workunit.client.1.smithi193.stdout:6/814: fsync d1/fa 0 2024-05-09T06:38:40.653 INFO:tasks.workunit.client.1.smithi193.stdout:0/869: fsync d18/d4c/d64/fcd 0 2024-05-09T06:38:40.655 INFO:tasks.workunit.client.1.smithi193.stdout:3/974: creat d1/d44/f144 x:0 0 0 2024-05-09T06:38:40.663 INFO:tasks.workunit.client.1.smithi193.stdout:6/815: creat d1/de/d11/d2d/ddf/f121 x:0 0 0 2024-05-09T06:38:40.665 INFO:tasks.workunit.client.1.smithi193.stdout:4/999: dread fa [0,4194304] 0 2024-05-09T06:38:40.667 INFO:tasks.workunit.client.1.smithi193.stdout:0/870: creat d18/d4c/d47/d59/d76/f10d x:0 0 0 2024-05-09T06:38:40.668 INFO:tasks.workunit.client.1.smithi193.stdout:2/997: write db/d25/d38/fdd [662993,83810] 0 2024-05-09T06:38:40.669 INFO:tasks.workunit.client.1.smithi193.stdout:2/998: chown db/df/l11 6890424 1 2024-05-09T06:38:40.673 INFO:tasks.workunit.client.1.smithi193.stdout:6/816: creat d1/de/d11/d18/d94/dac/f122 x:0 0 0 2024-05-09T06:38:40.673 INFO:tasks.workunit.client.1.smithi193.stdout:3/975: dwrite d1/f11 [0,4194304] 0 2024-05-09T06:38:40.679 INFO:tasks.workunit.client.1.smithi193.stdout:2/999: rename db/def/dfd/d80/d60/d7d/db3/cc6 to db/df/d54/c14c 0 2024-05-09T06:38:40.683 INFO:tasks.workunit.client.1.smithi193.stdout:3/976: write d1/db/d1b/fca [2156212,95122] 0 2024-05-09T06:38:40.683 INFO:tasks.workunit.client.1.smithi193.stdout:0/871: dwrite d18/d4c/d64/d8b/fe3 [0,4194304] 0 2024-05-09T06:38:40.684 INFO:tasks.workunit.client.1.smithi193.stdout:3/977: fsync d1/d44/d7d/f9c 0 2024-05-09T06:38:40.689 INFO:tasks.workunit.client.1.smithi193.stdout:6/817: dread d1/d25/d2c/d3c/d10c/f3f [0,4194304] 0 2024-05-09T06:38:40.689 INFO:tasks.workunit.client.1.smithi193.stdout:0/872: rmdir d18/d4c/d47 39 2024-05-09T06:38:40.690 INFO:tasks.workunit.client.1.smithi193.stdout:0/873: stat d18/d4c/f4b 0 2024-05-09T06:38:40.695 INFO:tasks.workunit.client.1.smithi193.stdout:0/874: symlink d18/dc9/l10e 0 2024-05-09T06:38:40.705 INFO:tasks.workunit.client.1.smithi193.stdout:6/818: getdents d1/d25/d2c/d81/d37/d4f 0 2024-05-09T06:38:40.706 INFO:tasks.workunit.client.1.smithi193.stdout:3/978: write d1/db/d2f/d78/d86/fa8 [3847942,94001] 0 2024-05-09T06:38:40.710 INFO:tasks.workunit.client.1.smithi193.stdout:6/819: creat d1/d25/d2c/d3c/d10c/d58/dc9/f123 x:0 0 0 2024-05-09T06:38:40.710 INFO:tasks.workunit.client.1.smithi193.stdout:3/979: chown d1/db/d59/d63/c89 25811765 1 2024-05-09T06:38:40.712 INFO:tasks.workunit.client.1.smithi193.stdout:6/820: chown d1/d25/d4c/l63 4 1 2024-05-09T06:38:40.714 INFO:tasks.workunit.client.1.smithi193.stdout:0/875: dwrite d18/d4c/d64/fcd [0,4194304] 0 2024-05-09T06:38:40.716 INFO:tasks.workunit.client.1.smithi193.stdout:3/980: chown d1/db/d2f/d78/d86/ddc/d51/d10c/c11e 40 1 2024-05-09T06:38:40.717 INFO:tasks.workunit.client.1.smithi193.stdout:6/821: creat d1/de/f124 x:0 0 0 2024-05-09T06:38:40.723 INFO:tasks.workunit.client.1.smithi193.stdout:0/876: dread d18/d4c/f8d [0,4194304] 0 2024-05-09T06:38:40.723 INFO:tasks.workunit.client.1.smithi193.stdout:6/822: rmdir d1/d25/d2c/d3c/d10c/d58 39 2024-05-09T06:38:40.724 INFO:tasks.workunit.client.1.smithi193.stdout:3/981: mkdir d1/db/d1b/d145 0 2024-05-09T06:38:40.726 INFO:tasks.workunit.client.1.smithi193.stdout:0/877: mknod d18/d1d/d56/c10f 0 2024-05-09T06:38:40.729 INFO:tasks.workunit.client.1.smithi193.stdout:3/982: mknod d1/db/d2f/d84/d138/c146 0 2024-05-09T06:38:40.735 INFO:tasks.workunit.client.1.smithi193.stdout:0/878: unlink d18/d4c/d47/d59/fcb 0 2024-05-09T06:38:40.736 INFO:tasks.workunit.client.1.smithi193.stdout:0/879: stat d18/dc9/ff4 0 2024-05-09T06:38:40.737 INFO:tasks.workunit.client.1.smithi193.stdout:6/823: dwrite d1/d25/d2c/d3c/fbb [0,4194304] 0 2024-05-09T06:38:40.739 INFO:tasks.workunit.client.1.smithi193.stdout:3/983: creat d1/db/d2f/d78/d86/ddc/d51/f147 x:0 0 0 2024-05-09T06:38:40.744 INFO:tasks.workunit.client.1.smithi193.stdout:0/880: mknod d18/d4c/d47/d59/d76/c110 0 2024-05-09T06:38:40.745 INFO:tasks.workunit.client.1.smithi193.stdout:0/881: fdatasync d18/d4c/d47/d59/faa 0 2024-05-09T06:38:40.746 INFO:tasks.workunit.client.1.smithi193.stdout:6/824: mknod d1/d25/d2c/d3c/d10c/dea/c125 0 2024-05-09T06:38:40.750 INFO:tasks.workunit.client.1.smithi193.stdout:0/882: fdatasync d18/d1d/f1e 0 2024-05-09T06:38:40.752 INFO:tasks.workunit.client.1.smithi193.stdout:6/825: symlink d1/d25/d4c/d9e/l126 0 2024-05-09T06:38:40.757 INFO:tasks.workunit.client.1.smithi193.stdout:3/984: link d1/db/d59/d63/fae d1/db/d2f/d78/d86/d94/f148 0 2024-05-09T06:38:40.760 INFO:tasks.workunit.client.1.smithi193.stdout:6/826: unlink d1/de/d11/d2d/d3a/d47/fa1 0 2024-05-09T06:38:40.762 INFO:tasks.workunit.client.1.smithi193.stdout:0/883: symlink d18/d4c/d64/d66/l111 0 2024-05-09T06:38:40.764 INFO:tasks.workunit.client.1.smithi193.stdout:3/985: symlink d1/d44/ddf/l149 0 2024-05-09T06:38:40.765 INFO:tasks.workunit.client.1.smithi193.stdout:3/986: chown d1/db/d2f/d78/d86/ffe 287274 1 2024-05-09T06:38:40.765 INFO:tasks.workunit.client.1.smithi193.stdout:6/827: creat d1/d25/d2c/d3c/f127 x:0 0 0 2024-05-09T06:38:40.765 INFO:tasks.workunit.client.1.smithi193.stdout:3/987: stat d1/db/d2f/d78/d86/l8f 0 2024-05-09T06:38:40.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/884: symlink d18/dc9/l112 0 2024-05-09T06:38:40.773 INFO:tasks.workunit.client.1.smithi193.stdout:6/828: dwrite d1/de/d11/d18/d94/f106 [0,4194304] 0 2024-05-09T06:38:40.777 INFO:tasks.workunit.client.1.smithi193.stdout:3/988: getdents d1/d44/ddf/d111 0 2024-05-09T06:38:40.777 INFO:tasks.workunit.client.1.smithi193.stdout:3/989: readlink d1/db/l67 0 2024-05-09T06:38:40.782 INFO:tasks.workunit.client.1.smithi193.stdout:6/829: symlink d1/de/d11/d2d/ddf/l128 0 2024-05-09T06:38:40.787 INFO:tasks.workunit.client.1.smithi193.stdout:0/885: dread f8 [0,4194304] 0 2024-05-09T06:38:40.793 INFO:tasks.workunit.client.1.smithi193.stdout:3/990: rmdir d1/db/d2f/d84/d13d 0 2024-05-09T06:38:40.794 INFO:tasks.workunit.client.1.smithi193.stdout:0/886: fdatasync d18/d1d/d56/f109 0 2024-05-09T06:38:40.797 INFO:tasks.workunit.client.1.smithi193.stdout:3/991: write d1/db/d2f/d78/d86/fa8 [1932211,63557] 0 2024-05-09T06:38:40.798 INFO:tasks.workunit.client.1.smithi193.stdout:6/830: dwrite d1/d25/d2c/d3c/d10c/dea/fe0 [0,4194304] 0 2024-05-09T06:38:40.806 INFO:tasks.workunit.client.1.smithi193.stdout:0/887: creat d18/d27/f113 x:0 0 0 2024-05-09T06:38:40.809 INFO:tasks.workunit.client.1.smithi193.stdout:3/992: rename d1/db/d2f/d78/d86/c8e to d1/db/c14a 0 2024-05-09T06:38:40.810 INFO:tasks.workunit.client.1.smithi193.stdout:3/993: read - d1/db/f3f zero size 2024-05-09T06:38:40.811 INFO:tasks.workunit.client.1.smithi193.stdout:0/888: mkdir d18/d4c/d47/d59/d114 0 2024-05-09T06:38:40.813 INFO:tasks.workunit.client.1.smithi193.stdout:3/994: rename d1/l15 to d1/db/d59/db8/l14b 0 2024-05-09T06:38:40.817 INFO:tasks.workunit.client.1.smithi193.stdout:6/831: dread d1/de/d11/d2d/d3a/f105 [0,4194304] 0 2024-05-09T06:38:40.820 INFO:tasks.workunit.client.1.smithi193.stdout:0/889: dwrite d18/d27/f53 [0,4194304] 0 2024-05-09T06:38:40.829 INFO:tasks.workunit.client.1.smithi193.stdout:3/995: truncate d1/db/d1b/f7a 1908308 0 2024-05-09T06:38:40.835 INFO:tasks.workunit.client.1.smithi193.stdout:0/890: rename d18/d4c/d47/d55/fb9 to d18/d4c/d47/d59/f115 0 2024-05-09T06:38:40.844 INFO:tasks.workunit.client.1.smithi193.stdout:6/832: link d1/d25/d2c/d3c/d10c/d58/c8a d1/de/d11/d18/c129 0 2024-05-09T06:38:40.848 INFO:tasks.workunit.client.1.smithi193.stdout:0/891: write d18/d4c/d64/d66/d10a/f10b [3324053,16858] 0 2024-05-09T06:38:40.848 INFO:tasks.workunit.client.1.smithi193.stdout:3/996: dwrite d1/db/d1b/f3b [0,4194304] 0 2024-05-09T06:38:40.852 INFO:tasks.workunit.client.1.smithi193.stdout:6/833: creat d1/de/d11/d2d/ddf/d116/d118/f12a x:0 0 0 2024-05-09T06:38:40.855 INFO:tasks.workunit.client.1.smithi193.stdout:0/892: symlink d18/d4c/d64/d66/d10a/l116 0 2024-05-09T06:38:40.856 INFO:tasks.workunit.client.1.smithi193.stdout:6/834: mkdir d1/d25/d4c/d12b 0 2024-05-09T06:38:40.857 INFO:tasks.workunit.client.1.smithi193.stdout:3/997: creat d1/db/f14c x:0 0 0 2024-05-09T06:38:40.860 INFO:tasks.workunit.client.1.smithi193.stdout:6/835: mkdir d1/d25/d12c 0 2024-05-09T06:38:40.861 INFO:tasks.workunit.client.1.smithi193.stdout:3/998: chown d1/db/d2f/d78/d86/ddc/d51/d11d 85820194 1 2024-05-09T06:38:40.863 INFO:tasks.workunit.client.1.smithi193.stdout:6/836: mkdir d1/de/d11/d18/d94/d12d 0 2024-05-09T06:38:40.867 INFO:tasks.workunit.client.1.smithi193.stdout:3/999: unlink d1/db/d59/c6f 0 2024-05-09T06:38:40.871 INFO:tasks.workunit.client.1.smithi193.stdout:5/883: sync 2024-05-09T06:38:40.879 INFO:tasks.workunit.client.1.smithi193.stdout:5/884: rename d0/d13/d2c/d37/d2b/d47/d60/d87/fa5 to d0/d13/d2c/d37/d2b/d6a/d90/def/db4/f125 0 2024-05-09T06:38:40.880 INFO:tasks.workunit.client.1.smithi193.stdout:0/893: write d18/d1d/f44 [792446,48459] 0 2024-05-09T06:38:40.882 INFO:tasks.workunit.client.1.smithi193.stdout:0/894: write d18/dc9/ff4 [4837771,56079] 0 2024-05-09T06:38:40.888 INFO:tasks.workunit.client.1.smithi193.stdout:5/885: mkdir d0/d13/d2c/d37/d5b/d4e/d102/d126 0 2024-05-09T06:38:40.890 INFO:tasks.workunit.client.1.smithi193.stdout:6/837: dwrite d1/de/fb8 [0,4194304] 0 2024-05-09T06:38:40.893 INFO:tasks.workunit.client.1.smithi193.stdout:5/886: truncate d0/d13/d2c/d37/d2b/d47/d60/d9a/fde 508258 0 2024-05-09T06:38:40.901 INFO:tasks.workunit.client.1.smithi193.stdout:5/887: symlink d0/d13/d2c/d37/d74/l127 0 2024-05-09T06:38:40.904 INFO:tasks.workunit.client.1.smithi193.stdout:0/895: rename d18/d4c/d5f/f8c to d18/d27/f117 0 2024-05-09T06:38:40.906 INFO:tasks.workunit.client.1.smithi193.stdout:0/896: write d18/d1d/d56/fe5 [383784,4789] 0 2024-05-09T06:38:40.925 INFO:tasks.workunit.client.1.smithi193.stdout:0/897: link d18/d4c/d5f/l104 d18/d65/l118 0 2024-05-09T06:38:40.933 INFO:tasks.workunit.client.1.smithi193.stdout:6/838: dwrite d1/d25/d2c/d3c/f93 [0,4194304] 0 2024-05-09T06:38:40.938 INFO:tasks.workunit.client.1.smithi193.stdout:0/898: getdents d18/d4c/d64/d66/d9d/da5 0 2024-05-09T06:38:40.942 INFO:tasks.workunit.client.1.smithi193.stdout:5/888: dwrite d0/f8 [0,4194304] 0 2024-05-09T06:38:40.946 INFO:tasks.workunit.client.1.smithi193.stdout:0/899: mknod d18/d27/c119 0 2024-05-09T06:38:40.963 INFO:tasks.workunit.client.1.smithi193.stdout:6/839: dread d1/de/d11/d18/d94/dac/ffe [0,4194304] 0 2024-05-09T06:38:40.974 INFO:tasks.workunit.client.1.smithi193.stdout:5/889: dwrite d0/f9 [0,4194304] 0 2024-05-09T06:38:40.975 INFO:tasks.workunit.client.1.smithi193.stdout:6/840: dwrite d1/f70 [4194304,4194304] 0 2024-05-09T06:38:40.975 INFO:tasks.workunit.client.1.smithi193.stdout:0/900: dwrite d18/f4f [0,4194304] 0 2024-05-09T06:38:40.979 INFO:tasks.workunit.client.1.smithi193.stdout:6/841: dread - d1/d25/d2c/d3c/f127 zero size 2024-05-09T06:38:40.980 INFO:tasks.workunit.client.1.smithi193.stdout:6/842: dread - d1/de/d11/d20/fbf zero size 2024-05-09T06:38:40.983 INFO:tasks.workunit.client.1.smithi193.stdout:0/901: symlink d18/d4c/d64/d74/dad/l11a 0 2024-05-09T06:38:40.984 INFO:tasks.workunit.client.1.smithi193.stdout:6/843: unlink d1/fa 0 2024-05-09T06:38:40.985 INFO:tasks.workunit.client.1.smithi193.stdout:6/844: chown d1/d25/lca 24123 1 2024-05-09T06:38:40.995 INFO:tasks.workunit.client.1.smithi193.stdout:6/845: getdents d1/d25/d2c/d81 0 2024-05-09T06:38:40.996 INFO:tasks.workunit.client.1.smithi193.stdout:6/846: symlink d1/d25/d2c/d3c/d10c/dea/l12e 0 2024-05-09T06:38:40.998 INFO:tasks.workunit.client.1.smithi193.stdout:6/847: write d1/de/d11/f101 [51848,12513] 0 2024-05-09T06:38:40.999 INFO:tasks.workunit.client.1.smithi193.stdout:6/848: chown d1/d25/d2c/d3c/fbb 41862 1 2024-05-09T06:38:41.002 INFO:tasks.workunit.client.1.smithi193.stdout:6/849: unlink d1/d25/d2c/d3c/d10c/d58/dc9/df6/ff8 0 2024-05-09T06:38:41.003 INFO:tasks.workunit.client.1.smithi193.stdout:5/890: truncate d0/d13/d2c/d37/d2b/d6a/d90/fc3 205577 0 2024-05-09T06:38:41.005 INFO:tasks.workunit.client.1.smithi193.stdout:6/850: creat d1/de/d11/d2d/f12f x:0 0 0 2024-05-09T06:38:41.006 INFO:tasks.workunit.client.1.smithi193.stdout:5/891: stat d0/d13/d2c/d37/d2b/d47/d66/cf 0 2024-05-09T06:38:41.008 INFO:tasks.workunit.client.1.smithi193.stdout:5/892: write d0/d13/d2c/d37/d5b/ffb [324775,38842] 0 2024-05-09T06:38:41.011 INFO:tasks.workunit.client.1.smithi193.stdout:0/902: dwrite d18/d4c/d64/d66/d9d/ff0 [0,4194304] 0 2024-05-09T06:38:41.013 INFO:tasks.workunit.client.1.smithi193.stdout:6/851: rename d1/de/d11/d18/d94/c10e to d1/de/d11/d2d/d3a/c130 0 2024-05-09T06:38:41.028 INFO:tasks.workunit.client.1.smithi193.stdout:6/852: unlink d1/d25/d2c/d3c/d10c/d58/fed 0 2024-05-09T06:38:41.030 INFO:tasks.workunit.client.1.smithi193.stdout:5/893: rename d0/d13/d2c/d37/d2b/d47/d60/d75/da9/l113 to d0/d13/d2c/d37/d2b/d47/l128 0 2024-05-09T06:38:41.036 INFO:tasks.workunit.client.1.smithi193.stdout:5/894: dread - d0/d13/d2c/db6/fd6 zero size 2024-05-09T06:38:41.036 INFO:tasks.workunit.client.1.smithi193.stdout:0/903: link d18/d4c/d64/d74/dad/d70/dc4/lcf d18/d4c/l11b 0 2024-05-09T06:38:41.038 INFO:tasks.workunit.client.1.smithi193.stdout:6/853: creat d1/d25/d12c/f131 x:0 0 0 2024-05-09T06:38:41.038 INFO:tasks.workunit.client.1.smithi193.stdout:0/904: truncate d18/d4c/d64/d74/dad/d70/dc4/ffe 5805 0 2024-05-09T06:38:41.039 INFO:tasks.workunit.client.1.smithi193.stdout:5/895: creat d0/d13/d2c/d37/d5b/d4e/d5d/f129 x:0 0 0 2024-05-09T06:38:41.048 INFO:tasks.workunit.client.1.smithi193.stdout:0/905: dwrite d18/d1d/f100 [0,4194304] 0 2024-05-09T06:38:41.048 INFO:tasks.workunit.client.1.smithi193.stdout:0/906: dread - d18/d27/f113 zero size 2024-05-09T06:38:41.050 INFO:tasks.workunit.client.1.smithi193.stdout:0/907: write d18/d1d/f10c [717777,39360] 0 2024-05-09T06:38:41.052 INFO:tasks.workunit.client.1.smithi193.stdout:0/908: write d18/d4c/d64/d74/dad/d70/f108 [831627,68986] 0 2024-05-09T06:38:41.053 INFO:tasks.workunit.client.1.smithi193.stdout:5/896: link d0/d13/d2c/d37/c41 d0/d13/d2c/d37/d2b/d6a/d90/def/db4/c12a 0 2024-05-09T06:38:41.057 INFO:tasks.workunit.client.1.smithi193.stdout:6/854: write d1/d25/d2c/d3c/d10c/dea/f11b [736820,74069] 0 2024-05-09T06:38:41.058 INFO:tasks.workunit.client.1.smithi193.stdout:6/855: readlink d1/d25/d2c/d3c/lae 0 2024-05-09T06:38:41.059 INFO:tasks.workunit.client.1.smithi193.stdout:5/897: mkdir d0/d13/d2c/d37/d2b/d47/d60/d12b 0 2024-05-09T06:38:41.060 INFO:tasks.workunit.client.1.smithi193.stdout:6/856: rmdir d1/d25/d4c 39 2024-05-09T06:38:41.063 INFO:tasks.workunit.client.1.smithi193.stdout:5/898: dread d0/d13/d2c/d37/d2b/d47/f79 [0,4194304] 0 2024-05-09T06:38:41.068 INFO:tasks.workunit.client.1.smithi193.stdout:0/909: write d18/d1d/f1f [477486,72996] 0 2024-05-09T06:38:41.070 INFO:tasks.workunit.client.1.smithi193.stdout:6/857: rmdir d1/de/d11/d18/d94/d12d 0 2024-05-09T06:38:41.071 INFO:tasks.workunit.client.1.smithi193.stdout:6/858: chown d1/de/d11/d2d 361319 1 2024-05-09T06:38:41.072 INFO:tasks.workunit.client.1.smithi193.stdout:5/899: dread d0/d13/d2c/d37/d5b/d4e/f7a [0,4194304] 0 2024-05-09T06:38:41.073 INFO:tasks.workunit.client.1.smithi193.stdout:0/910: symlink d18/d27/df1/l11c 0 2024-05-09T06:38:41.074 INFO:tasks.workunit.client.1.smithi193.stdout:6/859: fsync d1/d25/d2c/d3c/d10c/ff1 0 2024-05-09T06:38:41.076 INFO:tasks.workunit.client.1.smithi193.stdout:5/900: mkdir d0/d13/d2c/d37/d5b/d89/de0/d12c 0 2024-05-09T06:38:41.077 INFO:tasks.workunit.client.1.smithi193.stdout:0/911: creat d18/d4c/d47/d59/d114/f11d x:0 0 0 2024-05-09T06:38:41.078 INFO:tasks.workunit.client.1.smithi193.stdout:5/901: unlink d0/f7 0 2024-05-09T06:38:41.093 INFO:tasks.workunit.client.1.smithi193.stdout:0/912: chown d18/d4c/d64/l6d 24620 1 2024-05-09T06:38:41.095 INFO:tasks.workunit.client.1.smithi193.stdout:5/902: unlink d0/d13/d2c/d37/d2b/d47/f79 0 2024-05-09T06:38:41.096 INFO:tasks.workunit.client.1.smithi193.stdout:0/913: creat d18/dc9/f11e x:0 0 0 2024-05-09T06:38:41.099 INFO:tasks.workunit.client.1.smithi193.stdout:5/903: dread d0/f9 [0,4194304] 0 2024-05-09T06:38:41.107 INFO:tasks.workunit.client.1.smithi193.stdout:5/904: dwrite d0/f9 [0,4194304] 0 2024-05-09T06:38:41.115 INFO:tasks.workunit.client.1.smithi193.stdout:0/914: write d18/d4c/d64/f85 [278338,27669] 0 2024-05-09T06:38:41.116 INFO:tasks.workunit.client.1.smithi193.stdout:0/915: write d18/d1d/f1f [930112,102246] 0 2024-05-09T06:38:41.120 INFO:tasks.workunit.client.1.smithi193.stdout:0/916: mknod d18/d4c/d64/d66/d9d/c11f 0 2024-05-09T06:38:41.121 INFO:tasks.workunit.client.1.smithi193.stdout:5/905: dread d0/d13/d2c/d37/d2b/d47/f63 [0,4194304] 0 2024-05-09T06:38:41.129 INFO:tasks.workunit.client.1.smithi193.stdout:5/906: creat d0/d13/d2c/d37/d2b/d6a/d90/def/d10b/f12d x:0 0 0 2024-05-09T06:38:41.130 INFO:tasks.workunit.client.1.smithi193.stdout:0/917: write d18/d4c/f45 [4652847,38214] 0 2024-05-09T06:38:41.133 INFO:tasks.workunit.client.1.smithi193.stdout:5/907: symlink d0/d13/d2c/d37/d5b/d4e/d102/dbd/dd0/l12e 0 2024-05-09T06:38:41.134 INFO:tasks.workunit.client.1.smithi193.stdout:5/908: chown d0/d13/d2c/d37/d5b/l27 53876 1 2024-05-09T06:38:41.135 INFO:tasks.workunit.client.1.smithi193.stdout:5/909: write d0/d13/d2c/d37/d5b/d4e/d5d/f129 [874694,48502] 0 2024-05-09T06:38:41.138 INFO:tasks.workunit.client.1.smithi193.stdout:5/910: fdatasync d0/d13/d2c/d37/d2b/d6a/d90/def/db4/f125 0 2024-05-09T06:38:41.139 INFO:tasks.workunit.client.1.smithi193.stdout:0/918: write d18/d1d/d56/f109 [633513,35459] 0 2024-05-09T06:38:41.144 INFO:tasks.workunit.client.1.smithi193.stdout:0/919: rename d18/d1d/d56/fe5 to d18/d4c/d47/d59/d76/f120 0 2024-05-09T06:38:41.149 INFO:tasks.workunit.client.1.smithi193.stdout:0/920: rmdir d18/d4c/d64/d66/db5 39 2024-05-09T06:38:41.151 INFO:tasks.workunit.client.1.smithi193.stdout:0/921: stat d18/d4c/d64/d66/db5 0 2024-05-09T06:38:41.152 INFO:tasks.workunit.client.1.smithi193.stdout:0/922: write d18/d27/f41 [3250342,121943] 0 2024-05-09T06:38:41.153 INFO:tasks.workunit.client.1.smithi193.stdout:5/911: dwrite d0/d13/d2c/f4a [0,4194304] 0 2024-05-09T06:38:41.172 INFO:tasks.workunit.client.1.smithi193.stdout:5/912: dread d0/d13/d2c/d37/d5b/d89/f95 [0,4194304] 0 2024-05-09T06:38:41.182 INFO:tasks.workunit.client.1.smithi193.stdout:5/913: dwrite d0/d13/d2c/f4f [0,4194304] 0 2024-05-09T06:38:41.185 INFO:tasks.workunit.client.1.smithi193.stdout:5/914: write d0/d13/d2c/d4d/f101 [4047218,2631] 0 2024-05-09T06:38:41.190 INFO:tasks.workunit.client.1.smithi193.stdout:5/915: mkdir d0/d13/d2c/d37/d74/d12f 0 2024-05-09T06:38:41.191 INFO:tasks.workunit.client.1.smithi193.stdout:5/916: write d0/d13/d2c/d37/d5b/d4e/f78 [1894594,5321] 0 2024-05-09T06:38:41.198 INFO:tasks.workunit.client.1.smithi193.stdout:5/917: symlink d0/d10f/l130 0 2024-05-09T06:38:41.200 INFO:tasks.workunit.client.1.smithi193.stdout:5/918: creat d0/d13/d2c/d37/d2b/d6a/d90/f131 x:0 0 0 2024-05-09T06:38:41.202 INFO:tasks.workunit.client.1.smithi193.stdout:5/919: write d0/d13/d2c/d37/d2b/d6a/d90/def/db4/ffa [731997,74114] 0 2024-05-09T06:38:41.205 INFO:tasks.workunit.client.1.smithi193.stdout:5/920: stat d0/d13/d2c/d37/d2b/d6a/d90/def/db4/cc0 0 2024-05-09T06:38:41.206 INFO:tasks.workunit.client.1.smithi193.stdout:5/921: dread - d0/d13/d2c/d37/d2b/d6a/d90/f131 zero size 2024-05-09T06:38:41.209 INFO:tasks.workunit.client.1.smithi193.stdout:5/922: rmdir d0/d13/d2c/d37/d2b/d47/d60/d87 39 2024-05-09T06:38:41.210 INFO:tasks.workunit.client.1.smithi193.stdout:5/923: creat d0/d13/d2c/d4d/d67/f132 x:0 0 0 2024-05-09T06:38:41.213 INFO:tasks.workunit.client.1.smithi193.stdout:5/924: unlink d0/d13/d2c/d37/d5b/fbb 0 2024-05-09T06:38:41.219 INFO:tasks.workunit.client.1.smithi193.stdout:5/925: creat d0/d13/d2c/d37/d2b/d47/d66/f133 x:0 0 0 2024-05-09T06:38:41.220 INFO:tasks.workunit.client.1.smithi193.stdout:5/926: chown d0/d13/d2c/d37/d5b/d4e/d102/f121 111601 1 2024-05-09T06:38:41.227 INFO:tasks.workunit.client.1.smithi193.stdout:5/927: dwrite d0/d13/d2c/d37/d5b/d4e/d102/dbd/dd0/fe3 [4194304,4194304] 0 2024-05-09T06:38:41.249 INFO:tasks.workunit.client.1.smithi193.stdout:5/928: dread d0/f5c [4194304,4194304] 0 2024-05-09T06:38:41.252 INFO:tasks.workunit.client.1.smithi193.stdout:5/929: creat d0/d13/d2c/d37/d5b/d4e/d102/dbd/dd0/f134 x:0 0 0 2024-05-09T06:38:41.253 INFO:tasks.workunit.client.1.smithi193.stdout:5/930: readlink d0/d13/d2c/d37/d5b/l48 0 2024-05-09T06:38:41.253 INFO:tasks.workunit.client.1.smithi193.stdout:5/931: chown d0/d13/d2c/db6/dc6 708137 1 2024-05-09T06:38:41.259 INFO:tasks.workunit.client.1.smithi193.stdout:5/932: dwrite d0/d13/d2c/d37/d2b/d6a/d90/def/db4/ffa [0,4194304] 0 2024-05-09T06:38:41.260 INFO:tasks.workunit.client.1.smithi193.stdout:5/933: stat d0/d13/d2c/d37/d5b/d4e/d5d/f129 0 2024-05-09T06:38:41.263 INFO:tasks.workunit.client.1.smithi193.stdout:5/934: mkdir d0/d13/d2c/d37/d2b/d135 0 2024-05-09T06:38:41.265 INFO:tasks.workunit.client.1.smithi193.stdout:5/935: dread - d0/d13/d2c/d37/d2b/d6a/d90/def/fba zero size 2024-05-09T06:38:41.294 INFO:tasks.workunit.client.1.smithi193.stdout:5/936: dwrite d0/d13/d2c/d4d/fd7 [0,4194304] 0 2024-05-09T06:38:41.300 INFO:tasks.workunit.client.1.smithi193.stdout:5/937: mknod d0/d13/d2c/d37/d2b/d47/d60/d9a/c136 0 2024-05-09T06:38:41.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/938: write d0/d13/d2c/d37/d5b/d4e/d5d/f69 [1728177,87372] 0 2024-05-09T06:38:41.317 INFO:tasks.workunit.client.1.smithi193.stdout:5/939: dwrite d0/d13/d2c/d37/d2b/d6a/d90/def/db4/f105 [0,4194304] 0 2024-05-09T06:38:41.321 INFO:tasks.workunit.client.1.smithi193.stdout:5/940: mkdir d0/d10f/d137 0 2024-05-09T06:38:41.325 INFO:tasks.workunit.client.1.smithi193.stdout:5/941: mknod d0/d13/d2c/d37/d2b/d47/d66/d3a/c138 0 2024-05-09T06:38:41.326 INFO:tasks.workunit.client.1.smithi193.stdout:5/942: chown d0/d13/d2c/d37/d2b/d47/d66/d3a/c104 57253626 1 2024-05-09T06:38:41.331 INFO:tasks.workunit.client.1.smithi193.stdout:5/943: creat d0/d13/d2c/d37/d5b/d89/de0/d12c/f139 x:0 0 0 2024-05-09T06:38:41.331 INFO:tasks.workunit.client.1.smithi193.stdout:5/944: readlink d0/d13/d2c/db6/dc6/ldf 0 2024-05-09T06:38:41.332 INFO:tasks.workunit.client.1.smithi193.stdout:5/945: chown d0/d13/d2c/d37/d2b/d47/d60/d9a/fdb 467277 1 2024-05-09T06:38:41.335 INFO:tasks.workunit.client.1.smithi193.stdout:5/946: mknod d0/d13/d2c/d37/d5b/d89/de0/d12c/c13a 0 2024-05-09T06:38:41.336 INFO:tasks.workunit.client.1.smithi193.stdout:5/947: readlink d0/d13/d2c/d37/d2b/d6a/d90/def/le1 0 2024-05-09T06:38:41.339 INFO:tasks.workunit.client.1.smithi193.stdout:5/948: unlink d0/d13/d2c/d37/d2b/d6a/d90/def/l11d 0 2024-05-09T06:38:41.339 INFO:tasks.workunit.client.1.smithi193.stdout:5/949: chown d0/d13/d2c/db6/dc6 18569 1 2024-05-09T06:38:41.341 INFO:tasks.workunit.client.1.smithi193.stdout:5/950: write d0/d13/d2c/d4d/fd7 [2267203,112992] 0 2024-05-09T06:38:41.347 INFO:tasks.workunit.client.1.smithi193.stdout:5/951: creat d0/d13/d2c/d37/d5b/f13b x:0 0 0 2024-05-09T06:38:41.347 INFO:tasks.workunit.client.1.smithi193.stdout:5/952: fsync d0/ff8 0 2024-05-09T06:38:41.348 INFO:tasks.workunit.client.1.smithi193.stdout:5/953: readlink d0/d13/d2c/d37/d2b/d47/laa 0 2024-05-09T06:38:41.354 INFO:tasks.workunit.client.1.smithi193.stdout:5/954: creat d0/d13/d2c/d37/d5b/d4e/d102/d126/f13c x:0 0 0 2024-05-09T06:38:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:40 smithi063 ceph-mon[27171]: pgmap v23: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 61 MiB/s rd, 145 MiB/s wr, 357 op/s 2024-05-09T06:38:41.359 INFO:tasks.workunit.client.1.smithi193.stdout:5/955: rmdir d0/d13/d2c/d37/d2b/d135 0 2024-05-09T06:38:41.360 INFO:tasks.workunit.client.1.smithi193.stdout:5/956: fdatasync d0/d13/d2c/d37/d5b/d4e/d5d/f69 0 2024-05-09T06:38:41.366 INFO:tasks.workunit.client.1.smithi193.stdout:5/957: rename d0/d13/d2c/d37/d2b/d47/d60/dc8/ld2 to d0/d13/d2c/l13d 0 2024-05-09T06:38:41.368 INFO:tasks.workunit.client.1.smithi193.stdout:5/958: unlink d0/d13/d2c/d37/d2b/d47/d60/d9a/fdb 0 2024-05-09T06:38:41.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:40 smithi193 ceph-mon[35141]: pgmap v23: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 61 MiB/s rd, 145 MiB/s wr, 357 op/s 2024-05-09T06:38:41.375 INFO:tasks.workunit.client.1.smithi193.stdout:5/959: dwrite d0/d13/d2c/da0/f110 [0,4194304] 0 2024-05-09T06:38:41.384 INFO:tasks.workunit.client.1.smithi193.stdout:5/960: creat d0/d13/d2c/d37/d5b/f13e x:0 0 0 2024-05-09T06:38:41.387 INFO:tasks.workunit.client.1.smithi193.stdout:5/961: unlink d0/d13/d2c/d4d/d67/f76 0 2024-05-09T06:38:41.392 INFO:tasks.workunit.client.1.smithi193.stdout:5/962: mknod d0/d13/d2c/d37/d2b/d47/d6f/c13f 0 2024-05-09T06:38:41.392 INFO:tasks.workunit.client.1.smithi193.stdout:5/963: readlink d0/d13/l62 0 2024-05-09T06:38:41.395 INFO:tasks.workunit.client.1.smithi193.stdout:5/964: rmdir d0/d13/d2c/d37/d5b/d4e/d102/d126 39 2024-05-09T06:38:41.401 INFO:tasks.workunit.client.1.smithi193.stdout:5/965: dwrite d0/d13/d2c/f4a [0,4194304] 0 2024-05-09T06:38:41.406 INFO:tasks.workunit.client.1.smithi193.stdout:5/966: unlink d0/d13/d2c/d37/d5b/f21 0 2024-05-09T06:38:41.409 INFO:tasks.workunit.client.1.smithi193.stdout:5/967: truncate d0/d13/d2c/d37/d2b/d47/d60/d9a/fde 618987 0 2024-05-09T06:38:41.413 INFO:tasks.workunit.client.1.smithi193.stdout:5/968: link d0/d13/d2c/d4d/d67/c98 d0/d13/d2c/d37/d2b/d47/d60/d75/c140 0 2024-05-09T06:38:41.414 INFO:tasks.workunit.client.1.smithi193.stdout:5/969: chown d0/d13/d2c/d37/d5b/d89/de0/dff 2 1 2024-05-09T06:38:41.420 INFO:tasks.workunit.client.1.smithi193.stdout:5/970: dread d0/f29 [0,4194304] 0 2024-05-09T06:38:41.424 INFO:tasks.workunit.client.1.smithi193.stdout:5/971: rmdir d0/d13/d2c/d37/d5b 39 2024-05-09T06:38:41.428 INFO:tasks.workunit.client.1.smithi193.stdout:5/972: mknod d0/d13/d2c/d37/d5b/d89/de0/d12c/c141 0 2024-05-09T06:38:41.429 INFO:tasks.workunit.client.1.smithi193.stdout:5/973: chown d0/d13/d2c/d37/d2b/d47/d60/d75 191715 1 2024-05-09T06:38:41.429 INFO:tasks.workunit.client.1.smithi193.stdout:5/974: fdatasync d0/d13/d2c/d4d/d67/fc7 0 2024-05-09T06:38:41.446 INFO:tasks.workunit.client.1.smithi193.stdout:5/975: dread d0/d13/d2c/d37/d5b/d4e/d5d/fdd [0,4194304] 0 2024-05-09T06:38:41.449 INFO:tasks.workunit.client.1.smithi193.stdout:5/976: creat d0/d13/d2c/d37/d2b/d6a/d90/def/f142 x:0 0 0 2024-05-09T06:38:41.453 INFO:tasks.workunit.client.1.smithi193.stdout:5/977: getdents d0/d13/d2c/d37/d5b/d4e/d102/d126 0 2024-05-09T06:38:41.455 INFO:tasks.workunit.client.1.smithi193.stdout:5/978: mknod d0/d13/d2c/d37/d5b/d89/de0/c143 0 2024-05-09T06:38:41.459 INFO:tasks.workunit.client.1.smithi193.stdout:5/979: dread d0/d13/d2c/d37/d5b/d4e/d5d/f129 [0,4194304] 0 2024-05-09T06:38:41.468 INFO:tasks.workunit.client.1.smithi193.stdout:5/980: link d0/d13/d2c/d37/d74/f91 d0/d13/d2c/db6/f144 0 2024-05-09T06:38:41.470 INFO:tasks.workunit.client.1.smithi193.stdout:5/981: unlink d0/d13/d2c/d7e/fd3 0 2024-05-09T06:38:41.473 INFO:tasks.workunit.client.1.smithi193.stdout:5/982: read d0/d13/d2c/d4d/fca [2366525,13788] 0 2024-05-09T06:38:41.481 INFO:tasks.workunit.client.1.smithi193.stdout:5/983: dread d0/f9 [0,4194304] 0 2024-05-09T06:38:41.481 INFO:tasks.workunit.client.1.smithi193.stdout:5/984: dread - d0/d13/d2c/d37/d5b/d4e/d102/dbd/dd0/f134 zero size 2024-05-09T06:38:41.482 INFO:tasks.workunit.client.1.smithi193.stdout:5/985: fdatasync d0/d13/d2c/d37/d2b/d6a/d90/def/f142 0 2024-05-09T06:38:41.492 INFO:tasks.workunit.client.1.smithi193.stdout:5/986: dwrite d0/d13/d2c/d37/d2b/d47/d66/f15 [0,4194304] 0 2024-05-09T06:38:41.497 INFO:tasks.workunit.client.1.smithi193.stdout:5/987: symlink d0/d13/d2c/d37/d5b/d4e/d102/dbd/dd0/l145 0 2024-05-09T06:38:41.505 INFO:tasks.workunit.client.1.smithi193.stdout:5/988: creat d0/d13/d2c/d37/d2b/d47/d60/d75/da9/dc1/f146 x:0 0 0 2024-05-09T06:38:41.545 INFO:tasks.workunit.client.1.smithi193.stdout:5/989: creat d0/d13/d2c/d37/d5b/d4e/d102/dbd/f147 x:0 0 0 2024-05-09T06:38:41.551 INFO:tasks.workunit.client.1.smithi193.stdout:5/990: dread d0/d13/d2c/d37/f11 [0,4194304] 0 2024-05-09T06:38:41.551 INFO:tasks.workunit.client.1.smithi193.stdout:5/991: dread - d0/d13/d2c/d37/d2b/d6a/d90/def/d10b/f12d zero size 2024-05-09T06:38:41.552 INFO:tasks.workunit.client.1.smithi193.stdout:5/992: chown d0/d13/d2c/f4a 43 1 2024-05-09T06:38:41.561 INFO:tasks.workunit.client.1.smithi193.stdout:5/993: dwrite d0/d13/d2c/d37/d2b/d6a/d90/def/db4/f105 [0,4194304] 0 2024-05-09T06:38:41.589 INFO:tasks.workunit.client.1.smithi193.stdout:5/994: dread d0/d13/d2c/d4d/f101 [0,4194304] 0 2024-05-09T06:38:41.592 INFO:tasks.workunit.client.1.smithi193.stdout:5/995: mknod d0/d13/d2c/d37/d2b/d47/d60/d75/da9/c148 0 2024-05-09T06:38:41.594 INFO:tasks.workunit.client.1.smithi193.stdout:5/996: truncate d0/d13/d2c/d37/d2b/d47/d66/f26 2793114 0 2024-05-09T06:38:41.596 INFO:tasks.workunit.client.1.smithi193.stdout:5/997: creat d0/d13/d2c/db6/dc6/f149 x:0 0 0 2024-05-09T06:38:41.599 INFO:tasks.workunit.client.1.smithi193.stdout:5/998: dread - d0/d13/d2c/d37/d74/f91 zero size 2024-05-09T06:38:41.600 INFO:tasks.workunit.client.1.smithi193.stdout:5/999: write d0/d13/d2c/d7e/dac/ffd [4731485,70700] 0 2024-05-09T06:38:41.650 INFO:tasks.workunit.client.1.smithi193.stdout:0/923: sync 2024-05-09T06:38:41.650 INFO:tasks.workunit.client.1.smithi193.stdout:6/860: sync 2024-05-09T06:38:41.661 INFO:tasks.workunit.client.1.smithi193.stdout:0/924: dwrite d18/d4c/f4b [0,4194304] 0 2024-05-09T06:38:41.662 INFO:tasks.workunit.client.1.smithi193.stdout:6/861: dread d1/d25/d2c/d81/d37/d4f/fd3 [0,4194304] 0 2024-05-09T06:38:41.667 INFO:tasks.workunit.client.1.smithi193.stdout:6/862: symlink d1/d25/d12c/l132 0 2024-05-09T06:38:41.672 INFO:tasks.workunit.client.1.smithi193.stdout:0/925: creat d18/d4c/d64/ddb/f121 x:0 0 0 2024-05-09T06:38:41.682 INFO:tasks.workunit.client.1.smithi193.stdout:6/863: dwrite d1/d25/d2c/d76/d69/f88 [0,4194304] 0 2024-05-09T06:38:41.686 INFO:tasks.workunit.client.1.smithi193.stdout:6/864: mknod d1/d25/d2c/d76/c133 0 2024-05-09T06:38:42.090 INFO:tasks.workunit.client.1.smithi193.stdout:6/865: sync 2024-05-09T06:38:42.090 INFO:tasks.workunit.client.1.smithi193.stdout:0/926: sync 2024-05-09T06:38:42.091 INFO:tasks.workunit.client.1.smithi193.stdout:6/866: chown d1/d25/d2c/d3c/d10c/dea/f10d 1969354 1 2024-05-09T06:38:42.092 INFO:tasks.workunit.client.1.smithi193.stdout:6/867: fdatasync d1/de/d11/d2d/d3a/d47/f5f 0 2024-05-09T06:38:42.095 INFO:tasks.workunit.client.1.smithi193.stdout:6/868: dread - d1/de/d11/d2d/fdb zero size 2024-05-09T06:38:42.096 INFO:tasks.workunit.client.1.smithi193.stdout:0/927: write d18/d4c/d64/d66/fb1 [520124,28804] 0 2024-05-09T06:38:42.100 INFO:tasks.workunit.client.1.smithi193.stdout:0/928: dread d18/d4c/f4b [0,4194304] 0 2024-05-09T06:38:42.101 INFO:tasks.workunit.client.1.smithi193.stdout:6/869: dread - d1/d25/d2c/d3c/d10c/d58/dc9/f103 zero size 2024-05-09T06:38:42.106 INFO:tasks.workunit.client.1.smithi193.stdout:0/929: dwrite d18/d4c/d47/d59/d76/fda [0,4194304] 0 2024-05-09T06:38:42.107 INFO:tasks.workunit.client.1.smithi193.stdout:6/870: creat d1/d25/d4c/d9e/f134 x:0 0 0 2024-05-09T06:38:42.110 INFO:tasks.workunit.client.1.smithi193.stdout:6/871: mkdir d1/de/d11/d20/d135 0 2024-05-09T06:38:42.114 INFO:tasks.workunit.client.1.smithi193.stdout:6/872: write d1/de/d11/d2d/d3a/f105 [4366014,39497] 0 2024-05-09T06:38:42.115 INFO:tasks.workunit.client.1.smithi193.stdout:0/930: dwrite d18/d65/fee [0,4194304] 0 2024-05-09T06:38:42.119 INFO:tasks.workunit.client.1.smithi193.stdout:6/873: creat d1/de/d11/d2d/f136 x:0 0 0 2024-05-09T06:38:42.121 INFO:tasks.workunit.client.1.smithi193.stdout:6/874: write d1/d25/d2c/d3c/d10c/dea/fe0 [673628,12599] 0 2024-05-09T06:38:42.125 INFO:tasks.workunit.client.1.smithi193.stdout:0/931: getdents d18/d27 0 2024-05-09T06:38:42.126 INFO:tasks.workunit.client.1.smithi193.stdout:6/875: rename d1/de/ff to d1/d25/d4c/d12b/f137 0 2024-05-09T06:38:42.127 INFO:tasks.workunit.client.1.smithi193.stdout:0/932: readlink d18/d4c/d64/d74/l9b 0 2024-05-09T06:38:42.128 INFO:tasks.workunit.client.1.smithi193.stdout:0/933: fsync d18/d27/fa0 0 2024-05-09T06:38:42.147 INFO:tasks.workunit.client.1.smithi193.stdout:6/876: dread d1/d25/d4c/d9e/fb0 [0,4194304] 0 2024-05-09T06:38:42.147 INFO:tasks.workunit.client.1.smithi193.stdout:0/934: dwrite d18/d4c/fa4 [4194304,4194304] 0 2024-05-09T06:38:42.149 INFO:tasks.workunit.client.1.smithi193.stdout:6/877: mknod d1/d25/d2c/d3c/c138 0 2024-05-09T06:38:42.152 INFO:tasks.workunit.client.1.smithi193.stdout:0/935: creat d18/f122 x:0 0 0 2024-05-09T06:38:42.168 INFO:tasks.workunit.client.1.smithi193.stdout:0/936: dread d18/d27/f41 [0,4194304] 0 2024-05-09T06:38:42.168 INFO:tasks.workunit.client.1.smithi193.stdout:0/937: stat d18/d27/ffd 0 2024-05-09T06:38:42.170 INFO:tasks.workunit.client.1.smithi193.stdout:0/938: creat d18/d65/d82/f123 x:0 0 0 2024-05-09T06:38:42.183 INFO:tasks.workunit.client.1.smithi193.stdout:0/939: dread d18/d4c/f38 [0,4194304] 0 2024-05-09T06:38:42.185 INFO:tasks.workunit.client.1.smithi193.stdout:0/940: symlink d18/d4c/d47/l124 0 2024-05-09T06:38:42.186 INFO:tasks.workunit.client.1.smithi193.stdout:0/941: chown d18/dc9/fd8 106092 1 2024-05-09T06:38:42.191 INFO:tasks.workunit.client.1.smithi193.stdout:0/942: getdents d18/d4c/d47 0 2024-05-09T06:38:42.193 INFO:tasks.workunit.client.1.smithi193.stdout:0/943: unlink d18/d4c/d47/d55/fae 0 2024-05-09T06:38:42.195 INFO:tasks.workunit.client.1.smithi193.stdout:0/944: creat d18/d27/f125 x:0 0 0 2024-05-09T06:38:42.209 INFO:tasks.workunit.client.1.smithi193.stdout:0/945: dread d18/d4c/d47/fed [0,4194304] 0 2024-05-09T06:38:42.213 INFO:tasks.workunit.client.1.smithi193.stdout:0/946: dread d18/d4c/d47/d59/d76/fda [0,4194304] 0 2024-05-09T06:38:42.219 INFO:tasks.workunit.client.1.smithi193.stdout:0/947: write d18/d4c/f8d [203993,64794] 0 2024-05-09T06:38:42.221 INFO:tasks.workunit.client.1.smithi193.stdout:0/948: write d18/d27/f125 [707853,49142] 0 2024-05-09T06:38:42.227 INFO:tasks.workunit.client.1.smithi193.stdout:0/949: dwrite d18/d1d/d56/f109 [0,4194304] 0 2024-05-09T06:38:42.252 INFO:tasks.workunit.client.1.smithi193.stdout:0/950: dread d18/d4c/f45 [4194304,4194304] 0 2024-05-09T06:38:42.253 INFO:tasks.workunit.client.1.smithi193.stdout:0/951: chown d18/d4c/d64/d66/d9d/da5/dd0/fd7 122 1 2024-05-09T06:38:42.255 INFO:tasks.workunit.client.1.smithi193.stdout:0/952: symlink d18/l126 0 2024-05-09T06:38:42.258 INFO:tasks.workunit.client.1.smithi193.stdout:0/953: rename d18/d1d/lc5 to d18/d4c/d47/d59/d114/l127 0 2024-05-09T06:38:42.269 INFO:tasks.workunit.client.1.smithi193.stdout:0/954: dwrite d18/d4c/d64/d74/dad/d70/f83 [0,4194304] 0 2024-05-09T06:38:42.272 INFO:tasks.workunit.client.1.smithi193.stdout:0/955: mknod d18/d4c/d64/d74/dad/c128 0 2024-05-09T06:38:42.273 INFO:tasks.workunit.client.1.smithi193.stdout:0/956: write d18/d4c/d64/d66/fb1 [1175375,39910] 0 2024-05-09T06:38:42.287 INFO:tasks.workunit.client.1.smithi193.stdout:0/957: write d18/d4c/d47/d59/f91 [1885117,68691] 0 2024-05-09T06:38:42.289 INFO:tasks.workunit.client.1.smithi193.stdout:0/958: mkdir d18/d4c/d64/d8b/d129 0 2024-05-09T06:38:42.293 INFO:tasks.workunit.client.1.smithi193.stdout:0/959: rename d18/d27/f113 to d18/d4c/d64/d66/d9d/da5/dd0/f12a 0 2024-05-09T06:38:42.301 INFO:tasks.workunit.client.1.smithi193.stdout:0/960: dread d18/f28 [0,4194304] 0 2024-05-09T06:38:42.303 INFO:tasks.workunit.client.1.smithi193.stdout:0/961: mkdir d18/d4c/d64/ddb/d12b 0 2024-05-09T06:38:42.304 INFO:tasks.workunit.client.1.smithi193.stdout:0/962: truncate d18/d4c/d64/f85 931178 0 2024-05-09T06:38:42.311 INFO:tasks.workunit.client.1.smithi193.stdout:0/963: getdents d18/d1d/d56/da7 0 2024-05-09T06:38:42.316 INFO:tasks.workunit.client.1.smithi193.stdout:0/964: rename d18/d4c/d64/d66/d9d to d18/d1d/d56/da7/d12c 0 2024-05-09T06:38:42.331 INFO:tasks.workunit.client.1.smithi193.stdout:0/965: rename d18/d27/df1 to d18/d65/d12d 0 2024-05-09T06:38:42.332 INFO:tasks.workunit.client.1.smithi193.stdout:6/878: sync 2024-05-09T06:38:42.335 INFO:tasks.workunit.client.1.smithi193.stdout:0/966: mknod d18/d4c/c12e 0 2024-05-09T06:38:42.338 INFO:tasks.workunit.client.1.smithi193.stdout:0/967: mknod d18/d4c/d47/d59/d114/c12f 0 2024-05-09T06:38:42.341 INFO:tasks.workunit.client.1.smithi193.stdout:0/968: dread d18/d1d/f1f [0,4194304] 0 2024-05-09T06:38:42.346 INFO:tasks.workunit.client.1.smithi193.stdout:6/879: write d1/d25/fb7 [857258,57251] 0 2024-05-09T06:38:42.346 INFO:tasks.workunit.client.1.smithi193.stdout:0/969: creat d18/d1d/f130 x:0 0 0 2024-05-09T06:38:42.351 INFO:tasks.workunit.client.1.smithi193.stdout:0/970: rename d18/d1d/d56/da7/d12c/da5/dd0/f12a to d18/d4c/d64/d8b/d129/f131 0 2024-05-09T06:38:42.352 INFO:tasks.workunit.client.1.smithi193.stdout:6/880: write d1/d25/d4c/d9e/fb0 [2957144,116578] 0 2024-05-09T06:38:42.354 INFO:tasks.workunit.client.1.smithi193.stdout:0/971: mknod d18/d4c/d64/d66/db5/c132 0 2024-05-09T06:38:42.355 INFO:tasks.workunit.client.1.smithi193.stdout:6/881: read d1/d25/d2c/f50 [161036,101450] 0 2024-05-09T06:38:42.356 INFO:tasks.workunit.client.1.smithi193.stdout:6/882: chown d1/d25/d2c/d76/d69/f88 5 1 2024-05-09T06:38:42.357 INFO:tasks.workunit.client.1.smithi193.stdout:0/972: mkdir d18/d4c/d47/d59/d133 0 2024-05-09T06:38:42.359 INFO:tasks.workunit.client.1.smithi193.stdout:0/973: readlink d18/d4c/d64/l6e 0 2024-05-09T06:38:42.362 INFO:tasks.workunit.client.1.smithi193.stdout:0/974: unlink d18/d1d/d56/f109 0 2024-05-09T06:38:42.364 INFO:tasks.workunit.client.1.smithi193.stdout:6/883: link d1/de/f10 d1/de/d11/d2d/ddf/d116/f139 0 2024-05-09T06:38:42.366 INFO:tasks.workunit.client.1.smithi193.stdout:0/975: rmdir d18/d4c/d64/ddb 39 2024-05-09T06:38:42.366 INFO:tasks.workunit.client.1.smithi193.stdout:0/976: chown d18/d1d/d56/c93 57026 1 2024-05-09T06:38:42.373 INFO:tasks.workunit.client.1.smithi193.stdout:6/884: rename d1/de/d11/f6c to d1/de/d11/d2d/d3a/f13a 0 2024-05-09T06:38:42.377 INFO:tasks.workunit.client.1.smithi193.stdout:6/885: mknod d1/de/d11/d18/ddc/c13b 0 2024-05-09T06:38:42.378 INFO:tasks.workunit.client.1.smithi193.stdout:0/977: getdents d18/d4c/d47/d59 0 2024-05-09T06:38:42.383 INFO:tasks.workunit.client.1.smithi193.stdout:0/978: write d18/f25 [2820860,46758] 0 2024-05-09T06:38:42.389 INFO:tasks.workunit.client.1.smithi193.stdout:0/979: dwrite d18/d4c/f8d [0,4194304] 0 2024-05-09T06:38:42.390 INFO:tasks.workunit.client.1.smithi193.stdout:0/980: write d18/d4c/ffb [55064,101627] 0 2024-05-09T06:38:42.392 INFO:tasks.workunit.client.1.smithi193.stdout:0/981: unlink d18/d4c/d47/lf5 0 2024-05-09T06:38:42.394 INFO:tasks.workunit.client.1.smithi193.stdout:0/982: mkdir d18/d4c/d64/d74/dad/d70/d134 0 2024-05-09T06:38:42.396 INFO:tasks.workunit.client.1.smithi193.stdout:0/983: read d18/d4c/d64/d66/fd9 [474326,69379] 0 2024-05-09T06:38:42.399 INFO:tasks.workunit.client.1.smithi193.stdout:0/984: mknod d18/d4c/d47/d54/c135 0 2024-05-09T06:38:42.406 INFO:tasks.workunit.client.1.smithi193.stdout:0/985: dwrite d18/d4c/d47/fea [0,4194304] 0 2024-05-09T06:38:42.424 INFO:tasks.workunit.client.1.smithi193.stdout:0/986: dwrite d18/d27/fa9 [0,4194304] 0 2024-05-09T06:38:42.427 INFO:tasks.workunit.client.1.smithi193.stdout:0/987: symlink d18/d1d/d56/l136 0 2024-05-09T06:38:42.433 INFO:tasks.workunit.client.1.smithi193.stdout:0/988: dread d18/d27/f53 [0,4194304] 0 2024-05-09T06:38:42.437 INFO:tasks.workunit.client.1.smithi193.stdout:0/989: creat d18/d4c/d47/d59/f137 x:0 0 0 2024-05-09T06:38:42.441 INFO:tasks.workunit.client.1.smithi193.stdout:0/990: truncate d18/d4c/d64/ddb/f121 400091 0 2024-05-09T06:38:42.443 INFO:tasks.workunit.client.1.smithi193.stdout:0/991: stat d18/d27/ffd 0 2024-05-09T06:38:42.451 INFO:tasks.workunit.client.1.smithi193.stdout:0/992: fsync d18/d4c/d47/d59/d76/f10d 0 2024-05-09T06:38:42.452 INFO:tasks.workunit.client.1.smithi193.stdout:0/993: read d18/d4c/d47/d54/fd1 [3712729,91055] 0 2024-05-09T06:38:42.452 INFO:tasks.workunit.client.1.smithi193.stdout:0/994: chown d18/d1d/d56/cdf 43826335 1 2024-05-09T06:38:42.460 INFO:tasks.workunit.client.1.smithi193.stdout:0/995: dwrite d18/d4c/d47/f58 [0,4194304] 0 2024-05-09T06:38:42.465 INFO:tasks.workunit.client.1.smithi193.stdout:0/996: dread d18/d65/fee [0,4194304] 0 2024-05-09T06:38:42.471 INFO:tasks.workunit.client.1.smithi193.stdout:0/997: dwrite d18/d4c/f8d [0,4194304] 0 2024-05-09T06:38:42.477 INFO:tasks.workunit.client.1.smithi193.stdout:0/998: creat d18/f138 x:0 0 0 2024-05-09T06:38:42.480 INFO:tasks.workunit.client.1.smithi193.stdout:0/999: mknod d18/d4c/d64/d74/dad/c139 0 2024-05-09T06:38:42.552 INFO:tasks.workunit.client.1.smithi193.stdout:6/886: sync 2024-05-09T06:38:42.592 INFO:tasks.workunit.client.1.smithi193.stdout:6/887: dwrite d1/d25/d4c/d12b/f137 [0,4194304] 0 2024-05-09T06:38:42.594 INFO:tasks.workunit.client.1.smithi193.stdout:6/888: creat d1/d25/d2c/d3c/f13c x:0 0 0 2024-05-09T06:38:42.620 INFO:tasks.workunit.client.1.smithi193.stdout:6/889: dwrite d1/d25/d4c/d9e/feb [0,4194304] 0 2024-05-09T06:38:42.625 INFO:tasks.workunit.client.1.smithi193.stdout:6/890: creat d1/de/d11/d18/ddc/f13d x:0 0 0 2024-05-09T06:38:42.631 INFO:tasks.workunit.client.1.smithi193.stdout:6/891: dwrite d1/d25/d2c/d3c/d10c/ff1 [0,4194304] 0 2024-05-09T06:38:42.633 INFO:tasks.workunit.client.1.smithi193.stdout:6/892: truncate d1/d25/d4c/fd5 2031632 0 2024-05-09T06:38:42.643 INFO:tasks.workunit.client.1.smithi193.stdout:6/893: truncate d1/de/d11/f22 3888853 0 2024-05-09T06:38:42.645 INFO:tasks.workunit.client.1.smithi193.stdout:6/894: creat d1/d25/d4c/d12b/f13e x:0 0 0 2024-05-09T06:38:42.648 INFO:tasks.workunit.client.1.smithi193.stdout:6/895: rename d1/d25/d2c/d3c/f127 to d1/d25/d12c/f13f 0 2024-05-09T06:38:42.649 INFO:tasks.workunit.client.1.smithi193.stdout:6/896: write d1/d25/d12c/f131 [877065,58522] 0 2024-05-09T06:38:42.654 INFO:tasks.workunit.client.1.smithi193.stdout:6/897: getdents d1/de/d11/d18 0 2024-05-09T06:38:42.672 INFO:tasks.workunit.client.1.smithi193.stdout:6/898: dread d1/d25/d2c/d3c/d10c/d58/fce [0,4194304] 0 2024-05-09T06:38:42.675 INFO:tasks.workunit.client.1.smithi193.stdout:6/899: mkdir d1/d25/d140 0 2024-05-09T06:38:42.677 INFO:tasks.workunit.client.1.smithi193.stdout:6/900: read - d1/de/d11/d18/d94/fcd zero size 2024-05-09T06:38:42.689 INFO:tasks.workunit.client.1.smithi193.stdout:6/901: dread d1/de/d11/d2d/da4/fd0 [0,4194304] 0 2024-05-09T06:38:42.694 INFO:tasks.workunit.client.1.smithi193.stdout:6/902: dwrite d1/de/fa6 [0,4194304] 0 2024-05-09T06:38:42.696 INFO:tasks.workunit.client.1.smithi193.stdout:6/903: chown d1/de/d11/d18/d94/dac/ffe 688 1 2024-05-09T06:38:42.697 INFO:tasks.workunit.client.1.smithi193.stdout:6/904: truncate d1/de/d11/f101 525185 0 2024-05-09T06:38:42.702 INFO:tasks.workunit.client.1.smithi193.stdout:6/905: write d1/de/d11/f5d [975124,124482] 0 2024-05-09T06:38:42.709 INFO:tasks.workunit.client.1.smithi193.stdout:6/906: dwrite d1/d25/d2c/d3c/f13c [0,4194304] 0 2024-05-09T06:38:42.714 INFO:tasks.workunit.client.1.smithi193.stdout:6/907: link d1/de/d11/d18/d94/f106 d1/d25/d4c/d9e/dc8/de3/f141 0 2024-05-09T06:38:42.717 INFO:tasks.workunit.client.1.smithi193.stdout:6/908: fdatasync d1/de/d11/d20/fbf 0 2024-05-09T06:38:42.721 INFO:tasks.workunit.client.1.smithi193.stdout:6/909: rmdir d1/de/d11/d18/d94/dac 39 2024-05-09T06:38:43.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:42 smithi063 ceph-mon[27171]: pgmap v24: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 84 MiB/s rd, 202 MiB/s wr, 506 op/s 2024-05-09T06:38:43.143 INFO:tasks.workunit.client.1.smithi193.stdout:6/910: sync 2024-05-09T06:38:43.146 INFO:tasks.workunit.client.1.smithi193.stdout:6/911: fdatasync d1/d25/d2c/d81/d37/f10a 0 2024-05-09T06:38:43.161 INFO:tasks.workunit.client.1.smithi193.stdout:6/912: dread d1/de/d11/d2d/f34 [0,4194304] 0 2024-05-09T06:38:43.162 INFO:tasks.workunit.client.1.smithi193.stdout:6/913: write d1/de/d11/f101 [791340,67180] 0 2024-05-09T06:38:43.168 INFO:tasks.workunit.client.1.smithi193.stdout:6/914: link d1/d25/d2c/f50 d1/d25/d2c/d81/d37/d4f/f142 0 2024-05-09T06:38:43.172 INFO:tasks.workunit.client.1.smithi193.stdout:6/915: creat d1/d25/d4c/d9e/dc8/de3/f143 x:0 0 0 2024-05-09T06:38:43.174 INFO:tasks.workunit.client.1.smithi193.stdout:6/916: write d1/d25/d2c/d3c/d10c/dea/f11b [893032,77876] 0 2024-05-09T06:38:43.175 INFO:tasks.workunit.client.1.smithi193.stdout:6/917: chown d1/de/d11/d2d/f40 0 1 2024-05-09T06:38:43.175 INFO:tasks.workunit.client.1.smithi193.stdout:6/918: fsync d1/de/d11/f101 0 2024-05-09T06:38:43.182 INFO:tasks.workunit.client.1.smithi193.stdout:6/919: symlink d1/de/d11/d18/d94/l144 0 2024-05-09T06:38:43.183 INFO:tasks.workunit.client.1.smithi193.stdout:6/920: truncate d1/d25/d12c/f13f 33242 0 2024-05-09T06:38:43.187 INFO:tasks.workunit.client.1.smithi193.stdout:6/921: link d1/d25/d2c/d3c/d10c/dea/f11b d1/d25/d140/f145 0 2024-05-09T06:38:43.235 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:42 smithi193 ceph-mon[35141]: pgmap v24: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 84 MiB/s rd, 202 MiB/s wr, 506 op/s 2024-05-09T06:38:43.252 INFO:tasks.workunit.client.1.smithi193.stdout:6/922: dwrite d1/d25/d2c/d3c/ffb [0,4194304] 0 2024-05-09T06:38:43.256 INFO:tasks.workunit.client.1.smithi193.stdout:6/923: mknod d1/d25/d2c/d81/c146 0 2024-05-09T06:38:43.258 INFO:tasks.workunit.client.1.smithi193.stdout:6/924: chown d1/d25/lca 82 1 2024-05-09T06:38:43.277 INFO:tasks.workunit.client.1.smithi193.stdout:6/925: read d1/d25/d2c/d81/fb5 [196650,1450] 0 2024-05-09T06:38:43.280 INFO:tasks.workunit.client.1.smithi193.stdout:6/926: mkdir d1/d25/d2c/d81/d37/db3/d147 0 2024-05-09T06:38:43.288 INFO:tasks.workunit.client.1.smithi193.stdout:6/927: rename d1/de/d11/d2d/f136 to d1/de/d11/d18/d94/f148 0 2024-05-09T06:38:43.289 INFO:tasks.workunit.client.1.smithi193.stdout:6/928: fsync d1/de/d11/d2d/ddf/d116/d118/f12a 0 2024-05-09T06:38:43.309 INFO:tasks.workunit.client.1.smithi193.stdout:6/929: dwrite d1/d25/d2c/d81/d37/d4f/fd3 [0,4194304] 0 2024-05-09T06:38:43.313 INFO:tasks.workunit.client.1.smithi193.stdout:6/930: symlink d1/de/d11/d20/d135/l149 0 2024-05-09T06:38:43.331 INFO:tasks.workunit.client.1.smithi193.stdout:6/931: truncate d1/de/d11/f5d 250207 0 2024-05-09T06:38:43.334 INFO:tasks.workunit.client.1.smithi193.stdout:6/932: truncate d1/de/d11/d2d/fa8 2020018 0 2024-05-09T06:38:43.339 INFO:tasks.workunit.client.1.smithi193.stdout:6/933: link d1/de/d11/d18/d94/lde d1/de/d11/d2d/l14a 0 2024-05-09T06:38:43.341 INFO:tasks.workunit.client.1.smithi193.stdout:6/934: chown d1/de/d11/d18/d94/dac/f122 14758 1 2024-05-09T06:38:43.356 INFO:tasks.workunit.client.1.smithi193.stdout:6/935: dread d1/de/d11/d2d/f52 [0,4194304] 0 2024-05-09T06:38:43.358 INFO:tasks.workunit.client.1.smithi193.stdout:6/936: write d1/d25/d12c/f13f [96633,40283] 0 2024-05-09T06:38:43.361 INFO:tasks.workunit.client.1.smithi193.stdout:6/937: truncate d1/de/d11/d18/d94/f106 1754172 0 2024-05-09T06:38:43.361 INFO:tasks.workunit.client.1.smithi193.stdout:6/938: fdatasync d1/de/d11/f101 0 2024-05-09T06:38:43.364 INFO:tasks.workunit.client.1.smithi193.stdout:6/939: creat d1/de/d11/d2d/ddf/d116/f14b x:0 0 0 2024-05-09T06:38:43.366 INFO:tasks.workunit.client.1.smithi193.stdout:6/940: read d1/de/d11/d18/f61 [70828,125640] 0 2024-05-09T06:38:43.369 INFO:tasks.workunit.client.1.smithi193.stdout:6/941: rename d1/de/d11/d2d/da4/fe5 to d1/d25/d4c/d9e/dc5/f14c 0 2024-05-09T06:38:43.388 INFO:tasks.workunit.client.1.smithi193.stdout:6/942: write d1/de/d11/d2d/ddf/d116/f139 [1639829,36611] 0 2024-05-09T06:38:43.834 INFO:tasks.workunit.client.1.smithi193.stdout:6/943: sync 2024-05-09T06:38:43.845 INFO:tasks.workunit.client.1.smithi193.stdout:6/944: dread d1/d25/f30 [0,4194304] 0 2024-05-09T06:38:43.847 INFO:tasks.workunit.client.1.smithi193.stdout:6/945: read d1/d25/fb7 [77461,49157] 0 2024-05-09T06:38:43.848 INFO:tasks.workunit.client.1.smithi193.stdout:6/946: readlink d1/d25/d2c/d81/d37/d4f/l114 0 2024-05-09T06:38:43.853 INFO:tasks.workunit.client.1.smithi193.stdout:6/947: mknod d1/de/c14d 0 2024-05-09T06:38:43.855 INFO:tasks.workunit.client.1.smithi193.stdout:6/948: write d1/d25/d2c/d3c/d10c/d58/dc9/f123 [669105,91639] 0 2024-05-09T06:38:43.858 INFO:tasks.workunit.client.1.smithi193.stdout:6/949: fdatasync d1/de/d11/f68 0 2024-05-09T06:38:43.860 INFO:tasks.workunit.client.1.smithi193.stdout:6/950: symlink d1/de/d11/d18/d94/dac/l14e 0 2024-05-09T06:38:43.861 INFO:tasks.workunit.client.1.smithi193.stdout:6/951: stat d1/d25/d2c/d81/d37/d4f/f66 0 2024-05-09T06:38:43.869 INFO:tasks.workunit.client.1.smithi193.stdout:6/952: getdents d1/d25 0 2024-05-09T06:38:43.869 INFO:tasks.workunit.client.1.smithi193.stdout:6/953: fdatasync d1/d25/d2c/d76/d69/f88 0 2024-05-09T06:38:43.873 INFO:tasks.workunit.client.1.smithi193.stdout:6/954: mknod d1/d25/d4c/d12b/c14f 0 2024-05-09T06:38:43.874 INFO:tasks.workunit.client.1.smithi193.stdout:6/955: readlink d1/d25/d2c/l7f 0 2024-05-09T06:38:43.878 INFO:tasks.workunit.client.1.smithi193.stdout:6/956: creat d1/d25/d2c/d81/d37/d4f/f150 x:0 0 0 2024-05-09T06:38:43.882 INFO:tasks.workunit.client.1.smithi193.stdout:6/957: unlink d1/de/d11/d2d/ddf/l128 0 2024-05-09T06:38:43.884 INFO:tasks.workunit.client.1.smithi193.stdout:6/958: mknod d1/de/d11/d2d/ddf/d116/c151 0 2024-05-09T06:38:43.884 INFO:tasks.workunit.client.1.smithi193.stdout:6/959: chown d1/d25/d2c/d3c/d10c/d58/dc9 128 1 2024-05-09T06:38:43.889 INFO:tasks.workunit.client.1.smithi193.stdout:6/960: rename d1/d25/d2c/d3c/d10c/dea/fe0 to d1/d25/d2c/d81/d37/db3/f152 0 2024-05-09T06:38:43.890 INFO:tasks.workunit.client.1.smithi193.stdout:6/961: stat d1/de/d11/d2d/ddf/d116/d118/f12a 0 2024-05-09T06:38:43.894 INFO:tasks.workunit.client.1.smithi193.stdout:6/962: symlink d1/de/d11/l153 0 2024-05-09T06:38:43.897 INFO:tasks.workunit.client.1.smithi193.stdout:6/963: mknod d1/d25/d2c/d76/d69/c154 0 2024-05-09T06:38:43.900 INFO:tasks.workunit.client.1.smithi193.stdout:6/964: unlink d1/d25/d4c/d9e/dc8/de3/f143 0 2024-05-09T06:38:43.904 INFO:tasks.workunit.client.1.smithi193.stdout:6/965: creat d1/d25/d2c/d3c/d10c/d58/f155 x:0 0 0 2024-05-09T06:38:43.923 INFO:tasks.workunit.client.1.smithi193.stdout:6/966: dread d1/f23 [0,4194304] 0 2024-05-09T06:38:43.937 INFO:tasks.workunit.client.1.smithi193.stdout:6/967: rename d1/d25/d2c/d3c/d10c/f113 to d1/f156 0 2024-05-09T06:38:43.973 INFO:tasks.workunit.client.1.smithi193.stdout:6/968: dwrite d1/de/d11/d2d/f40 [0,4194304] 0 2024-05-09T06:38:43.975 INFO:tasks.workunit.client.1.smithi193.stdout:6/969: read - d1/de/d11/f68 zero size 2024-05-09T06:38:44.274 INFO:tasks.workunit.client.1.smithi193.stdout:6/970: sync 2024-05-09T06:38:44.277 INFO:tasks.workunit.client.1.smithi193.stdout:6/971: creat d1/d25/d4c/d9e/dc8/f157 x:0 0 0 2024-05-09T06:38:44.280 INFO:tasks.workunit.client.1.smithi193.stdout:6/972: mknod d1/d25/d2c/d3c/c158 0 2024-05-09T06:38:44.284 INFO:tasks.workunit.client.1.smithi193.stdout:6/973: rename d1/de/d11/d2d/d3a/d47/f7a to d1/d25/d2c/d76/d69/f159 0 2024-05-09T06:38:44.286 INFO:tasks.workunit.client.1.smithi193.stdout:6/974: symlink d1/de/d11/d2d/l15a 0 2024-05-09T06:38:44.314 INFO:tasks.workunit.client.1.smithi193.stdout:6/975: dwrite d1/d25/d2c/d76/d69/fd4 [0,4194304] 0 2024-05-09T06:38:44.320 INFO:tasks.workunit.client.1.smithi193.stdout:6/976: mknod d1/d25/d2c/d3c/d10c/d58/c15b 0 2024-05-09T06:38:44.323 INFO:tasks.workunit.client.1.smithi193.stdout:6/977: dread - d1/d25/d2c/d81/d37/f104 zero size 2024-05-09T06:38:44.327 INFO:tasks.workunit.client.1.smithi193.stdout:6/978: creat d1/d25/d2c/d81/d37/d4f/f15c x:0 0 0 2024-05-09T06:38:44.328 INFO:tasks.workunit.client.1.smithi193.stdout:6/979: chown d1/de/d11/d20/l11c 47995609 1 2024-05-09T06:38:44.328 INFO:tasks.workunit.client.1.smithi193.stdout:6/980: chown d1/fd 1 1 2024-05-09T06:38:44.330 INFO:tasks.workunit.client.1.smithi193.stdout:6/981: creat d1/de/d11/d18/ddc/f15d x:0 0 0 2024-05-09T06:38:44.331 INFO:tasks.workunit.client.1.smithi193.stdout:6/982: stat d1/de/c60 0 2024-05-09T06:38:44.333 INFO:tasks.workunit.client.1.smithi193.stdout:6/983: creat d1/de/d11/d18/f15e x:0 0 0 2024-05-09T06:38:44.334 INFO:tasks.workunit.client.1.smithi193.stdout:6/984: truncate d1/de/f124 32575 0 2024-05-09T06:38:44.335 INFO:tasks.workunit.client.1.smithi193.stdout:6/985: chown d1/d25/d2c/d76/f11f 618843 1 2024-05-09T06:38:44.337 INFO:tasks.workunit.client.1.smithi193.stdout:6/986: chown d1/d25/d2c/d81/c64 8551 1 2024-05-09T06:38:44.341 INFO:tasks.workunit.client.1.smithi193.stdout:6/987: link d1/d25/d2c/d76/d69/fd4 d1/d25/d4c/d9e/f15f 0 2024-05-09T06:38:44.344 INFO:tasks.workunit.client.1.smithi193.stdout:6/988: mknod d1/d25/d2c/d81/d37/db3/d147/c160 0 2024-05-09T06:38:44.346 INFO:tasks.workunit.client.1.smithi193.stdout:6/989: fsync d1/d25/d2c/d81/d37/ffd 0 2024-05-09T06:38:44.348 INFO:tasks.workunit.client.1.smithi193.stdout:6/990: chown d1/de/f53 3745 1 2024-05-09T06:38:44.356 INFO:tasks.workunit.client.1.smithi193.stdout:6/991: mknod d1/de/d11/d18/ddc/c161 0 2024-05-09T06:38:44.358 INFO:tasks.workunit.client.1.smithi193.stdout:6/992: mkdir d1/d25/d4c/d9e/dc5/d162 0 2024-05-09T06:38:44.359 INFO:tasks.workunit.client.1.smithi193.stdout:6/993: chown d1/de/d11/d18/cb6 164978 1 2024-05-09T06:38:44.363 INFO:tasks.workunit.client.1.smithi193.stdout:6/994: dread d1/f42 [0,4194304] 0 2024-05-09T06:38:44.365 INFO:tasks.workunit.client.1.smithi193.stdout:6/995: chown d1/de/d11/d18/c129 170301715 1 2024-05-09T06:38:44.367 INFO:tasks.workunit.client.1.smithi193.stdout:6/996: fdatasync d1/d25/d2c/fc4 0 2024-05-09T06:38:44.370 INFO:tasks.workunit.client.1.smithi193.stdout:6/997: symlink d1/de/d11/d2d/d3a/d47/l163 0 2024-05-09T06:38:44.373 INFO:tasks.workunit.client.1.smithi193.stdout:6/998: creat d1/d25/d2c/d81/d37/f164 x:0 0 0 2024-05-09T06:38:44.388 INFO:tasks.workunit.client.1.smithi193.stdout:6/999: dwrite d1/de/d11/d18/f61 [0,4194304] 0 2024-05-09T06:38:44.391 INFO:tasks.workunit.client.1.smithi193.stderr:+ rm -rf -- ./tmp.CAkSB7U89v 2024-05-09T06:38:44.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: Adjusting osd_memory_target on smithi193 to 2240M 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: pgmap v25: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 43 MiB/s rd, 113 MiB/s wr, 300 op/s 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:38:44.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:38:44.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:38:44.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:38:44.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:44 smithi063 ceph-mon[27171]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:38:44.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:44.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:44.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:44.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:44.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T06:38:44.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: Adjusting osd_memory_target on smithi193 to 2240M 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: pgmap v25: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 43 MiB/s rd, 113 MiB/s wr, 300 op/s 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:38:44.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:44 smithi193 ceph-mon[35141]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:38:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.236+0000 7f4e43ffe700 1 -- 172.21.15.63:0/1286157763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c103fe0 msgr2=0x7f4e3c1063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.236+0000 7f4e43ffe700 1 --2- 172.21.15.63:0/1286157763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c103fe0 0x7f4e3c1063d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009910 tx=0x7f4e2c009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:45.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.236+0000 7f4e43ffe700 1 -- 172.21.15.63:0/1286157763 shutdown_connections 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.236+0000 7f4e43ffe700 1 --2- 172.21.15.63:0/1286157763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c103fe0 0x7f4e3c1063d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.236+0000 7f4e43ffe700 1 --2- 172.21.15.63:0/1286157763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3c1016b0 0x7f4e3c103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.236+0000 7f4e43ffe700 1 -- 172.21.15.63:0/1286157763 >> 172.21.15.63:0/1286157763 conn(0x7f4e3c0fb0c0 msgr2=0x7f4e3c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.236+0000 7f4e43ffe700 1 -- 172.21.15.63:0/1286157763 shutdown_connections 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.236+0000 7f4e43ffe700 1 -- 172.21.15.63:0/1286157763 wait complete. 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e43ffe700 1 Processor -- start 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e43ffe700 1 -- start start 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e43ffe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c1016b0 0x7f4e3c1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e43ffe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3c103fe0 0x7f4e3c196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e43ffe700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c197140 con 0x7f4e3c1016b0 2024-05-09T06:38:45.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e43ffe700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c197280 con 0x7f4e3c103fe0 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e41d9a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c1016b0 0x7f4e3c1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e41d9a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c1016b0 0x7f4e3c1965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45838/0 (socket says 172.21.15.63:45838) 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e41d9a700 1 -- 172.21.15.63:0/273227730 learned_addr learned my addr 172.21.15.63:0/273227730 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e41599700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3c103fe0 0x7f4e3c196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e41d9a700 1 -- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3c103fe0 msgr2=0x7f4e3c196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e41d9a700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3c103fe0 0x7f4e3c196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e41d9a700 1 -- 172.21.15.63:0/273227730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c0095f0 con 0x7f4e3c1016b0 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.237+0000 7f4e41599700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3c103fe0 0x7f4e3c196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.238+0000 7f4e41d9a700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c1016b0 0x7f4e3c1965e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e920 tx=0x7f4e3800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.238+0000 7f4e32ffd700 1 -- 172.21.15.63:0/273227730 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e3800c9d0 con 0x7f4e3c1016b0 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.238+0000 7f4e32ffd700 1 -- 172.21.15.63:0/273227730 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e38010430 con 0x7f4e3c1016b0 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.238+0000 7f4e43ffe700 1 -- 172.21.15.63:0/273227730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c0ff500 con 0x7f4e3c1016b0 2024-05-09T06:38:45.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.238+0000 7f4e32ffd700 1 -- 172.21.15.63:0/273227730 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e38018630 con 0x7f4e3c1016b0 2024-05-09T06:38:45.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.238+0000 7f4e43ffe700 1 -- 172.21.15.63:0/273227730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c0ffad0 con 0x7f4e3c1016b0 2024-05-09T06:38:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.239+0000 7f4e30ff9700 1 -- 172.21.15.63:0/273227730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e240052f0 con 0x7f4e3c1016b0 2024-05-09T06:38:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.240+0000 7f4e32ffd700 1 -- 172.21.15.63:0/273227730 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f4e380105a0 con 0x7f4e3c1016b0 2024-05-09T06:38:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.240+0000 7f4e32ffd700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f4e2806d6f0 0x7f4e2806fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.240+0000 7f4e32ffd700 1 -- 172.21.15.63:0/273227730 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4e38014070 con 0x7f4e3c1016b0 2024-05-09T06:38:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.241+0000 7f4e41599700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f4e2806d6f0 0x7f4e2806fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:45.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.241+0000 7f4e41599700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f4e2806d6f0 0x7f4e2806fbb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b800 tx=0x7f4e2c009f70 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:45.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.245+0000 7f4e32ffd700 1 -- 172.21.15.63:0/273227730 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f4e38053720 con 0x7f4e3c1016b0 2024-05-09T06:38:45.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.494+0000 7f4e30ff9700 1 -- 172.21.15.63:0/273227730 --> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e24000bc0 con 0x7f4e2806d6f0 2024-05-09T06:38:45.475 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:45 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.475 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:45 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.475 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:45 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.475 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:45 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.475 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:45 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.475 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:45 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:38:45.476 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:45 smithi063 ceph-mon[27171]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-05-09T06:38:45.476 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:45 smithi063 ceph-mon[27171]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-05-09T06:38:45.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.495+0000 7f4e32ffd700 1 -- 172.21.15.63:0/273227730 <== mgr.24415 v2:172.21.15.193:6828/1432929717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f4e24000bc0 con 0x7f4e2806d6f0 2024-05-09T06:38:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 -- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f4e2806d6f0 msgr2=0x7f4e2806fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f4e2806d6f0 0x7f4e2806fbb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b800 tx=0x7f4e2c009f70 comp rx=0 tx=0).stop 2024-05-09T06:38:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 -- 172.21.15.63:0/273227730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c1016b0 msgr2=0x7f4e3c1965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c1016b0 0x7f4e3c1965e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e920 tx=0x7f4e3800ec30 comp rx=0 tx=0).stop 2024-05-09T06:38:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 -- 172.21.15.63:0/273227730 shutdown_connections 2024-05-09T06:38:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f4e2806d6f0 0x7f4e2806fbb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e3c1016b0 0x7f4e3c1965e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 --2- 172.21.15.63:0/273227730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3c103fe0 0x7f4e3c196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 -- 172.21.15.63:0/273227730 >> 172.21.15.63:0/273227730 conn(0x7f4e3c0fb0c0 msgr2=0x7f4e3c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 -- 172.21.15.63:0/273227730 shutdown_connections 2024-05-09T06:38:45.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.497+0000 7f4e30ff9700 1 -- 172.21.15.63:0/273227730 wait complete. 2024-05-09T06:38:45.489 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:38:45.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.612+0000 7ff9fc714700 1 -- 172.21.15.63:0/1713389369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f4100da0 msgr2=0x7ff9f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.612+0000 7ff9fc714700 1 --2- 172.21.15.63:0/1713389369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f4100da0 0x7ff9f4101200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009910 tx=0x7ff9e4009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:45.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.612+0000 7ff9fc714700 1 -- 172.21.15.63:0/1713389369 shutdown_connections 2024-05-09T06:38:45.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.612+0000 7ff9fc714700 1 --2- 172.21.15.63:0/1713389369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f4100da0 0x7ff9f4101200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.612+0000 7ff9fc714700 1 --2- 172.21.15.63:0/1713389369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f40ffba0 0x7ff9f40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.612+0000 7ff9fc714700 1 -- 172.21.15.63:0/1713389369 >> 172.21.15.63:0/1713389369 conn(0x7ff9f40fb120 msgr2=0x7ff9f40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.612+0000 7ff9fc714700 1 -- 172.21.15.63:0/1713389369 shutdown_connections 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.612+0000 7ff9fc714700 1 -- 172.21.15.63:0/1713389369 wait complete. 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fc714700 1 Processor -- start 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fc714700 1 -- start start 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fc714700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f40ffba0 0x7ff9f41966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fc714700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f4100da0 0x7ff9f4196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fc714700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4197210 con 0x7ff9f40ffba0 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fc714700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4197350 con 0x7ff9f4100da0 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fa4b0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f40ffba0 0x7ff9f41966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fa4b0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f40ffba0 0x7ff9f41966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45856/0 (socket says 172.21.15.63:45856) 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fa4b0700 1 -- 172.21.15.63:0/1449890221 learned_addr learned my addr 172.21.15.63:0/1449890221 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:45.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9f9caf700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f4100da0 0x7ff9f4196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:45.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fa4b0700 1 -- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f4100da0 msgr2=0x7ff9f4196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fa4b0700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f4100da0 0x7ff9f4196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.613+0000 7ff9fa4b0700 1 -- 172.21.15.63:0/1449890221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40095f0 con 0x7ff9f40ffba0 2024-05-09T06:38:45.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.614+0000 7ff9fa4b0700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f40ffba0 0x7ff9f41966b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000d900 tx=0x7ff9f000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:45.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.614+0000 7ff9eb7fe700 1 -- 172.21.15.63:0/1449890221 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f00041f0 con 0x7ff9f40ffba0 2024-05-09T06:38:45.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.614+0000 7ff9eb7fe700 1 -- 172.21.15.63:0/1449890221 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9f0004350 con 0x7ff9f40ffba0 2024-05-09T06:38:45.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.614+0000 7ff9eb7fe700 1 -- 172.21.15.63:0/1449890221 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f0003cf0 con 0x7ff9f40ffba0 2024-05-09T06:38:45.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.614+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f419bdb0 con 0x7ff9f40ffba0 2024-05-09T06:38:45.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.614+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f419c270 con 0x7ff9f40ffba0 2024-05-09T06:38:45.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.615+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f404ebb0 con 0x7ff9f40ffba0 2024-05-09T06:38:45.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.615+0000 7ff9eb7fe700 1 -- 172.21.15.63:0/1449890221 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7ff9f0009730 con 0x7ff9f40ffba0 2024-05-09T06:38:45.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.616+0000 7ff9eb7fe700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7ff9e006d6f0 0x7ff9e006fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:45.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.616+0000 7ff9eb7fe700 1 -- 172.21.15.63:0/1449890221 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff9f008cf60 con 0x7ff9f40ffba0 2024-05-09T06:38:45.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.616+0000 7ff9f9caf700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7ff9e006d6f0 0x7ff9e006fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:45.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.616+0000 7ff9f9caf700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7ff9e006d6f0 0x7ff9e006fbb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4005090 tx=0x7ff9e4005960 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:45.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.621+0000 7ff9eb7fe700 1 -- 172.21.15.63:0/1449890221 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7ff9f00c1720 con 0x7ff9f40ffba0 2024-05-09T06:38:45.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:45 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:45 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:45 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:45 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:45 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:45.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:45 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:38:45.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:45 smithi193 ceph-mon[35141]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-05-09T06:38:45.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:45 smithi193 ceph-mon[35141]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-05-09T06:38:45.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.869+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 --> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9f4068d50 con 0x7ff9e006d6f0 2024-05-09T06:38:45.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.870+0000 7ff9eb7fe700 1 -- 172.21.15.63:0/1449890221 <== mgr.24415 v2:172.21.15.193:6828/1432929717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7ff9f4068d50 con 0x7ff9e006d6f0 2024-05-09T06:38:45.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7ff9e006d6f0 msgr2=0x7ff9e006fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7ff9e006d6f0 0x7ff9e006fbb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4005090 tx=0x7ff9e4005960 comp rx=0 tx=0).stop 2024-05-09T06:38:45.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f40ffba0 msgr2=0x7ff9f41966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f40ffba0 0x7ff9f41966b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000d900 tx=0x7ff9f000dc10 comp rx=0 tx=0).stop 2024-05-09T06:38:45.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 shutdown_connections 2024-05-09T06:38:45.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7ff9e006d6f0 0x7ff9e006fbb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f40ffba0 0x7ff9f41966b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 --2- 172.21.15.63:0/1449890221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9f4100da0 0x7ff9f4196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 >> 172.21.15.63:0/1449890221 conn(0x7ff9f40fb120 msgr2=0x7ff9f4068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:45.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 shutdown_connections 2024-05-09T06:38:45.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:45.873+0000 7ff9fc714700 1 -- 172.21.15.63:0/1449890221 wait complete. 2024-05-09T06:38:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.011+0000 7fe189137700 1 -- 172.21.15.63:0/3754138857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 msgr2=0x7fe18410c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.011+0000 7fe189137700 1 --2- 172.21.15.63:0/3754138857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 0x7fe18410c8d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe174009910 tx=0x7fe174009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.011+0000 7fe189137700 1 -- 172.21.15.63:0/3754138857 shutdown_connections 2024-05-09T06:38:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.011+0000 7fe189137700 1 --2- 172.21.15.63:0/3754138857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 0x7fe18410c8d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.011+0000 7fe189137700 1 --2- 172.21.15.63:0/3754138857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe184072c10 0x7fe184109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.012+0000 7fe189137700 1 -- 172.21.15.63:0/3754138857 >> 172.21.15.63:0/3754138857 conn(0x7fe18406db90 msgr2=0x7fe18406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.012+0000 7fe189137700 1 -- 172.21.15.63:0/3754138857 shutdown_connections 2024-05-09T06:38:45.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.012+0000 7fe189137700 1 -- 172.21.15.63:0/3754138857 wait complete. 2024-05-09T06:38:45.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.014+0000 7fe189137700 1 Processor -- start 2024-05-09T06:38:45.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.014+0000 7fe189137700 1 -- start start 2024-05-09T06:38:45.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.014+0000 7fe189137700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe184072c10 0x7fe1841a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:45.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.014+0000 7fe189137700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 0x7fe1841a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:45.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.014+0000 7fe189137700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1841a5f60 con 0x7fe18410a440 2024-05-09T06:38:45.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.014+0000 7fe189137700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1841a60a0 con 0x7fe184072c10 2024-05-09T06:38:45.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.015+0000 7fe17a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 0x7fe1841a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:45.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.015+0000 7fe17a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 0x7fe1841a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45886/0 (socket says 172.21.15.63:45886) 2024-05-09T06:38:45.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.015+0000 7fe17a59c700 1 -- 172.21.15.63:0/685351190 learned_addr learned my addr 172.21.15.63:0/685351190 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:45.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.015+0000 7fe17a59c700 1 -- 172.21.15.63:0/685351190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe184072c10 msgr2=0x7fe1841a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:45.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.015+0000 7fe17a59c700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe184072c10 0x7fe1841a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:45.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.015+0000 7fe17a59c700 1 -- 172.21.15.63:0/685351190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740095f0 con 0x7fe18410a440 2024-05-09T06:38:45.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.016+0000 7fe17a59c700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 0x7fe1841a5940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe1740098e0 tx=0x7fe17400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.016+0000 7fe180d99700 1 -- 172.21.15.63:0/685351190 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe17401d070 con 0x7fe18410a440 2024-05-09T06:38:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.016+0000 7fe180d99700 1 -- 172.21.15.63:0/685351190 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe17400f460 con 0x7fe18410a440 2024-05-09T06:38:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.016+0000 7fe180d99700 1 -- 172.21.15.63:0/685351190 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe174021640 con 0x7fe18410a440 2024-05-09T06:38:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.017+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1841aaaa0 con 0x7fe18410a440 2024-05-09T06:38:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.017+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1841aaf50 con 0x7fe18410a440 2024-05-09T06:38:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.018+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18419f5c0 con 0x7fe18410a440 2024-05-09T06:38:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.019+0000 7fe180d99700 1 -- 172.21.15.63:0/685351190 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7fe17400fad0 con 0x7fe18410a440 2024-05-09T06:38:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.020+0000 7fe180d99700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fe17006d730 0x7fe17006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.020+0000 7fe180d99700 1 -- 172.21.15.63:0/685351190 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe17408e3f0 con 0x7fe18410a440 2024-05-09T06:38:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.021+0000 7fe182d9d700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fe17006d730 0x7fe17006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:46.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.022+0000 7fe182d9d700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fe17006d730 0x7fe17006fbf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00a990 tx=0x7fe16c005c80 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:46.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.028+0000 7fe180d99700 1 -- 172.21.15.63:0/685351190 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7fe17405b6e0 con 0x7fe18410a440 2024-05-09T06:38:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.273+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 --> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe18402cf00 con 0x7fe17006d730 2024-05-09T06:38:46.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.286+0000 7fe180d99700 1 -- 172.21.15.63:0/685351190 <== mgr.24415 v2:172.21.15.193:6828/1432929717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe18402cf00 con 0x7fe17006d730 2024-05-09T06:38:46.267 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:38:46.267 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (7m) 26s ago 9m 25.1M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:38:46.267 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (9m) 26s ago 9m 7684k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:38:46.267 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (8m) 22s ago 8m 7684k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:38:46.267 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (9m) 26s ago 9m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:38:46.267 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (8m) 22s ago 8m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:38:46.267 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (7m) 26s ago 8m 86.3M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (3m) 26s ago 3m 13.5M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (3m) 26s ago 3m 114M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (3m) 22s ago 3m 176M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (3m) 22s ago 3m 15.0M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:9283,8765,8443 running (10m) 26s ago 10m 398M - 18.2.2-1297-g1758e8ea 196422f1b641 a66e4acc828f 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (71s) 22s ago 8m 488M - 19.0.0-2515-g96805dcf d7acac7d5657 b3fcf25fc9f0 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (10m) 26s ago 10m 48.9M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 7d88727e4bbb 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (8m) 22s ago 8m 40.6M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (9m) 26s ago 9m 16.1M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (8m) 22s ago 8m 15.9M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (7m) 26s ago 7m 191M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (6m) 26s ago 6m 177M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (6m) 26s ago 6m 134M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (5m) 22s ago 5m 200M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:38:46.268 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (5m) 22s ago 5m 168M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:38:46.269 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (5m) 22s ago 5m 177M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:38:46.269 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (7m) 26s ago 8m 62.3M - 2.43.0 a07b618ecd1d 9f4deb8cc8b4 2024-05-09T06:38:46.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fe17006d730 msgr2=0x7fe17006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:46.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fe17006d730 0x7fe17006fbf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00a990 tx=0x7fe16c005c80 comp rx=0 tx=0).stop 2024-05-09T06:38:46.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 msgr2=0x7fe1841a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:46.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 0x7fe1841a5940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe1740098e0 tx=0x7fe17400be20 comp rx=0 tx=0).stop 2024-05-09T06:38:46.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 shutdown_connections 2024-05-09T06:38:46.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fe17006d730 0x7fe17006fbf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe184072c10 0x7fe1841a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 --2- 172.21.15.63:0/685351190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe18410a440 0x7fe1841a5940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 >> 172.21.15.63:0/685351190 conn(0x7fe18406db90 msgr2=0x7fe18410b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:46.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 shutdown_connections 2024-05-09T06:38:46.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.288+0000 7fe189137700 1 -- 172.21.15.63:0/685351190 wait complete. 2024-05-09T06:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.406+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2513192873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 msgr2=0x7f1e301064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.406+0000 7f1e37bbe700 1 --2- 172.21.15.63:0/2513192873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 0x7f1e301064f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009910 tx=0x7f1e20009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.406+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2513192873 shutdown_connections 2024-05-09T06:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.406+0000 7f1e37bbe700 1 --2- 172.21.15.63:0/2513192873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 0x7f1e301064f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.406+0000 7f1e37bbe700 1 --2- 172.21.15.63:0/2513192873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e301017d0 0x7f1e30103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.406+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2513192873 >> 172.21.15.63:0/2513192873 conn(0x7f1e300fb120 msgr2=0x7f1e300fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.406+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2513192873 shutdown_connections 2024-05-09T06:38:46.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.406+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2513192873 wait complete. 2024-05-09T06:38:46.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e37bbe700 1 Processor -- start 2024-05-09T06:38:46.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e37bbe700 1 -- start start 2024-05-09T06:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e37bbe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e301017d0 0x7f1e30196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e37bbe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 0x7f1e30196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e37bbe700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e301971d0 con 0x7f1e30104100 2024-05-09T06:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e37bbe700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30197310 con 0x7f1e301017d0 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e35159700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 0x7f1e30196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e35159700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 0x7f1e30196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45912/0 (socket says 172.21.15.63:45912) 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e35159700 1 -- 172.21.15.63:0/2116072411 learned_addr learned my addr 172.21.15.63:0/2116072411 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.407+0000 7f1e3595a700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e301017d0 0x7f1e30196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.408+0000 7f1e35159700 1 -- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e301017d0 msgr2=0x7f1e30196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.408+0000 7f1e35159700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e301017d0 0x7f1e30196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.408+0000 7f1e35159700 1 -- 172.21.15.63:0/2116072411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200095f0 con 0x7f1e30104100 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.408+0000 7f1e35159700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 0x7f1e30196bb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000b800 tx=0x7f1e20005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.408+0000 7f1e26ffd700 1 -- 172.21.15.63:0/2116072411 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e2001d070 con 0x7f1e30104100 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.408+0000 7f1e26ffd700 1 -- 172.21.15.63:0/2116072411 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e200052a0 con 0x7f1e30104100 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.408+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3019bd10 con 0x7f1e30104100 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.408+0000 7f1e26ffd700 1 -- 172.21.15.63:0/2116072411 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e20017400 con 0x7f1e30104100 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.408+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3019c200 con 0x7f1e30104100 2024-05-09T06:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.409+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3004ebb0 con 0x7f1e30104100 2024-05-09T06:38:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.410+0000 7f1e26ffd700 1 -- 172.21.15.63:0/2116072411 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f1e20003780 con 0x7f1e30104100 2024-05-09T06:38:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.410+0000 7f1e26ffd700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f1e1c06d6f0 0x7f1e1c06fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.410+0000 7f1e26ffd700 1 -- 172.21.15.63:0/2116072411 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1e2008e5e0 con 0x7f1e30104100 2024-05-09T06:38:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.411+0000 7f1e3595a700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f1e1c06d6f0 0x7f1e1c06fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.411+0000 7f1e3595a700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f1e1c06d6f0 0x7f1e1c06fbb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009c60 tx=0x7f1e2c009260 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:46.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.415+0000 7f1e26ffd700 1 -- 172.21.15.63:0/2116072411 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f1e2005b850 con 0x7f1e30104100 2024-05-09T06:38:46.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:46 smithi063 ceph-mon[27171]: pgmap v26: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 43 MiB/s rd, 113 MiB/s wr, 300 op/s 2024-05-09T06:38:46.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:46 smithi063 ceph-mon[27171]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:46.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:46 smithi063 ceph-mon[27171]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:46.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:46 smithi193 ceph-mon[35141]: pgmap v26: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 43 MiB/s rd, 113 MiB/s wr, 300 op/s 2024-05-09T06:38:46.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:46 smithi193 ceph-mon[35141]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:46.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:46 smithi193 ceph-mon[35141]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:46.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.729+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1e301099c0 con 0x7f1e30104100 2024-05-09T06:38:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.730+0000 7f1e26ffd700 1 -- 172.21.15.63:0/2116072411 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+841 (secure 0 0 0) 0x7f1e2005b3e0 con 0x7f1e30104100 2024-05-09T06:38:46.711 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:38:46.711 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:38:46.711 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:38:46.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 1, 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 1 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 13, 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 1 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:38:46.712 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:38:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f1e1c06d6f0 msgr2=0x7f1e1c06fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f1e1c06d6f0 0x7f1e1c06fbb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009c60 tx=0x7f1e2c009260 comp rx=0 tx=0).stop 2024-05-09T06:38:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 msgr2=0x7f1e30196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 0x7f1e30196bb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000b800 tx=0x7f1e20005f00 comp rx=0 tx=0).stop 2024-05-09T06:38:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 shutdown_connections 2024-05-09T06:38:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f1e1c06d6f0 0x7f1e1c06fbb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e301017d0 0x7f1e30196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 --2- 172.21.15.63:0/2116072411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e30104100 0x7f1e30196bb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 >> 172.21.15.63:0/2116072411 conn(0x7f1e300fb120 msgr2=0x7f1e300fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 shutdown_connections 2024-05-09T06:38:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.732+0000 7f1e37bbe700 1 -- 172.21.15.63:0/2116072411 wait complete. 2024-05-09T06:38:46.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.847+0000 7f50e52a4700 1 -- 172.21.15.63:0/1731052887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e01016b0 msgr2=0x7f50e0103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:46.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.847+0000 7f50e52a4700 1 --2- 172.21.15.63:0/1731052887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e01016b0 0x7f50e0103aa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009910 tx=0x7f50d0009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:46.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.847+0000 7f50e52a4700 1 -- 172.21.15.63:0/1731052887 shutdown_connections 2024-05-09T06:38:46.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.847+0000 7f50e52a4700 1 --2- 172.21.15.63:0/1731052887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e0103fe0 0x7f50e01063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.847+0000 7f50e52a4700 1 --2- 172.21.15.63:0/1731052887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e01016b0 0x7f50e0103aa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.847+0000 7f50e52a4700 1 -- 172.21.15.63:0/1731052887 >> 172.21.15.63:0/1731052887 conn(0x7f50e00fb0c0 msgr2=0x7f50e00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:46.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.847+0000 7f50e52a4700 1 -- 172.21.15.63:0/1731052887 shutdown_connections 2024-05-09T06:38:46.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.847+0000 7f50e52a4700 1 -- 172.21.15.63:0/1731052887 wait complete. 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.848+0000 7f50e52a4700 1 Processor -- start 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.848+0000 7f50e52a4700 1 -- start start 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.848+0000 7f50e52a4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e01016b0 0x7f50e0100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.848+0000 7f50e52a4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e0103fe0 0x7f50e00ff660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.848+0000 7f50e52a4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00ffba0 con 0x7f50e0103fe0 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.848+0000 7f50e52a4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00ffce0 con 0x7f50e01016b0 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50de7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e0103fe0 0x7f50e00ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50de7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e0103fe0 0x7f50e00ff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45926/0 (socket says 172.21.15.63:45926) 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50de7fc700 1 -- 172.21.15.63:0/2472744701 learned_addr learned my addr 172.21.15.63:0/2472744701 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:46.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50deffd700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e01016b0 0x7f50e0100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50de7fc700 1 -- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e01016b0 msgr2=0x7f50e0100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50de7fc700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e01016b0 0x7f50e0100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50de7fc700 1 -- 172.21.15.63:0/2472744701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00095f0 con 0x7f50e0103fe0 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50deffd700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e01016b0 0x7f50e0100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50de7fc700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e0103fe0 0x7f50e00ff660 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f50d400d8d0 tx=0x7f50d400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.849+0000 7f50c7fff700 1 -- 172.21.15.63:0/2472744701 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50d40098a0 con 0x7f50e0103fe0 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.850+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e00fffc0 con 0x7f50e0103fe0 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.850+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e019e6a0 con 0x7f50e0103fe0 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.850+0000 7f50c7fff700 1 -- 172.21.15.63:0/2472744701 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50d4010460 con 0x7f50e0103fe0 2024-05-09T06:38:46.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.850+0000 7f50c7fff700 1 -- 172.21.15.63:0/2472744701 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50d400f5f0 con 0x7f50e0103fe0 2024-05-09T06:38:46.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.851+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50cc005320 con 0x7f50e0103fe0 2024-05-09T06:38:46.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.851+0000 7f50c7fff700 1 -- 172.21.15.63:0/2472744701 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f50d400f750 con 0x7f50e0103fe0 2024-05-09T06:38:46.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.852+0000 7f50c7fff700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f50c806d6f0 0x7f50c806fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:46.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.852+0000 7f50c7fff700 1 -- 172.21.15.63:0/2472744701 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f50d408cf30 con 0x7f50e0103fe0 2024-05-09T06:38:46.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.852+0000 7f50deffd700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f50c806d6f0 0x7f50c806fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:46.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.853+0000 7f50deffd700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f50c806d6f0 0x7f50c806fbb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005090 tx=0x7f50d0005960 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:46.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:46.857+0000 7f50c7fff700 1 -- 172.21.15.63:0/2472744701 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f50d40c1720 con 0x7f50e0103fe0 2024-05-09T06:38:47.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.226+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f50cc0058a0 con 0x7f50e0103fe0 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.226+0000 7f50c7fff700 1 -- 172.21.15.63:0/2472744701 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f50d4016770 con 0x7f50e0103fe0 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:38:47.208 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:38:47.209 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:38:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:38:47.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.228+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f50c806d6f0 msgr2=0x7f50c806fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.228+0000 7f50e52a4700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f50c806d6f0 0x7f50c806fbb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005090 tx=0x7f50d0005960 comp rx=0 tx=0).stop 2024-05-09T06:38:47.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.228+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e0103fe0 msgr2=0x7f50e00ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.228+0000 7f50e52a4700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e0103fe0 0x7f50e00ff660 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f50d400d8d0 tx=0x7f50d400dbe0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.229+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 shutdown_connections 2024-05-09T06:38:47.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.229+0000 7f50e52a4700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f50c806d6f0 0x7f50c806fbb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.229+0000 7f50e52a4700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e01016b0 0x7f50e0100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.229+0000 7f50e52a4700 1 --2- 172.21.15.63:0/2472744701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50e0103fe0 0x7f50e00ff660 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.229+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 >> 172.21.15.63:0/2472744701 conn(0x7f50e00fb0c0 msgr2=0x7f50e00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.229+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 shutdown_connections 2024-05-09T06:38:47.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.229+0000 7f50e52a4700 1 -- 172.21.15.63:0/2472744701 wait complete. 2024-05-09T06:38:47.212 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 -- 172.21.15.63:0/2393214307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 msgr2=0x7fadc40ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 --2- 172.21.15.63:0/2393214307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 0x7fadc40ff0e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009910 tx=0x7fadb4009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 -- 172.21.15.63:0/2393214307 shutdown_connections 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 --2- 172.21.15.63:0/2393214307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 0x7fadc40ff0e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 --2- 172.21.15.63:0/2393214307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadc4102200 0x7fadc4102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 -- 172.21.15.63:0/2393214307 >> 172.21.15.63:0/2393214307 conn(0x7fadc40fa840 msgr2=0x7fadc40fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 -- 172.21.15.63:0/2393214307 shutdown_connections 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 -- 172.21.15.63:0/2393214307 wait complete. 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 Processor -- start 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.349+0000 7fadcc66e700 1 -- start start 2024-05-09T06:38:47.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadcc66e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 0x7fadc4196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadcc66e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadc4102200 0x7fadc4196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadcc66e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41971a0 con 0x7fadc40fec80 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadcc66e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41972e0 con 0x7fadc4102200 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadca40a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 0x7fadc4196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadca40a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 0x7fadc4196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45944/0 (socket says 172.21.15.63:45944) 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadca40a700 1 -- 172.21.15.63:0/2648407639 learned_addr learned my addr 172.21.15.63:0/2648407639 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadc9c09700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadc4102200 0x7fadc4196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadca40a700 1 -- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadc4102200 msgr2=0x7fadc4196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadca40a700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadc4102200 0x7fadc4196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadca40a700 1 -- 172.21.15.63:0/2648407639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb40095f0 con 0x7fadc40fec80 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadca40a700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 0x7fadc4196640 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fadc000e8f0 tx=0x7fadc000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.350+0000 7fadbb7fe700 1 -- 172.21.15.63:0/2648407639 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc0009790 con 0x7fadc40fec80 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.351+0000 7fadbb7fe700 1 -- 172.21.15.63:0/2648407639 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadc000c810 con 0x7fadc40fec80 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.351+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc419bd40 con 0x7fadc40fec80 2024-05-09T06:38:47.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.351+0000 7fadbb7fe700 1 -- 172.21.15.63:0/2648407639 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc0003d20 con 0x7fadc40fec80 2024-05-09T06:38:47.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.351+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc419c2b0 con 0x7fadc40fec80 2024-05-09T06:38:47.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.352+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc404ec20 con 0x7fadc40fec80 2024-05-09T06:38:47.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.352+0000 7fadbb7fe700 1 -- 172.21.15.63:0/2648407639 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7fadc00041f0 con 0x7fadc40fec80 2024-05-09T06:38:47.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.353+0000 7fadbb7fe700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fadb006d6f0 0x7fadb006fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.353+0000 7fadbb7fe700 1 -- 172.21.15.63:0/2648407639 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fadc0014070 con 0x7fadc40fec80 2024-05-09T06:38:47.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.353+0000 7fadc9c09700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fadb006d6f0 0x7fadb006fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.354+0000 7fadc9c09700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fadb006d6f0 0x7fadb006fbb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b7a0 tx=0x7fadb4009f70 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.358+0000 7fadbb7fe700 1 -- 172.21.15.63:0/2648407639 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7fadc404ec20 con 0x7fadc40fec80 2024-05-09T06:38:47.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.609+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 --> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadc404ec20 con 0x7fadb006d6f0 2024-05-09T06:38:47.591 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:47 smithi063 ceph-mon[27171]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:47.591 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:47 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2116072411' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:38:47.591 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:47 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2472744701' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:38:47.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.610+0000 7fadbb7fe700 1 -- 172.21.15.63:0/2648407639 <== mgr.24415 v2:172.21.15.193:6828/1432929717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fadc404ec20 con 0x7fadb006d6f0 2024-05-09T06:38:47.592 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:38:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:38:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:38:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:38:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-05-09T06:38:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "1/23 daemons upgraded", 2024-05-09T06:38:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-05-09T06:38:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:38:47.592 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fadb006d6f0 msgr2=0x7fadb006fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fadb006d6f0 0x7fadb006fbb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b7a0 tx=0x7fadb4009f70 comp rx=0 tx=0).stop 2024-05-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 msgr2=0x7fadc4196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 0x7fadc4196640 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fadc000e8f0 tx=0x7fadc000ec00 comp rx=0 tx=0).stop 2024-05-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 shutdown_connections 2024-05-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7fadb006d6f0 0x7fadb006fbb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc40fec80 0x7fadc4196640 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 --2- 172.21.15.63:0/2648407639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadc4102200 0x7fadc4196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 >> 172.21.15.63:0/2648407639 conn(0x7fadc40fa840 msgr2=0x7fadc41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 shutdown_connections 2024-05-09T06:38:47.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.613+0000 7fadcc66e700 1 -- 172.21.15.63:0/2648407639 wait complete. 2024-05-09T06:38:47.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:47 smithi193 ceph-mon[35141]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:47.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:47 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2116072411' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:38:47.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:47 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2472744701' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:38:47.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.732+0000 7f77970a3700 1 -- 172.21.15.63:0/168735004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100040 msgr2=0x7f7790100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.732+0000 7f77970a3700 1 --2- 172.21.15.63:0/168735004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100040 0x7f7790100460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7784009910 tx=0x7f7784009c20 comp rx=0 tx=0).stop 2024-05-09T06:38:47.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.733+0000 7f77970a3700 1 -- 172.21.15.63:0/168735004 shutdown_connections 2024-05-09T06:38:47.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.733+0000 7f77970a3700 1 --2- 172.21.15.63:0/168735004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7790100d60 0x7f77901011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.733+0000 7f77970a3700 1 --2- 172.21.15.63:0/168735004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100040 0x7f7790100460 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.733+0000 7f77970a3700 1 -- 172.21.15.63:0/168735004 >> 172.21.15.63:0/168735004 conn(0x7f77900fa840 msgr2=0x7f77900fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:47.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.733+0000 7f77970a3700 1 -- 172.21.15.63:0/168735004 shutdown_connections 2024-05-09T06:38:47.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.733+0000 7f77970a3700 1 -- 172.21.15.63:0/168735004 wait complete. 2024-05-09T06:38:47.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.733+0000 7f77970a3700 1 Processor -- start 2024-05-09T06:38:47.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.733+0000 7f77970a3700 1 -- start start 2024-05-09T06:38:47.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f77970a3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7790100040 0x7f7790196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f77970a3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100d60 0x7f7790196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f77970a3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77901971c0 con 0x7f7790100d60 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f77970a3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7790197300 con 0x7f7790100040 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f778ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100d60 0x7f7790196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f778ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100d60 0x7f7790196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45962/0 (socket says 172.21.15.63:45962) 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f778ffff700 1 -- 172.21.15.63:0/4066367171 learned_addr learned my addr 172.21.15.63:0/4066367171 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f7794e3f700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7790100040 0x7f7790196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f778ffff700 1 -- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7790100040 msgr2=0x7f7790196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f778ffff700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7790100040 0x7f7790196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f778ffff700 1 -- 172.21.15.63:0/4066367171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77840095f0 con 0x7f7790100d60 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f778ffff700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100d60 0x7f7790196ba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f777c00e8f0 tx=0x7f777c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f778dffb700 1 -- 172.21.15.63:0/4066367171 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f777c009790 con 0x7f7790100d60 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.734+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779019bd60 con 0x7f7790100d60 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.735+0000 7f778dffb700 1 -- 172.21.15.63:0/4066367171 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f777c00c810 con 0x7f7790100d60 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.735+0000 7f778dffb700 1 -- 172.21.15.63:0/4066367171 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f777c003d20 con 0x7f7790100d60 2024-05-09T06:38:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.735+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779019c330 con 0x7f7790100d60 2024-05-09T06:38:47.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.736+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77901907d0 con 0x7f7790100d60 2024-05-09T06:38:47.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.736+0000 7f778dffb700 1 -- 172.21.15.63:0/4066367171 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 91561+0+0 (secure 0 0 0) 0x7f777c0041f0 con 0x7f7790100d60 2024-05-09T06:38:47.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.737+0000 7f778dffb700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f778006d6e0 0x7f778006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:38:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.737+0000 7f778dffb700 1 -- 172.21.15.63:0/4066367171 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f777c014070 con 0x7f7790100d60 2024-05-09T06:38:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.737+0000 7f7794e3f700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f778006d6e0 0x7f778006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:38:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.738+0000 7f7794e3f700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f778006d6e0 0x7f778006fba0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f778400b800 tx=0x7f7784009f70 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:38:47.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:47.743+0000 7f778dffb700 1 -- 172.21.15.63:0/4066367171 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f777c05a6f0 con 0x7f7790100d60 2024-05-09T06:38:48.099 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:38:48.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.117+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f779004ebb0 con 0x7f7790100d60 2024-05-09T06:38:48.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.118+0000 7f778dffb700 1 -- 172.21.15.63:0/4066367171 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f777c05fe60 con 0x7f7790100d60 2024-05-09T06:38:48.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.122+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f778006d6e0 msgr2=0x7f778006fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.122+0000 7f77970a3700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f778006d6e0 0x7f778006fba0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f778400b800 tx=0x7f7784009f70 comp rx=0 tx=0).stop 2024-05-09T06:38:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.122+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100d60 msgr2=0x7f7790196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:38:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.122+0000 7f77970a3700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100d60 0x7f7790196ba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f777c00e8f0 tx=0x7f777c00ec00 comp rx=0 tx=0).stop 2024-05-09T06:38:48.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.126+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 shutdown_connections 2024-05-09T06:38:48.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.126+0000 7f77970a3700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:6828/1432929717,v1:172.21.15.193:6829/1432929717] conn(0x7f778006d6e0 0x7f778006fba0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:48.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.126+0000 7f77970a3700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7790100040 0x7f7790196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:48.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.126+0000 7f77970a3700 1 --2- 172.21.15.63:0/4066367171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7790100d60 0x7f7790196ba0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:38:48.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.126+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 >> 172.21.15.63:0/4066367171 conn(0x7f77900fa840 msgr2=0x7f7790104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:38:48.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.126+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 shutdown_connections 2024-05-09T06:38:48.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:38:48.127+0000 7f77970a3700 1 -- 172.21.15.63:0/4066367171 wait complete. 2024-05-09T06:38:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:48 smithi063 ceph-mon[27171]: pgmap v27: 65 pgs: 65 active+clean; 3.1 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 52 MiB/s rd, 133 MiB/s wr, 394 op/s 2024-05-09T06:38:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:48 smithi063 ceph-mon[27171]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:48.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:48 smithi193 ceph-mon[35141]: pgmap v27: 65 pgs: 65 active+clean; 3.1 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 52 MiB/s rd, 133 MiB/s wr, 394 op/s 2024-05-09T06:38:48.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:48 smithi193 ceph-mon[35141]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:38:48.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:48 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/4066367171' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:38:48.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:48 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/4066367171' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:38:50.154 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T06:38:50.154 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T06:38:50.555 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:50 smithi063 ceph-mon[27171]: pgmap v28: 65 pgs: 65 active+clean; 3.1 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 31 MiB/s rd, 76 MiB/s wr, 243 op/s 2024-05-09T06:38:50.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:50 smithi193 ceph-mon[35141]: pgmap v28: 65 pgs: 65 active+clean; 3.1 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 31 MiB/s rd, 76 MiB/s wr, 243 op/s 2024-05-09T06:38:53.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:53 smithi063 ceph-mon[27171]: pgmap v29: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 32 MiB/s rd, 77 MiB/s wr, 325 op/s 2024-05-09T06:38:53.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:53 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:53.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:53 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:53.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:53 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:38:53.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:53 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:38:53.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:53 smithi193 ceph-mon[35141]: pgmap v29: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 32 MiB/s rd, 77 MiB/s wr, 325 op/s 2024-05-09T06:38:53.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:53 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:53.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:53 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:53.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:53 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:38:53.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:53 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:38:54.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:54 smithi063 ceph-mon[27171]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:38:54.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:54 smithi063 ceph-mon[27171]: Upgrade: Need to upgrade myself (mgr.smithi193.mmmqzh) 2024-05-09T06:38:54.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:54 smithi063 ceph-mon[27171]: Upgrade: Need to upgrade myself (mgr.smithi193.mmmqzh) 2024-05-09T06:38:54.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:54 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:54.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:54 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.egfguy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:38:54.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:54 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:38:54.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:54 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:38:54.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:54 smithi193 ceph-mon[35141]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:38:54.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:54 smithi193 ceph-mon[35141]: Upgrade: Need to upgrade myself (mgr.smithi193.mmmqzh) 2024-05-09T06:38:54.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:54 smithi193 ceph-mon[35141]: Upgrade: Need to upgrade myself (mgr.smithi193.mmmqzh) 2024-05-09T06:38:54.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:54 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:38:54.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:54 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.egfguy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:38:54.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:54 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:38:54.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:54 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:38:55.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:55 smithi063 ceph-mon[27171]: pgmap v30: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 175 op/s 2024-05-09T06:38:55.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:55 smithi063 ceph-mon[27171]: Upgrade: Updating mgr.smithi063.egfguy 2024-05-09T06:38:55.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:55 smithi063 ceph-mon[27171]: Deploying daemon mgr.smithi063.egfguy on smithi063 2024-05-09T06:38:55.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:55 smithi193 ceph-mon[35141]: pgmap v30: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 175 op/s 2024-05-09T06:38:55.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:55 smithi193 ceph-mon[35141]: Upgrade: Updating mgr.smithi063.egfguy 2024-05-09T06:38:55.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:55 smithi193 ceph-mon[35141]: Deploying daemon mgr.smithi063.egfguy on smithi063 2024-05-09T06:38:57.563 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:57 smithi063 ceph-mon[27171]: pgmap v31: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 175 op/s 2024-05-09T06:38:57.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:57 smithi193 ceph-mon[35141]: pgmap v31: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 175 op/s 2024-05-09T06:38:59.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:38:59 smithi063 ceph-mon[27171]: pgmap v32: 65 pgs: 65 active+clean; 1.0 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 10 MiB/s rd, 21 MiB/s wr, 277 op/s 2024-05-09T06:38:59.433 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T06:38:59.434 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-05-09T06:38:59.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:38:59 smithi193 ceph-mon[35141]: pgmap v32: 65 pgs: 65 active+clean; 1.0 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 10 MiB/s rd, 21 MiB/s wr, 277 op/s 2024-05-09T06:39:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:00 smithi063 ceph-mon[27171]: pgmap v33: 65 pgs: 65 active+clean; 1.0 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2024-05-09T06:39:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:00.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:00 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:00 smithi193 ceph-mon[35141]: pgmap v33: 65 pgs: 65 active+clean; 1.0 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2024-05-09T06:39:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:00.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:00 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:02.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:02 smithi063 ceph-mon[27171]: pgmap v34: 65 pgs: 65 active+clean; 370 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-05-09T06:39:02.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:02 smithi193 ceph-mon[35141]: pgmap v34: 65 pgs: 65 active+clean; 370 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-05-09T06:39:04.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:04 smithi063 ceph-mon[27171]: pgmap v35: 65 pgs: 65 active+clean; 370 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-05-09T06:39:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:04 smithi193 ceph-mon[35141]: pgmap v35: 65 pgs: 65 active+clean; 370 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-05-09T06:39:07.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:07 smithi063 ceph-mon[27171]: pgmap v36: 65 pgs: 65 active+clean; 370 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-05-09T06:39:07.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:07 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:07.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:07 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:07.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:07 smithi193 ceph-mon[35141]: pgmap v36: 65 pgs: 65 active+clean; 370 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-05-09T06:39:07.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:07 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:07.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:07 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:08.498 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:08 smithi063 ceph-mon[27171]: pgmap v37: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 378 op/s 2024-05-09T06:39:08.498 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:08 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:08.498 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:08 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:08.498 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:08 smithi063 ceph-mon[27171]: Standby manager daemon smithi063.egfguy restarted 2024-05-09T06:39:08.498 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:08 smithi063 ceph-mon[27171]: Standby manager daemon smithi063.egfguy started 2024-05-09T06:39:08.498 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:08 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.egfguy/crt"}]: dispatch 2024-05-09T06:39:08.498 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:08 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:39:08.498 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:08 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.egfguy/key"}]: dispatch 2024-05-09T06:39:08.498 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:08 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:39:08.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:08 smithi193 ceph-mon[35141]: pgmap v37: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 378 op/s 2024-05-09T06:39:08.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:08 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:08.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:08 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:08.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:08 smithi193 ceph-mon[35141]: Standby manager daemon smithi063.egfguy restarted 2024-05-09T06:39:08.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:08 smithi193 ceph-mon[35141]: Standby manager daemon smithi063.egfguy started 2024-05-09T06:39:08.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:08 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.egfguy/crt"}]: dispatch 2024-05-09T06:39:08.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:08 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:39:08.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:08 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.egfguy/key"}]: dispatch 2024-05-09T06:39:08.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:08 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:39:09.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:09 smithi063 ceph-mon[27171]: mgrmap e26: smithi193.mmmqzh(active, since 69s), standbys: smithi063.egfguy 2024-05-09T06:39:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:09 smithi193 ceph-mon[35141]: mgrmap e26: smithi193.mmmqzh(active, since 69s), standbys: smithi063.egfguy 2024-05-09T06:39:10.852 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:10 smithi063 ceph-mon[27171]: pgmap v38: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 277 op/s 2024-05-09T06:39:10.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:10 smithi193 ceph-mon[35141]: pgmap v38: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 277 op/s 2024-05-09T06:39:12.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:12.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:12.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:39:12.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:39:12.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mgr fail", "who": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: Activating manager daemon smithi063.egfguy 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: osdmap e49: 6 total, 6 up, 6 in 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd='[{"prefix": "mgr fail", "who": "smithi193.mmmqzh"}]': finished 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: mgrmap e27: smithi063.egfguy(active, starting, since 0.00693588s) 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:39:12.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: Manager daemon smithi063.egfguy is now available 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:39:12.975 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:12 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/trash_purge_schedule"}]: dispatch 2024-05-09T06:39:13.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "mgr fail", "who": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: Activating manager daemon smithi063.egfguy 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: osdmap e49: 6 total, 6 up, 6 in 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24415 172.21.15.193:0/504744941' entity='mgr.smithi193.mmmqzh' cmd='[{"prefix": "mgr fail", "who": "smithi193.mmmqzh"}]': finished 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: mgrmap e27: smithi063.egfguy(active, starting, since 0.00693588s) 2024-05-09T06:39:13.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:39:13.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:39:13.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: Manager daemon smithi063.egfguy is now available 2024-05-09T06:39:13.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:13.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:13.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:39:13.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:12 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/trash_purge_schedule"}]: dispatch 2024-05-09T06:39:14.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:13 smithi063 ceph-mon[27171]: [09/May/2024:06:39:12] ENGINE Bus STARTING 2024-05-09T06:39:14.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:13 smithi063 ceph-mon[27171]: [09/May/2024:06:39:12] ENGINE Serving on http://172.21.15.63:8765 2024-05-09T06:39:14.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:13 smithi063 ceph-mon[27171]: mgrmap e28: smithi063.egfguy(active, since 1.01089s) 2024-05-09T06:39:14.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:13 smithi063 ceph-mon[27171]: pgmap v3: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:39:14.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:13 smithi063 ceph-mon[27171]: [09/May/2024:06:39:12] ENGINE Serving on https://172.21.15.63:7150 2024-05-09T06:39:14.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:13 smithi063 ceph-mon[27171]: [09/May/2024:06:39:12] ENGINE Bus STARTED 2024-05-09T06:39:14.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:13 smithi193 ceph-mon[35141]: [09/May/2024:06:39:12] ENGINE Bus STARTING 2024-05-09T06:39:14.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:13 smithi193 ceph-mon[35141]: [09/May/2024:06:39:12] ENGINE Serving on http://172.21.15.63:8765 2024-05-09T06:39:14.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:13 smithi193 ceph-mon[35141]: mgrmap e28: smithi063.egfguy(active, since 1.01089s) 2024-05-09T06:39:14.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:13 smithi193 ceph-mon[35141]: pgmap v3: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:39:14.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:13 smithi193 ceph-mon[35141]: [09/May/2024:06:39:12] ENGINE Serving on https://172.21.15.63:7150 2024-05-09T06:39:14.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:13 smithi193 ceph-mon[35141]: [09/May/2024:06:39:12] ENGINE Bus STARTED 2024-05-09T06:39:14.909 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:14 smithi193 ceph-mon[35141]: pgmap v4: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:39:14.909 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:14 smithi193 ceph-mon[35141]: mgrmap e29: smithi063.egfguy(active, since 2s) 2024-05-09T06:39:15.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:14 smithi063 ceph-mon[27171]: pgmap v4: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:39:15.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:14 smithi063 ceph-mon[27171]: mgrmap e29: smithi063.egfguy(active, since 2s) 2024-05-09T06:39:17.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:16 smithi193 ceph-mon[35141]: pgmap v5: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:39:17.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:16 smithi193 ceph-mon[35141]: mgrmap e30: smithi063.egfguy(active, since 4s) 2024-05-09T06:39:17.166 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:16 smithi063 ceph-mon[27171]: pgmap v5: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:39:17.166 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:16 smithi063 ceph-mon[27171]: mgrmap e30: smithi063.egfguy(active, since 4s) 2024-05-09T06:39:18.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.260+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/859163905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 msgr2=0x7f5e041022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.260+0000 7f5e0a6a0700 1 --2- 172.21.15.63:0/859163905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 0x7f5e041022b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009910 tx=0x7f5df8009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:18.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.260+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/859163905 shutdown_connections 2024-05-09T06:39:18.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.260+0000 7f5e0a6a0700 1 --2- 172.21.15.63:0/859163905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 0x7f5e041022b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.260+0000 7f5e0a6a0700 1 --2- 172.21.15.63:0/859163905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e04100c50 0x7f5e04101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.260+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/859163905 >> 172.21.15.63:0/859163905 conn(0x7f5e040fc210 msgr2=0x7f5e040fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.260+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/859163905 shutdown_connections 2024-05-09T06:39:18.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.260+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/859163905 wait complete. 2024-05-09T06:39:18.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.260+0000 7f5e0a6a0700 1 Processor -- start 2024-05-09T06:39:18.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.261+0000 7f5e0a6a0700 1 -- start start 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.261+0000 7f5e0a6a0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e04100c50 0x7f5e04196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.261+0000 7f5e0a6a0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 0x7f5e04196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.261+0000 7f5e0a6a0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04197170 con 0x7f5e04101e50 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.261+0000 7f5e0a6a0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041972b0 con 0x7f5e04100c50 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.261+0000 7f5e037fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 0x7f5e04196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.261+0000 7f5e037fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 0x7f5e04196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54708/0 (socket says 172.21.15.63:54708) 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.261+0000 7f5e037fe700 1 -- 172.21.15.63:0/2424551138 learned_addr learned my addr 172.21.15.63:0/2424551138 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.261+0000 7f5e03fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e04100c50 0x7f5e04196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.262+0000 7f5e037fe700 1 -- 172.21.15.63:0/2424551138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e04100c50 msgr2=0x7f5e04196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.262+0000 7f5e037fe700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e04100c50 0x7f5e04196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.262+0000 7f5e037fe700 1 -- 172.21.15.63:0/2424551138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80095f0 con 0x7f5e04101e50 2024-05-09T06:39:18.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.262+0000 7f5e037fe700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 0x7f5e04196b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5df8005b40 tx=0x7f5df800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.262+0000 7f5e017fa700 1 -- 172.21.15.63:0/2424551138 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5df801d070 con 0x7f5e04101e50 2024-05-09T06:39:18.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.262+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0419bcb0 con 0x7f5e04101e50 2024-05-09T06:39:18.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.262+0000 7f5e017fa700 1 -- 172.21.15.63:0/2424551138 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5df80050d0 con 0x7f5e04101e50 2024-05-09T06:39:18.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.262+0000 7f5e017fa700 1 -- 172.21.15.63:0/2424551138 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5df8022470 con 0x7f5e04101e50 2024-05-09T06:39:18.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.262+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0419c220 con 0x7f5e04101e50 2024-05-09T06:39:18.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.263+0000 7f5e017fa700 1 -- 172.21.15.63:0/2424551138 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 30) v1 ==== 46434+0+0 (secure 0 0 0) 0x7f5df8017400 con 0x7f5e04101e50 2024-05-09T06:39:18.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.264+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e041907d0 con 0x7f5e04101e50 2024-05-09T06:39:18.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.264+0000 7f5e017fa700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5dec039360 0x7f5dec03b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.264+0000 7f5e017fa700 1 -- 172.21.15.63:0/2424551138 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5df804ebf0 con 0x7f5e04101e50 2024-05-09T06:39:18.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.264+0000 7f5e03fff700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5dec039360 0x7f5dec03b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.264+0000 7f5e03fff700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5dec039360 0x7f5dec03b820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5e04101cb0 tx=0x7f5df4009260 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.269+0000 7f5e017fa700 1 -- 172.21.15.63:0/2424551138 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f5df8020a60 con 0x7f5e04101e50 2024-05-09T06:39:18.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.521+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 --> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e0402cef0 con 0x7f5dec039360 2024-05-09T06:39:18.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.523+0000 7f5e017fa700 1 -- 172.21.15.63:0/2424551138 <== mgr.24513 v2:172.21.15.63:6800/1655504434 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f5e0402cef0 con 0x7f5dec039360 2024-05-09T06:39:18.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5dec039360 msgr2=0x7f5dec03b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5dec039360 0x7f5dec03b820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5e04101cb0 tx=0x7f5df4009260 comp rx=0 tx=0).stop 2024-05-09T06:39:18.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 msgr2=0x7f5e04196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 0x7f5e04196b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5df8005b40 tx=0x7f5df800f690 comp rx=0 tx=0).stop 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 shutdown_connections 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5dec039360 0x7f5dec03b820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e04100c50 0x7f5e04196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 --2- 172.21.15.63:0/2424551138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e04101e50 0x7f5e04196b50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 >> 172.21.15.63:0/2424551138 conn(0x7f5e040fc210 msgr2=0x7f5e04105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 shutdown_connections 2024-05-09T06:39:18.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.525+0000 7f5e0a6a0700 1 -- 172.21.15.63:0/2424551138 wait complete. 2024-05-09T06:39:18.517 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:39:18.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.649+0000 7f6e74b2d700 1 -- 172.21.15.63:0/4272973211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700ff5a0 msgr2=0x7f6e700ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.649+0000 7f6e74b2d700 1 --2- 172.21.15.63:0/4272973211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700ff5a0 0x7f6e700ff9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009910 tx=0x7f6e60009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:18.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.649+0000 7f6e74b2d700 1 -- 172.21.15.63:0/4272973211 shutdown_connections 2024-05-09T06:39:18.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.649+0000 7f6e74b2d700 1 --2- 172.21.15.63:0/4272973211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e700fff00 0x7f6e70100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.649+0000 7f6e74b2d700 1 --2- 172.21.15.63:0/4272973211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700ff5a0 0x7f6e700ff9c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.649+0000 7f6e74b2d700 1 -- 172.21.15.63:0/4272973211 >> 172.21.15.63:0/4272973211 conn(0x7f6e700fb120 msgr2=0x7f6e700fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.649+0000 7f6e74b2d700 1 -- 172.21.15.63:0/4272973211 shutdown_connections 2024-05-09T06:39:18.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.649+0000 7f6e74b2d700 1 -- 172.21.15.63:0/4272973211 wait complete. 2024-05-09T06:39:18.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e74b2d700 1 Processor -- start 2024-05-09T06:39:18.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e74b2d700 1 -- start start 2024-05-09T06:39:18.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e74b2d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e700ff5a0 0x7f6e70198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e74b2d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700fff00 0x7f6e70198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e74b2d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e701993f0 con 0x7f6e700fff00 2024-05-09T06:39:18.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e74b2d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70199530 con 0x7f6e700ff5a0 2024-05-09T06:39:18.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e6dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700fff00 0x7f6e70198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e6dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700fff00 0x7f6e70198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54724/0 (socket says 172.21.15.63:54724) 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e6dd9b700 1 -- 172.21.15.63:0/1648127555 learned_addr learned my addr 172.21.15.63:0/1648127555 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e6dd9b700 1 -- 172.21.15.63:0/1648127555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e700ff5a0 msgr2=0x7f6e70198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e6dd9b700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e700ff5a0 0x7f6e70198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.650+0000 7f6e6dd9b700 1 -- 172.21.15.63:0/1648127555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e600095f0 con 0x7f6e700fff00 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.651+0000 7f6e6dd9b700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700fff00 0x7f6e70198dd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400e920 tx=0x7f6e6400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.651+0000 7f6e5f7fe700 1 -- 172.21.15.63:0/1648127555 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e64009790 con 0x7f6e700fff00 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.651+0000 7f6e5f7fe700 1 -- 172.21.15.63:0/1648127555 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e6400cba0 con 0x7f6e700fff00 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.651+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7019df90 con 0x7f6e700fff00 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.651+0000 7f6e5f7fe700 1 -- 172.21.15.63:0/1648127555 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e64018650 con 0x7f6e700fff00 2024-05-09T06:39:18.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.651+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7019e4b0 con 0x7f6e700fff00 2024-05-09T06:39:18.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.652+0000 7f6e5f7fe700 1 -- 172.21.15.63:0/1648127555 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 30) v1 ==== 46434+0+0 (secure 0 0 0) 0x7f6e64018910 con 0x7f6e700fff00 2024-05-09T06:39:18.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.652+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e70192a20 con 0x7f6e700fff00 2024-05-09T06:39:18.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.653+0000 7f6e5f7fe700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f6e580393b0 0x7f6e5803b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:18.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.653+0000 7f6e5f7fe700 1 -- 172.21.15.63:0/1648127555 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6e64014070 con 0x7f6e700fff00 2024-05-09T06:39:18.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.653+0000 7f6e6e59c700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f6e580393b0 0x7f6e5803b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:18.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.654+0000 7f6e6e59c700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f6e580393b0 0x7f6e5803b870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009fd0 tx=0x7f6e6000b7c0 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:18.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.663+0000 7f6e5f7fe700 1 -- 172.21.15.63:0/1648127555 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f6e6401fdd0 con 0x7f6e700fff00 2024-05-09T06:39:18.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.970+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 --> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e7002ce80 con 0x7f6e580393b0 2024-05-09T06:39:18.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.971+0000 7f6e5f7fe700 1 -- 172.21.15.63:0/1648127555 <== mgr.24513 v2:172.21.15.63:6800/1655504434 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f6e7002ce80 con 0x7f6e580393b0 2024-05-09T06:39:18.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f6e580393b0 msgr2=0x7f6e5803b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f6e580393b0 0x7f6e5803b870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009fd0 tx=0x7f6e6000b7c0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700fff00 msgr2=0x7f6e70198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:18.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700fff00 0x7f6e70198dd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400e920 tx=0x7f6e6400ece0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 shutdown_connections 2024-05-09T06:39:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f6e580393b0 0x7f6e5803b870 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e700ff5a0 0x7f6e70198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 --2- 172.21.15.63:0/1648127555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e700fff00 0x7f6e70198dd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 >> 172.21.15.63:0/1648127555 conn(0x7f6e700fb120 msgr2=0x7f6e70107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 shutdown_connections 2024-05-09T06:39:18.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:18.974+0000 7f6e74b2d700 1 -- 172.21.15.63:0/1648127555 wait complete. 2024-05-09T06:39:19.032 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:18 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:19.032 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:18 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:19.032 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:18 smithi193 ceph-mon[35141]: pgmap v6: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:39:19.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.107+0000 7f405ba11700 1 -- 172.21.15.63:0/1881942225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054101700 msgr2=0x7f4054103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:19.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.107+0000 7f405ba11700 1 --2- 172.21.15.63:0/1881942225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054101700 0x7f4054103af0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4050009910 tx=0x7f4050009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:19.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.107+0000 7f405ba11700 1 -- 172.21.15.63:0/1881942225 shutdown_connections 2024-05-09T06:39:19.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.107+0000 7f405ba11700 1 --2- 172.21.15.63:0/1881942225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4054104030 0x7f4054106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.108+0000 7f405ba11700 1 --2- 172.21.15.63:0/1881942225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054101700 0x7f4054103af0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.108+0000 7f405ba11700 1 -- 172.21.15.63:0/1881942225 >> 172.21.15.63:0/1881942225 conn(0x7f40540fb0d0 msgr2=0x7f40540fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:19.090 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:18 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:19.090 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:18 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:19.090 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:18 smithi063 ceph-mon[27171]: pgmap v6: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:39:19.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.108+0000 7f405ba11700 1 -- 172.21.15.63:0/1881942225 shutdown_connections 2024-05-09T06:39:19.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.108+0000 7f405ba11700 1 -- 172.21.15.63:0/1881942225 wait complete. 2024-05-09T06:39:19.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f405ba11700 1 Processor -- start 2024-05-09T06:39:19.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f405ba11700 1 -- start start 2024-05-09T06:39:19.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f405ba11700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4054101700 0x7f4054101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f405ba11700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054104030 0x7f40540ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f405ba11700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40540ffbf0 con 0x7f4054104030 2024-05-09T06:39:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f405ba11700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40540ffd30 con 0x7f4054101700 2024-05-09T06:39:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f40597ad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4054101700 0x7f4054101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f4058fac700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054104030 0x7f40540ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f4058fac700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054104030 0x7f40540ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54734/0 (socket says 172.21.15.63:54734) 2024-05-09T06:39:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.109+0000 7f4058fac700 1 -- 172.21.15.63:0/2017524130 learned_addr learned my addr 172.21.15.63:0/2017524130 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.110+0000 7f4058fac700 1 -- 172.21.15.63:0/2017524130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4054101700 msgr2=0x7f4054101010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.110+0000 7f4058fac700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4054101700 0x7f4054101010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.110+0000 7f4058fac700 1 -- 172.21.15.63:0/2017524130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40500095f0 con 0x7f4054104030 2024-05-09T06:39:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.110+0000 7f4058fac700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054104030 0x7f40540ff6b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f404400e920 tx=0x7f404400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.110+0000 7f404a7fc700 1 -- 172.21.15.63:0/2017524130 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f404400c9d0 con 0x7f4054104030 2024-05-09T06:39:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.110+0000 7f404a7fc700 1 -- 172.21.15.63:0/2017524130 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4044010430 con 0x7f4054104030 2024-05-09T06:39:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.110+0000 7f404a7fc700 1 -- 172.21.15.63:0/2017524130 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4044018630 con 0x7f4054104030 2024-05-09T06:39:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.111+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4054100010 con 0x7f4054104030 2024-05-09T06:39:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.111+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405419e690 con 0x7f4054104030 2024-05-09T06:39:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.112+0000 7f404a7fc700 1 -- 172.21.15.63:0/2017524130 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 30) v1 ==== 46434+0+0 (secure 0 0 0) 0x7f40440105a0 con 0x7f4054104030 2024-05-09T06:39:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.112+0000 7f404a7fc700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f4040039360 0x7f404003b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.112+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405404ebb0 con 0x7f4054104030 2024-05-09T06:39:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.113+0000 7f404a7fc700 1 -- 172.21.15.63:0/2017524130 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4044014070 con 0x7f4054104030 2024-05-09T06:39:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.113+0000 7f40597ad700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f4040039360 0x7f404003b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.113+0000 7f40597ad700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f4040039360 0x7f404003b820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f405000b800 tx=0x7f4050009f70 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:19.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.121+0000 7f404a7fc700 1 -- 172.21.15.63:0/2017524130 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f4044018790 con 0x7f4054104030 2024-05-09T06:39:19.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.360+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 --> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4054104a10 con 0x7f4040039360 2024-05-09T06:39:19.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.373+0000 7f404a7fc700 1 -- 172.21.15.63:0/2017524130 <== mgr.24513 v2:172.21.15.63:6800/1655504434 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f4054104a10 con 0x7f4040039360 2024-05-09T06:39:19.354 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:39:19.354 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (8m) 1s ago 9m 25.1M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:39:19.354 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (10m) 1s ago 9m 7768k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:39:19.354 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (8m) 55s ago 8m 7684k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:39:19.354 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (9m) 1s ago 9m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:39:19.354 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (8m) 55s ago 8m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:39:19.354 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (8m) 1s ago 9m 87.8M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:39:19.354 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (4m) 1s ago 4m 14.1M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (4m) 1s ago 4m 140M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (4m) 55s ago 4m 176M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (3m) 55s ago 3m 15.0M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (20s) 1s ago 11m 479M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (104s) 55s ago 8m 488M - 19.0.0-2515-g96805dcf d7acac7d5657 b3fcf25fc9f0 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (11m) 1s ago 11m 53.6M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 7d88727e4bbb 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (8m) 55s ago 8m 40.6M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (9m) 1s ago 9m 15.9M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (8m) 55s ago 8m 15.9M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (7m) 1s ago 7m 198M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (7m) 1s ago 7m 212M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (6m) 1s ago 6m 182M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (6m) 55s ago 6m 200M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (6m) 55s ago 6m 168M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:39:19.355 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (5m) 55s ago 5m 177M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:39:19.356 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (29s) 1s ago 9m 47.0M - 2.43.0 a07b618ecd1d 0ea57198b7a1 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.376+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f4040039360 msgr2=0x7f404003b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.376+0000 7f405ba11700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f4040039360 0x7f404003b820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f405000b800 tx=0x7f4050009f70 comp rx=0 tx=0).stop 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.376+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054104030 msgr2=0x7f40540ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.376+0000 7f405ba11700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054104030 0x7f40540ff6b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f404400e920 tx=0x7f404400ec30 comp rx=0 tx=0).stop 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.377+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 shutdown_connections 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.377+0000 7f405ba11700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f4040039360 0x7f404003b820 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.377+0000 7f405ba11700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4054101700 0x7f4054101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.377+0000 7f405ba11700 1 --2- 172.21.15.63:0/2017524130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4054104030 0x7f40540ff6b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.377+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 >> 172.21.15.63:0/2017524130 conn(0x7f40540fb0d0 msgr2=0x7f40540fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.377+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 shutdown_connections 2024-05-09T06:39:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.377+0000 7f405ba11700 1 -- 172.21.15.63:0/2017524130 wait complete. 2024-05-09T06:39:19.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.502+0000 7ff3735a2700 1 -- 172.21.15.63:0/2414514969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 msgr2=0x7ff36c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:19.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.502+0000 7ff3735a2700 1 --2- 172.21.15.63:0/2414514969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 0x7ff36c0fffc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff368009910 tx=0x7ff368009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:19.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.502+0000 7ff3735a2700 1 -- 172.21.15.63:0/2414514969 shutdown_connections 2024-05-09T06:39:19.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.502+0000 7ff3735a2700 1 --2- 172.21.15.63:0/2414514969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff36c100da0 0x7ff36c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.502+0000 7ff3735a2700 1 --2- 172.21.15.63:0/2414514969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 0x7ff36c0fffc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.502+0000 7ff3735a2700 1 -- 172.21.15.63:0/2414514969 >> 172.21.15.63:0/2414514969 conn(0x7ff36c0fb120 msgr2=0x7ff36c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:19.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.503+0000 7ff3735a2700 1 -- 172.21.15.63:0/2414514969 shutdown_connections 2024-05-09T06:39:19.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.503+0000 7ff3735a2700 1 -- 172.21.15.63:0/2414514969 wait complete. 2024-05-09T06:39:19.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.503+0000 7ff3735a2700 1 Processor -- start 2024-05-09T06:39:19.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.503+0000 7ff3735a2700 1 -- start start 2024-05-09T06:39:19.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.503+0000 7ff3735a2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 0x7ff36c107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:19.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.503+0000 7ff3735a2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff36c100da0 0x7ff36c1061a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:19.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.503+0000 7ff3735a2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c19a340 con 0x7ff36c0ffba0 2024-05-09T06:39:19.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.503+0000 7ff3735a2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c19a480 con 0x7ff36c100da0 2024-05-09T06:39:19.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff37133e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 0x7ff36c107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:19.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff37133e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 0x7ff36c107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54744/0 (socket says 172.21.15.63:54744) 2024-05-09T06:39:19.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff37133e700 1 -- 172.21.15.63:0/1636158844 learned_addr learned my addr 172.21.15.63:0/1636158844 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff370b3d700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff36c100da0 0x7ff36c1061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff37133e700 1 -- 172.21.15.63:0/1636158844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff36c100da0 msgr2=0x7ff36c1061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff37133e700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff36c100da0 0x7ff36c1061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff37133e700 1 -- 172.21.15.63:0/1636158844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3680095f0 con 0x7ff36c0ffba0 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff37133e700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 0x7ff36c107ab0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff36800b800 tx=0x7ff36800b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff3627fc700 1 -- 172.21.15.63:0/1636158844 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff36801d070 con 0x7ff36c0ffba0 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36c1066e0 con 0x7ff36c0ffba0 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.504+0000 7ff3627fc700 1 -- 172.21.15.63:0/1636158844 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3680052a0 con 0x7ff36c0ffba0 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.505+0000 7ff3627fc700 1 -- 172.21.15.63:0/1636158844 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff368017400 con 0x7ff36c0ffba0 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.505+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c106ba0 con 0x7ff36c0ffba0 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.505+0000 7ff3627fc700 1 -- 172.21.15.63:0/1636158844 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 30) v1 ==== 46434+0+0 (secure 0 0 0) 0x7ff3680176c0 con 0x7ff36c0ffba0 2024-05-09T06:39:19.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.506+0000 7ff3627fc700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7ff358039360 0x7ff35803b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:19.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.506+0000 7ff3627fc700 1 -- 172.21.15.63:0/1636158844 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff368003680 con 0x7ff36c0ffba0 2024-05-09T06:39:19.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.506+0000 7ff370b3d700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7ff358039360 0x7ff35803b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:19.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.507+0000 7ff370b3d700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7ff358039360 0x7ff35803b820 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009c60 tx=0x7ff35c009260 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:19.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.507+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36c190820 con 0x7ff36c0ffba0 2024-05-09T06:39:19.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.512+0000 7ff3627fc700 1 -- 172.21.15.63:0/1636158844 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7ff36800d340 con 0x7ff36c0ffba0 2024-05-09T06:39:19.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.918+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff36c04ec10 con 0x7ff36c0ffba0 2024-05-09T06:39:19.899 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:19 smithi063 ceph-mon[27171]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:19.899 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:19 smithi063 ceph-mon[27171]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:19.899 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:19 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:19.899 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:19 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:19.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.919+0000 7ff3627fc700 1 -- 172.21.15.63:0/1636158844 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7ff368021e40 con 0x7ff36c0ffba0 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 1 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:39:19.900 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:39:19.901 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 12, 2024-05-09T06:39:19.901 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 1 2024-05-09T06:39:19.901 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:39:19.901 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:39:19.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7ff358039360 msgr2=0x7ff35803b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7ff358039360 0x7ff35803b820 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009c60 tx=0x7ff35c009260 comp rx=0 tx=0).stop 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 msgr2=0x7ff36c107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 0x7ff36c107ab0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff36800b800 tx=0x7ff36800b600 comp rx=0 tx=0).stop 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 shutdown_connections 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7ff358039360 0x7ff35803b820 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff36c0ffba0 0x7ff36c107ab0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 --2- 172.21.15.63:0/1636158844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff36c100da0 0x7ff36c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 >> 172.21.15.63:0/1636158844 conn(0x7ff36c0fb120 msgr2=0x7ff36c1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 shutdown_connections 2024-05-09T06:39:19.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:19.921+0000 7ff3735a2700 1 -- 172.21.15.63:0/1636158844 wait complete. 2024-05-09T06:39:20.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.039+0000 7f0bc53db700 1 -- 172.21.15.63:0/107229759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 msgr2=0x7f0bc0103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.039+0000 7f0bc53db700 1 --2- 172.21.15.63:0/107229759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 0x7f0bc0103b20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009910 tx=0x7f0bb0009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:20.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.039+0000 7f0bc53db700 1 -- 172.21.15.63:0/107229759 shutdown_connections 2024-05-09T06:39:20.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.039+0000 7f0bc53db700 1 --2- 172.21.15.63:0/107229759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc0104060 0x7f0bc0106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.039+0000 7f0bc53db700 1 --2- 172.21.15.63:0/107229759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 0x7f0bc0103b20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.039+0000 7f0bc53db700 1 -- 172.21.15.63:0/107229759 >> 172.21.15.63:0/107229759 conn(0x7f0bc00fb120 msgr2=0x7f0bc00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:20.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.039+0000 7f0bc53db700 1 -- 172.21.15.63:0/107229759 shutdown_connections 2024-05-09T06:39:20.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.039+0000 7f0bc53db700 1 -- 172.21.15.63:0/107229759 wait complete. 2024-05-09T06:39:20.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.040+0000 7f0bc53db700 1 Processor -- start 2024-05-09T06:39:20.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.040+0000 7f0bc53db700 1 -- start start 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.040+0000 7f0bc53db700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 0x7f0bc0196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.040+0000 7f0bc53db700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc0104060 0x7f0bc0196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.040+0000 7f0bc53db700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0197190 con 0x7f0bc0101730 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.040+0000 7f0bc53db700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01972d0 con 0x7f0bc0104060 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.041+0000 7f0bbeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 0x7f0bc0196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.041+0000 7f0bbeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 0x7f0bc0196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54766/0 (socket says 172.21.15.63:54766) 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.041+0000 7f0bbeffd700 1 -- 172.21.15.63:0/2237758592 learned_addr learned my addr 172.21.15.63:0/2237758592 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.041+0000 7f0bbe7fc700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc0104060 0x7f0bc0196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.041+0000 7f0bbeffd700 1 -- 172.21.15.63:0/2237758592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc0104060 msgr2=0x7f0bc0196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.041+0000 7f0bbeffd700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc0104060 0x7f0bc0196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.041+0000 7f0bbeffd700 1 -- 172.21.15.63:0/2237758592 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00095f0 con 0x7f0bc0101730 2024-05-09T06:39:20.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.041+0000 7f0bbe7fc700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc0104060 0x7f0bc0196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:20.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.041+0000 7f0bbeffd700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 0x7f0bc0196630 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000b800 tx=0x7f0bb0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:20.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.042+0000 7f0ba7fff700 1 -- 172.21.15.63:0/2237758592 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bb001d070 con 0x7f0bc0101730 2024-05-09T06:39:20.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.042+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc019bcd0 con 0x7f0bc0101730 2024-05-09T06:39:20.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.042+0000 7f0ba7fff700 1 -- 172.21.15.63:0/2237758592 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bb00052a0 con 0x7f0bc0101730 2024-05-09T06:39:20.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.042+0000 7f0ba7fff700 1 -- 172.21.15.63:0/2237758592 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bb0017400 con 0x7f0bc0101730 2024-05-09T06:39:20.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.042+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc019c1c0 con 0x7f0bc0101730 2024-05-09T06:39:20.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.043+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc004ebb0 con 0x7f0bc0101730 2024-05-09T06:39:20.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.043+0000 7f0ba7fff700 1 -- 172.21.15.63:0/2237758592 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 30) v1 ==== 46434+0+0 (secure 0 0 0) 0x7f0bb0003780 con 0x7f0bc0101730 2024-05-09T06:39:20.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.044+0000 7f0ba7fff700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f0ba8039360 0x7f0ba803b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:20.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.044+0000 7f0ba7fff700 1 -- 172.21.15.63:0/2237758592 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0bb004eb80 con 0x7f0bc0101730 2024-05-09T06:39:20.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.044+0000 7f0bbe7fc700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f0ba8039360 0x7f0ba803b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:20.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.045+0000 7f0bbe7fc700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f0ba8039360 0x7f0ba803b820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4011490 tx=0x7f0bb4009260 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:20.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.051+0000 7f0ba7fff700 1 -- 172.21.15.63:0/2237758592 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f0bb000f820 con 0x7f0bc0101730 2024-05-09T06:39:20.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:19 smithi193 ceph-mon[35141]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:20.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:19 smithi193 ceph-mon[35141]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:20.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:19 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:20.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:19 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:20.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.360+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0bc0109800 con 0x7f0bc0101730 2024-05-09T06:39:20.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.360+0000 7f0ba7fff700 1 -- 172.21.15.63:0/2237758592 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f0bb00163b0 con 0x7f0bc0101730 2024-05-09T06:39:20.341 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:39:20.341 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:39:20.341 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:39:20.341 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:39:20.341 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:39:20.341 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:39:20.341 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:39:20.342 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:39:20.343 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:39:20.344 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:39:20.344 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:39:20.344 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:39:20.344 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:39:20.344 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:39:20.344 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:39:20.344 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:39:20.344 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f0ba8039360 msgr2=0x7f0ba803b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f0ba8039360 0x7f0ba803b820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4011490 tx=0x7f0bb4009260 comp rx=0 tx=0).stop 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 msgr2=0x7f0bc0196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 0x7f0bc0196630 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000b800 tx=0x7f0bb0005f00 comp rx=0 tx=0).stop 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 shutdown_connections 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f0ba8039360 0x7f0ba803b820 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bc0101730 0x7f0bc0196630 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 --2- 172.21.15.63:0/2237758592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc0104060 0x7f0bc0196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 >> 172.21.15.63:0/2237758592 conn(0x7f0bc00fb120 msgr2=0x7f0bc00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 shutdown_connections 2024-05-09T06:39:20.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.363+0000 7f0bc53db700 1 -- 172.21.15.63:0/2237758592 wait complete. 2024-05-09T06:39:20.346 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:39:20.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.478+0000 7fbfda814700 1 -- 172.21.15.63:0/3719697792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 msgr2=0x7fbfd4102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.478+0000 7fbfda814700 1 --2- 172.21.15.63:0/3719697792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 0x7fbfd4102280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009910 tx=0x7fbfc8009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:20.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.478+0000 7fbfda814700 1 -- 172.21.15.63:0/3719697792 shutdown_connections 2024-05-09T06:39:20.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.478+0000 7fbfda814700 1 --2- 172.21.15.63:0/3719697792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 0x7fbfd4102280 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.478+0000 7fbfda814700 1 --2- 172.21.15.63:0/3719697792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd4100c20 0x7fbfd4101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.478+0000 7fbfda814700 1 -- 172.21.15.63:0/3719697792 >> 172.21.15.63:0/3719697792 conn(0x7fbfd40fc200 msgr2=0x7fbfd40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:20.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.478+0000 7fbfda814700 1 -- 172.21.15.63:0/3719697792 shutdown_connections 2024-05-09T06:39:20.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.479+0000 7fbfda814700 1 -- 172.21.15.63:0/3719697792 wait complete. 2024-05-09T06:39:20.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.479+0000 7fbfda814700 1 Processor -- start 2024-05-09T06:39:20.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.479+0000 7fbfda814700 1 -- start start 2024-05-09T06:39:20.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfda814700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd4100c20 0x7fbfd4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:20.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfda814700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 0x7fbfd4198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:20.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfda814700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41993c0 con 0x7fbfd4101e20 2024-05-09T06:39:20.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfda814700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4199500 con 0x7fbfd4100c20 2024-05-09T06:39:20.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfd37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 0x7fbfd4198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:20.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfd37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 0x7fbfd4198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54786/0 (socket says 172.21.15.63:54786) 2024-05-09T06:39:20.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfd37fe700 1 -- 172.21.15.63:0/602702343 learned_addr learned my addr 172.21.15.63:0/602702343 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:20.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfd3fff700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd4100c20 0x7fbfd4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:20.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfd37fe700 1 -- 172.21.15.63:0/602702343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd4100c20 msgr2=0x7fbfd4198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfd37fe700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd4100c20 0x7fbfd4198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfd37fe700 1 -- 172.21.15.63:0/602702343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80095f0 con 0x7fbfd4101e20 2024-05-09T06:39:20.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfd3fff700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd4100c20 0x7fbfd4198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:20.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.480+0000 7fbfd37fe700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 0x7fbfd4198da0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800bd70 tx=0x7fbfc800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:20.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.481+0000 7fbfd17fa700 1 -- 172.21.15.63:0/602702343 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc801d070 con 0x7fbfd4101e20 2024-05-09T06:39:20.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.481+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd419deb0 con 0x7fbfd4101e20 2024-05-09T06:39:20.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.481+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd419e370 con 0x7fbfd4101e20 2024-05-09T06:39:20.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.481+0000 7fbfd17fa700 1 -- 172.21.15.63:0/602702343 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfc8005dc0 con 0x7fbfd4101e20 2024-05-09T06:39:20.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.481+0000 7fbfd17fa700 1 -- 172.21.15.63:0/602702343 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc8017400 con 0x7fbfd4101e20 2024-05-09T06:39:20.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.482+0000 7fbfd17fa700 1 -- 172.21.15.63:0/602702343 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 30) v1 ==== 46434+0+0 (secure 0 0 0) 0x7fbfc8017560 con 0x7fbfd4101e20 2024-05-09T06:39:20.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.482+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc0005320 con 0x7fbfd4101e20 2024-05-09T06:39:20.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.482+0000 7fbfd17fa700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fbfbc039360 0x7fbfbc03b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:20.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.482+0000 7fbfd17fa700 1 -- 172.21.15.63:0/602702343 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbfc804ea10 con 0x7fbfd4101e20 2024-05-09T06:39:20.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.482+0000 7fbfd3fff700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fbfbc039360 0x7fbfbc03b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:20.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.483+0000 7fbfd3fff700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fbfbc039360 0x7fbfbc03b820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4101c80 tx=0x7fbfc4007400 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:20.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.487+0000 7fbfd17fa700 1 -- 172.21.15.63:0/602702343 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7fbfc801f900 con 0x7fbfd4101e20 2024-05-09T06:39:20.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.805+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 --> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfc0000bf0 con 0x7fbfbc039360 2024-05-09T06:39:20.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.806+0000 7fbfd17fa700 1 -- 172.21.15.63:0/602702343 <== mgr.24513 v2:172.21.15.63:6800/1655504434 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fbfc0000bf0 con 0x7fbfbc039360 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/23 daemons upgraded", 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:39:20.787 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:39:20.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fbfbc039360 msgr2=0x7fbfbc03b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fbfbc039360 0x7fbfbc03b820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4101c80 tx=0x7fbfc4007400 comp rx=0 tx=0).stop 2024-05-09T06:39:20.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 msgr2=0x7fbfd4198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 0x7fbfd4198da0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800bd70 tx=0x7fbfc800bda0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 shutdown_connections 2024-05-09T06:39:20.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fbfbc039360 0x7fbfbc03b820 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfd4100c20 0x7fbfd4198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 --2- 172.21.15.63:0/602702343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbfd4101e20 0x7fbfd4198da0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 >> 172.21.15.63:0/602702343 conn(0x7fbfd40fc200 msgr2=0x7fbfd4073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:20.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 shutdown_connections 2024-05-09T06:39:20.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.809+0000 7fbfda814700 1 -- 172.21.15.63:0/602702343 wait complete. 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.928+0000 7f2035fb0700 1 -- 172.21.15.63:0/203368691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030100c50 msgr2=0x7f2030101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.928+0000 7f2035fb0700 1 --2- 172.21.15.63:0/203368691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030100c50 0x7f2030101070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2020009910 tx=0x7f2020009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.928+0000 7f2035fb0700 1 -- 172.21.15.63:0/203368691 shutdown_connections 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.928+0000 7f2035fb0700 1 --2- 172.21.15.63:0/203368691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2030101e50 0x7f20301022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.928+0000 7f2035fb0700 1 --2- 172.21.15.63:0/203368691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030100c50 0x7f2030101070 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.928+0000 7f2035fb0700 1 -- 172.21.15.63:0/203368691 >> 172.21.15.63:0/203368691 conn(0x7f20300fc210 msgr2=0x7f20300fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.928+0000 7f2035fb0700 1 -- 172.21.15.63:0/203368691 shutdown_connections 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.928+0000 7f2035fb0700 1 -- 172.21.15.63:0/203368691 wait complete. 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.929+0000 7f2035fb0700 1 Processor -- start 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.929+0000 7f2035fb0700 1 -- start start 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.929+0000 7f2035fb0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2030100c50 0x7f2030196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:20.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.929+0000 7f2035fb0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030101e50 0x7f2030196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.929+0000 7f2035fb0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20301971c0 con 0x7f2030101e50 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.929+0000 7f2035fb0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030197300 con 0x7f2030100c50 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030101e50 0x7f2030196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2030100c50 0x7f2030196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2030100c50 0x7f2030196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.63:44084/0 (socket says 172.21.15.63:44084) 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202f7fe700 1 -- 172.21.15.63:0/757045349 learned_addr learned my addr 172.21.15.63:0/757045349 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030101e50 0x7f2030196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54804/0 (socket says 172.21.15.63:54804) 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202f7fe700 1 -- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030101e50 msgr2=0x7f2030196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202f7fe700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030101e50 0x7f2030196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202f7fe700 1 -- 172.21.15.63:0/757045349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20200095f0 con 0x7f2030100c50 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202effd700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030101e50 0x7f2030196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:20.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202f7fe700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2030100c50 0x7f2030196660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f202000b620 tx=0x7f2020005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202cff9700 1 -- 172.21.15.63:0/757045349 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f202001d070 con 0x7f2030100c50 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203019bcb0 con 0x7f2030100c50 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202cff9700 1 -- 172.21.15.63:0/757045349 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20200052a0 con 0x7f2030100c50 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f202cff9700 1 -- 172.21.15.63:0/757045349 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2020017400 con 0x7f2030100c50 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.930+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203019c1a0 con 0x7f2030100c50 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.931+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20301907d0 con 0x7f2030100c50 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.932+0000 7f202cff9700 1 -- 172.21.15.63:0/757045349 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f2020003680 con 0x7f2030100c50 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.932+0000 7f202cff9700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f201806dfb0 0x7f2018070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.933+0000 7f202cff9700 1 -- 172.21.15.63:0/757045349 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f202008f290 con 0x7f2030100c50 2024-05-09T06:39:20.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.933+0000 7f202effd700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f201806dfb0 0x7f2018070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:20.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.933+0000 7f202effd700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f201806dfb0 0x7f2018070470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2024011490 tx=0x7f2024009260 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:20.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:20.938+0000 7f202cff9700 1 -- 172.21.15.63:0/757045349 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f202005c310 con 0x7f2030100c50 2024-05-09T06:39:21.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:20 smithi063 ceph-mon[27171]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:21.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:20 smithi063 ceph-mon[27171]: pgmap v7: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 981 KiB/s wr, 276 op/s 2024-05-09T06:39:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:20 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/1636158844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:39:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:20 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/2237758592' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:39:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:20 smithi063 ceph-mon[27171]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:39:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:20 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/3316247662' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:39:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:20 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/3316247662' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:39:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:20 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/3316247662' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:39:21.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:20 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/3316247662' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:39:21.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:20 smithi193 ceph-mon[35141]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:21.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:20 smithi193 ceph-mon[35141]: pgmap v7: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 981 KiB/s wr, 276 op/s 2024-05-09T06:39:21.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:20 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/1636158844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:39:21.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:20 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/2237758592' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:39:21.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:20 smithi193 ceph-mon[35141]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:39:21.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:20 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/3316247662' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:39:21.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:20 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/3316247662' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:39:21.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:20 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/3316247662' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:39:21.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:20 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/3316247662' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:39:21.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.278+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f203004ebb0 con 0x7f2030100c50 2024-05-09T06:39:21.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.279+0000 7f202cff9700 1 -- 172.21.15.63:0/757045349 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f202005bea0 con 0x7f2030100c50 2024-05-09T06:39:21.260 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:39:21.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.281+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f201806dfb0 msgr2=0x7f2018070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.281+0000 7f2035fb0700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f201806dfb0 0x7f2018070470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2024011490 tx=0x7f2024009260 comp rx=0 tx=0).stop 2024-05-09T06:39:21.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.281+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2030100c50 msgr2=0x7f2030196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:21.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.281+0000 7f2035fb0700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2030100c50 0x7f2030196660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f202000b620 tx=0x7f2020005f00 comp rx=0 tx=0).stop 2024-05-09T06:39:21.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.281+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 shutdown_connections 2024-05-09T06:39:21.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.281+0000 7f2035fb0700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f201806dfb0 0x7f2018070470 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.281+0000 7f2035fb0700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2030100c50 0x7f2030196660 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.282+0000 7f2035fb0700 1 --2- 172.21.15.63:0/757045349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2030101e50 0x7f2030196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:21.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.282+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 >> 172.21.15.63:0/757045349 conn(0x7f20300fc210 msgr2=0x7f2030105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:21.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.282+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 shutdown_connections 2024-05-09T06:39:21.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:21.282+0000 7f2035fb0700 1 -- 172.21.15.63:0/757045349 wait complete. 2024-05-09T06:39:22.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:21 smithi063 ceph-mon[27171]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:22.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:21 smithi063 ceph-mon[27171]: mgrmap e31: smithi063.egfguy(active, since 8s), standbys: smithi193.mmmqzh 2024-05-09T06:39:22.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:21 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mmmqzh", "id": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:39:22.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:21 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/757045349' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:39:22.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:21 smithi193 ceph-mon[35141]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:22.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:21 smithi193 ceph-mon[35141]: mgrmap e31: smithi063.egfguy(active, since 8s), standbys: smithi193.mmmqzh 2024-05-09T06:39:22.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:21 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mmmqzh", "id": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:39:22.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:21 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/757045349' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:39:23.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:22 smithi063 ceph-mon[27171]: pgmap v8: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 827 KiB/s rd, 763 KiB/s wr, 214 op/s 2024-05-09T06:39:23.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:22 smithi193 ceph-mon[35141]: pgmap v8: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 827 KiB/s rd, 763 KiB/s wr, 214 op/s 2024-05-09T06:39:25.227 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:24 smithi063 ceph-mon[27171]: pgmap v9: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 345 op/s 2024-05-09T06:39:25.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:24 smithi193 ceph-mon[35141]: pgmap v9: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 345 op/s 2024-05-09T06:39:26.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:25 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:26.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:25 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:26.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:25 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:26.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:25 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:27 smithi063 ceph-mon[27171]: pgmap v10: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-05-09T06:39:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:27 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:27 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:27 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:27 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:27.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:27 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:39:27.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:27 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:39:27.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:27 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:27.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:27 smithi193 ceph-mon[35141]: pgmap v10: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-05-09T06:39:27.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:27 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:27.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:27 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:27.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:27 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:27.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:27 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:27.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:27 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:39:27.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:27 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:39:27.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:27 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:28.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-09T06:39:28.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-09T06:39:28.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:39:28.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:39:28.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:39:28.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:39:28.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:28.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:28.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:28.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:28.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:28 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:28.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-09T06:39:28.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-09T06:39:28.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:39:28.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:39:28.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:39:28.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:39:28.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:28.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:28.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:28.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:28.373 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:28 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:29.256 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:29 smithi063 ceph-mon[27171]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:39:29.256 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:29 smithi063 ceph-mon[27171]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:39:29.256 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:29 smithi063 ceph-mon[27171]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-05-09T06:39:29.256 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:29 smithi063 ceph-mon[27171]: pgmap v11: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-05-09T06:39:29.256 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:29 smithi063 ceph-mon[27171]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-05-09T06:39:29.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:29 smithi193 ceph-mon[35141]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:39:29.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:29 smithi193 ceph-mon[35141]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:39:29.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:29 smithi193 ceph-mon[35141]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-05-09T06:39:29.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:29 smithi193 ceph-mon[35141]: pgmap v11: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-05-09T06:39:29.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:29 smithi193 ceph-mon[35141]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-05-09T06:39:31.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:31 smithi063 ceph-mon[27171]: pgmap v12: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 744 op/s 2024-05-09T06:39:31.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:31 smithi193 ceph-mon[35141]: pgmap v12: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 744 op/s 2024-05-09T06:39:33.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:33 smithi063 ceph-mon[27171]: pgmap v13: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 583 op/s 2024-05-09T06:39:33.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:33 smithi193 ceph-mon[35141]: pgmap v13: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 583 op/s 2024-05-09T06:39:35.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:35 smithi193 ceph-mon[35141]: pgmap v14: 65 pgs: 65 active+clean; 269 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 1.03k op/s 2024-05-09T06:39:35.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:35 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:35.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:35 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:35.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:35 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:39:35.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:35 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:35.322 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:35 smithi063 ceph-mon[27171]: pgmap v14: 65 pgs: 65 active+clean; 269 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 1.03k op/s 2024-05-09T06:39:35.323 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:35 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:35.323 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:35 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:35.323 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:35 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:39:35.323 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:35 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:36.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:36 smithi193 ceph-mon[35141]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:39:36.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:36 smithi063 ceph-mon[27171]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:39:37.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:37 smithi063 ceph-mon[27171]: pgmap v15: 65 pgs: 65 active+clean; 269 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 872 op/s 2024-05-09T06:39:37.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:37 smithi063 ceph-mon[27171]: Upgrade: Updating mgr.smithi193.mmmqzh 2024-05-09T06:39:37.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:37 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:39:37.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:37 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:39:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:37 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:39:37.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:37 smithi063 ceph-mon[27171]: Deploying daemon mgr.smithi193.mmmqzh on smithi193 2024-05-09T06:39:37.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:37 smithi193 ceph-mon[35141]: pgmap v15: 65 pgs: 65 active+clean; 269 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 872 op/s 2024-05-09T06:39:37.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:37 smithi193 ceph-mon[35141]: Upgrade: Updating mgr.smithi193.mmmqzh 2024-05-09T06:39:37.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:37 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:39:37.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:37 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:39:37.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:37 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:39:37.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:37 smithi193 ceph-mon[35141]: Deploying daemon mgr.smithi193.mmmqzh on smithi193 2024-05-09T06:39:39.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:39 smithi063 ceph-mon[27171]: pgmap v16: 65 pgs: 65 active+clean; 269 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 872 op/s 2024-05-09T06:39:39.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:39 smithi193 ceph-mon[35141]: pgmap v16: 65 pgs: 65 active+clean; 269 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 872 op/s 2024-05-09T06:39:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:41 smithi063 ceph-mon[27171]: pgmap v17: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 1.11k op/s 2024-05-09T06:39:41.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:41 smithi193 ceph-mon[35141]: pgmap v17: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 1.11k op/s 2024-05-09T06:39:42.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:42 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:42.367 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:42 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:43 smithi063 ceph-mon[27171]: pgmap v18: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 682 op/s 2024-05-09T06:39:43.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:43 smithi193 ceph-mon[35141]: pgmap v18: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 682 op/s 2024-05-09T06:39:44.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:44 smithi193 ceph-mon[35141]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 833 op/s 2024-05-09T06:39:44.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:44 smithi063 ceph-mon[27171]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 833 op/s 2024-05-09T06:39:46.342 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:46 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:46.343 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:46 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:46.343 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:46 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:46.343 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:46 smithi193 ceph-mon[35141]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 388 op/s 2024-05-09T06:39:46.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:46 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:46.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:46 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:46.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:46 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:46.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:46 smithi063 ceph-mon[27171]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 388 op/s 2024-05-09T06:39:49.217 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:48 smithi193 ceph-mon[35141]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 388 op/s 2024-05-09T06:39:49.217 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:48 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:49.217 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:48 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:49.235 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:48 smithi063 ceph-mon[27171]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 388 op/s 2024-05-09T06:39:49.235 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:48 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:49.235 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:48 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:51.161 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:50 smithi193 ceph-mon[35141]: pgmap v22: 65 pgs: 65 active+clean; 256 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 518 op/s 2024-05-09T06:39:51.161 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:50 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:51.161 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:50 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:50 smithi063 ceph-mon[27171]: pgmap v22: 65 pgs: 65 active+clean; 256 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 518 op/s 2024-05-09T06:39:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:50 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:50 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:51.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.449+0000 7f21a414f700 1 -- 172.21.15.63:0/815780495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f219c100da0 msgr2=0x7f219c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.449+0000 7f21a414f700 1 --2- 172.21.15.63:0/815780495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f219c100da0 0x7f219c101200 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f218c009910 tx=0x7f218c009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.449+0000 7f21a414f700 1 -- 172.21.15.63:0/815780495 shutdown_connections 2024-05-09T06:39:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.449+0000 7f21a414f700 1 --2- 172.21.15.63:0/815780495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f219c100da0 0x7f219c101200 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.449+0000 7f21a414f700 1 --2- 172.21.15.63:0/815780495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f219c0ffba0 0x7f219c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.449+0000 7f21a414f700 1 -- 172.21.15.63:0/815780495 >> 172.21.15.63:0/815780495 conn(0x7f219c0fb120 msgr2=0x7f219c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.449+0000 7f21a414f700 1 -- 172.21.15.63:0/815780495 shutdown_connections 2024-05-09T06:39:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.449+0000 7f21a414f700 1 -- 172.21.15.63:0/815780495 wait complete. 2024-05-09T06:39:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a414f700 1 Processor -- start 2024-05-09T06:39:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a414f700 1 -- start start 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a414f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f219c0ffba0 0x7f219c103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a414f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f219c100da0 0x7f219c1025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a414f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c102ae0 con 0x7f219c100da0 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a414f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c102c20 con 0x7f219c0ffba0 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a1eeb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f219c0ffba0 0x7f219c103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a16ea700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f219c100da0 0x7f219c1025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a1eeb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f219c0ffba0 0x7f219c103f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.63:46870/0 (socket says 172.21.15.63:46870) 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.450+0000 7f21a1eeb700 1 -- 172.21.15.63:0/400342893 learned_addr learned my addr 172.21.15.63:0/400342893 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f21a1eeb700 1 -- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f219c100da0 msgr2=0x7f219c1025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f21a1eeb700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f219c100da0 0x7f219c1025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f21a1eeb700 1 -- 172.21.15.63:0/400342893 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c0095f0 con 0x7f219c0ffba0 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f21a16ea700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f219c100da0 0x7f219c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f21a1eeb700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f219c0ffba0 0x7f219c103f00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f219800e8f0 tx=0x7f219800ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:51.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f2192ffd700 1 -- 172.21.15.63:0/400342893 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2198009790 con 0x7f219c0ffba0 2024-05-09T06:39:51.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c102f00 con 0x7f219c0ffba0 2024-05-09T06:39:51.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f2192ffd700 1 -- 172.21.15.63:0/400342893 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f219800c810 con 0x7f219c0ffba0 2024-05-09T06:39:51.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f2192ffd700 1 -- 172.21.15.63:0/400342893 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2198003d20 con 0x7f219c0ffba0 2024-05-09T06:39:51.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.451+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c19e6e0 con 0x7f219c0ffba0 2024-05-09T06:39:51.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.452+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c04ebb0 con 0x7f219c0ffba0 2024-05-09T06:39:51.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.453+0000 7f2192ffd700 1 -- 172.21.15.63:0/400342893 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f21980041f0 con 0x7f219c0ffba0 2024-05-09T06:39:51.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.453+0000 7f2192ffd700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f218806dfc0 0x7f2188070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:51.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.453+0000 7f2192ffd700 1 -- 172.21.15.63:0/400342893 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2198014070 con 0x7f219c0ffba0 2024-05-09T06:39:51.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.454+0000 7f21a16ea700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f218806dfc0 0x7f2188070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:51.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.454+0000 7f21a16ea700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f218806dfc0 0x7f2188070480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f218c00b800 tx=0x7f218c009f70 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:51.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.459+0000 7f2192ffd700 1 -- 172.21.15.63:0/400342893 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f219805b260 con 0x7f219c0ffba0 2024-05-09T06:39:51.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.787+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 --> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f219c068ce0 con 0x7f218806dfc0 2024-05-09T06:39:51.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.788+0000 7f2192ffd700 1 -- 172.21.15.63:0/400342893 <== mgr.24513 v2:172.21.15.63:6800/1655504434 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f219c068ce0 con 0x7f218806dfc0 2024-05-09T06:39:51.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f218806dfc0 msgr2=0x7f2188070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:51.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f218806dfc0 0x7f2188070480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f218c00b800 tx=0x7f218c009f70 comp rx=0 tx=0).stop 2024-05-09T06:39:51.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f219c0ffba0 msgr2=0x7f219c103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:51.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f219c0ffba0 0x7f219c103f00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f219800e8f0 tx=0x7f219800ec00 comp rx=0 tx=0).stop 2024-05-09T06:39:51.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 shutdown_connections 2024-05-09T06:39:51.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f218806dfc0 0x7f2188070480 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:51.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f219c0ffba0 0x7f219c103f00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:51.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 --2- 172.21.15.63:0/400342893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f219c100da0 0x7f219c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:51.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 >> 172.21.15.63:0/400342893 conn(0x7f219c0fb120 msgr2=0x7f219c0685d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:51.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 shutdown_connections 2024-05-09T06:39:51.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.790+0000 7f21a414f700 1 -- 172.21.15.63:0/400342893 wait complete. 2024-05-09T06:39:51.783 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:39:51.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.909+0000 7f5cc22c0700 1 -- 172.21.15.63:0/759409636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 msgr2=0x7f5cbc103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:51.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.909+0000 7f5cc22c0700 1 --2- 172.21.15.63:0/759409636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 0x7f5cbc103b20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009910 tx=0x7f5cac009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:51.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.909+0000 7f5cc22c0700 1 -- 172.21.15.63:0/759409636 shutdown_connections 2024-05-09T06:39:51.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.909+0000 7f5cc22c0700 1 --2- 172.21.15.63:0/759409636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cbc104060 0x7f5cbc106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:51.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.909+0000 7f5cc22c0700 1 --2- 172.21.15.63:0/759409636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 0x7f5cbc103b20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:51.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.909+0000 7f5cc22c0700 1 -- 172.21.15.63:0/759409636 >> 172.21.15.63:0/759409636 conn(0x7f5cbc0fb120 msgr2=0x7f5cbc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:51.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.909+0000 7f5cc22c0700 1 -- 172.21.15.63:0/759409636 shutdown_connections 2024-05-09T06:39:51.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.909+0000 7f5cc22c0700 1 -- 172.21.15.63:0/759409636 wait complete. 2024-05-09T06:39:51.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cc22c0700 1 Processor -- start 2024-05-09T06:39:51.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cc22c0700 1 -- start start 2024-05-09T06:39:51.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cc22c0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 0x7f5cbc196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:51.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cc22c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cbc104060 0x7f5cbc196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:51.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cc22c0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc197190 con 0x7f5cbc101730 2024-05-09T06:39:51.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cc22c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc1972d0 con 0x7f5cbc104060 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cbbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 0x7f5cbc196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cbbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 0x7f5cbc196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57530/0 (socket says 172.21.15.63:57530) 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cbbfff700 1 -- 172.21.15.63:0/2955226037 learned_addr learned my addr 172.21.15.63:0/2955226037 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.910+0000 7f5cbb7fe700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cbc104060 0x7f5cbc196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.911+0000 7f5cbbfff700 1 -- 172.21.15.63:0/2955226037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cbc104060 msgr2=0x7f5cbc196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.911+0000 7f5cbbfff700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cbc104060 0x7f5cbc196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.911+0000 7f5cbbfff700 1 -- 172.21.15.63:0/2955226037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0095f0 con 0x7f5cbc101730 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.911+0000 7f5cbbfff700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 0x7f5cbc196630 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00b800 tx=0x7f5cac005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.911+0000 7f5cb97fa700 1 -- 172.21.15.63:0/2955226037 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cac01d070 con 0x7f5cbc101730 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.911+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc19bcd0 con 0x7f5cbc101730 2024-05-09T06:39:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.911+0000 7f5cb97fa700 1 -- 172.21.15.63:0/2955226037 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cac0052a0 con 0x7f5cbc101730 2024-05-09T06:39:51.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.912+0000 7f5cb97fa700 1 -- 172.21.15.63:0/2955226037 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cac017400 con 0x7f5cbc101730 2024-05-09T06:39:51.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.912+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc19c1c0 con 0x7f5cbc101730 2024-05-09T06:39:51.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.913+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc04ebb0 con 0x7f5cbc101730 2024-05-09T06:39:51.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.913+0000 7f5cb97fa700 1 -- 172.21.15.63:0/2955226037 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f5cac003780 con 0x7f5cbc101730 2024-05-09T06:39:51.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.913+0000 7f5cb97fa700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5ca406dfc0 0x7f5ca4070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:51.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.914+0000 7f5cb97fa700 1 -- 172.21.15.63:0/2955226037 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5cac08f250 con 0x7f5cbc101730 2024-05-09T06:39:51.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.914+0000 7f5cbb7fe700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5ca406dfc0 0x7f5ca4070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:51.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.914+0000 7f5cbb7fe700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5ca406dfc0 0x7f5ca4070480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0009c60 tx=0x7f5cb0009260 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:51.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:51.918+0000 7f5cb97fa700 1 -- 172.21.15.63:0/2955226037 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f5cac05c370 con 0x7f5cbc101730 2024-05-09T06:39:52.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.223+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 --> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cbc105f10 con 0x7f5ca406dfc0 2024-05-09T06:39:52.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.224+0000 7f5cb97fa700 1 -- 172.21.15.63:0/2955226037 <== mgr.24513 v2:172.21.15.63:6800/1655504434 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f5cbc105f10 con 0x7f5ca406dfc0 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.226+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5ca406dfc0 msgr2=0x7f5ca4070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.226+0000 7f5cc22c0700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5ca406dfc0 0x7f5ca4070480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0009c60 tx=0x7f5cb0009260 comp rx=0 tx=0).stop 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.226+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 msgr2=0x7f5cbc196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.226+0000 7f5cc22c0700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 0x7f5cbc196630 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00b800 tx=0x7f5cac005f00 comp rx=0 tx=0).stop 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.227+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 shutdown_connections 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.227+0000 7f5cc22c0700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f5ca406dfc0 0x7f5ca4070480 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.227+0000 7f5cc22c0700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5cbc101730 0x7f5cbc196630 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.227+0000 7f5cc22c0700 1 --2- 172.21.15.63:0/2955226037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cbc104060 0x7f5cbc196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.227+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 >> 172.21.15.63:0/2955226037 conn(0x7f5cbc0fb120 msgr2=0x7f5cbc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.227+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 shutdown_connections 2024-05-09T06:39:52.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.227+0000 7f5cc22c0700 1 -- 172.21.15.63:0/2955226037 wait complete. 2024-05-09T06:39:52.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.345+0000 7f8b57474700 1 -- 172.21.15.63:0/1110851405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 msgr2=0x7f8b50101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:52.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.345+0000 7f8b57474700 1 --2- 172.21.15.63:0/1110851405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 0x7f8b50101190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009910 tx=0x7f8b40009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:52.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.346+0000 7f8b57474700 1 -- 172.21.15.63:0/1110851405 shutdown_connections 2024-05-09T06:39:52.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.346+0000 7f8b57474700 1 --2- 172.21.15.63:0/1110851405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 0x7f8b50101190 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.346+0000 7f8b57474700 1 --2- 172.21.15.63:0/1110851405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b500ffb30 0x7f8b500fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.346+0000 7f8b57474700 1 -- 172.21.15.63:0/1110851405 >> 172.21.15.63:0/1110851405 conn(0x7f8b500fb0f0 msgr2=0x7f8b500fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:52.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.346+0000 7f8b57474700 1 -- 172.21.15.63:0/1110851405 shutdown_connections 2024-05-09T06:39:52.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.346+0000 7f8b57474700 1 -- 172.21.15.63:0/1110851405 wait complete. 2024-05-09T06:39:52.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.346+0000 7f8b57474700 1 Processor -- start 2024-05-09T06:39:52.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.346+0000 7f8b57474700 1 -- start start 2024-05-09T06:39:52.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b57474700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b500ffb30 0x7f8b50069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:52.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b57474700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 0x7f8b5019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:52.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b57474700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5019b8f0 con 0x7f8b50100d30 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b57474700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5019ba30 con 0x7f8b500ffb30 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b54a0f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 0x7f8b5019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b54a0f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 0x7f8b5019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57556/0 (socket says 172.21.15.63:57556) 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b54a0f700 1 -- 172.21.15.63:0/493442622 learned_addr learned my addr 172.21.15.63:0/493442622 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b54a0f700 1 -- 172.21.15.63:0/493442622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b500ffb30 msgr2=0x7f8b50069cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b54a0f700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b500ffb30 0x7f8b50069cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b54a0f700 1 -- 172.21.15.63:0/493442622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400095f0 con 0x7f8b50100d30 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b54a0f700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 0x7f8b5019b320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005f30 tx=0x7f8b4000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b467fc700 1 -- 172.21.15.63:0/493442622 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b4001d070 con 0x7f8b50100d30 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b467fc700 1 -- 172.21.15.63:0/493442622 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b4000fe30 con 0x7f8b50100d30 2024-05-09T06:39:52.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5019bc60 con 0x7f8b50100d30 2024-05-09T06:39:52.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.347+0000 7f8b467fc700 1 -- 172.21.15.63:0/493442622 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b40017400 con 0x7f8b50100d30 2024-05-09T06:39:52.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.348+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5019c1d0 con 0x7f8b50100d30 2024-05-09T06:39:52.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.349+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5018f6b0 con 0x7f8b50100d30 2024-05-09T06:39:52.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.349+0000 7f8b467fc700 1 -- 172.21.15.63:0/493442622 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f8b40003880 con 0x7f8b50100d30 2024-05-09T06:39:52.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.350+0000 7f8b467fc700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f8b3c06dfc0 0x7f8b3c070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:52.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.350+0000 7f8b467fc700 1 -- 172.21.15.63:0/493442622 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8b4008f230 con 0x7f8b50100d30 2024-05-09T06:39:52.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.350+0000 7f8b55210700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f8b3c06dfc0 0x7f8b3c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:52.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.351+0000 7f8b55210700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f8b3c06dfc0 0x7f8b3c070480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8b50100b90 tx=0x7f8b4c009260 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:52.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.355+0000 7f8b467fc700 1 -- 172.21.15.63:0/493442622 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f8b4005c3d0 con 0x7f8b50100d30 2024-05-09T06:39:52.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.590+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 --> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b5002ce50 con 0x7f8b3c06dfc0 2024-05-09T06:39:52.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.602+0000 7f8b467fc700 1 -- 172.21.15.63:0/493442622 <== mgr.24513 v2:172.21.15.63:6800/1655504434 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8b5002ce50 con 0x7f8b3c06dfc0 2024-05-09T06:39:52.582 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:39:52.582 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (8m) 0s ago 10m 25.1M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:39:52.582 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (10m) 0s ago 10m 7818k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (9m) 3s ago 9m 7826k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (10m) 0s ago 10m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (9m) 3s ago 9m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (8m) 0s ago 10m 87.9M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (4m) 0s ago 4m 14.4M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (4m) 0s ago 4m 113M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (4m) 3s ago 4m 180M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (4m) 3s ago 4m 15.9M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (54s) 0s ago 11m 496M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (8s) 3s ago 9m 52.0M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (11m) 0s ago 11m 55.9M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 7d88727e4bbb 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (9m) 3s ago 9m 44.7M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:39:52.583 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (10m) 0s ago 10m 16.0M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:39:52.584 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (9m) 3s ago 9m 16.0M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:39:52.584 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (8m) 0s ago 8m 220M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:39:52.584 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (7m) 0s ago 7m 237M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:39:52.584 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (7m) 0s ago 7m 198M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:39:52.584 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (7m) 3s ago 7m 271M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:39:52.584 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (6m) 3s ago 6m 249M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:39:52.584 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (6m) 3s ago 6m 223M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:39:52.584 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (19s) 0s ago 9m 45.1M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:39:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f8b3c06dfc0 msgr2=0x7f8b3c070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f8b3c06dfc0 0x7f8b3c070480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8b50100b90 tx=0x7f8b4c009260 comp rx=0 tx=0).stop 2024-05-09T06:39:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 msgr2=0x7f8b5019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 0x7f8b5019b320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005f30 tx=0x7f8b4000f510 comp rx=0 tx=0).stop 2024-05-09T06:39:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 shutdown_connections 2024-05-09T06:39:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f8b3c06dfc0 0x7f8b3c070480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b500ffb30 0x7f8b50069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 --2- 172.21.15.63:0/493442622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b50100d30 0x7f8b5019b320 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 >> 172.21.15.63:0/493442622 conn(0x7f8b500fb0f0 msgr2=0x7f8b501042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:52.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 shutdown_connections 2024-05-09T06:39:52.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.605+0000 7f8b57474700 1 -- 172.21.15.63:0/493442622 wait complete. 2024-05-09T06:39:52.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.719+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/1037353675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 msgr2=0x7f1bc4103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:52.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.719+0000 7f1bc8f8c700 1 --2- 172.21.15.63:0/1037353675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 0x7f1bc4103b80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009910 tx=0x7f1bb4009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:52.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.719+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/1037353675 shutdown_connections 2024-05-09T06:39:52.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.719+0000 7f1bc8f8c700 1 --2- 172.21.15.63:0/1037353675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bc41040c0 0x7f1bc41064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.719+0000 7f1bc8f8c700 1 --2- 172.21.15.63:0/1037353675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 0x7f1bc4103b80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.719+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/1037353675 >> 172.21.15.63:0/1037353675 conn(0x7f1bc40fb120 msgr2=0x7f1bc40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:52.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.720+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/1037353675 shutdown_connections 2024-05-09T06:39:52.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.720+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/1037353675 wait complete. 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.720+0000 7f1bc8f8c700 1 Processor -- start 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc8f8c700 1 -- start start 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc8f8c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 0x7f1bc4196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc8f8c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bc41040c0 0x7f1bc4196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc8f8c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc41971a0 con 0x7f1bc4101790 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc8f8c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc41972e0 con 0x7f1bc41040c0 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 0x7f1bc4196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 0x7f1bc4196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57578/0 (socket says 172.21.15.63:57578) 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc259c700 1 -- 172.21.15.63:0/681538599 learned_addr learned my addr 172.21.15.63:0/681538599 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc1d9b700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bc41040c0 0x7f1bc4196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:52.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc259c700 1 -- 172.21.15.63:0/681538599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bc41040c0 msgr2=0x7f1bc4196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc259c700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bc41040c0 0x7f1bc4196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc259c700 1 -- 172.21.15.63:0/681538599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40095f0 con 0x7f1bc4101790 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.721+0000 7f1bc1d9b700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bc41040c0 0x7f1bc4196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.722+0000 7f1bc259c700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 0x7f1bc4196640 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400b800 tx=0x7f1bb4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.722+0000 7f1bb37fe700 1 -- 172.21.15.63:0/681538599 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bb401d070 con 0x7f1bc4101790 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.722+0000 7f1bb37fe700 1 -- 172.21.15.63:0/681538599 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bb40052a0 con 0x7f1bc4101790 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.722+0000 7f1bb37fe700 1 -- 172.21.15.63:0/681538599 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bb4017400 con 0x7f1bc4101790 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.722+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc419bce0 con 0x7f1bc4101790 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.722+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc419c180 con 0x7f1bc4101790 2024-05-09T06:39:52.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.723+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc404ebb0 con 0x7f1bc4101790 2024-05-09T06:39:52.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.724+0000 7f1bb37fe700 1 -- 172.21.15.63:0/681538599 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f1bb40176c0 con 0x7f1bc4101790 2024-05-09T06:39:52.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.725+0000 7f1bb37fe700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f1bac06dfc0 0x7f1bac070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:52.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.725+0000 7f1bb37fe700 1 -- 172.21.15.63:0/681538599 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1bb408f2c0 con 0x7f1bc4101790 2024-05-09T06:39:52.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.725+0000 7f1bc1d9b700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f1bac06dfc0 0x7f1bac070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:52.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.726+0000 7f1bc1d9b700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f1bac06dfc0 0x7f1bac070480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8011490 tx=0x7f1bb8009260 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:52.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:52.732+0000 7f1bb37fe700 1 -- 172.21.15.63:0/681538599 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f1bb405c3e0 con 0x7f1bc4101790 2024-05-09T06:39:52.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:52 smithi063 ceph-mon[27171]: from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:52.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:52 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:52.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:52 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:52.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:52 smithi063 ceph-mon[27171]: pgmap v23: 65 pgs: 65 active+clean; 256 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 959 KiB/s rd, 998 KiB/s wr, 280 op/s 2024-05-09T06:39:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:52 smithi193 ceph-mon[35141]: from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:52 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:52 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:52 smithi193 ceph-mon[35141]: pgmap v23: 65 pgs: 65 active+clean; 256 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 959 KiB/s rd, 998 KiB/s wr, 280 op/s 2024-05-09T06:39:53.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.115+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1bc4109a20 con 0x7f1bc4101790 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.116+0000 7f1bb37fe700 1 -- 172.21.15.63:0/681538599 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f1bb405bf70 con 0x7f1bc4101790 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:39:53.096 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:39:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:39:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:39:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 12, 2024-05-09T06:39:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:39:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:39:53.097 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f1bac06dfc0 msgr2=0x7f1bac070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f1bac06dfc0 0x7f1bac070480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8011490 tx=0x7f1bb8009260 comp rx=0 tx=0).stop 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 msgr2=0x7f1bc4196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 0x7f1bc4196640 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400b800 tx=0x7f1bb4005f00 comp rx=0 tx=0).stop 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 shutdown_connections 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f1bac06dfc0 0x7f1bac070480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bc4101790 0x7f1bc4196640 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 --2- 172.21.15.63:0/681538599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bc41040c0 0x7f1bc4196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 >> 172.21.15.63:0/681538599 conn(0x7f1bc40fb120 msgr2=0x7f1bc40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 shutdown_connections 2024-05-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.118+0000 7f1bc8f8c700 1 -- 172.21.15.63:0/681538599 wait complete. 2024-05-09T06:39:53.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.236+0000 7fafb62c6700 1 -- 172.21.15.63:0/3242574357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0100c50 msgr2=0x7fafb0101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.236+0000 7fafb62c6700 1 --2- 172.21.15.63:0/3242574357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0100c50 0x7fafb0101070 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009910 tx=0x7fafa0009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:53.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.236+0000 7fafb62c6700 1 -- 172.21.15.63:0/3242574357 shutdown_connections 2024-05-09T06:39:53.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.236+0000 7fafb62c6700 1 --2- 172.21.15.63:0/3242574357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb0101e50 0x7fafb01022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.236+0000 7fafb62c6700 1 --2- 172.21.15.63:0/3242574357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0100c50 0x7fafb0101070 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.237+0000 7fafb62c6700 1 -- 172.21.15.63:0/3242574357 >> 172.21.15.63:0/3242574357 conn(0x7fafb00fc210 msgr2=0x7fafb00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:53.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.237+0000 7fafb62c6700 1 -- 172.21.15.63:0/3242574357 shutdown_connections 2024-05-09T06:39:53.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.237+0000 7fafb62c6700 1 -- 172.21.15.63:0/3242574357 wait complete. 2024-05-09T06:39:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.237+0000 7fafb62c6700 1 Processor -- start 2024-05-09T06:39:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.237+0000 7fafb62c6700 1 -- start start 2024-05-09T06:39:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafb62c6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb0100c50 0x7fafb0074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafb62c6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0101e50 0x7fafb00732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafb62c6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb0073820 con 0x7fafb0101e50 2024-05-09T06:39:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafb62c6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb0073960 con 0x7fafb0100c50 2024-05-09T06:39:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafaf7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0101e50 0x7fafb00732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:53.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafaf7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0101e50 0x7fafb00732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57592/0 (socket says 172.21.15.63:57592) 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafaf7fe700 1 -- 172.21.15.63:0/624751095 learned_addr learned my addr 172.21.15.63:0/624751095 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafaffff700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb0100c50 0x7fafb0074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafaf7fe700 1 -- 172.21.15.63:0/624751095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb0100c50 msgr2=0x7fafb0074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafaf7fe700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb0100c50 0x7fafb0074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafaf7fe700 1 -- 172.21.15.63:0/624751095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa00095f0 con 0x7fafb0101e50 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.238+0000 7fafaf7fe700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0101e50 0x7fafb00732e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d8d0 tx=0x7fafa400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.239+0000 7fafad7fa700 1 -- 172.21.15.63:0/624751095 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa40098a0 con 0x7fafb0101e50 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.239+0000 7fafad7fa700 1 -- 172.21.15.63:0/624751095 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafa4010460 con 0x7fafb0101e50 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.239+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb0073c40 con 0x7fafb0101e50 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.239+0000 7fafad7fa700 1 -- 172.21.15.63:0/624751095 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa400f5f0 con 0x7fafb0101e50 2024-05-09T06:39:53.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.239+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb019e670 con 0x7fafb0101e50 2024-05-09T06:39:53.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.240+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb01907d0 con 0x7fafb0101e50 2024-05-09T06:39:53.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.240+0000 7fafad7fa700 1 -- 172.21.15.63:0/624751095 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7fafa4009a00 con 0x7fafb0101e50 2024-05-09T06:39:53.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.241+0000 7fafad7fa700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7faf9806dfc0 0x7faf98070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:53.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.241+0000 7fafad7fa700 1 -- 172.21.15.63:0/624751095 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fafa408dde0 con 0x7fafb0101e50 2024-05-09T06:39:53.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.241+0000 7fafaffff700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7faf9806dfc0 0x7faf98070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:53.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.241+0000 7fafaffff700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7faf9806dfc0 0x7faf98070480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fafa000b5c0 tx=0x7fafa0005960 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:53.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.245+0000 7fafad7fa700 1 -- 172.21.15.63:0/624751095 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7fafa405ae60 con 0x7fafb0101e50 2024-05-09T06:39:53.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.541+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fafb01097b0 con 0x7fafb0101e50 2024-05-09T06:39:53.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.542+0000 7fafad7fa700 1 -- 172.21.15.63:0/624751095 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fafa405a9f0 con 0x7fafb0101e50 2024-05-09T06:39:53.522 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:39:53.522 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:39:53.522 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:39:53.523 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:39:53.524 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:39:53.525 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:39:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7faf9806dfc0 msgr2=0x7faf98070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7faf9806dfc0 0x7faf98070480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fafa000b5c0 tx=0x7fafa0005960 comp rx=0 tx=0).stop 2024-05-09T06:39:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0101e50 msgr2=0x7fafb00732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0101e50 0x7fafb00732e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d8d0 tx=0x7fafa400dbe0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 shutdown_connections 2024-05-09T06:39:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7faf9806dfc0 0x7faf98070480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb0100c50 0x7fafb0074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 --2- 172.21.15.63:0/624751095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb0101e50 0x7fafb00732e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 >> 172.21.15.63:0/624751095 conn(0x7fafb00fc210 msgr2=0x7fafb0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 shutdown_connections 2024-05-09T06:39:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.545+0000 7fafb62c6700 1 -- 172.21.15.63:0/624751095 wait complete. 2024-05-09T06:39:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:39:53.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.668+0000 7fc65f1f2700 1 -- 172.21.15.63:0/1612407580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 msgr2=0x7fc658100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.668+0000 7fc65f1f2700 1 --2- 172.21.15.63:0/1612407580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 0x7fc658100fd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009910 tx=0x7fc64c009c20 comp rx=0 tx=0).stop 2024-05-09T06:39:53.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.668+0000 7fc65f1f2700 1 -- 172.21.15.63:0/1612407580 shutdown_connections 2024-05-09T06:39:53.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.668+0000 7fc65f1f2700 1 --2- 172.21.15.63:0/1612407580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc658101db0 0x7fc658102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.668+0000 7fc65f1f2700 1 --2- 172.21.15.63:0/1612407580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 0x7fc658100fd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.668+0000 7fc65f1f2700 1 -- 172.21.15.63:0/1612407580 >> 172.21.15.63:0/1612407580 conn(0x7fc6580fc190 msgr2=0x7fc6580fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:53.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.668+0000 7fc65f1f2700 1 -- 172.21.15.63:0/1612407580 shutdown_connections 2024-05-09T06:39:53.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.668+0000 7fc65f1f2700 1 -- 172.21.15.63:0/1612407580 wait complete. 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc65f1f2700 1 Processor -- start 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc65f1f2700 1 -- start start 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc65f1f2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 0x7fc658196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc65f1f2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc658101db0 0x7fc658196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc65f1f2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6581971b0 con 0x7fc658100bb0 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc65f1f2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6581972f0 con 0x7fc658101db0 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc65cf8e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 0x7fc658196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc65cf8e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 0x7fc658196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57610/0 (socket says 172.21.15.63:57610) 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc65cf8e700 1 -- 172.21.15.63:0/3664415762 learned_addr learned my addr 172.21.15.63:0/3664415762 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc657fff700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc658101db0 0x7fc658196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc657fff700 1 -- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 msgr2=0x7fc658196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.669+0000 7fc657fff700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 0x7fc658196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.670+0000 7fc657fff700 1 -- 172.21.15.63:0/3664415762 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c0095f0 con 0x7fc658101db0 2024-05-09T06:39:53.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.670+0000 7fc65cf8e700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 0x7fc658196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:39:53.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.670+0000 7fc657fff700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc658101db0 0x7fc658196b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e970 tx=0x7fc64400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:53.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.670+0000 7fc655ffb700 1 -- 172.21.15.63:0/3664415762 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc64400c9d0 con 0x7fc658101db0 2024-05-09T06:39:53.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.670+0000 7fc655ffb700 1 -- 172.21.15.63:0/3664415762 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc644010430 con 0x7fc658101db0 2024-05-09T06:39:53.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.670+0000 7fc655ffb700 1 -- 172.21.15.63:0/3664415762 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc644018630 con 0x7fc658101db0 2024-05-09T06:39:53.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.670+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65819bcb0 con 0x7fc658101db0 2024-05-09T06:39:53.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.670+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65819c230 con 0x7fc658101db0 2024-05-09T06:39:53.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.671+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc658190730 con 0x7fc658101db0 2024-05-09T06:39:53.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.672+0000 7fc655ffb700 1 -- 172.21.15.63:0/3664415762 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7fc6440105a0 con 0x7fc658101db0 2024-05-09T06:39:53.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.673+0000 7fc655ffb700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fc64806dfc0 0x7fc648070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:53.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.673+0000 7fc655ffb700 1 -- 172.21.15.63:0/3664415762 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc644014070 con 0x7fc658101db0 2024-05-09T06:39:53.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.673+0000 7fc65cf8e700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fc64806dfc0 0x7fc648070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:53.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.673+0000 7fc65cf8e700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fc64806dfc0 0x7fc648070480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00b800 tx=0x7fc64c009f70 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:53.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.677+0000 7fc655ffb700 1 -- 172.21.15.63:0/3664415762 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7fc64405b700 con 0x7fc658101db0 2024-05-09T06:39:53.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.923+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 --> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc65802ceb0 con 0x7fc64806dfc0 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.925+0000 7fc655ffb700 1 -- 172.21.15.63:0/3664415762 <== mgr.24513 v2:172.21.15.63:6800/1655504434 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc65802ceb0 con 0x7fc64806dfc0 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/23 daemons upgraded", 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mgr daemons", 2024-05-09T06:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:39:53.906 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:39:53.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fc64806dfc0 msgr2=0x7fc648070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fc64806dfc0 0x7fc648070480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00b800 tx=0x7fc64c009f70 comp rx=0 tx=0).stop 2024-05-09T06:39:53.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc658101db0 msgr2=0x7fc658196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:53.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc658101db0 0x7fc658196b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e970 tx=0x7fc64400ec80 comp rx=0 tx=0).stop 2024-05-09T06:39:53.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 shutdown_connections 2024-05-09T06:39:53.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7fc64806dfc0 0x7fc648070480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc658100bb0 0x7fc658196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 --2- 172.21.15.63:0/3664415762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc658101db0 0x7fc658196b90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:53.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 >> 172.21.15.63:0/3664415762 conn(0x7fc6580fc190 msgr2=0x7fc658105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:53.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 shutdown_connections 2024-05-09T06:39:53.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:53.927+0000 7fc65f1f2700 1 -- 172.21.15.63:0/3664415762 wait complete. 2024-05-09T06:39:54.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.079+0000 7f60df36c700 1 -- 172.21.15.63:0/2691256338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d81004b0 msgr2=0x7f60d8100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:54.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.079+0000 7f60df36c700 1 --2- 172.21.15.63:0/2691256338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d81004b0 0x7f60d8100910 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009870 tx=0x7f60d4009b80 comp rx=0 tx=0).stop 2024-05-09T06:39:54.059 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:53 smithi063 ceph-mon[27171]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:54.060 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:53 smithi063 ceph-mon[27171]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:54.060 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:53 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/681538599' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:39:54.060 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:53 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/624751095' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:39:54.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.079+0000 7f60df36c700 1 -- 172.21.15.63:0/2691256338 shutdown_connections 2024-05-09T06:39:54.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.079+0000 7f60df36c700 1 --2- 172.21.15.63:0/2691256338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d81004b0 0x7f60d8100910 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:54.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.079+0000 7f60df36c700 1 --2- 172.21.15.63:0/2691256338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d80ff3c0 0x7f60d80ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:54.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.079+0000 7f60df36c700 1 -- 172.21.15.63:0/2691256338 >> 172.21.15.63:0/2691256338 conn(0x7f60d80fa430 msgr2=0x7f60d80fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:54.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.079+0000 7f60df36c700 1 -- 172.21.15.63:0/2691256338 shutdown_connections 2024-05-09T06:39:54.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.079+0000 7f60df36c700 1 -- 172.21.15.63:0/2691256338 wait complete. 2024-05-09T06:39:54.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.080+0000 7f60df36c700 1 Processor -- start 2024-05-09T06:39:54.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.080+0000 7f60df36c700 1 -- start start 2024-05-09T06:39:54.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.080+0000 7f60df36c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d80ff3c0 0x7f60d819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:54.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.080+0000 7f60df36c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d81004b0 0x7f60d819d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:54.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.080+0000 7f60df36c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d819d7c0 con 0x7f60d80ff3c0 2024-05-09T06:39:54.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.080+0000 7f60df36c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81a2170 con 0x7f60d81004b0 2024-05-09T06:39:54.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.081+0000 7f60dc907700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d81004b0 0x7f60d819d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:54.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.081+0000 7f60dc907700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d81004b0 0x7f60d819d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.63:46974/0 (socket says 172.21.15.63:46974) 2024-05-09T06:39:54.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.081+0000 7f60dc907700 1 -- 172.21.15.63:0/4021602517 learned_addr learned my addr 172.21.15.63:0/4021602517 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:39:54.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.081+0000 7f60dd108700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d80ff3c0 0x7f60d819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:54.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.082+0000 7f60dd108700 1 -- 172.21.15.63:0/4021602517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d81004b0 msgr2=0x7f60d819d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:54.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.082+0000 7f60dd108700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d81004b0 0x7f60d819d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:54.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.082+0000 7f60dd108700 1 -- 172.21.15.63:0/4021602517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d4009520 con 0x7f60d80ff3c0 2024-05-09T06:39:54.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.083+0000 7f60dd108700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d80ff3c0 0x7f60d819cc60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005fd0 tx=0x7f60c800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:54.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.083+0000 7f60ce7fc700 1 -- 172.21.15.63:0/4021602517 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c800cd60 con 0x7f60d80ff3c0 2024-05-09T06:39:54.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.083+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d81a2450 con 0x7f60d80ff3c0 2024-05-09T06:39:54.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.083+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d81a2a20 con 0x7f60d80ff3c0 2024-05-09T06:39:54.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.083+0000 7f60ce7fc700 1 -- 172.21.15.63:0/4021602517 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60c800ee90 con 0x7f60d80ff3c0 2024-05-09T06:39:54.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.083+0000 7f60ce7fc700 1 -- 172.21.15.63:0/4021602517 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60c800cd60 con 0x7f60d80ff3c0 2024-05-09T06:39:54.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.085+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc0052f0 con 0x7f60d80ff3c0 2024-05-09T06:39:54.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.086+0000 7f60ce7fc700 1 -- 172.21.15.63:0/4021602517 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 92166+0+0 (secure 0 0 0) 0x7f60c8017480 con 0x7f60d80ff3c0 2024-05-09T06:39:54.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.086+0000 7f60ce7fc700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f60c406e070 0x7f60c4070530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:39:54.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.086+0000 7f60ce7fc700 1 -- 172.21.15.63:0/4021602517 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f60c808de30 con 0x7f60d80ff3c0 2024-05-09T06:39:54.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.087+0000 7f60dc907700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f60c406e070 0x7f60c4070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:39:54.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.087+0000 7f60dc907700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f60c406e070 0x7f60c4070530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009f50 tx=0x7f60d4010040 comp rx=0 tx=0).ready entity=mgr.24513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:39:54.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.095+0000 7f60ce7fc700 1 -- 172.21.15.63:0/4021602517 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186127 (secure 0 0 0) 0x7f60c8058260 con 0x7f60d80ff3c0 2024-05-09T06:39:54.102 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:53 smithi193 ceph-mon[35141]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:54.102 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:53 smithi193 ceph-mon[35141]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:54.102 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:53 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/681538599' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:39:54.102 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:53 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/624751095' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:39:54.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.449+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f60bc005870 con 0x7f60d80ff3c0 2024-05-09T06:39:54.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.450+0000 7f60ce7fc700 1 -- 172.21.15.63:0/4021602517 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f60c8004270 con 0x7f60d80ff3c0 2024-05-09T06:39:54.430 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:39:54.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f60c406e070 msgr2=0x7f60c4070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f60c406e070 0x7f60c4070530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009f50 tx=0x7f60d4010040 comp rx=0 tx=0).stop 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d80ff3c0 msgr2=0x7f60d819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d80ff3c0 0x7f60d819cc60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005fd0 tx=0x7f60c800ed70 comp rx=0 tx=0).stop 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 shutdown_connections 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:6800/1655504434,v1:172.21.15.63:6801/1655504434] conn(0x7f60c406e070 0x7f60c4070530 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d80ff3c0 0x7f60d819cc60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 --2- 172.21.15.63:0/4021602517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d81004b0 0x7f60d819d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 >> 172.21.15.63:0/4021602517 conn(0x7f60d80fa430 msgr2=0x7f60d8072440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 shutdown_connections 2024-05-09T06:39:54.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:39:54.452+0000 7f60df36c700 1 -- 172.21.15.63:0/4021602517 wait complete. 2024-05-09T06:39:55.069 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: pgmap v24: 65 pgs: 65 active+clean; 240 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 419 op/s 2024-05-09T06:39:55.069 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:55.069 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.egfguy"}]: dispatch 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.egfguy"}]': finished 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mmmqzh"}]: dispatch 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mmmqzh"}]': finished 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='client.? 172.21.15.63:0/4021602517' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:39:55.070 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: Standby manager daemon smithi193.mmmqzh restarted 2024-05-09T06:39:55.071 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:39:55.071 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/2557897653' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:39:55.071 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/2557897653' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:39:55.071 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/2557897653' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:39:55.071 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:54 smithi063 ceph-mon[27171]: from='mgr.? 172.21.15.193:0/2557897653' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:39:55.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: pgmap v24: 65 pgs: 65 active+clean; 240 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 419 op/s 2024-05-09T06:39:55.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:39:55.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:55.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.egfguy"}]: dispatch 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.egfguy"}]': finished 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mmmqzh"}]: dispatch 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mmmqzh"}]': finished 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='client.? 172.21.15.63:0/4021602517' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: Standby manager daemon smithi193.mmmqzh restarted 2024-05-09T06:39:55.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:39:55.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/2557897653' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:39:55.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/2557897653' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:39:55.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/2557897653' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:39:55.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:54 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/2557897653' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:39:56.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:55 smithi063 ceph-mon[27171]: Upgrade: Setting container_image for all mgr 2024-05-09T06:39:56.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:55 smithi063 ceph-mon[27171]: mgrmap e32: smithi063.egfguy(active, since 43s), standbys: smithi193.mmmqzh 2024-05-09T06:39:56.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:55 smithi193 ceph-mon[35141]: Upgrade: Setting container_image for all mgr 2024-05-09T06:39:56.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:55 smithi193 ceph-mon[35141]: mgrmap e32: smithi063.egfguy(active, since 43s), standbys: smithi193.mmmqzh 2024-05-09T06:39:57.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:56 smithi063 ceph-mon[27171]: pgmap v25: 65 pgs: 65 active+clean; 240 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 972 KiB/s rd, 1011 KiB/s wr, 268 op/s 2024-05-09T06:39:57.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:56 smithi193 ceph-mon[35141]: pgmap v25: 65 pgs: 65 active+clean; 240 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 972 KiB/s rd, 1011 KiB/s wr, 268 op/s 2024-05-09T06:39:58.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:57 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:58.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:57 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:39:59.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:58 smithi063 ceph-mon[27171]: pgmap v26: 65 pgs: 65 active+clean; 238 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 972 KiB/s rd, 1011 KiB/s wr, 289 op/s 2024-05-09T06:39:59.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:58 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:59.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:58 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:39:59.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:58 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:39:59.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:58 smithi063 ceph-mon[27171]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:39:59.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:58 smithi193 ceph-mon[35141]: pgmap v26: 65 pgs: 65 active+clean; 238 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 972 KiB/s rd, 1011 KiB/s wr, 289 op/s 2024-05-09T06:39:59.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:58 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' 2024-05-09T06:39:59.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:58 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:39:59.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:58 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:39:59.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:58 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:00.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:59 smithi063 ceph-mon[27171]: Upgrade: Updating mon.smithi063 2024-05-09T06:40:00.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:39:59 smithi063 ceph-mon[27171]: Deploying daemon mon.smithi063 on smithi063 2024-05-09T06:40:00.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:59 smithi193 ceph-mon[35141]: Upgrade: Updating mon.smithi063 2024-05-09T06:40:00.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:39:59 smithi193 ceph-mon[35141]: Deploying daemon mon.smithi063 on smithi063 2024-05-09T06:40:01.064 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:00 smithi063 ceph-mon[27171]: pgmap v27: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 417 op/s 2024-05-09T06:40:01.064 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:00 smithi063 ceph-mon[27171]: overall HEALTH_OK 2024-05-09T06:40:01.064 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:00 smithi063 systemd[1]: Stopping Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:40:01.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:00 smithi193 ceph-mon[35141]: pgmap v27: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 417 op/s 2024-05-09T06:40:01.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:00 smithi193 ceph-mon[35141]: overall HEALTH_OK 2024-05-09T06:40:01.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:01 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063[27167]: 2024-05-09T06:40:01.151+0000 7f6da38fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi063 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:40:01.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:01 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063[27167]: 2024-05-09T06:40:01.151+0000 7f6da38fa700 -1 mon.smithi063@0(leader) e2 *** Got Signal Terminated *** 2024-05-09T06:40:01.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:01 smithi063 podman[92571]: 2024-05-09 06:40:01.426009014 +0000 UTC m=+0.419493281 container died 7d88727e4bbbd85e71c5790a652c29c6566266eae7cb5bedc66f7483ccdb0c5f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=reef-1758e8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_CLEAN=True) 2024-05-09T06:40:02.079 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:01 smithi063 podman[92571]: 2024-05-09 06:40:01.805513301 +0000 UTC m=+0.798997555 container cleanup 7d88727e4bbbd85e71c5790a652c29c6566266eae7cb5bedc66f7483ccdb0c5f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=reef-1758e8e, ceph=True) 2024-05-09T06:40:02.079 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:01 smithi063 bash[92571]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063 2024-05-09T06:40:02.081 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-05-09T06:40:02.082 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T06:40:02.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:02 smithi063 podman[92597]: 2024-05-09 06:40:02.113605434 +0000 UTC m=+0.682439043 container remove 7d88727e4bbbd85e71c5790a652c29c6566266eae7cb5bedc66f7483ccdb0c5f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-1758e8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:40:02.335 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:02 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063.service: Deactivated successfully. 2024-05-09T06:40:02.335 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:02 smithi063 systemd[1]: Stopped Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:40:02.335 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:02 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063.service: Consumed 13.071s CPU time. 2024-05-09T06:40:02.470 DEBUG:teuthology.parallel:result is None 2024-05-09T06:40:02.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:02 smithi063 systemd[1]: Starting Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:40:03.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:02 smithi063 podman[92746]: 2024-05-09 06:40:02.711039503 +0000 UTC m=+0.020184514 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:40:03.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:02 smithi063 podman[92746]: 2024-05-09 06:40:02.830088944 +0000 UTC m=+0.139233956 container create 8b7491de89ee74a84df6f094de48da563508e1e520281437387c946370db3770 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:40:03.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 podman[92746]: 2024-05-09 06:40:03.066852776 +0000 UTC m=+0.375997782 container init 8b7491de89ee74a84df6f094de48da563508e1e520281437387c946370db3770 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-09T06:40:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 podman[92746]: 2024-05-09 06:40:03.073596394 +0000 UTC m=+0.382741395 container start 8b7491de89ee74a84df6f094de48da563508e1e520281437387c946370db3770 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc) 2024-05-09T06:40:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T06:40:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev), process ceph-mon, pid 2 2024-05-09T06:40:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: pidfile_write: ignore empty --pid-file 2024-05-09T06:40:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: load: jerasure load: lrc 2024-05-09T06:40:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: RocksDB version: 7.9.2 2024-05-09T06:40:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Git sha 0 2024-05-09T06:40:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Compile date 2024-05-07 19:19:25 2024-05-09T06:40:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: DB SUMMARY 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: DB Session ID: FYPG0QKZQOI18614VF09 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: CURRENT file: CURRENT 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1254 Bytes 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi063/store.db dir, Total Num: 1, files: 000029.sst 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi063/store.db: 000027.log size: 2743407 ; 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.error_if_exists: 0 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.create_if_missing: 0 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.paranoid_checks: 1 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T06:40:03.357 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.env: 0x55bb94803c60 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.info_log: 0x55bb96bad980 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.statistics: (nil) 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.use_fsync: 0 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_log_file_size: 0 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T06:40:03.358 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.allow_fallocate: 1 2024-05-09T06:40:03.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T06:40:03.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T06:40:03.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.use_direct_reads: 0 2024-05-09T06:40:03.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T06:40:03.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T06:40:03.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.db_log_dir: 2024-05-09T06:40:03.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.wal_dir: 2024-05-09T06:40:03.360 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.write_buffer_manager: 0x55bb96b801e0 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T06:40:03.361 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.unordered_write: 0 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.row_cache: None 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.wal_filter: None 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.two_write_queues: 0 2024-05-09T06:40:03.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.wal_compression: 0 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.atomic_flush: 0 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.log_readahead_size: 0 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T06:40:03.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_background_jobs: 2 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_background_compactions: -1 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_subcompactions: 1 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_open_files: -1 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T06:40:03.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_background_flushes: -1 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Compression algorithms supported: 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: kZSTD supported: 0 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: kXpressCompression supported: 0 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: kZlibCompression supported: 1 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: kSnappyCompression supported: 1 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: kLZ4Compression supported: 1 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: kBZip2Compression supported: 0 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000015 2024-05-09T06:40:03.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.merge_operator: 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_filter: None 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bb96bada80) 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks: 1 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_top_level_index_and_filter: 1 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_type: 0 2024-05-09T06:40:03.366 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_index_type: 0 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_shortening: 1 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: checksum: 4 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: no_block_cache: 0 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache: 0x55bb96bb1090 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_name: BinnedLRUCache 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_options: 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: capacity : 536870912 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_shard_bits : 4 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: strict_capacity_limit : 0 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: high_pri_pool_ratio: 0.000 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_compressed: (nil) 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: persistent_cache: (nil) 2024-05-09T06:40:03.367 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size: 4096 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size_deviation: 10 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_restart_interval: 16 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_block_restart_interval: 1 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: metadata_block_size: 4096 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: partition_filters: 0 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: use_delta_encoding: 1 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: filter_policy: bloomfilter 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: whole_key_filtering: 1 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: verify_compression: 0 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: read_amp_bytes_per_bit: 0 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: format_version: 5 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_index_compression: 1 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_align: 0 2024-05-09T06:40:03.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_auto_readahead_size: 262144 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: prepopulate_block_cache: 0 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: initial_auto_readahead_size: 8192 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression: NoCompression 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.num_levels: 7 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T06:40:03.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T06:40:03.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T06:40:03.371 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T06:40:03.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.table_properties_collectors: 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.inplace_update_support: 0 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.bloom_locality: 0 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.max_successive_merges: 0 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T06:40:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.ttl: 2592000 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.enable_blob_files: false 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.min_blob_size: 0 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T06:40:03.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 31, last_sequence is 10419, log_number is 27,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 27 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 27 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3aeef634-646c-4050-9255-05b80ab7fcac 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236803123596, "job": 1, "event": "recovery_started", "wal_files": [27]} 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #27 mode 2 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236803140258, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 32, "file_size": 2501496, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10420, "largest_seqno": 11553, "table_properties": {"data_size": 2495334, "index_size": 3799, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1349, "raw_key_size": 11919, "raw_average_key_size": 23, "raw_value_size": 2483918, "raw_average_value_size": 4823, "num_data_blocks": 173, "num_entries": 515, "num_filter_entries": 515, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715236803, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3aeef634-646c-4050-9255-05b80ab7fcac", "db_session_id": "FYPG0QKZQOI18614VF09", "orig_file_number": 32, "seqno_to_time_mapping": "N/A"}} 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236803140914, "job": 1, "event": "recovery_finished"} 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/version_set.cc:5047] Creating manifest 34 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi063/store.db/000027.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bb96ca2000 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: DB pointer 0x55bb96c8c000 2024-05-09T06:40:03.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** DB Stats ** 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T06:40:03.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L0 1/0 2.39 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 406.0 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L6 1/0 5.35 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Sum 2/0 7.73 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 406.0 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 406.0 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 406.0 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T06:40:03.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative compaction: 0.00 GB write, 75.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval compaction: 0.00 GB write, 75.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Block cache BinnedLRUCache@0x55bb96bb1090#2 capacity: 512.00 MB usage: 13.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,9.44 KB,0.00180006%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: starting mon.smithi063 rank 0 at public addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] at bind addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi063 fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: mon.smithi063@-1(???) e2 preinit fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: mon.smithi063@-1(???).mds e15 new map 2024-05-09T06:40:03.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: mon.smithi063@-1(???).mds e15 print_map 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: e15 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: legacy client fscid: 1 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Filesystem 'cephfs' (1) 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: fs_name cephfs 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: epoch 15 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:40:03.379 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: tableserver 0 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: root 0 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: session_timeout 60 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: session_autoclose 300 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_file_size 1099511627776 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_xattr_size 65536 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: required_client_features {} 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: last_failure 0 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: last_failure_osd_epoch 47 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_mds 1 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: in 0 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: up {0=24277} 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: failed 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: damaged 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: stopped 2024-05-09T06:40:03.380 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_pools [3] 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: metadata_pool 2 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: inline_data disabled 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: balancer 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: bal_rank_mask -1 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: standby_count_wanted 1 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: qdb_cluster leader: 0 members: 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Standby daemons: 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:03.381 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: mon.smithi063@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-09T06:40:03.382 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: mon.smithi063@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-09T06:40:03.382 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: mon.smithi063@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-09T06:40:03.382 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: mon.smithi063@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-09T06:40:03.382 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 ceph-mon[92765]: mon.smithi063@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-05-09T06:40:03.382 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 bash[92746]: 8b7491de89ee74a84df6f094de48da563508e1e520281437387c946370db3770 2024-05-09T06:40:03.382 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:03 smithi063 systemd[1]: Started Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:40:04.763 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:40:04.763 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:40:04.763 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: pgmap v28: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1008 KiB/s wr, 287 op/s 2024-05-09T06:40:04.763 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: pgmap v29: 65 pgs: 65 active+clean; 226 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 446 op/s 2024-05-09T06:40:04.763 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: mon.smithi063 calling monitor election 2024-05-09T06:40:04.763 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: mon.smithi063 is new leader, mons smithi063,smithi193 in quorum (ranks 0,1) 2024-05-09T06:40:04.763 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: monmap epoch 2 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: last_changed 2024-05-09T06:30:43.500943+0000 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: created 2024-05-09T06:27:48.192363+0000 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: min_mon_release 18 (reef) 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: election_strategy: 1 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.smithi063 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: osdmap e49: 6 total, 6 up, 6 in 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: mgrmap e32: smithi063.egfguy(active, since 52s), standbys: smithi193.mmmqzh 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: overall HEALTH_OK 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: from='mgr.24513 ' entity='' 2024-05-09T06:40:04.764 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:04 smithi193 ceph-mon[35141]: mgrmap e33: smithi063.egfguy(active, since 52s), standbys: smithi193.mmmqzh 2024-05-09T06:40:04.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:40:04.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: from='mgr.24513 172.21.15.63:0/918939029' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:40:04.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: pgmap v28: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1008 KiB/s wr, 287 op/s 2024-05-09T06:40:04.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: pgmap v29: 65 pgs: 65 active+clean; 226 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 446 op/s 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: mon.smithi063 calling monitor election 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: mon.smithi063 is new leader, mons smithi063,smithi193 in quorum (ranks 0,1) 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: monmap epoch 2 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: last_changed 2024-05-09T06:30:43.500943+0000 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: created 2024-05-09T06:27:48.192363+0000 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: min_mon_release 18 (reef) 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: election_strategy: 1 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.smithi063 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: osdmap e49: 6 total, 6 up, 6 in 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: mgrmap e32: smithi063.egfguy(active, since 52s), standbys: smithi193.mmmqzh 2024-05-09T06:40:04.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: overall HEALTH_OK 2024-05-09T06:40:04.778 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: from='mgr.24513 ' entity='' 2024-05-09T06:40:04.778 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:04 smithi063 ceph-mon[92765]: mgrmap e33: smithi063.egfguy(active, since 52s), standbys: smithi193.mmmqzh 2024-05-09T06:40:05.692 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-05-09T06:40:05.693 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-05-09T06:40:06.001 DEBUG:teuthology.parallel:result is None 2024-05-09T06:40:06.002 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T06:40:12.081 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:11 smithi063 ceph-mon[92765]: Standby manager daemon smithi193.mmmqzh restarted 2024-05-09T06:40:12.082 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:11 smithi063 ceph-mon[92765]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:40:12.082 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:11 smithi063 ceph-mon[92765]: from='mgr.? 172.21.15.193:0/3906594631' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:40:12.082 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:11 smithi063 ceph-mon[92765]: from='mgr.? 172.21.15.193:0/3906594631' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:40:12.082 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:11 smithi063 ceph-mon[92765]: from='mgr.? 172.21.15.193:0/3906594631' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:40:12.082 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:11 smithi063 ceph-mon[92765]: from='mgr.? 172.21.15.193:0/3906594631' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:40:12.083 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T06:40:12.083 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-05-09T06:40:12.114 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-05-09T06:40:12.114 DEBUG:teuthology.parallel:result is None 2024-05-09T06:40:12.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:11 smithi193 ceph-mon[35141]: Standby manager daemon smithi193.mmmqzh restarted 2024-05-09T06:40:12.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:11 smithi193 ceph-mon[35141]: Standby manager daemon smithi193.mmmqzh started 2024-05-09T06:40:12.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:11 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/3906594631' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/crt"}]: dispatch 2024-05-09T06:40:12.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:11 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/3906594631' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T06:40:12.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:11 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/3906594631' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mmmqzh/key"}]: dispatch 2024-05-09T06:40:12.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:11 smithi193 ceph-mon[35141]: from='mgr.? 172.21.15.193:0/3906594631' entity='mgr.smithi193.mmmqzh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T06:40:13.079 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: mgrmap e34: smithi063.egfguy(active, since 60s), standbys: smithi193.mmmqzh 2024-05-09T06:40:13.079 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: Active manager daemon smithi063.egfguy restarted 2024-05-09T06:40:13.079 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: Activating manager daemon smithi063.egfguy 2024-05-09T06:40:13.079 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: osdmap e50: 6 total, 6 up, 6 in 2024-05-09T06:40:13.079 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: mgrmap e35: smithi063.egfguy(active, starting, since 0.00725176s), standbys: smithi193.mmmqzh 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mmmqzh", "id": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:40:13.080 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:40:13.081 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:40:13.081 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:40:13.081 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: Manager daemon smithi063.egfguy is now available 2024-05-09T06:40:13.081 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:40:13.081 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:40:13.081 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:40:13.081 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/trash_purge_schedule"}]: dispatch 2024-05-09T06:40:13.246 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: mgrmap e34: smithi063.egfguy(active, since 60s), standbys: smithi193.mmmqzh 2024-05-09T06:40:13.246 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: Active manager daemon smithi063.egfguy restarted 2024-05-09T06:40:13.246 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: Activating manager daemon smithi063.egfguy 2024-05-09T06:40:13.246 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: osdmap e50: 6 total, 6 up, 6 in 2024-05-09T06:40:13.246 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: mgrmap e35: smithi063.egfguy(active, starting, since 0.00725176s), standbys: smithi193.mmmqzh 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi063.egfguy", "id": "smithi063.egfguy"}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mmmqzh", "id": "smithi193.mmmqzh"}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T06:40:13.247 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T06:40:13.248 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T06:40:13.248 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: Manager daemon smithi063.egfguy is now available 2024-05-09T06:40:13.248 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:40:13.248 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:40:13.248 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/mirror_snapshot_schedule"}]: dispatch 2024-05-09T06:40:13.248 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:12 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.egfguy/trash_purge_schedule"}]: dispatch 2024-05-09T06:40:14.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:14 smithi063 ceph-mon[92765]: mgrmap e36: smithi063.egfguy(active, since 1.01208s), standbys: smithi193.mmmqzh 2024-05-09T06:40:14.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:14 smithi063 ceph-mon[92765]: pgmap v3: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:40:14.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:14 smithi193 ceph-mon[35141]: mgrmap e36: smithi063.egfguy(active, since 1.01208s), standbys: smithi193.mmmqzh 2024-05-09T06:40:14.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:14 smithi193 ceph-mon[35141]: pgmap v3: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:40:15.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:15 smithi063 ceph-mon[92765]: mgrmap e37: smithi063.egfguy(active, since 2s), standbys: smithi193.mmmqzh 2024-05-09T06:40:15.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:15 smithi063 ceph-mon[92765]: [09/May/2024:06:40:14] ENGINE Bus STARTING 2024-05-09T06:40:15.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:15 smithi063 ceph-mon[92765]: [09/May/2024:06:40:14] ENGINE Serving on http://172.21.15.63:8765 2024-05-09T06:40:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:15 smithi193 ceph-mon[35141]: mgrmap e37: smithi063.egfguy(active, since 2s), standbys: smithi193.mmmqzh 2024-05-09T06:40:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:15 smithi193 ceph-mon[35141]: [09/May/2024:06:40:14] ENGINE Bus STARTING 2024-05-09T06:40:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:15 smithi193 ceph-mon[35141]: [09/May/2024:06:40:14] ENGINE Serving on http://172.21.15.63:8765 2024-05-09T06:40:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:16 smithi193 ceph-mon[35141]: [09/May/2024:06:40:14] ENGINE Serving on https://172.21.15.63:7150 2024-05-09T06:40:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:16 smithi193 ceph-mon[35141]: [09/May/2024:06:40:14] ENGINE Bus STARTED 2024-05-09T06:40:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:16 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:16 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:16.833 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:16 smithi063 ceph-mon[92765]: [09/May/2024:06:40:14] ENGINE Serving on https://172.21.15.63:7150 2024-05-09T06:40:16.833 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:16 smithi063 ceph-mon[92765]: [09/May/2024:06:40:14] ENGINE Bus STARTED 2024-05-09T06:40:16.833 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:16 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:16.833 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:16 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:17.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:17 smithi063 ceph-mon[92765]: pgmap v5: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:40:17.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:17 smithi063 ceph-mon[92765]: mgrmap e38: smithi063.egfguy(active, since 4s), standbys: smithi193.mmmqzh 2024-05-09T06:40:17.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:17 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:17.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:17 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:17.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:17 smithi193 ceph-mon[35141]: pgmap v5: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:40:17.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:17 smithi193 ceph-mon[35141]: mgrmap e38: smithi063.egfguy(active, since 4s), standbys: smithi193.mmmqzh 2024-05-09T06:40:17.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:17 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:17.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:17 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:18.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:18 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:18.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:18 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:18.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:18 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:18.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:18 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:19.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:19 smithi063 ceph-mon[92765]: pgmap v6: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:40:19.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:19 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:19.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:19 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:19.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:19 smithi193 ceph-mon[35141]: pgmap v6: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:40:19.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:19 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:19.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:19 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:21.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:21.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:21.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:21.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:40:21.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-09T06:40:21.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-09T06:40:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: pgmap v7: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1 KiB/s wr, 19 op/s 2024-05-09T06:40:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:40:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:40:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:40:21.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:21 smithi063 ceph-mon[92765]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:40:21.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:21.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:21.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:21.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:40:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-09T06:40:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-09T06:40:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: pgmap v7: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1 KiB/s wr, 19 op/s 2024-05-09T06:40:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:40:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T06:40:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:40:21.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:21 smithi193 ceph-mon[35141]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:40:22.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:22 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-09T06:40:22.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: Updating smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:40:22.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: Updating smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T06:40:22.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:22.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:40:22.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:40:22.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-09T06:40:23.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:23 smithi193 ceph-mon[35141]: pgmap v8: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 796 B/s wr, 15 op/s 2024-05-09T06:40:23.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:23 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:23.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:23 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:40:23.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:23 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:40:23.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:23 smithi193 ceph-mon[35141]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:23 smithi063 ceph-mon[92765]: pgmap v8: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 796 B/s wr, 15 op/s 2024-05-09T06:40:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:40:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:40:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:24.308 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:24 smithi193 ceph-mon[35141]: Upgrade: Updating mon.smithi193 2024-05-09T06:40:24.308 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:24 smithi193 ceph-mon[35141]: Deploying daemon mon.smithi193 on smithi193 2024-05-09T06:40:24.559 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:24 smithi063 ceph-mon[92765]: Upgrade: Updating mon.smithi193 2024-05-09T06:40:24.559 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:24 smithi063 ceph-mon[92765]: Deploying daemon mon.smithi193 on smithi193 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 -- 172.21.15.63:0/1405973388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 msgr2=0x7f0e18101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 --2- 172.21.15.63:0/1405973388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 0x7f0e18101200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009910 tx=0x7f0e0c009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 -- 172.21.15.63:0/1405973388 shutdown_connections 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 --2- 172.21.15.63:0/1405973388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 0x7f0e18101200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 --2- 172.21.15.63:0/1405973388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e180ffba0 0x7f0e180fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 -- 172.21.15.63:0/1405973388 >> 172.21.15.63:0/1405973388 conn(0x7f0e180fb120 msgr2=0x7f0e180fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 -- 172.21.15.63:0/1405973388 shutdown_connections 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 -- 172.21.15.63:0/1405973388 wait complete. 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 Processor -- start 2024-05-09T06:40:24.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.579+0000 7f0e20026700 1 -- start start 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e20026700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e180ffba0 0x7f0e180690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e20026700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 0x7f0e1819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e20026700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e181971d0 con 0x7f0e18100da0 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e20026700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1819dd30 con 0x7f0e180ffba0 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e1d5c1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 0x7f0e1819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e1d5c1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 0x7f0e1819d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49518/0 (socket says 172.21.15.63:49518) 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e1d5c1700 1 -- 172.21.15.63:0/604149950 learned_addr learned my addr 172.21.15.63:0/604149950 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e1ddc2700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e180ffba0 0x7f0e180690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e1d5c1700 1 -- 172.21.15.63:0/604149950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e180ffba0 msgr2=0x7f0e180690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e1d5c1700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e180ffba0 0x7f0e180690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:24.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e1d5c1700 1 -- 172.21.15.63:0/604149950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c0095f0 con 0x7f0e18100da0 2024-05-09T06:40:24.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.580+0000 7f0e1d5c1700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 0x7f0e1819d7f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000c00 tx=0x7f0e0c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:24.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.581+0000 7f0e16ffd700 1 -- 172.21.15.63:0/604149950 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c01d070 con 0x7f0e18100da0 2024-05-09T06:40:24.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.581+0000 7f0e16ffd700 1 -- 172.21.15.63:0/604149950 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e0c0052a0 con 0x7f0e18100da0 2024-05-09T06:40:24.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.581+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1819df30 con 0x7f0e18100da0 2024-05-09T06:40:24.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.581+0000 7f0e16ffd700 1 -- 172.21.15.63:0/604149950 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c017400 con 0x7f0e18100da0 2024-05-09T06:40:24.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.581+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1819e3f0 con 0x7f0e18100da0 2024-05-09T06:40:24.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.582+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1818f720 con 0x7f0e18100da0 2024-05-09T06:40:24.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.582+0000 7f0e16ffd700 1 -- 172.21.15.63:0/604149950 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0e0c003810 con 0x7f0e18100da0 2024-05-09T06:40:24.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.583+0000 7f0e16ffd700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0e0006e340 0x7f0e00070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:24.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.583+0000 7f0e16ffd700 1 -- 172.21.15.63:0/604149950 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0e0c08f7f0 con 0x7f0e18100da0 2024-05-09T06:40:24.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.583+0000 7f0e1ddc2700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0e0006e340 0x7f0e00070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:24.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.583+0000 7f0e1ddc2700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0e0006e340 0x7f0e00070800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e18100c00 tx=0x7f0e0800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:24.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.588+0000 7f0e16ffd700 1 -- 172.21.15.63:0/604149950 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0e0c05c430 con 0x7f0e18100da0 2024-05-09T06:40:24.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.813+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e1802ce50 con 0x7f0e0006e340 2024-05-09T06:40:24.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.814+0000 7f0e16ffd700 1 -- 172.21.15.63:0/604149950 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f0e1802ce50 con 0x7f0e0006e340 2024-05-09T06:40:24.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.816+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0e0006e340 msgr2=0x7f0e00070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:24.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.816+0000 7f0e20026700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0e0006e340 0x7f0e00070800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e18100c00 tx=0x7f0e0800b040 comp rx=0 tx=0).stop 2024-05-09T06:40:24.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.816+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 msgr2=0x7f0e1819d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:24.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.816+0000 7f0e20026700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 0x7f0e1819d7f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000c00 tx=0x7f0e0c005f00 comp rx=0 tx=0).stop 2024-05-09T06:40:24.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.816+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 shutdown_connections 2024-05-09T06:40:24.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.817+0000 7f0e20026700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0e0006e340 0x7f0e00070800 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:24.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.817+0000 7f0e20026700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e180ffba0 0x7f0e180690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:24.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.817+0000 7f0e20026700 1 --2- 172.21.15.63:0/604149950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e18100da0 0x7f0e1819d7f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:24.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.817+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 >> 172.21.15.63:0/604149950 conn(0x7f0e180fb120 msgr2=0x7f0e181043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:24.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.817+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 shutdown_connections 2024-05-09T06:40:24.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.817+0000 7f0e20026700 1 -- 172.21.15.63:0/604149950 wait complete. 2024-05-09T06:40:24.807 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:40:24.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:24 smithi193 systemd[1]: Stopping Ceph mon.smithi193 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:40:24.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:24 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193[35125]: 2024-05-09T06:40:24.858+0000 7f480f371700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi193 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:40:24.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:24 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193[35125]: 2024-05-09T06:40:24.858+0000 7f480f371700 -1 mon.smithi193@1(peon) e2 *** Got Signal Terminated *** 2024-05-09T06:40:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.930+0000 7f59a104e700 1 -- 172.21.15.63:0/2655806609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 msgr2=0x7f599c1064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.930+0000 7f59a104e700 1 --2- 172.21.15.63:0/2655806609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 0x7f599c1064b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5990009910 tx=0x7f5990009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.930+0000 7f59a104e700 1 -- 172.21.15.63:0/2655806609 shutdown_connections 2024-05-09T06:40:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.930+0000 7f59a104e700 1 --2- 172.21.15.63:0/2655806609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 0x7f599c1064b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.930+0000 7f59a104e700 1 --2- 172.21.15.63:0/2655806609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f599c101790 0x7f599c103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.930+0000 7f59a104e700 1 -- 172.21.15.63:0/2655806609 >> 172.21.15.63:0/2655806609 conn(0x7f599c0fb120 msgr2=0x7f599c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:24.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.930+0000 7f59a104e700 1 -- 172.21.15.63:0/2655806609 shutdown_connections 2024-05-09T06:40:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.930+0000 7f59a104e700 1 -- 172.21.15.63:0/2655806609 wait complete. 2024-05-09T06:40:24.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.930+0000 7f59a104e700 1 Processor -- start 2024-05-09T06:40:24.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f59a104e700 1 -- start start 2024-05-09T06:40:24.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f59a104e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f599c101790 0x7f599c1010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:24.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f59a104e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 0x7f599c0ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:24.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f59a104e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0ffc80 con 0x7f599c1040c0 2024-05-09T06:40:24.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f59a104e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0ffdc0 con 0x7f599c101790 2024-05-09T06:40:24.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f599a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 0x7f599c0ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f599ad9d700 1 -- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f599c101790 msgr2=0x7f599c1010a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.193:3300/0 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f599a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 0x7f599c0ff740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47544/0 (socket says 172.21.15.63:47544) 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f599ad9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f599c101790 0x7f599c1010a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f599a59c700 1 -- 172.21.15.63:0/2082915231 learned_addr learned my addr 172.21.15.63:0/2082915231 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f599a59c700 1 -- 172.21.15.63:0/2082915231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f599c101790 msgr2=0x7f599c1010a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f599a59c700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f599c101790 0x7f599c1010a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.931+0000 7f599a59c700 1 -- 172.21.15.63:0/2082915231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59900095f0 con 0x7f599c1040c0 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.932+0000 7f599a59c700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 0x7f599c0ff740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5990005090 tx=0x7f599000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.932+0000 7f5983fff700 1 -- 172.21.15.63:0/2082915231 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f599001d070 con 0x7f599c1040c0 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.932+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c100040 con 0x7f599c1040c0 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.932+0000 7f5983fff700 1 -- 172.21.15.63:0/2082915231 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59900048e0 con 0x7f599c1040c0 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.932+0000 7f5983fff700 1 -- 172.21.15.63:0/2082915231 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5990003c10 con 0x7f599c1040c0 2024-05-09T06:40:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.932+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c100550 con 0x7f599c1040c0 2024-05-09T06:40:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.933+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c04ebb0 con 0x7f599c1040c0 2024-05-09T06:40:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.933+0000 7f5983fff700 1 -- 172.21.15.63:0/2082915231 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f5990004a50 con 0x7f599c1040c0 2024-05-09T06:40:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.934+0000 7f5983fff700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f598406e2e0 0x7f59840707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.934+0000 7f599ad9d700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f598406e2e0 0x7f59840707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.934+0000 7f5983fff700 1 -- 172.21.15.63:0/2082915231 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f599008f9d0 con 0x7f599c1040c0 2024-05-09T06:40:24.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.934+0000 7f599ad9d700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f598406e2e0 0x7f59840707a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f598c001050 tx=0x7f598c006df0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:24.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:24.938+0000 7f5983fff700 1 -- 172.21.15.63:0/2082915231 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f599005c610 con 0x7f599c1040c0 2024-05-09T06:40:25.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.168+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f599c104b20 con 0x7f598406e2e0 2024-05-09T06:40:25.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.169+0000 7f5983fff700 1 -- 172.21.15.63:0/2082915231 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f599c104b20 con 0x7f598406e2e0 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.171+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f598406e2e0 msgr2=0x7f59840707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.171+0000 7f59a104e700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f598406e2e0 0x7f59840707a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f598c001050 tx=0x7f598c006df0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.171+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 msgr2=0x7f599c0ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.171+0000 7f59a104e700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 0x7f599c0ff740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5990005090 tx=0x7f599000be20 comp rx=0 tx=0).stop 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.171+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 shutdown_connections 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.171+0000 7f59a104e700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f598406e2e0 0x7f59840707a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.171+0000 7f59a104e700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f599c101790 0x7f599c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.171+0000 7f59a104e700 1 --2- 172.21.15.63:0/2082915231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f599c1040c0 0x7f599c0ff740 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.171+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 >> 172.21.15.63:0/2082915231 conn(0x7f599c0fb120 msgr2=0x7f599c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:25.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.172+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 shutdown_connections 2024-05-09T06:40:25.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.172+0000 7f59a104e700 1 -- 172.21.15.63:0/2082915231 wait complete. 2024-05-09T06:40:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.280+0000 7efc29ef1700 1 -- 172.21.15.63:0/3311355889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc240fef40 msgr2=0x7efc240ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.280+0000 7efc29ef1700 1 --2- 172.21.15.63:0/3311355889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc240fef40 0x7efc240ff360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efc14009910 tx=0x7efc14009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.280+0000 7efc29ef1700 1 -- 172.21.15.63:0/3311355889 shutdown_connections 2024-05-09T06:40:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.280+0000 7efc29ef1700 1 --2- 172.21.15.63:0/3311355889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc24101fc0 0x7efc24102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.280+0000 7efc29ef1700 1 --2- 172.21.15.63:0/3311355889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc240fef40 0x7efc240ff360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.280+0000 7efc29ef1700 1 -- 172.21.15.63:0/3311355889 >> 172.21.15.63:0/3311355889 conn(0x7efc240fa8a0 msgr2=0x7efc240fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:25.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.280+0000 7efc29ef1700 1 -- 172.21.15.63:0/3311355889 shutdown_connections 2024-05-09T06:40:25.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.280+0000 7efc29ef1700 1 -- 172.21.15.63:0/3311355889 wait complete. 2024-05-09T06:40:25.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc29ef1700 1 Processor -- start 2024-05-09T06:40:25.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc29ef1700 1 -- start start 2024-05-09T06:40:25.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc29ef1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc240fef40 0x7efc24196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc29ef1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc24101fc0 0x7efc24196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc29ef1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc24197280 con 0x7efc24101fc0 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc29ef1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc241973c0 con 0x7efc240fef40 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc237fe700 1 -- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc240fef40 msgr2=0x7efc24196720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.193:3300/0 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc237fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc240fef40 0x7efc24196720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc1affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc24101fc0 0x7efc24196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc1affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc24101fc0 0x7efc24196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47570/0 (socket says 172.21.15.63:47570) 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.281+0000 7efc1affd700 1 -- 172.21.15.63:0/308055922 learned_addr learned my addr 172.21.15.63:0/308055922 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.282+0000 7efc1affd700 1 -- 172.21.15.63:0/308055922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc240fef40 msgr2=0x7efc24196720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.282+0000 7efc1affd700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc240fef40 0x7efc24196720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.282+0000 7efc1affd700 1 -- 172.21.15.63:0/308055922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140095f0 con 0x7efc24101fc0 2024-05-09T06:40:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.282+0000 7efc1affd700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc24101fc0 0x7efc24196c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00b700 tx=0x7efc0c00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.282+0000 7efc217fa700 1 -- 172.21.15.63:0/308055922 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0c011800 con 0x7efc24101fc0 2024-05-09T06:40:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.282+0000 7efc217fa700 1 -- 172.21.15.63:0/308055922 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc0c011e20 con 0x7efc24101fc0 2024-05-09T06:40:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.282+0000 7efc217fa700 1 -- 172.21.15.63:0/308055922 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0c00f2e0 con 0x7efc24101fc0 2024-05-09T06:40:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.282+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2419be20 con 0x7efc24101fc0 2024-05-09T06:40:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.282+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2419c370 con 0x7efc24101fc0 2024-05-09T06:40:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.283+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc24190880 con 0x7efc24101fc0 2024-05-09T06:40:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.284+0000 7efc217fa700 1 -- 172.21.15.63:0/308055922 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7efc0c010380 con 0x7efc24101fc0 2024-05-09T06:40:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.284+0000 7efc217fa700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efc1006e2e0 0x7efc100707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.284+0000 7efc217fa700 1 -- 172.21.15.63:0/308055922 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7efc0c08df00 con 0x7efc24101fc0 2024-05-09T06:40:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.284+0000 7efc237fe700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efc1006e2e0 0x7efc100707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.285+0000 7efc237fe700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efc1006e2e0 0x7efc100707a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efc14005090 tx=0x7efc14009fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:25.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.289+0000 7efc217fa700 1 -- 172.21.15.63:0/308055922 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efc0c05ac50 con 0x7efc24101fc0 2024-05-09T06:40:25.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:25 smithi193 podman[81501]: 2024-05-09 06:40:25.094291764 +0000 UTC m=+0.326779955 container died bb042f0315eb935cfb36dca64555b1b7784f5bbb2412aeb9d906c2e1f7600a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=reef-1758e8e, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-09T06:40:25.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.502+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efc2402cf00 con 0x7efc1006e2e0 2024-05-09T06:40:25.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.513+0000 7efc217fa700 1 -- 172.21.15.63:0/308055922 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7efc2402cf00 con 0x7efc1006e2e0 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (9m) 7s ago 10m 25.1M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (11m) 7s ago 11m 7852k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (10m) 10s ago 10m 7856k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (11m) 7s ago 11m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (9m) 10s ago 9m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (9m) 7s ago 10m 85.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (5m) 7s ago 5m 15.0M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (5m) 7s ago 5m 90.1M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (5m) 10s ago 5m 142M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (5m) 10s ago 5m 16.2M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (86s) 7s ago 12m 476M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:40:25.494 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (41s) 10s ago 9m 448M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (22s) 7s ago 12m 29.1M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (9m) 10s ago 9m 49.7M 2048M 18.2.2-1297-g1758e8ea 196422f1b641 bb042f0315eb 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (10m) 7s ago 10m 15.7M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (9m) 10s ago 9m 16.2M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (8m) 7s ago 8m 229M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (8m) 7s ago 8m 244M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (8m) 7s ago 8m 208M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (7m) 10s ago 7m 291M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (7m) 10s ago 7m 258M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (6m) 10s ago 6m 233M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:40:25.495 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (51s) 7s ago 10m 45.7M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efc1006e2e0 msgr2=0x7efc100707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efc1006e2e0 0x7efc100707a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efc14005090 tx=0x7efc14009fd0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc24101fc0 msgr2=0x7efc24196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc24101fc0 0x7efc24196c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00b700 tx=0x7efc0c00bac0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 shutdown_connections 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efc1006e2e0 0x7efc100707a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc240fef40 0x7efc24196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 --2- 172.21.15.63:0/308055922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc24101fc0 0x7efc24196c60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 >> 172.21.15.63:0/308055922 conn(0x7efc240fa8a0 msgr2=0x7efc24105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:25.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 shutdown_connections 2024-05-09T06:40:25.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.516+0000 7efc29ef1700 1 -- 172.21.15.63:0/308055922 wait complete. 2024-05-09T06:40:25.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.622+0000 7fb2b244f700 1 -- 172.21.15.63:0/2127453178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 msgr2=0x7fb2ac1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:25.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.622+0000 7fb2b244f700 1 --2- 172.21.15.63:0/2127453178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 0x7fb2ac1022b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009910 tx=0x7fb2a0009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:25.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.623+0000 7fb2b244f700 1 -- 172.21.15.63:0/2127453178 shutdown_connections 2024-05-09T06:40:25.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.623+0000 7fb2b244f700 1 --2- 172.21.15.63:0/2127453178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 0x7fb2ac1022b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.623+0000 7fb2b244f700 1 --2- 172.21.15.63:0/2127453178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ac100c50 0x7fb2ac101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.623+0000 7fb2b244f700 1 -- 172.21.15.63:0/2127453178 >> 172.21.15.63:0/2127453178 conn(0x7fb2ac0fc210 msgr2=0x7fb2ac0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:25.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.623+0000 7fb2b244f700 1 -- 172.21.15.63:0/2127453178 shutdown_connections 2024-05-09T06:40:25.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.623+0000 7fb2b244f700 1 -- 172.21.15.63:0/2127453178 wait complete. 2024-05-09T06:40:25.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.623+0000 7fb2b244f700 1 Processor -- start 2024-05-09T06:40:25.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.623+0000 7fb2b244f700 1 -- start start 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2b244f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ac100c50 0x7fb2ac196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2b244f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 0x7fb2ac196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2b244f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac1971f0 con 0x7fb2ac101e50 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2b244f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac197330 con 0x7fb2ac100c50 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2ab7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 0x7fb2ac196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2abfff700 1 -- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ac100c50 msgr2=0x7fb2ac196690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.193:3300/0 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ac100c50 0x7fb2ac196690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2ab7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 0x7fb2ac196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47590/0 (socket says 172.21.15.63:47590) 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2ab7fe700 1 -- 172.21.15.63:0/3871596644 learned_addr learned my addr 172.21.15.63:0/3871596644 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2ab7fe700 1 -- 172.21.15.63:0/3871596644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ac100c50 msgr2=0x7fb2ac196690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2ab7fe700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ac100c50 0x7fb2ac196690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2ab7fe700 1 -- 172.21.15.63:0/3871596644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00095f0 con 0x7fb2ac101e50 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2ab7fe700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 0x7fb2ac196bd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005090 tx=0x7fb2a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2a97fa700 1 -- 172.21.15.63:0/3871596644 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2a001d070 con 0x7fb2ac101e50 2024-05-09T06:40:25.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2a97fa700 1 -- 172.21.15.63:0/3871596644 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2a00048e0 con 0x7fb2ac101e50 2024-05-09T06:40:25.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2a97fa700 1 -- 172.21.15.63:0/3871596644 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2a0003c10 con 0x7fb2ac101e50 2024-05-09T06:40:25.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.624+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac19bd30 con 0x7fb2ac101e50 2024-05-09T06:40:25.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.625+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac19c240 con 0x7fb2ac101e50 2024-05-09T06:40:25.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.625+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac04ec20 con 0x7fb2ac101e50 2024-05-09T06:40:25.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.626+0000 7fb2a97fa700 1 -- 172.21.15.63:0/3871596644 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fb2a0004a50 con 0x7fb2ac101e50 2024-05-09T06:40:25.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.627+0000 7fb2a97fa700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb29406e2e0 0x7fb2940707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:25.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.627+0000 7fb2abfff700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb29406e2e0 0x7fb2940707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:25.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.627+0000 7fb2a97fa700 1 -- 172.21.15.63:0/3871596644 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb2a008f910 con 0x7fb2ac101e50 2024-05-09T06:40:25.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.627+0000 7fb2abfff700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb29406e2e0 0x7fb2940707a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb29c006fd0 tx=0x7fb29c006dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:25.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.632+0000 7fb2a97fa700 1 -- 172.21.15.63:0/3871596644 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb2ac04ec20 con 0x7fb2ac101e50 2024-05-09T06:40:25.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:25 smithi193 podman[81501]: 2024-05-09 06:40:25.48334759 +0000 UTC m=+0.715835768 container cleanup bb042f0315eb935cfb36dca64555b1b7784f5bbb2412aeb9d906c2e1f7600a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=reef-1758e8e) 2024-05-09T06:40:25.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:25 smithi193 bash[81501]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193 2024-05-09T06:40:25.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.926+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb2ac109910 con 0x7fb2ac101e50 2024-05-09T06:40:25.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.926+0000 7fb2a97fa700 1 -- 172.21.15.63:0/3871596644 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fb2a005c5d0 con 0x7fb2ac101e50 2024-05-09T06:40:25.906 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:40:25.906 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:40:25.906 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 1, 2024-05-09T06:40:25.906 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 1 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 11, 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 3 2024-05-09T06:40:25.907 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:40:25.908 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:40:25.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb29406e2e0 msgr2=0x7fb2940707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb29406e2e0 0x7fb2940707a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb29c006fd0 tx=0x7fb29c006dc0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 msgr2=0x7fb2ac196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 0x7fb2ac196bd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005090 tx=0x7fb2a000be20 comp rx=0 tx=0).stop 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 shutdown_connections 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb29406e2e0 0x7fb2940707a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2ac100c50 0x7fb2ac196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 --2- 172.21.15.63:0/3871596644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2ac101e50 0x7fb2ac196bd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 >> 172.21.15.63:0/3871596644 conn(0x7fb2ac0fc210 msgr2=0x7fb2ac105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 shutdown_connections 2024-05-09T06:40:25.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:25.929+0000 7fb2b244f700 1 -- 172.21.15.63:0/3871596644 wait complete. 2024-05-09T06:40:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.037+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/4196189956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 msgr2=0x7f4b58106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.037+0000 7f4b5e3ad700 1 --2- 172.21.15.63:0/4196189956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 0x7f4b58106400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009910 tx=0x7f4b4c009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.037+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/4196189956 shutdown_connections 2024-05-09T06:40:26.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.037+0000 7f4b5e3ad700 1 --2- 172.21.15.63:0/4196189956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 0x7f4b58106400 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.037+0000 7f4b5e3ad700 1 --2- 172.21.15.63:0/4196189956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b581016e0 0x7f4b58103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.037+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/4196189956 >> 172.21.15.63:0/4196189956 conn(0x7f4b580fb0f0 msgr2=0x7f4b580fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.037+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/4196189956 shutdown_connections 2024-05-09T06:40:26.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.037+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/4196189956 wait complete. 2024-05-09T06:40:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b5e3ad700 1 Processor -- start 2024-05-09T06:40:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b5e3ad700 1 -- start start 2024-05-09T06:40:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b5e3ad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b581016e0 0x7f4b58100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b5e3ad700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 0x7f4b580ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:26.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b5e3ad700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580ffbd0 con 0x7f4b58104010 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b5e3ad700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580ffd10 con 0x7f4b581016e0 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b57fff700 1 -- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b581016e0 msgr2=0x7f4b58100ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.193:3300/0 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b577fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 0x7f4b580ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b57fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b581016e0 0x7f4b58100ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b577fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 0x7f4b580ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47616/0 (socket says 172.21.15.63:47616) 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.038+0000 7f4b577fe700 1 -- 172.21.15.63:0/774946251 learned_addr learned my addr 172.21.15.63:0/774946251 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.039+0000 7f4b577fe700 1 -- 172.21.15.63:0/774946251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b581016e0 msgr2=0x7f4b58100ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.039+0000 7f4b577fe700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b581016e0 0x7f4b58100ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.039+0000 7f4b577fe700 1 -- 172.21.15.63:0/774946251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c0095f0 con 0x7f4b58104010 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.039+0000 7f4b577fe700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 0x7f4b580ff690 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c005090 tx=0x7f4b4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.039+0000 7f4b557fa700 1 -- 172.21.15.63:0/774946251 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b4c01d070 con 0x7f4b58104010 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.039+0000 7f4b557fa700 1 -- 172.21.15.63:0/774946251 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b4c0048e0 con 0x7f4b58104010 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.039+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b580fff90 con 0x7f4b58104010 2024-05-09T06:40:26.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.039+0000 7f4b557fa700 1 -- 172.21.15.63:0/774946251 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b4c003c10 con 0x7f4b58104010 2024-05-09T06:40:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.039+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b581004a0 con 0x7f4b58104010 2024-05-09T06:40:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.040+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5804ebb0 con 0x7f4b58104010 2024-05-09T06:40:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.041+0000 7f4b557fa700 1 -- 172.21.15.63:0/774946251 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f4b4c004a50 con 0x7f4b58104010 2024-05-09T06:40:26.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.041+0000 7f4b557fa700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b4006e2e0 0x7f4b400707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:26.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.041+0000 7f4b557fa700 1 -- 172.21.15.63:0/774946251 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4b4c08f9d0 con 0x7f4b58104010 2024-05-09T06:40:26.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.041+0000 7f4b57fff700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b4006e2e0 0x7f4b400707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:26.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.042+0000 7f4b57fff700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b4006e2e0 0x7f4b400707a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b48001050 tx=0x7f4b48006df0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:26.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.046+0000 7f4b557fa700 1 -- 172.21.15.63:0/774946251 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4b4c05c610 con 0x7f4b58104010 2024-05-09T06:40:26.228 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:25 smithi193 podman[81515]: 2024-05-09 06:40:25.911487993 +0000 UTC m=+0.817341092 container remove bb042f0315eb935cfb36dca64555b1b7784f5bbb2412aeb9d906c2e1f7600a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, RELEASE=reef-1758e8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, ceph=True) 2024-05-09T06:40:26.228 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi193.service: Deactivated successfully. 2024-05-09T06:40:26.228 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 systemd[1]: Stopped Ceph mon.smithi193 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:40:26.228 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi193.service: Consumed 6.247s CPU time. 2024-05-09T06:40:26.228 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 systemd[1]: Starting Ceph mon.smithi193 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:40:26.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.299+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b58109ac0 con 0x7f4b58104010 2024-05-09T06:40:26.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.299+0000 7f4b557fa700 1 -- 172.21.15.63:0/774946251 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1905 (secure 0 0 0) 0x7f4b4c05c1a0 con 0x7f4b58104010 2024-05-09T06:40:26.279 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:40:26.279 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:40:26.280 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:40:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:26.282 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.301+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b4006e2e0 msgr2=0x7f4b400707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.301+0000 7f4b5e3ad700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b4006e2e0 0x7f4b400707a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b48001050 tx=0x7f4b48006df0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.301+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 msgr2=0x7f4b580ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.301+0000 7f4b5e3ad700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 0x7f4b580ff690 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c005090 tx=0x7f4b4c00be20 comp rx=0 tx=0).stop 2024-05-09T06:40:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.302+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 shutdown_connections 2024-05-09T06:40:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.302+0000 7f4b5e3ad700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b4006e2e0 0x7f4b400707a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.302+0000 7f4b5e3ad700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b581016e0 0x7f4b58100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.302+0000 7f4b5e3ad700 1 --2- 172.21.15.63:0/774946251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b58104010 0x7f4b580ff690 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.302+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 >> 172.21.15.63:0/774946251 conn(0x7f4b580fb0f0 msgr2=0x7f4b580fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:26.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.302+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 shutdown_connections 2024-05-09T06:40:26.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.302+0000 7f4b5e3ad700 1 -- 172.21.15.63:0/774946251 wait complete. 2024-05-09T06:40:26.288 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:40:26.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.411+0000 7fbe6b572700 1 -- 172.21.15.63:0/1838865931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 msgr2=0x7fbe640fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:26.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.411+0000 7fbe6b572700 1 --2- 172.21.15.63:0/1838865931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 0x7fbe640fffc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009910 tx=0x7fbe60009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:26.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.411+0000 7fbe6b572700 1 -- 172.21.15.63:0/1838865931 shutdown_connections 2024-05-09T06:40:26.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.411+0000 7fbe6b572700 1 --2- 172.21.15.63:0/1838865931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe64100da0 0x7fbe64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.411+0000 7fbe6b572700 1 --2- 172.21.15.63:0/1838865931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 0x7fbe640fffc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.411+0000 7fbe6b572700 1 -- 172.21.15.63:0/1838865931 >> 172.21.15.63:0/1838865931 conn(0x7fbe640fb120 msgr2=0x7fbe640fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:26.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.411+0000 7fbe6b572700 1 -- 172.21.15.63:0/1838865931 shutdown_connections 2024-05-09T06:40:26.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.412+0000 7fbe6b572700 1 -- 172.21.15.63:0/1838865931 wait complete. 2024-05-09T06:40:26.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.412+0000 7fbe6b572700 1 Processor -- start 2024-05-09T06:40:26.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.412+0000 7fbe6b572700 1 -- start start 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.412+0000 7fbe6b572700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 0x7fbe64195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.412+0000 7fbe6b572700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe64100da0 0x7fbe64195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.412+0000 7fbe6b572700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64196190 con 0x7fbe640ffba0 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.412+0000 7fbe6b572700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe641962d0 con 0x7fbe64100da0 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe6930e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 0x7fbe64195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe6930e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 0x7fbe64195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47646/0 (socket says 172.21.15.63:47646) 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe6930e700 1 -- 172.21.15.63:0/1526155475 learned_addr learned my addr 172.21.15.63:0/1526155475 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe68b0d700 1 -- 172.21.15.63:0/1526155475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe64100da0 msgr2=0x7fbe64195b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.193:3300/0 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe68b0d700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe64100da0 0x7fbe64195b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe6930e700 1 -- 172.21.15.63:0/1526155475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe64100da0 msgr2=0x7fbe64195b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe6930e700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe64100da0 0x7fbe64195b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe6930e700 1 -- 172.21.15.63:0/1526155475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe600095f0 con 0x7fbe640ffba0 2024-05-09T06:40:26.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe6930e700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 0x7fbe64195630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000bd70 tx=0x7fbe6000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:26.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe5a7fc700 1 -- 172.21.15.63:0/1526155475 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe6001d070 con 0x7fbe640ffba0 2024-05-09T06:40:26.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe5a7fc700 1 -- 172.21.15.63:0/1526155475 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe6000f460 con 0x7fbe640ffba0 2024-05-09T06:40:26.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6419acd0 con 0x7fbe640ffba0 2024-05-09T06:40:26.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe5a7fc700 1 -- 172.21.15.63:0/1526155475 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe60021620 con 0x7fbe640ffba0 2024-05-09T06:40:26.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.413+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6419b240 con 0x7fbe640ffba0 2024-05-09T06:40:26.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.414+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6418f7c0 con 0x7fbe640ffba0 2024-05-09T06:40:26.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.415+0000 7fbe5a7fc700 1 -- 172.21.15.63:0/1526155475 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fbe6000f5d0 con 0x7fbe640ffba0 2024-05-09T06:40:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.415+0000 7fbe5a7fc700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe5006e280 0x7fbe50070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.415+0000 7fbe5a7fc700 1 -- 172.21.15.63:0/1526155475 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbe6008f6b0 con 0x7fbe640ffba0 2024-05-09T06:40:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.415+0000 7fbe68b0d700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe5006e280 0x7fbe50070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:26.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.416+0000 7fbe68b0d700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe5006e280 0x7fbe50070740 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe54001050 tx=0x7fbe54006df0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:26.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.420+0000 7fbe5a7fc700 1 -- 172.21.15.63:0/1526155475 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbe6005c0e0 con 0x7fbe640ffba0 2024-05-09T06:40:26.481 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 podman[81646]: 2024-05-09 06:40:26.369840154 +0000 UTC m=+0.017983510 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:40:26.482 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 podman[81646]: 2024-05-09 06:40:26.502603415 +0000 UTC m=+0.150746762 container create dafb35e76458fefc325a774ca0e6ae888b1c9c92d7d4706055a45b0b7ecb3ba0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, RELEASE=squid-96805dc, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-09T06:40:26.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.644+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe6402ced0 con 0x7fbe5006e280 2024-05-09T06:40:26.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.646+0000 7fbe5a7fc700 1 -- 172.21.15.63:0/1526155475 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fbe6402ced0 con 0x7fbe5006e280 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "3/23 daemons upgraded", 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mon daemons", 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:40:26.627 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:40:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe5006e280 msgr2=0x7fbe50070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe5006e280 0x7fbe50070740 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe54001050 tx=0x7fbe54006df0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 msgr2=0x7fbe64195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 0x7fbe64195630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000bd70 tx=0x7fbe6000be50 comp rx=0 tx=0).stop 2024-05-09T06:40:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 shutdown_connections 2024-05-09T06:40:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe5006e280 0x7fbe50070740 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe640ffba0 0x7fbe64195630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 --2- 172.21.15.63:0/1526155475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe64100da0 0x7fbe64195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 >> 172.21.15.63:0/1526155475 conn(0x7fbe640fb120 msgr2=0x7fbe641043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 shutdown_connections 2024-05-09T06:40:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.648+0000 7fbe6b572700 1 -- 172.21.15.63:0/1526155475 wait complete. 2024-05-09T06:40:26.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.756+0000 7f9999b34700 1 -- 172.21.15.63:0/3496180945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99941016c0 msgr2=0x7f9994103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.756+0000 7f9999b34700 1 --2- 172.21.15.63:0/3496180945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99941016c0 0x7f9994103ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9984009910 tx=0x7f9984009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.756+0000 7f9999b34700 1 -- 172.21.15.63:0/3496180945 shutdown_connections 2024-05-09T06:40:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.756+0000 7f9999b34700 1 --2- 172.21.15.63:0/3496180945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9994103ff0 0x7f99941063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.756+0000 7f9999b34700 1 --2- 172.21.15.63:0/3496180945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99941016c0 0x7f9994103ab0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.756+0000 7f9999b34700 1 -- 172.21.15.63:0/3496180945 >> 172.21.15.63:0/3496180945 conn(0x7f99940fb0f0 msgr2=0x7f99940fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.756+0000 7f9999b34700 1 -- 172.21.15.63:0/3496180945 shutdown_connections 2024-05-09T06:40:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.756+0000 7f9999b34700 1 -- 172.21.15.63:0/3496180945 wait complete. 2024-05-09T06:40:26.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.757+0000 7f9999b34700 1 Processor -- start 2024-05-09T06:40:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.757+0000 7f9999b34700 1 -- start start 2024-05-09T06:40:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.757+0000 7f9999b34700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99941016c0 0x7f9994100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.757+0000 7f9999b34700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9994103ff0 0x7f99940ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.757+0000 7f9999b34700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940ffbb0 con 0x7f9994103ff0 2024-05-09T06:40:26.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.757+0000 7f9999b34700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940ffcf0 con 0x7f99941016c0 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.757+0000 7f9992ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9994103ff0 0x7f99940ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9992ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9994103ff0 0x7f99940ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47658/0 (socket says 172.21.15.63:47658) 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9992ffd700 1 -- 172.21.15.63:0/3932786625 learned_addr learned my addr 172.21.15.63:0/3932786625 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f99937fe700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99941016c0 0x7f9994100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f99937fe700 1 -- 172.21.15.63:0/3932786625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99941016c0 msgr2=0x7f9994100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f99937fe700 1 -- 172.21.15.63:0/3932786625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99941016c0 msgr2=0x7f9994100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f99937fe700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99941016c0 0x7f9994100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f99937fe700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99941016c0 0x7f9994100fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9992ffd700 1 -- 172.21.15.63:0/3932786625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99941016c0 msgr2=0x7f9994100fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9992ffd700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99941016c0 0x7f9994100fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9992ffd700 1 -- 172.21.15.63:0/3932786625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840095f0 con 0x7f9994103ff0 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9992ffd700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9994103ff0 0x7f99940ff670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f998800d8d0 tx=0x7f998800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:26.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9990ff9700 1 -- 172.21.15.63:0/3932786625 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9988009940 con 0x7f9994103ff0 2024-05-09T06:40:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99940fffd0 con 0x7f9994103ff0 2024-05-09T06:40:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9990ff9700 1 -- 172.21.15.63:0/3932786625 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9988010460 con 0x7f9994103ff0 2024-05-09T06:40:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9990ff9700 1 -- 172.21.15.63:0/3932786625 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f998800f5f0 con 0x7f9994103ff0 2024-05-09T06:40:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.758+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999419e650 con 0x7f9994103ff0 2024-05-09T06:40:26.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.759+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999404ebb0 con 0x7f9994103ff0 2024-05-09T06:40:26.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.760+0000 7f9990ff9700 1 -- 172.21.15.63:0/3932786625 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f99880105d0 con 0x7f9994103ff0 2024-05-09T06:40:26.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.760+0000 7f9990ff9700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f997c06e340 0x7f997c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:26.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.761+0000 7f9990ff9700 1 -- 172.21.15.63:0/3932786625 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f998808e370 con 0x7f9994103ff0 2024-05-09T06:40:26.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.761+0000 7f99937fe700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f997c06e340 0x7f997c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:26.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.761+0000 7f99937fe700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f997c06e340 0x7f997c070800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f99840051c0 tx=0x7f9984005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:26.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:26.765+0000 7f9990ff9700 1 -- 172.21.15.63:0/3932786625 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f998805b0e0 con 0x7f9994103ff0 2024-05-09T06:40:26.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 podman[81646]: 2024-05-09 06:40:26.705065042 +0000 UTC m=+0.353208387 container init dafb35e76458fefc325a774ca0e6ae888b1c9c92d7d4706055a45b0b7ecb3ba0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:40:26.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 podman[81646]: 2024-05-09 06:40:26.708320591 +0000 UTC m=+0.356463937 container start dafb35e76458fefc325a774ca0e6ae888b1c9c92d7d4706055a45b0b7ecb3ba0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS) 2024-05-09T06:40:26.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T06:40:26.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev), process ceph-mon, pid 2 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: pidfile_write: ignore empty --pid-file 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: load: jerasure load: lrc 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: RocksDB version: 7.9.2 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Git sha 0 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Compile date 2024-05-07 19:19:25 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: DB SUMMARY 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: DB Session ID: 7FVW20R02UXEM5ASFLEX 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: CURRENT file: CURRENT 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1386 Bytes 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi193/store.db dir, Total Num: 1, files: 000027.sst 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi193/store.db: 000025.log size: 544360 ; 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.error_if_exists: 0 2024-05-09T06:40:26.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.create_if_missing: 0 2024-05-09T06:40:26.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.paranoid_checks: 1 2024-05-09T06:40:26.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T06:40:26.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T06:40:26.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T06:40:26.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.env: 0x563a3001fc60 2024-05-09T06:40:26.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T06:40:26.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.info_log: 0x563a31131980 2024-05-09T06:40:26.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T06:40:26.779 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.statistics: (nil) 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.use_fsync: 0 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_log_file_size: 0 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.allow_fallocate: 1 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.use_direct_reads: 0 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T06:40:26.780 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.db_log_dir: 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.wal_dir: 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.write_buffer_manager: 0x563a311041e0 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T06:40:26.781 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.unordered_write: 0 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.row_cache: None 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.wal_filter: None 2024-05-09T06:40:26.782 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.two_write_queues: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.wal_compression: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.atomic_flush: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.log_readahead_size: 0 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T06:40:26.783 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_background_jobs: 2 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_background_compactions: -1 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_subcompactions: 1 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T06:40:26.784 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_open_files: -1 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_background_flushes: -1 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Compression algorithms supported: 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: kZSTD supported: 0 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: kXpressCompression supported: 0 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: kZlibCompression supported: 1 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: kSnappyCompression supported: 1 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: kLZ4Compression supported: 1 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: kBZip2Compression supported: 0 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T06:40:26.785 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000010 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.merge_operator: 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_filter: None 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563a31131a80) 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-05-09T06:40:26.786 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_type: 0 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_index_type: 0 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_shortening: 1 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: checksum: 4 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: no_block_cache: 0 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache: 0x563a31135090 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_options: 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: capacity : 536870912 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: num_shard_bits : 4 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: strict_capacity_limit : 0 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_compressed: (nil) 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: persistent_cache: (nil) 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size: 4096 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size_deviation: 10 2024-05-09T06:40:26.787 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_restart_interval: 16 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_block_restart_interval: 1 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: metadata_block_size: 4096 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: partition_filters: 0 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: use_delta_encoding: 1 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: filter_policy: bloomfilter 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: whole_key_filtering: 1 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: verify_compression: 0 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: format_version: 5 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_index_compression: 1 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_align: 0 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_auto_readahead_size: 262144 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: prepopulate_block_cache: 0 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: initial_auto_readahead_size: 8192 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T06:40:26.788 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression: NoCompression 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.num_levels: 7 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T06:40:26.789 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T06:40:26.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T06:40:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.table_properties_collectors: 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.inplace_update_support: 0 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T06:40:26.792 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.bloom_locality: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.max_successive_merges: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.ttl: 2592000 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.enable_blob_files: false 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.min_blob_size: 0 2024-05-09T06:40:26.793 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12178, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7e83f109-0ae1-4fa8-8338-a0979228845b 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236826737837, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236826739771, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 318534, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12183, "largest_seqno": 12385, "table_properties": {"data_size": 316522, "index_size": 744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 261, "raw_key_size": 2334, "raw_average_key_size": 24, "raw_value_size": 314185, "raw_average_value_size": 3342, "num_data_blocks": 33, "num_entries": 94, "num_filter_entries": 94, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715236826, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7e83f109-0ae1-4fa8-8338-a0979228845b", "db_session_id": "7FVW20R02UXEM5ASFLEX", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715236826739853, "job": 1, "event": "recovery_finished"} 2024-05-09T06:40:26.794 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi193/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563a31226000 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: DB pointer 0x563a31210000 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** DB Stats ** 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T06:40:26.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: L0 1/0 311.07 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 270.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: L6 1/0 8.08 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Sum 2/0 8.38 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 270.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 270.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 270.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:40:26.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative compaction: 0.00 GB write, 74.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval compaction: 0.00 GB write, 74.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Block cache BinnedLRUCache@0x563a31135090#2 capacity: 512.00 MB usage: 1.14 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.31 KB,5.96046e-05%) IndexBlock(1,0.83 KB,0.000157952%) Misc(1,0.00 KB,0%) 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: starting mon.smithi193 rank 1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi193 fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: mon.smithi193@-1(???) e2 preinit fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: mon.smithi193@-1(???).mds e15 new map 2024-05-09T06:40:26.797 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: mon.smithi193@-1(???).mds e15 print_map 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: e15 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: legacy client fscid: 1 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Filesystem 'cephfs' (1) 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: fs_name cephfs 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: epoch 15 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: tableserver 0 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: root 0 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: session_timeout 60 2024-05-09T06:40:26.798 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: session_autoclose 300 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_file_size 1099511627776 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_xattr_size 65536 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: required_client_features {} 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: last_failure 0 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: last_failure_osd_epoch 47 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_mds 1 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: in 0 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: up {0=24277} 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: failed 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: damaged 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: stopped 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_pools [3] 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: metadata_pool 2 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: inline_data disabled 2024-05-09T06:40:26.799 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: balancer 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: bal_rank_mask -1 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: standby_count_wanted 1 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: qdb_cluster leader: 0 members: 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Standby daemons: 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: mon.smithi193@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: mon.smithi193@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: mon.smithi193@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: mon.smithi193@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 ceph-mon[81665]: mon.smithi193@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-05-09T06:40:26.800 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 bash[81646]: dafb35e76458fefc325a774ca0e6ae888b1c9c92d7d4706055a45b0b7ecb3ba0 2024-05-09T06:40:26.801 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:26 smithi193 systemd[1]: Started Ceph mon.smithi193 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:40:27.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.055+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9994109790 con 0x7f9994103ff0 2024-05-09T06:40:27.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.056+0000 7f9990ff9700 1 -- 172.21.15.63:0/3932786625 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f998805ac70 con 0x7f9994103ff0 2024-05-09T06:40:27.037 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:40:27.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f997c06e340 msgr2=0x7f997c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f997c06e340 0x7f997c070800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f99840051c0 tx=0x7f9984005a90 comp rx=0 tx=0).stop 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9994103ff0 msgr2=0x7f99940ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9994103ff0 0x7f99940ff670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f998800d8d0 tx=0x7f998800dc90 comp rx=0 tx=0).stop 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 shutdown_connections 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f997c06e340 0x7f997c070800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99941016c0 0x7f9994100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 --2- 172.21.15.63:0/3932786625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9994103ff0 0x7f99940ff670 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 >> 172.21.15.63:0/3932786625 conn(0x7f99940fb0f0 msgr2=0x7f99940fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.058+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 shutdown_connections 2024-05-09T06:40:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:27.059+0000 7f9999b34700 1 -- 172.21.15.63:0/3932786625 wait complete. 2024-05-09T06:40:28.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:28 smithi063 ceph-mon[92765]: pgmap v9: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 651 B/s wr, 13 op/s 2024-05-09T06:40:28.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:28 smithi193 ceph-mon[81665]: pgmap v9: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 651 B/s wr, 13 op/s 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: pgmap v11: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: mon.smithi063 calling monitor election 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: mon.smithi193 calling monitor election 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: mon.smithi063 is new leader, mons smithi063,smithi193 in quorum (ranks 0,1) 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: monmap epoch 3 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: last_changed 2024-05-09T06:40:28.174132+0000 2024-05-09T06:40:29.516 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: created 2024-05-09T06:27:48.192363+0000 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: min_mon_release 19 (squid) 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: election_strategy: 1 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.smithi063 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: osdmap e50: 6 total, 6 up, 6 in 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: mgrmap e38: smithi063.egfguy(active, since 16s), standbys: smithi193.mmmqzh 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: overall HEALTH_OK 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:40:29.517 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:29.518 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:29.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: pgmap v11: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: mon.smithi063 calling monitor election 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: mon.smithi193 calling monitor election 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: mon.smithi063 is new leader, mons smithi063,smithi193 in quorum (ranks 0,1) 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: monmap epoch 3 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: last_changed 2024-05-09T06:40:28.174132+0000 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: created 2024-05-09T06:27:48.192363+0000 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: min_mon_release 19 (squid) 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: election_strategy: 1 2024-05-09T06:40:29.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.smithi063 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: osdmap e50: 6 total, 6 up, 6 in 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: mgrmap e38: smithi063.egfguy(active, since 16s), standbys: smithi193.mmmqzh 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: overall HEALTH_OK 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:29.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:31.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:31 smithi193 ceph-mon[81665]: pgmap v12: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-09T06:40:31.438 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:31 smithi063 ceph-mon[92765]: pgmap v12: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-09T06:40:32.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:32 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:32.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:32 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:32.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:32 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:32.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:32 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:33.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:33 smithi063 ceph-mon[92765]: pgmap v13: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-09T06:40:33.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:33.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:33.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:33 smithi193 ceph-mon[81665]: pgmap v13: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-09T06:40:33.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:33.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:35.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:35 smithi193 ceph-mon[81665]: pgmap v14: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-05-09T06:40:35.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:35 smithi063 ceph-mon[92765]: pgmap v14: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-05-09T06:40:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: pgmap v15: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: Reconfiguring mon.smithi063 (monmap changed)... 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:37 smithi063 ceph-mon[92765]: Reconfiguring daemon mon.smithi063 on smithi063 2024-05-09T06:40:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: pgmap v15: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:40:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: Reconfiguring mon.smithi063 (monmap changed)... 2024-05-09T06:40:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:40:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:40:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:37.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:37 smithi193 ceph-mon[81665]: Reconfiguring daemon mon.smithi063 on smithi063 2024-05-09T06:40:39.545 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:39 smithi063 ceph-mon[92765]: pgmap v16: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:39.545 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:39 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:39.545 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:39 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:39.546 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:39 smithi063 ceph-mon[92765]: Reconfiguring mgr.smithi063.egfguy (monmap changed)... 2024-05-09T06:40:39.546 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:39 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.egfguy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:40:39.546 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:39 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:40:39.546 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:39 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:39.546 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:39 smithi063 ceph-mon[92765]: Reconfiguring daemon mgr.smithi063.egfguy on smithi063 2024-05-09T06:40:39.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:39 smithi193 ceph-mon[81665]: pgmap v16: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:39.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:39 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:39.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:39 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:39.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:39 smithi193 ceph-mon[81665]: Reconfiguring mgr.smithi063.egfguy (monmap changed)... 2024-05-09T06:40:39.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:39 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.egfguy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:40:39.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:39 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:40:39.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:39 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:39.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:39 smithi193 ceph-mon[81665]: Reconfiguring daemon mgr.smithi063.egfguy on smithi063 2024-05-09T06:40:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: Reconfiguring ceph-exporter.smithi063 (monmap changed)... 2024-05-09T06:40:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:40:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:40:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-09T06:40:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: Unable to update caps for client.ceph-exporter.smithi063 2024-05-09T06:40:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi063"}]: dispatch 2024-05-09T06:40:41.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:41.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: Reconfiguring daemon ceph-exporter.smithi063 on smithi063 2024-05-09T06:40:41.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:40 smithi063 ceph-mon[92765]: pgmap v17: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:40:41.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:41.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:41.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: Reconfiguring ceph-exporter.smithi063 (monmap changed)... 2024-05-09T06:40:41.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:40:41.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:40:41.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-09T06:40:41.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: Unable to update caps for client.ceph-exporter.smithi063 2024-05-09T06:40:41.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi063"}]: dispatch 2024-05-09T06:40:41.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:41.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: Reconfiguring daemon ceph-exporter.smithi063 on smithi063 2024-05-09T06:40:41.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:40 smithi193 ceph-mon[81665]: pgmap v17: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:40:42.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:42.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:42.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:42 smithi063 ceph-mon[92765]: Reconfiguring crash.smithi063 (monmap changed)... 2024-05-09T06:40:42.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:40:42.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:42.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:42 smithi063 ceph-mon[92765]: Reconfiguring daemon crash.smithi063 on smithi063 2024-05-09T06:40:42.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:40:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:42 smithi193 ceph-mon[81665]: Reconfiguring crash.smithi063 (monmap changed)... 2024-05-09T06:40:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:40:42.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:42.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:42 smithi193 ceph-mon[81665]: Reconfiguring daemon crash.smithi063 on smithi063 2024-05-09T06:40:42.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:40:43.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:43 smithi063 ceph-mon[92765]: pgmap v18: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:43.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:43.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:43.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T06:40:43.804 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:43.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:43 smithi193 ceph-mon[81665]: pgmap v18: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:43.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:43.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:43.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T06:40:43.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:44.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:44 smithi193 ceph-mon[81665]: Reconfiguring osd.0 (monmap changed)... 2024-05-09T06:40:44.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:44 smithi193 ceph-mon[81665]: Reconfiguring daemon osd.0 on smithi063 2024-05-09T06:40:44.844 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:44 smithi063 ceph-mon[92765]: Reconfiguring osd.0 (monmap changed)... 2024-05-09T06:40:44.844 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:44 smithi063 ceph-mon[92765]: Reconfiguring daemon osd.0 on smithi063 2024-05-09T06:40:45.830 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:45 smithi063 ceph-mon[92765]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:45.830 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:45.831 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:45.831 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T06:40:45.831 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:45.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:45 smithi193 ceph-mon[81665]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:45.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:45.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:45.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T06:40:45.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:46.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:46 smithi063 ceph-mon[92765]: Reconfiguring osd.1 (monmap changed)... 2024-05-09T06:40:46.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:46 smithi063 ceph-mon[92765]: Reconfiguring daemon osd.1 on smithi063 2024-05-09T06:40:46.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:46 smithi193 ceph-mon[81665]: Reconfiguring osd.1 (monmap changed)... 2024-05-09T06:40:46.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:46 smithi193 ceph-mon[81665]: Reconfiguring daemon osd.1 on smithi063 2024-05-09T06:40:47.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:47 smithi063 ceph-mon[92765]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:47.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:47.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:47.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T06:40:47.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:47.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:47 smithi193 ceph-mon[81665]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:47.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:47.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:47.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T06:40:47.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:48.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:48 smithi063 ceph-mon[92765]: Reconfiguring osd.2 (monmap changed)... 2024-05-09T06:40:48.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:48 smithi063 ceph-mon[92765]: Reconfiguring daemon osd.2 on smithi063 2024-05-09T06:40:48.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:48.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:48.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.aoiibw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:40:48.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:48.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:48 smithi193 ceph-mon[81665]: Reconfiguring osd.2 (monmap changed)... 2024-05-09T06:40:48.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:48 smithi193 ceph-mon[81665]: Reconfiguring daemon osd.2 on smithi063 2024-05-09T06:40:48.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:48.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:48.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.aoiibw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:40:48.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:49.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:49 smithi063 ceph-mon[92765]: pgmap v21: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:49.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:49 smithi063 ceph-mon[92765]: Reconfiguring mds.cephfs.smithi063.aoiibw (monmap changed)... 2024-05-09T06:40:49.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:49 smithi063 ceph-mon[92765]: Reconfiguring daemon mds.cephfs.smithi063.aoiibw on smithi063 2024-05-09T06:40:49.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:49 smithi193 ceph-mon[81665]: pgmap v21: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:49.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:49 smithi193 ceph-mon[81665]: Reconfiguring mds.cephfs.smithi063.aoiibw (monmap changed)... 2024-05-09T06:40:49.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:49 smithi193 ceph-mon[81665]: Reconfiguring daemon mds.cephfs.smithi063.aoiibw on smithi063 2024-05-09T06:40:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:51 smithi063 ceph-mon[92765]: pgmap v22: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:40:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:51 smithi063 ceph-mon[92765]: Reconfiguring mds.cephfs.smithi063.fbzxbf (monmap changed)... 2024-05-09T06:40:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.fbzxbf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:40:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:51.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:51 smithi063 ceph-mon[92765]: Reconfiguring daemon mds.cephfs.smithi063.fbzxbf on smithi063 2024-05-09T06:40:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:51 smithi193 ceph-mon[81665]: pgmap v22: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:40:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:51 smithi193 ceph-mon[81665]: Reconfiguring mds.cephfs.smithi063.fbzxbf (monmap changed)... 2024-05-09T06:40:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.fbzxbf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:40:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:51 smithi193 ceph-mon[81665]: Reconfiguring daemon mds.cephfs.smithi063.fbzxbf on smithi063 2024-05-09T06:40:53.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:53.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:53.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: Reconfiguring ceph-exporter.smithi193 (monmap changed)... 2024-05-09T06:40:53.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:40:53.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:40:53.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-09T06:40:53.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: Unable to update caps for client.ceph-exporter.smithi193 2024-05-09T06:40:53.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi193"}]: dispatch 2024-05-09T06:40:53.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:53.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: Reconfiguring daemon ceph-exporter.smithi193 on smithi193 2024-05-09T06:40:53.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:52 smithi063 ceph-mon[92765]: pgmap v23: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:53.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:53.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:53.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: Reconfiguring ceph-exporter.smithi193 (monmap changed)... 2024-05-09T06:40:53.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:40:53.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:40:53.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-09T06:40:53.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: Unable to update caps for client.ceph-exporter.smithi193 2024-05-09T06:40:53.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi193"}]: dispatch 2024-05-09T06:40:53.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:53.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: Reconfiguring daemon ceph-exporter.smithi193 on smithi193 2024-05-09T06:40:53.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:52 smithi193 ceph-mon[81665]: pgmap v23: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:54.842 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:54.842 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:54.842 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:54 smithi193 ceph-mon[81665]: Reconfiguring crash.smithi193 (monmap changed)... 2024-05-09T06:40:54.843 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:40:54.843 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:54.843 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:54 smithi193 ceph-mon[81665]: Reconfiguring daemon crash.smithi193 on smithi193 2024-05-09T06:40:54.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:54.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:54.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:54 smithi063 ceph-mon[92765]: Reconfiguring crash.smithi193 (monmap changed)... 2024-05-09T06:40:54.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:40:54.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:54.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:54 smithi063 ceph-mon[92765]: Reconfiguring daemon crash.smithi193 on smithi193 2024-05-09T06:40:55.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:55 smithi193 ceph-mon[81665]: pgmap v24: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:55.795 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:55.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:55.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:40:55.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:40:55.796 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:55.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:55 smithi063 ceph-mon[92765]: pgmap v24: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:55.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:55.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:55.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mmmqzh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T06:40:55.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T06:40:55.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:56.814 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:56 smithi193 ceph-mon[81665]: Reconfiguring mgr.smithi193.mmmqzh (monmap changed)... 2024-05-09T06:40:56.814 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:56 smithi193 ceph-mon[81665]: Reconfiguring daemon mgr.smithi193.mmmqzh on smithi193 2024-05-09T06:40:56.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:56 smithi063 ceph-mon[92765]: Reconfiguring mgr.smithi193.mmmqzh (monmap changed)... 2024-05-09T06:40:56.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:56 smithi063 ceph-mon[92765]: Reconfiguring daemon mgr.smithi193.mmmqzh on smithi193 2024-05-09T06:40:57.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.185+0000 7feedcd52700 1 -- 172.21.15.63:0/3107326037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 msgr2=0x7feed80fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.185+0000 7feedcd52700 1 --2- 172.21.15.63:0/3107326037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 0x7feed80fff50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feec800b0a0 tx=0x7feec800b3b0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.185+0000 7feedcd52700 1 -- 172.21.15.63:0/3107326037 shutdown_connections 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.185+0000 7feedcd52700 1 --2- 172.21.15.63:0/3107326037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed8100d30 0x7feed8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.185+0000 7feedcd52700 1 --2- 172.21.15.63:0/3107326037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 0x7feed80fff50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.185+0000 7feedcd52700 1 -- 172.21.15.63:0/3107326037 >> 172.21.15.63:0/3107326037 conn(0x7feed80fb0d0 msgr2=0x7feed80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.185+0000 7feedcd52700 1 -- 172.21.15.63:0/3107326037 shutdown_connections 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.185+0000 7feedcd52700 1 -- 172.21.15.63:0/3107326037 wait complete. 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.186+0000 7feedcd52700 1 Processor -- start 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.186+0000 7feedcd52700 1 -- start start 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.186+0000 7feedcd52700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 0x7feed8103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.186+0000 7feedcd52700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed8100d30 0x7feed8102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.186+0000 7feedcd52700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8102650 con 0x7feed80ffb30 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.186+0000 7feedcd52700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8102790 con 0x7feed8100d30 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feed659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 0x7feed8103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feed659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 0x7feed8103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57148/0 (socket says 172.21.15.63:57148) 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feed659c700 1 -- 172.21.15.63:0/183791909 learned_addr learned my addr 172.21.15.63:0/183791909 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feed5d9b700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed8100d30 0x7feed8102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:57.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feed659c700 1 -- 172.21.15.63:0/183791909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed8100d30 msgr2=0x7feed8102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feed659c700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed8100d30 0x7feed8102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feed659c700 1 -- 172.21.15.63:0/183791909 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8009d00 con 0x7feed80ffb30 2024-05-09T06:40:57.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feed659c700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 0x7feed8103a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feec8006130 tx=0x7feec800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:57.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feec77fe700 1 -- 172.21.15.63:0/183791909 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec800e040 con 0x7feed80ffb30 2024-05-09T06:40:57.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feec77fe700 1 -- 172.21.15.63:0/183791909 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feec8003c10 con 0x7feed80ffb30 2024-05-09T06:40:57.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed8102a10 con 0x7feed80ffb30 2024-05-09T06:40:57.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feec77fe700 1 -- 172.21.15.63:0/183791909 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec801d7b0 con 0x7feed80ffb30 2024-05-09T06:40:57.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.187+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed8102f80 con 0x7feed80ffb30 2024-05-09T06:40:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.188+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed804ebb0 con 0x7feed80ffb30 2024-05-09T06:40:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.189+0000 7feec77fe700 1 -- 172.21.15.63:0/183791909 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7feec8019070 con 0x7feed80ffb30 2024-05-09T06:40:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.189+0000 7feec77fe700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feec006e340 0x7feec0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.190+0000 7feec77fe700 1 -- 172.21.15.63:0/183791909 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7feec808f7e0 con 0x7feed80ffb30 2024-05-09T06:40:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.190+0000 7feed5d9b700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feec006e340 0x7feec0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.190+0000 7feed5d9b700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feec006e340 0x7feec0070800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feecc009c60 tx=0x7feecc009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:57.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.194+0000 7feec77fe700 1 -- 172.21.15.63:0/183791909 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feec805c420 con 0x7feed80ffb30 2024-05-09T06:40:57.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.426+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feed8102f80 con 0x7feec006e340 2024-05-09T06:40:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.427+0000 7feec77fe700 1 -- 172.21.15.63:0/183791909 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7feed8102f80 con 0x7feec006e340 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feec006e340 msgr2=0x7feec0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feec006e340 0x7feec0070800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feecc009c60 tx=0x7feecc009260 comp rx=0 tx=0).stop 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 msgr2=0x7feed8103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 0x7feed8103a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feec8006130 tx=0x7feec800bc60 comp rx=0 tx=0).stop 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 shutdown_connections 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feec006e340 0x7feec0070800 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed80ffb30 0x7feed8103a70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 --2- 172.21.15.63:0/183791909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed8100d30 0x7feed8102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 >> 172.21.15.63:0/183791909 conn(0x7feed80fb0d0 msgr2=0x7feed8069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 shutdown_connections 2024-05-09T06:40:57.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.429+0000 7feedcd52700 1 -- 172.21.15.63:0/183791909 wait complete. 2024-05-09T06:40:57.419 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:40:57.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.539+0000 7f9735de2700 1 -- 172.21.15.63:0/3489592540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 msgr2=0x7f97301022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.539+0000 7f9735de2700 1 --2- 172.21.15.63:0/3489592540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 0x7f97301022b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9724009910 tx=0x7f9724009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:57.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.539+0000 7f9735de2700 1 -- 172.21.15.63:0/3489592540 shutdown_connections 2024-05-09T06:40:57.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.539+0000 7f9735de2700 1 --2- 172.21.15.63:0/3489592540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 0x7f97301022b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.539+0000 7f9735de2700 1 --2- 172.21.15.63:0/3489592540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9730100c50 0x7f9730101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.539+0000 7f9735de2700 1 -- 172.21.15.63:0/3489592540 >> 172.21.15.63:0/3489592540 conn(0x7f97300fc210 msgr2=0x7f97300fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.539+0000 7f9735de2700 1 -- 172.21.15.63:0/3489592540 shutdown_connections 2024-05-09T06:40:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.539+0000 7f9735de2700 1 -- 172.21.15.63:0/3489592540 wait complete. 2024-05-09T06:40:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.540+0000 7f9735de2700 1 Processor -- start 2024-05-09T06:40:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.540+0000 7f9735de2700 1 -- start start 2024-05-09T06:40:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.540+0000 7f9735de2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9730100c50 0x7f9730074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.540+0000 7f9735de2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 0x7f97300732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.540+0000 7f9735de2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9730073820 con 0x7f9730101e50 2024-05-09T06:40:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.540+0000 7f9735de2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9730073960 con 0x7f9730100c50 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 0x7f97300732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9730100c50 0x7f9730074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 0x7f97300732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57166/0 (socket says 172.21.15.63:57166) 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972effd700 1 -- 172.21.15.63:0/487150485 learned_addr learned my addr 172.21.15.63:0/487150485 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972f7fe700 1 -- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 msgr2=0x7f97300732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972f7fe700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 0x7f97300732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972f7fe700 1 -- 172.21.15.63:0/487150485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240095f0 con 0x7f9730100c50 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972effd700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 0x7f97300732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972f7fe700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9730100c50 0x7f9730074bf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f972000d6c0 tx=0x7f972000d9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972cff9700 1 -- 172.21.15.63:0/487150485 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97200041f0 con 0x7f9730100c50 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972cff9700 1 -- 172.21.15.63:0/487150485 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9720004350 con 0x7f9730100c50 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9730073c40 con 0x7f9730100c50 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f972cff9700 1 -- 172.21.15.63:0/487150485 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9720003d20 con 0x7f9730100c50 2024-05-09T06:40:57.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.541+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973019e670 con 0x7f9730100c50 2024-05-09T06:40:57.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.542+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97301907d0 con 0x7f9730100c50 2024-05-09T06:40:57.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.543+0000 7f972cff9700 1 -- 172.21.15.63:0/487150485 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f972000f690 con 0x7f9730100c50 2024-05-09T06:40:57.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.543+0000 7f972cff9700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f971806e340 0x7f9718070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:57.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.543+0000 7f972cff9700 1 -- 172.21.15.63:0/487150485 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f972008de40 con 0x7f9730100c50 2024-05-09T06:40:57.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.543+0000 7f972effd700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f971806e340 0x7f9718070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:57.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.544+0000 7f972effd700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f971806e340 0x7f9718070800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9724005b40 tx=0x7f9724005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:57.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.549+0000 7f972cff9700 1 -- 172.21.15.63:0/487150485 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f972005aa80 con 0x7f9730100c50 2024-05-09T06:40:57.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:57 smithi193 ceph-mon[81665]: pgmap v25: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:57.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:57.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:57.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:40:57.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:40:57.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:57.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:40:57.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.776+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9730061330 con 0x7f971806e340 2024-05-09T06:40:57.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:57 smithi063 ceph-mon[92765]: pgmap v25: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:40:57.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:57.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:57.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T06:40:57.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T06:40:57.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:57.757 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:40:57.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.778+0000 7f972cff9700 1 -- 172.21.15.63:0/487150485 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f9730061330 con 0x7f971806e340 2024-05-09T06:40:57.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.780+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f971806e340 msgr2=0x7f9718070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.780+0000 7f9735de2700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f971806e340 0x7f9718070800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9724005b40 tx=0x7f9724005a90 comp rx=0 tx=0).stop 2024-05-09T06:40:57.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.780+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9730100c50 msgr2=0x7f9730074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.781+0000 7f9735de2700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9730100c50 0x7f9730074bf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f972000d6c0 tx=0x7f972000d9d0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.781+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 shutdown_connections 2024-05-09T06:40:57.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.781+0000 7f9735de2700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f971806e340 0x7f9718070800 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.781+0000 7f9735de2700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9730100c50 0x7f9730074bf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.781+0000 7f9735de2700 1 --2- 172.21.15.63:0/487150485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730101e50 0x7f97300732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.781+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 >> 172.21.15.63:0/487150485 conn(0x7f97300fc210 msgr2=0x7f97301053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:57.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.781+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 shutdown_connections 2024-05-09T06:40:57.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.781+0000 7f9735de2700 1 -- 172.21.15.63:0/487150485 wait complete. 2024-05-09T06:40:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.889+0000 7fa436930700 1 -- 172.21.15.63:0/888974575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 msgr2=0x7fa430101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.889+0000 7fa436930700 1 --2- 172.21.15.63:0/888974575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 0x7fa430101190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa424009910 tx=0x7fa424009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.889+0000 7fa436930700 1 -- 172.21.15.63:0/888974575 shutdown_connections 2024-05-09T06:40:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.889+0000 7fa436930700 1 --2- 172.21.15.63:0/888974575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 0x7fa430101190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.889+0000 7fa436930700 1 --2- 172.21.15.63:0/888974575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4300ffb30 0x7fa4300fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.889+0000 7fa436930700 1 -- 172.21.15.63:0/888974575 >> 172.21.15.63:0/888974575 conn(0x7fa4300fb0d0 msgr2=0x7fa4300fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.889+0000 7fa436930700 1 -- 172.21.15.63:0/888974575 shutdown_connections 2024-05-09T06:40:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.889+0000 7fa436930700 1 -- 172.21.15.63:0/888974575 wait complete. 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa436930700 1 Processor -- start 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa436930700 1 -- start start 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa436930700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4300ffb30 0x7fa430196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa436930700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 0x7fa430196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa436930700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4301971a0 con 0x7fa430100d30 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa436930700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4301972e0 con 0x7fa4300ffb30 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa42f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 0x7fa430196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa42f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 0x7fa430196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57180/0 (socket says 172.21.15.63:57180) 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa42f7fe700 1 -- 172.21.15.63:0/201285584 learned_addr learned my addr 172.21.15.63:0/201285584 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa42ffff700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4300ffb30 0x7fa430196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa42f7fe700 1 -- 172.21.15.63:0/201285584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4300ffb30 msgr2=0x7fa430196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa42f7fe700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4300ffb30 0x7fa430196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.890+0000 7fa42f7fe700 1 -- 172.21.15.63:0/201285584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4240095f0 con 0x7fa430100d30 2024-05-09T06:40:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.891+0000 7fa42f7fe700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 0x7fa430196b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa4240092e0 tx=0x7fa42400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.891+0000 7fa42d7fa700 1 -- 172.21.15.63:0/201285584 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa42401d070 con 0x7fa430100d30 2024-05-09T06:40:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.891+0000 7fa42d7fa700 1 -- 172.21.15.63:0/201285584 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa42400f460 con 0x7fa430100d30 2024-05-09T06:40:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.891+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43019bc90 con 0x7fa430100d30 2024-05-09T06:40:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.891+0000 7fa42d7fa700 1 -- 172.21.15.63:0/201285584 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa424021620 con 0x7fa430100d30 2024-05-09T06:40:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.891+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43019c200 con 0x7fa430100d30 2024-05-09T06:40:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.892+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43004ebb0 con 0x7fa430100d30 2024-05-09T06:40:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.892+0000 7fa42d7fa700 1 -- 172.21.15.63:0/201285584 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa42400f5d0 con 0x7fa430100d30 2024-05-09T06:40:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.893+0000 7fa42d7fa700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa41806e340 0x7fa418070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.893+0000 7fa42d7fa700 1 -- 172.21.15.63:0/201285584 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa42408f630 con 0x7fa430100d30 2024-05-09T06:40:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.893+0000 7fa42ffff700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa41806e340 0x7fa418070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.893+0000 7fa42ffff700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa41806e340 0x7fa418070800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa430100b90 tx=0x7fa42000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:57.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:57.898+0000 7fa42d7fa700 1 -- 172.21.15.63:0/201285584 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa42405c270 con 0x7fa430100d30 2024-05-09T06:40:58.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.114+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4300697d0 con 0x7fa41806e340 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.126+0000 7fa42d7fa700 1 -- 172.21.15.63:0/201285584 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fa4300697d0 con 0x7fa41806e340 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (9m) 40s ago 11m 25.1M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (11m) 40s ago 11m 7852k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (10m) 26s ago 10m 7876k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (11m) 40s ago 11m 6665k - 18.2.2-1297-g1758e8ea 196422f1b641 f2f8d93aacdd 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (10m) 26s ago 10m 6660k - 18.2.2-1297-g1758e8ea 196422f1b641 fcb163b69204 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (9m) 40s ago 11m 85.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (5m) 40s ago 5m 15.0M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (5m) 40s ago 5m 90.1M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (5m) 26s ago 5m 143M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (5m) 26s ago 5m 16.7M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:40:58.106 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (119s) 40s ago 12m 476M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (74s) 26s ago 10m 450M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (55s) 40s ago 13m 29.1M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (31s) 26s ago 10m 18.4M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (11m) 40s ago 11m 15.7M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (10m) 26s ago 10m 16.1M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (9m) 40s ago 9m 229M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (9m) 40s ago 9m 244M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (8m) 40s ago 8m 208M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (8m) 26s ago 8m 291M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (7m) 26s ago 7m 259M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (7m) 26s ago 7m 232M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:40:58.107 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (84s) 40s ago 10m 45.7M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:40:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa41806e340 msgr2=0x7fa418070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa41806e340 0x7fa418070800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa430100b90 tx=0x7fa42000b410 comp rx=0 tx=0).stop 2024-05-09T06:40:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 msgr2=0x7fa430196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 0x7fa430196b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa4240092e0 tx=0x7fa42400bd70 comp rx=0 tx=0).stop 2024-05-09T06:40:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 shutdown_connections 2024-05-09T06:40:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa41806e340 0x7fa418070800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4300ffb30 0x7fa430196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 --2- 172.21.15.63:0/201285584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa430100d30 0x7fa430196b80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 >> 172.21.15.63:0/201285584 conn(0x7fa4300fb0d0 msgr2=0x7fa4300690c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 shutdown_connections 2024-05-09T06:40:58.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.128+0000 7fa436930700 1 -- 172.21.15.63:0/201285584 wait complete. 2024-05-09T06:40:58.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 -- 172.21.15.63:0/1587659640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd41004e0 msgr2=0x7ffbd4100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 --2- 172.21.15.63:0/1587659640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd41004e0 0x7ffbd4100940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000aa20 tx=0x7ffbc000ad30 comp rx=0 tx=0).stop 2024-05-09T06:40:58.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 -- 172.21.15.63:0/1587659640 shutdown_connections 2024-05-09T06:40:58.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 --2- 172.21.15.63:0/1587659640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd41004e0 0x7ffbd4100940 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 --2- 172.21.15.63:0/1587659640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd40ff2e0 0x7ffbd40ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 -- 172.21.15.63:0/1587659640 >> 172.21.15.63:0/1587659640 conn(0x7ffbd40fa8a0 msgr2=0x7ffbd40fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:58.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 -- 172.21.15.63:0/1587659640 shutdown_connections 2024-05-09T06:40:58.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 -- 172.21.15.63:0/1587659640 wait complete. 2024-05-09T06:40:58.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 Processor -- start 2024-05-09T06:40:58.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.236+0000 7ffbdade0700 1 -- start start 2024-05-09T06:40:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbdade0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd40ff2e0 0x7ffbd4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbdade0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd41004e0 0x7ffbd4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbdade0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd41993a0 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbdade0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd41994e0 con 0x7ffbd41004e0 2024-05-09T06:40:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbd8b7c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd40ff2e0 0x7ffbd4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbd8b7c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd40ff2e0 0x7ffbd4198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57202/0 (socket says 172.21.15.63:57202) 2024-05-09T06:40:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbd8b7c700 1 -- 172.21.15.63:0/2137565021 learned_addr learned my addr 172.21.15.63:0/2137565021 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbd3fff700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd41004e0 0x7ffbd4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbd8b7c700 1 -- 172.21.15.63:0/2137565021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd41004e0 msgr2=0x7ffbd4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbd8b7c700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd41004e0 0x7ffbd4198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbd8b7c700 1 -- 172.21.15.63:0/2137565021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc000a700 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbd8b7c700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd40ff2e0 0x7ffbd4198840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800d900 tx=0x7ffbc800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:58.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbd1ffb700 1 -- 172.21.15.63:0/2137565021 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbc80041f0 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.237+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd419df40 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.238+0000 7ffbd1ffb700 1 -- 172.21.15.63:0/2137565021 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbc8004350 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.238+0000 7ffbd1ffb700 1 -- 172.21.15.63:0/2137565021 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbc8003dc0 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.238+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd419e400 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.239+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd41929e0 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.239+0000 7ffbd1ffb700 1 -- 172.21.15.63:0/2137565021 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ffbc8009730 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.239+0000 7ffbd1ffb700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffbc406e330 0x7ffbc40707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:58.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.240+0000 7ffbd1ffb700 1 -- 172.21.15.63:0/2137565021 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ffbc808e4f0 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.240+0000 7ffbd3fff700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffbc406e330 0x7ffbc40707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:58.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.240+0000 7ffbd3fff700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffbc406e330 0x7ffbc40707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00055e0 tx=0x7ffbc0005de0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:58.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.244+0000 7ffbd1ffb700 1 -- 172.21.15.63:0/2137565021 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ffbc805b1b0 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.548+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffbd4107650 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.549+0000 7ffbd1ffb700 1 -- 172.21.15.63:0/2137565021 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7ffbc805ad40 con 0x7ffbd40ff2e0 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:40:58.529 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 10, 2024-05-09T06:40:58.530 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 4 2024-05-09T06:40:58.530 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:40:58.530 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:40:58.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffbc406e330 msgr2=0x7ffbc40707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffbc406e330 0x7ffbc40707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00055e0 tx=0x7ffbc0005de0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd40ff2e0 msgr2=0x7ffbd4198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd40ff2e0 0x7ffbd4198840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800d900 tx=0x7ffbc800dcc0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 shutdown_connections 2024-05-09T06:40:58.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffbc406e330 0x7ffbc40707f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd40ff2e0 0x7ffbd4198840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 --2- 172.21.15.63:0/2137565021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd41004e0 0x7ffbd4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 >> 172.21.15.63:0/2137565021 conn(0x7ffbd40fa8a0 msgr2=0x7ffbd4103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:58.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 shutdown_connections 2024-05-09T06:40:58.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.551+0000 7ffbdade0700 1 -- 172.21.15.63:0/2137565021 wait complete. 2024-05-09T06:40:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.662+0000 7fcc6db9f700 1 -- 172.21.15.63:0/1090358514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 msgr2=0x7fcc68101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.662+0000 7fcc6db9f700 1 --2- 172.21.15.63:0/1090358514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 0x7fcc68101200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009910 tx=0x7fcc5c009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.662+0000 7fcc6db9f700 1 -- 172.21.15.63:0/1090358514 shutdown_connections 2024-05-09T06:40:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.662+0000 7fcc6db9f700 1 --2- 172.21.15.63:0/1090358514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 0x7fcc68101200 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.662+0000 7fcc6db9f700 1 --2- 172.21.15.63:0/1090358514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc680ffba0 0x7fcc680fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.662+0000 7fcc6db9f700 1 -- 172.21.15.63:0/1090358514 >> 172.21.15.63:0/1090358514 conn(0x7fcc680fb120 msgr2=0x7fcc680fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:58.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.662+0000 7fcc6db9f700 1 -- 172.21.15.63:0/1090358514 shutdown_connections 2024-05-09T06:40:58.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.662+0000 7fcc6db9f700 1 -- 172.21.15.63:0/1090358514 wait complete. 2024-05-09T06:40:58.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc6db9f700 1 Processor -- start 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc6db9f700 1 -- start start 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc6db9f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc680ffba0 0x7fcc68103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc6db9f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 0x7fcc68102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc6db9f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc681026c0 con 0x7fcc68100da0 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc6db9f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68102800 con 0x7fcc680ffba0 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc66ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 0x7fcc68102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc66ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 0x7fcc68102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57212/0 (socket says 172.21.15.63:57212) 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc66ffd700 1 -- 172.21.15.63:0/3700145318 learned_addr learned my addr 172.21.15.63:0/3700145318 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc677fe700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc680ffba0 0x7fcc68103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc66ffd700 1 -- 172.21.15.63:0/3700145318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc680ffba0 msgr2=0x7fcc68103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc66ffd700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc680ffba0 0x7fcc68103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.663+0000 7fcc66ffd700 1 -- 172.21.15.63:0/3700145318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c0095f0 con 0x7fcc68100da0 2024-05-09T06:40:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.664+0000 7fcc66ffd700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 0x7fcc68102180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0092e0 tx=0x7fcc5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.664+0000 7fcc64ff9700 1 -- 172.21.15.63:0/3700145318 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc5c01d070 con 0x7fcc68100da0 2024-05-09T06:40:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.664+0000 7fcc64ff9700 1 -- 172.21.15.63:0/3700145318 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc5c00f460 con 0x7fcc68100da0 2024-05-09T06:40:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.664+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc68102a80 con 0x7fcc68100da0 2024-05-09T06:40:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.664+0000 7fcc64ff9700 1 -- 172.21.15.63:0/3700145318 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc5c021620 con 0x7fcc68100da0 2024-05-09T06:40:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.664+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc68102f70 con 0x7fcc68100da0 2024-05-09T06:40:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.665+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6804ebb0 con 0x7fcc68100da0 2024-05-09T06:40:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.666+0000 7fcc64ff9700 1 -- 172.21.15.63:0/3700145318 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fcc5c00f5d0 con 0x7fcc68100da0 2024-05-09T06:40:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.666+0000 7fcc64ff9700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcc5006e340 0x7fcc50070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.666+0000 7fcc64ff9700 1 -- 172.21.15.63:0/3700145318 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcc5c08f640 con 0x7fcc68100da0 2024-05-09T06:40:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.666+0000 7fcc677fe700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcc5006e340 0x7fcc50070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.667+0000 7fcc677fe700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcc5006e340 0x7fcc50070800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc68100c00 tx=0x7fcc5800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.671+0000 7fcc64ff9700 1 -- 172.21.15.63:0/3700145318 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcc5c05c280 con 0x7fcc68100da0 2024-05-09T06:40:58.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:58 smithi063 ceph-mon[92765]: Reconfiguring mon.smithi193 (monmap changed)... 2024-05-09T06:40:58.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:58 smithi063 ceph-mon[92765]: Reconfiguring daemon mon.smithi193 on smithi193 2024-05-09T06:40:58.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:58 smithi063 ceph-mon[92765]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:40:58.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:58 smithi193 ceph-mon[81665]: Reconfiguring mon.smithi193 (monmap changed)... 2024-05-09T06:40:58.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:58 smithi193 ceph-mon[81665]: Reconfiguring daemon mon.smithi193 on smithi193 2024-05-09T06:40:58.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:58 smithi193 ceph-mon[81665]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:40:58.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.926+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcc681088b0 con 0x7fcc68100da0 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.927+0000 7fcc64ff9700 1 -- 172.21.15.63:0/3700145318 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1905 (secure 0 0 0) 0x7fcc5c05be10 con 0x7fcc68100da0 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:40:58.907 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:40:58.908 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:58.909 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:40:58.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcc5006e340 msgr2=0x7fcc50070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcc5006e340 0x7fcc50070800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc68100c00 tx=0x7fcc5800b410 comp rx=0 tx=0).stop 2024-05-09T06:40:58.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 msgr2=0x7fcc68102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:58.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 0x7fcc68102180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0092e0 tx=0x7fcc5c00bd70 comp rx=0 tx=0).stop 2024-05-09T06:40:58.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 shutdown_connections 2024-05-09T06:40:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcc5006e340 0x7fcc50070800 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc680ffba0 0x7fcc68103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 --2- 172.21.15.63:0/3700145318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcc68100da0 0x7fcc68102180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 >> 172.21.15.63:0/3700145318 conn(0x7fcc680fb120 msgr2=0x7fcc680685d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 shutdown_connections 2024-05-09T06:40:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:58.930+0000 7fcc6db9f700 1 -- 172.21.15.63:0/3700145318 wait complete. 2024-05-09T06:40:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:40:59.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.040+0000 7f4270a7c700 1 -- 172.21.15.63:0/4213711518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c104100 msgr2=0x7f426c1064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:59.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.040+0000 7f4270a7c700 1 --2- 172.21.15.63:0/4213711518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c104100 0x7f426c1064f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4260009910 tx=0x7f4260009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:59.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.040+0000 7f4270a7c700 1 -- 172.21.15.63:0/4213711518 shutdown_connections 2024-05-09T06:40:59.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.040+0000 7f4270a7c700 1 --2- 172.21.15.63:0/4213711518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c104100 0x7f426c1064f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.040+0000 7f4270a7c700 1 --2- 172.21.15.63:0/4213711518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426c1017d0 0x7f426c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.040+0000 7f4270a7c700 1 -- 172.21.15.63:0/4213711518 >> 172.21.15.63:0/4213711518 conn(0x7f426c0fb120 msgr2=0x7f426c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:59.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.040+0000 7f4270a7c700 1 -- 172.21.15.63:0/4213711518 shutdown_connections 2024-05-09T06:40:59.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.040+0000 7f4270a7c700 1 -- 172.21.15.63:0/4213711518 wait complete. 2024-05-09T06:40:59.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f4270a7c700 1 Processor -- start 2024-05-09T06:40:59.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f4270a7c700 1 -- start start 2024-05-09T06:40:59.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f4270a7c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c1017d0 0x7f426c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:59.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f4270a7c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426c104100 0x7f426c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:59.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f4270a7c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c1971d0 con 0x7f426c1017d0 2024-05-09T06:40:59.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f4270a7c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c197310 con 0x7f426c104100 2024-05-09T06:40:59.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f426a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c1017d0 0x7f426c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f426a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c1017d0 0x7f426c196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57222/0 (socket says 172.21.15.63:57222) 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f426a59c700 1 -- 172.21.15.63:0/3362162705 learned_addr learned my addr 172.21.15.63:0/3362162705 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.041+0000 7f4269d9b700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426c104100 0x7f426c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.042+0000 7f426a59c700 1 -- 172.21.15.63:0/3362162705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426c104100 msgr2=0x7f426c196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.042+0000 7f426a59c700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426c104100 0x7f426c196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.042+0000 7f426a59c700 1 -- 172.21.15.63:0/3362162705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600095f0 con 0x7f426c1017d0 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.042+0000 7f426a59c700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c1017d0 0x7f426c196670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f425c00d8d0 tx=0x7f425c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.042+0000 7f425b7fe700 1 -- 172.21.15.63:0/3362162705 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f425c0098a0 con 0x7f426c1017d0 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.042+0000 7f425b7fe700 1 -- 172.21.15.63:0/3362162705 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f425c010460 con 0x7f426c1017d0 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.042+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c19bd70 con 0x7f426c1017d0 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.042+0000 7f425b7fe700 1 -- 172.21.15.63:0/3362162705 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f425c00f5f0 con 0x7f426c1017d0 2024-05-09T06:40:59.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.042+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c19c260 con 0x7f426c1017d0 2024-05-09T06:40:59.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.043+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426c04ebb0 con 0x7f426c1017d0 2024-05-09T06:40:59.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.044+0000 7f425b7fe700 1 -- 172.21.15.63:0/3362162705 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f425c009a00 con 0x7f426c1017d0 2024-05-09T06:40:59.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.044+0000 7f425b7fe700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f425406e340 0x7f4254070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:59.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.044+0000 7f425b7fe700 1 -- 172.21.15.63:0/3362162705 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f425c08e1c0 con 0x7f426c1017d0 2024-05-09T06:40:59.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.044+0000 7f4269d9b700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f425406e340 0x7f4254070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:59.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.045+0000 7f4269d9b700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f425406e340 0x7f4254070800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4260005090 tx=0x7f4260005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:59.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.049+0000 7f425b7fe700 1 -- 172.21.15.63:0/3362162705 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f425c05ae80 con 0x7f426c1017d0 2024-05-09T06:40:59.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.276+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f426c104ac0 con 0x7f425406e340 2024-05-09T06:40:59.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.277+0000 7f425b7fe700 1 -- 172.21.15.63:0/3362162705 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f426c104ac0 con 0x7f425406e340 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "4/23 daemons upgraded", 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mon daemons", 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:40:59.257 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:40:59.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f425406e340 msgr2=0x7f4254070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:59.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f425406e340 0x7f4254070800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4260005090 tx=0x7f4260005960 comp rx=0 tx=0).stop 2024-05-09T06:40:59.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c1017d0 msgr2=0x7f426c196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:59.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c1017d0 0x7f426c196670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f425c00d8d0 tx=0x7f425c00dbe0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 shutdown_connections 2024-05-09T06:40:59.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f425406e340 0x7f4254070800 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f426c1017d0 0x7f426c196670 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 --2- 172.21.15.63:0/3362162705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426c104100 0x7f426c196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 >> 172.21.15.63:0/3362162705 conn(0x7f426c0fb120 msgr2=0x7f426c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:59.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 shutdown_connections 2024-05-09T06:40:59.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.280+0000 7f4270a7c700 1 -- 172.21.15.63:0/3362162705 wait complete. 2024-05-09T06:40:59.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.390+0000 7f049ca9d700 1 -- 172.21.15.63:0/2745700328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498100c20 msgr2=0x7f0498101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.390+0000 7f049ca9d700 1 --2- 172.21.15.63:0/2745700328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498100c20 0x7f0498101040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0488009910 tx=0x7f0488009c20 comp rx=0 tx=0).stop 2024-05-09T06:40:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.390+0000 7f049ca9d700 1 -- 172.21.15.63:0/2745700328 shutdown_connections 2024-05-09T06:40:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.390+0000 7f049ca9d700 1 --2- 172.21.15.63:0/2745700328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0498101e20 0x7f0498102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.390+0000 7f049ca9d700 1 --2- 172.21.15.63:0/2745700328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498100c20 0x7f0498101040 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.390+0000 7f049ca9d700 1 -- 172.21.15.63:0/2745700328 >> 172.21.15.63:0/2745700328 conn(0x7f04980fc200 msgr2=0x7f04980fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.390+0000 7f049ca9d700 1 -- 172.21.15.63:0/2745700328 shutdown_connections 2024-05-09T06:40:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.390+0000 7f049ca9d700 1 -- 172.21.15.63:0/2745700328 wait complete. 2024-05-09T06:40:59.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.391+0000 7f049ca9d700 1 Processor -- start 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.391+0000 7f049ca9d700 1 -- start start 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.391+0000 7f049ca9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0498100c20 0x7f0498074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.391+0000 7f049ca9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498101e20 0x7f04980732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.391+0000 7f049ca9d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498073820 con 0x7f0498101e20 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.391+0000 7f049ca9d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498073960 con 0x7f0498100c20 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f0495d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498101e20 0x7f04980732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f0495d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498101e20 0x7f04980732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57242/0 (socket says 172.21.15.63:57242) 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f0495d9b700 1 -- 172.21.15.63:0/4105871033 learned_addr learned my addr 172.21.15.63:0/4105871033 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f049659c700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0498100c20 0x7f0498074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f0495d9b700 1 -- 172.21.15.63:0/4105871033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0498100c20 msgr2=0x7f0498074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f0495d9b700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0498100c20 0x7f0498074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f0495d9b700 1 -- 172.21.15.63:0/4105871033 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880095f0 con 0x7f0498101e20 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f0495d9b700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498101e20 0x7f04980732e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f048000c8a0 tx=0x7f048000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f048f7fe700 1 -- 172.21.15.63:0/4105871033 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04800078e0 con 0x7f0498101e20 2024-05-09T06:40:59.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f048f7fe700 1 -- 172.21.15.63:0/4105871033 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f048000f450 con 0x7f0498101e20 2024-05-09T06:40:59.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0498073c40 con 0x7f0498101e20 2024-05-09T06:40:59.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f048f7fe700 1 -- 172.21.15.63:0/4105871033 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f048000e5e0 con 0x7f0498101e20 2024-05-09T06:40:59.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.392+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049819e6a0 con 0x7f0498101e20 2024-05-09T06:40:59.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.393+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04981907a0 con 0x7f0498101e20 2024-05-09T06:40:59.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.394+0000 7f048f7fe700 1 -- 172.21.15.63:0/4105871033 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0480007a40 con 0x7f0498101e20 2024-05-09T06:40:59.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.394+0000 7f048f7fe700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f048406e340 0x7f0484070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:40:59.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.394+0000 7f048f7fe700 1 -- 172.21.15.63:0/4105871033 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f048008e290 con 0x7f0498101e20 2024-05-09T06:40:59.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.394+0000 7f049659c700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f048406e340 0x7f0484070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:40:59.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.395+0000 7f049659c700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f048406e340 0x7f0484070800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0488000c00 tx=0x7f0488005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:40:59.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.399+0000 7f048f7fe700 1 -- 172.21.15.63:0/4105871033 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f048005af50 con 0x7f0498101e20 2024-05-09T06:40:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.695+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0498109780 con 0x7f0498101e20 2024-05-09T06:40:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.695+0000 7f048f7fe700 1 -- 172.21.15.63:0/4105871033 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f048005aae0 con 0x7f0498101e20 2024-05-09T06:40:59.675 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f048406e340 msgr2=0x7f0484070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f048406e340 0x7f0484070800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0488000c00 tx=0x7f0488005960 comp rx=0 tx=0).stop 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498101e20 msgr2=0x7f04980732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498101e20 0x7f04980732e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f048000c8a0 tx=0x7f048000cbb0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 shutdown_connections 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f048406e340 0x7f0484070800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0498100c20 0x7f0498074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 --2- 172.21.15.63:0/4105871033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0498101e20 0x7f04980732e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 >> 172.21.15.63:0/4105871033 conn(0x7f04980fc200 msgr2=0x7f0498105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 shutdown_connections 2024-05-09T06:40:59.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:40:59.698+0000 7f049ca9d700 1 -- 172.21.15.63:0/4105871033 wait complete. 2024-05-09T06:40:59.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:40:59.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:59.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:40:59.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:59.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2137565021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:40:59.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:59.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: Reconfiguring osd.3 (monmap changed)... 2024-05-09T06:40:59.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T06:40:59.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:59.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: Reconfiguring daemon osd.3 on smithi193 2024-05-09T06:40:59.857 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:40:59 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3700145318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:40:59.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:40:59.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:40:59.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:40:59.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:59.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2137565021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:40:59.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:40:59.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: Reconfiguring osd.3 (monmap changed)... 2024-05-09T06:40:59.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T06:40:59.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:40:59.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: Reconfiguring daemon osd.3 on smithi193 2024-05-09T06:40:59.873 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:40:59 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3700145318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:41:00.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:00 smithi063 ceph-mon[92765]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:00.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:00 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/4105871033' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:41:00.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:00 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:00.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:00 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:00.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:00 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T06:41:00.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:00 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:00.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:00 smithi193 ceph-mon[81665]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:00.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:00 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/4105871033' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:41:00.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:00 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:00.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:00 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:00.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:00 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T06:41:00.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:00 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:01.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:01 smithi063 ceph-mon[92765]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:41:01.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:01 smithi063 ceph-mon[92765]: Reconfiguring osd.4 (monmap changed)... 2024-05-09T06:41:01.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:01 smithi063 ceph-mon[92765]: Reconfiguring daemon osd.4 on smithi193 2024-05-09T06:41:01.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:01 smithi193 ceph-mon[81665]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:41:01.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:01 smithi193 ceph-mon[81665]: Reconfiguring osd.4 (monmap changed)... 2024-05-09T06:41:01.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:01 smithi193 ceph-mon[81665]: Reconfiguring daemon osd.4 on smithi193 2024-05-09T06:41:03.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:03.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:03.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:03 smithi063 ceph-mon[92765]: Reconfiguring osd.5 (monmap changed)... 2024-05-09T06:41:03.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T06:41:03.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:03.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:03 smithi063 ceph-mon[92765]: Reconfiguring daemon osd.5 on smithi193 2024-05-09T06:41:03.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:03 smithi063 ceph-mon[92765]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:03.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:03.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:03.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:03 smithi193 ceph-mon[81665]: Reconfiguring osd.5 (monmap changed)... 2024-05-09T06:41:03.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T06:41:03.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:03.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:03 smithi193 ceph-mon[81665]: Reconfiguring daemon osd.5 on smithi193 2024-05-09T06:41:03.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:03 smithi193 ceph-mon[81665]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:05.021 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:05.021 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:05.022 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:04 smithi193 ceph-mon[81665]: Reconfiguring mds.cephfs.smithi193.ggtmyb (monmap changed)... 2024-05-09T06:41:05.022 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ggtmyb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:41:05.022 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:05.022 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:04 smithi193 ceph-mon[81665]: Reconfiguring daemon mds.cephfs.smithi193.ggtmyb on smithi193 2024-05-09T06:41:05.022 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:04 smithi193 ceph-mon[81665]: pgmap v29: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:05.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:05.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:05.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:04 smithi063 ceph-mon[92765]: Reconfiguring mds.cephfs.smithi193.ggtmyb (monmap changed)... 2024-05-09T06:41:05.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ggtmyb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:41:05.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:05.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:04 smithi063 ceph-mon[92765]: Reconfiguring daemon mds.cephfs.smithi193.ggtmyb on smithi193 2024-05-09T06:41:05.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:04 smithi063 ceph-mon[92765]: pgmap v29: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:06.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:06 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:06.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:06 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:06.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:06 smithi063 ceph-mon[92765]: Reconfiguring mds.cephfs.smithi193.sygxnf (monmap changed)... 2024-05-09T06:41:06.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:06 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sygxnf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:41:06.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:06 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:06.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:06 smithi063 ceph-mon[92765]: Reconfiguring daemon mds.cephfs.smithi193.sygxnf on smithi193 2024-05-09T06:41:06.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:06 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:06.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:06 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:06.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:06 smithi193 ceph-mon[81665]: Reconfiguring mds.cephfs.smithi193.sygxnf (monmap changed)... 2024-05-09T06:41:06.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:06 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sygxnf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:41:06.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:06 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:06.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:06 smithi193 ceph-mon[81665]: Reconfiguring daemon mds.cephfs.smithi193.sygxnf on smithi193 2024-05-09T06:41:07.668 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: pgmap v30: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:07.668 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:07.668 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:07.668 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:07.669 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:07.669 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:07.669 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:07.669 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]: dispatch 2024-05-09T06:41:07.669 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]': finished 2024-05-09T06:41:07.669 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]: dispatch 2024-05-09T06:41:07.669 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]': finished 2024-05-09T06:41:07.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: pgmap v30: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]: dispatch 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]': finished 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]: dispatch 2024-05-09T06:41:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]': finished 2024-05-09T06:41:08.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:08 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all mon 2024-05-09T06:41:08.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:08 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all mon 2024-05-09T06:41:09.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:09 smithi063 ceph-mon[92765]: pgmap v31: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:09.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:09.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:41:09.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:09 smithi193 ceph-mon[81665]: pgmap v31: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:09.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:09.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:41:09.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:10.540 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:10 smithi063 ceph-mon[92765]: Upgrade: Updating crash.smithi063 (1/2) 2024-05-09T06:41:10.540 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:10 smithi063 ceph-mon[92765]: Deploying daemon crash.smithi063 on smithi063 2024-05-09T06:41:10.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:10 smithi193 ceph-mon[81665]: Upgrade: Updating crash.smithi063 (1/2) 2024-05-09T06:41:10.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:10 smithi193 ceph-mon[81665]: Deploying daemon crash.smithi063 on smithi063 2024-05-09T06:41:11.822 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:11 smithi063 ceph-mon[92765]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:11.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:11 smithi193 ceph-mon[81665]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:12.454 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:41:12.454 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:12.454 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:12.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:41:12.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:12.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:13.559 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:13 smithi193 ceph-mon[81665]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:13.747 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:13 smithi063 ceph-mon[92765]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:15.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:15 smithi063 ceph-mon[92765]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:15.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:15 smithi063 ceph-mon[92765]: Upgrade: Updating crash.smithi193 (2/2) 2024-05-09T06:41:15.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:15 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:15.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:15 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:41:15.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:15 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:15.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:15 smithi063 ceph-mon[92765]: Deploying daemon crash.smithi193 on smithi193 2024-05-09T06:41:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:15 smithi193 ceph-mon[81665]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:15 smithi193 ceph-mon[81665]: Upgrade: Updating crash.smithi193 (2/2) 2024-05-09T06:41:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:15 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:15 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-09T06:41:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:15 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:15.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:15 smithi193 ceph-mon[81665]: Deploying daemon crash.smithi193 on smithi193 2024-05-09T06:41:17.348 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:17 smithi193 ceph-mon[81665]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:17.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:17 smithi063 ceph-mon[92765]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:19.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:18 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:19.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:18 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:19.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:18 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:19.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:18 smithi063 ceph-mon[92765]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:19.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:18 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:19.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:18 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:19.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:18 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:19.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:18 smithi193 ceph-mon[81665]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:21.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:21 smithi063 ceph-mon[92765]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:21.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:21.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:21.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:21 smithi193 ceph-mon[81665]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:21.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:21.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:23.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:23 smithi193 ceph-mon[81665]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:23.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:23.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:23.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:23.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:23 smithi063 ceph-mon[92765]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:25.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:25 smithi063 ceph-mon[92765]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:25.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:25 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:25.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:25 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:25.529 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:25 smithi193 ceph-mon[81665]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:25.529 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:25 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:25.529 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:25 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:26.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:26.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:26.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:26.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:27.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:27 smithi063 ceph-mon[92765]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:27.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:41:27.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:27 smithi193 ceph-mon[81665]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:27.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:41:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:41:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all crash 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]: dispatch 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]': finished 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]: dispatch 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]': finished 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: Upgrade: osd.0 is safe to restart 2024-05-09T06:41:28.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:28 smithi063 ceph-mon[92765]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:29.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:29.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all crash 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]: dispatch 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]': finished 2024-05-09T06:41:29.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]: dispatch 2024-05-09T06:41:29.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]': finished 2024-05-09T06:41:29.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-09T06:41:29.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-09T06:41:29.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: Upgrade: osd.0 is safe to restart 2024-05-09T06:41:29.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:28 smithi193 ceph-mon[81665]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:41:29.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.812+0000 7ffb4740b700 1 -- 172.21.15.63:0/4284970413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40101e50 msgr2=0x7ffb401022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:29.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.812+0000 7ffb4740b700 1 --2- 172.21.15.63:0/4284970413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40101e50 0x7ffb401022b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009910 tx=0x7ffb34009c20 comp rx=0 tx=0).stop 2024-05-09T06:41:29.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.812+0000 7ffb4740b700 1 -- 172.21.15.63:0/4284970413 shutdown_connections 2024-05-09T06:41:29.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.812+0000 7ffb4740b700 1 --2- 172.21.15.63:0/4284970413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40101e50 0x7ffb401022b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:29.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.812+0000 7ffb4740b700 1 --2- 172.21.15.63:0/4284970413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40100c50 0x7ffb40101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:29.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.812+0000 7ffb4740b700 1 -- 172.21.15.63:0/4284970413 >> 172.21.15.63:0/4284970413 conn(0x7ffb400fc210 msgr2=0x7ffb400fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:29.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.812+0000 7ffb4740b700 1 -- 172.21.15.63:0/4284970413 shutdown_connections 2024-05-09T06:41:29.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.812+0000 7ffb4740b700 1 -- 172.21.15.63:0/4284970413 wait complete. 2024-05-09T06:41:29.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.813+0000 7ffb4740b700 1 Processor -- start 2024-05-09T06:41:29.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.813+0000 7ffb4740b700 1 -- start start 2024-05-09T06:41:29.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb4740b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40100c50 0x7ffb40074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:29.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb4740b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40101e50 0x7ffb400732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:29.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb4740b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400751f0 con 0x7ffb40100c50 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb4740b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40073820 con 0x7ffb40101e50 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb451a7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40100c50 0x7ffb40074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb451a7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40100c50 0x7ffb40074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49566/0 (socket says 172.21.15.63:49566) 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb451a7700 1 -- 172.21.15.63:0/2527523330 learned_addr learned my addr 172.21.15.63:0/2527523330 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb449a6700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40101e50 0x7ffb400732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb451a7700 1 -- 172.21.15.63:0/2527523330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40101e50 msgr2=0x7ffb400732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb451a7700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40101e50 0x7ffb400732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb451a7700 1 -- 172.21.15.63:0/2527523330 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340095f0 con 0x7ffb40100c50 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb451a7700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40100c50 0x7ffb40074bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b480 tx=0x7ffb3c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:29.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.814+0000 7ffb327fc700 1 -- 172.21.15.63:0/2527523330 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3c011800 con 0x7ffb40100c50 2024-05-09T06:41:29.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.815+0000 7ffb327fc700 1 -- 172.21.15.63:0/2527523330 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb3c011e20 con 0x7ffb40100c50 2024-05-09T06:41:29.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.815+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb40073b00 con 0x7ffb40100c50 2024-05-09T06:41:29.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.815+0000 7ffb327fc700 1 -- 172.21.15.63:0/2527523330 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3c010ea0 con 0x7ffb40100c50 2024-05-09T06:41:29.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.815+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb400740d0 con 0x7ffb40100c50 2024-05-09T06:41:29.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.816+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40190870 con 0x7ffb40100c50 2024-05-09T06:41:29.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.816+0000 7ffb327fc700 1 -- 172.21.15.63:0/2527523330 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ffb3c010380 con 0x7ffb40100c50 2024-05-09T06:41:29.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.817+0000 7ffb327fc700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffb2c06e340 0x7ffb2c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:29.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.817+0000 7ffb327fc700 1 -- 172.21.15.63:0/2527523330 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ffb3c08dd90 con 0x7ffb40100c50 2024-05-09T06:41:29.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.817+0000 7ffb449a6700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffb2c06e340 0x7ffb2c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:29.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.818+0000 7ffb449a6700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffb2c06e340 0x7ffb2c070800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb34005090 tx=0x7ffb34005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:29.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:29.822+0000 7ffb327fc700 1 -- 172.21.15.63:0/2527523330 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ffb3c05aae0 con 0x7ffb40100c50 2024-05-09T06:41:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.065+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb4002ce50 con 0x7ffb2c06e340 2024-05-09T06:41:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.066+0000 7ffb327fc700 1 -- 172.21.15.63:0/2527523330 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ffb4002ce50 con 0x7ffb2c06e340 2024-05-09T06:41:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffb2c06e340 msgr2=0x7ffb2c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffb2c06e340 0x7ffb2c070800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb34005090 tx=0x7ffb34005960 comp rx=0 tx=0).stop 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40100c50 msgr2=0x7ffb40074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40100c50 0x7ffb40074bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b480 tx=0x7ffb3c00b790 comp rx=0 tx=0).stop 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 shutdown_connections 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ffb2c06e340 0x7ffb2c070800 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb40100c50 0x7ffb40074bd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 --2- 172.21.15.63:0/2527523330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40101e50 0x7ffb400732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 >> 172.21.15.63:0/2527523330 conn(0x7ffb400fc210 msgr2=0x7ffb401054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 shutdown_connections 2024-05-09T06:41:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.069+0000 7ffb4740b700 1 -- 172.21.15.63:0/2527523330 wait complete. 2024-05-09T06:41:30.059 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:41:30.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.180+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/1682817111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4104070 msgr2=0x7fb5c4106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.180+0000 7fb5cc3b2700 1 --2- 172.21.15.63:0/1682817111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4104070 0x7fb5c4106460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009910 tx=0x7fb5b4009c20 comp rx=0 tx=0).stop 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/1682817111 shutdown_connections 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 --2- 172.21.15.63:0/1682817111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4104070 0x7fb5c4106460 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 --2- 172.21.15.63:0/1682817111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c4101740 0x7fb5c4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/1682817111 >> 172.21.15.63:0/1682817111 conn(0x7fb5c40fb110 msgr2=0x7fb5c40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/1682817111 shutdown_connections 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/1682817111 wait complete. 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 Processor -- start 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 -- start start 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4101740 0x7fb5c41965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c4104070 0x7fb5c4196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c41970c0 con 0x7fb5c4101740 2024-05-09T06:41:30.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.181+0000 7fb5cc3b2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4197200 con 0x7fb5c4104070 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5ca14e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4101740 0x7fb5c41965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5ca14e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4101740 0x7fb5c41965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49572/0 (socket says 172.21.15.63:49572) 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5ca14e700 1 -- 172.21.15.63:0/2235125582 learned_addr learned my addr 172.21.15.63:0/2235125582 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5c994d700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c4104070 0x7fb5c4196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5ca14e700 1 -- 172.21.15.63:0/2235125582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c4104070 msgr2=0x7fb5c4196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5ca14e700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c4104070 0x7fb5c4196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5ca14e700 1 -- 172.21.15.63:0/2235125582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b40095f0 con 0x7fb5c4101740 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5ca14e700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4101740 0x7fb5c41965f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c930 tx=0x7fb5c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5bb7fe700 1 -- 172.21.15.63:0/2235125582 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c0007ad0 con 0x7fb5c4101740 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5bb7fe700 1 -- 172.21.15.63:0/2235125582 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5c0007c30 con 0x7fb5c4101740 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c419bc70 con 0x7fb5c4101740 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5bb7fe700 1 -- 172.21.15.63:0/2235125582 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c0018610 con 0x7fb5c4101740 2024-05-09T06:41:30.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.182+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c419c190 con 0x7fb5c4101740 2024-05-09T06:41:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.183+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c404ebb0 con 0x7fb5c4101740 2024-05-09T06:41:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.184+0000 7fb5bb7fe700 1 -- 172.21.15.63:0/2235125582 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fb5c000f450 con 0x7fb5c4101740 2024-05-09T06:41:30.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.184+0000 7fb5bb7fe700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb5b006e340 0x7fb5b0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.184+0000 7fb5bb7fe700 1 -- 172.21.15.63:0/2235125582 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb5c008e5e0 con 0x7fb5c4101740 2024-05-09T06:41:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.184+0000 7fb5c994d700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb5b006e340 0x7fb5b0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:30.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.185+0000 7fb5c994d700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb5b006e340 0x7fb5b0070800 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4006010 tx=0x7fb5b400b580 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:30.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.189+0000 7fb5bb7fe700 1 -- 172.21.15.63:0/2235125582 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb5c005b220 con 0x7fb5c4101740 2024-05-09T06:41:30.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.425+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5c4105f50 con 0x7fb5b006e340 2024-05-09T06:41:30.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.426+0000 7fb5bb7fe700 1 -- 172.21.15.63:0/2235125582 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb5c4105f50 con 0x7fb5b006e340 2024-05-09T06:41:30.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.428+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb5b006e340 msgr2=0x7fb5b0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.428+0000 7fb5cc3b2700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb5b006e340 0x7fb5b0070800 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4006010 tx=0x7fb5b400b580 comp rx=0 tx=0).stop 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.428+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4101740 msgr2=0x7fb5c41965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.428+0000 7fb5cc3b2700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4101740 0x7fb5c41965f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c930 tx=0x7fb5c000ccf0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.429+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 shutdown_connections 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.429+0000 7fb5cc3b2700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb5b006e340 0x7fb5b0070800 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.429+0000 7fb5cc3b2700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5c4101740 0x7fb5c41965f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.429+0000 7fb5cc3b2700 1 --2- 172.21.15.63:0/2235125582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c4104070 0x7fb5c4196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.429+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 >> 172.21.15.63:0/2235125582 conn(0x7fb5c40fb110 msgr2=0x7fb5c40fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.429+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 shutdown_connections 2024-05-09T06:41:30.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.429+0000 7fb5cc3b2700 1 -- 172.21.15.63:0/2235125582 wait complete. 2024-05-09T06:41:30.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.540+0000 7f305d511700 1 -- 172.21.15.63:0/3140787882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 msgr2=0x7f30580fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.540+0000 7f305d511700 1 --2- 172.21.15.63:0/3140787882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 0x7f30580fffc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3048009910 tx=0x7f3048009c20 comp rx=0 tx=0).stop 2024-05-09T06:41:30.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.540+0000 7f305d511700 1 -- 172.21.15.63:0/3140787882 shutdown_connections 2024-05-09T06:41:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.540+0000 7f305d511700 1 --2- 172.21.15.63:0/3140787882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3058100da0 0x7f3058101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.540+0000 7f305d511700 1 --2- 172.21.15.63:0/3140787882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 0x7f30580fffc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.540+0000 7f305d511700 1 -- 172.21.15.63:0/3140787882 >> 172.21.15.63:0/3140787882 conn(0x7f30580fb120 msgr2=0x7f30580fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.540+0000 7f305d511700 1 -- 172.21.15.63:0/3140787882 shutdown_connections 2024-05-09T06:41:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.540+0000 7f305d511700 1 -- 172.21.15.63:0/3140787882 wait complete. 2024-05-09T06:41:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.540+0000 7f305d511700 1 Processor -- start 2024-05-09T06:41:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f305d511700 1 -- start start 2024-05-09T06:41:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f305d511700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 0x7f3058196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f305d511700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3058100da0 0x7f3058196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f305d511700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3058197290 con 0x7f30580ffba0 2024-05-09T06:41:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f305d511700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30581973d0 con 0x7f3058100da0 2024-05-09T06:41:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f3056ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 0x7f3058196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f3056ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 0x7f3058196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49596/0 (socket says 172.21.15.63:49596) 2024-05-09T06:41:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f3056ffd700 1 -- 172.21.15.63:0/2080509605 learned_addr learned my addr 172.21.15.63:0/2080509605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:41:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f30567fc700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3058100da0 0x7f3058196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f3056ffd700 1 -- 172.21.15.63:0/2080509605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3058100da0 msgr2=0x7f3058196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f3056ffd700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3058100da0 0x7f3058196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.541+0000 7f3056ffd700 1 -- 172.21.15.63:0/2080509605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30480095f0 con 0x7f30580ffba0 2024-05-09T06:41:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.542+0000 7f3056ffd700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 0x7f3058196730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f304800bd70 tx=0x7f304800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:30.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.544+0000 7f303ffff700 1 -- 172.21.15.63:0/2080509605 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304801d070 con 0x7f30580ffba0 2024-05-09T06:41:30.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.544+0000 7f303ffff700 1 -- 172.21.15.63:0/2080509605 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f304800f460 con 0x7f30580ffba0 2024-05-09T06:41:30.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.544+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305819bdd0 con 0x7f30580ffba0 2024-05-09T06:41:30.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.544+0000 7f303ffff700 1 -- 172.21.15.63:0/2080509605 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3048021640 con 0x7f30580ffba0 2024-05-09T06:41:30.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.544+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305819c2e0 con 0x7f30580ffba0 2024-05-09T06:41:30.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.545+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f305804ec20 con 0x7f30580ffba0 2024-05-09T06:41:30.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.545+0000 7f303ffff700 1 -- 172.21.15.63:0/2080509605 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f304802b440 con 0x7f30580ffba0 2024-05-09T06:41:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.546+0000 7f303ffff700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f304006e330 0x7f30400707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.546+0000 7f303ffff700 1 -- 172.21.15.63:0/2080509605 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f304808f5d0 con 0x7f30580ffba0 2024-05-09T06:41:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.546+0000 7f30567fc700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f304006e330 0x7f30400707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:30.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.546+0000 7f30567fc700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f304006e330 0x7f30400707f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f304c009a10 tx=0x7f304c009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:30.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.553+0000 7f303ffff700 1 -- 172.21.15.63:0/2080509605 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f305804ec20 con 0x7f30580ffba0 2024-05-09T06:41:30.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.772+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f305804ebb0 con 0x7f304006e330 2024-05-09T06:41:30.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:30 smithi063 ceph-mon[92765]: Upgrade: Updating osd.0 2024-05-09T06:41:30.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:30 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:30.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:30 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T06:41:30.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:30 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:30.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:30 smithi063 ceph-mon[92765]: Deploying daemon osd.0 on smithi063 2024-05-09T06:41:30.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.784+0000 7f303ffff700 1 -- 172.21.15.63:0/2080509605 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f305804ebb0 con 0x7f304006e330 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (10m) 7s ago 11m 25.1M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (12m) 7s ago 12m 7936k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (11m) 9s ago 11m 7944k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (18s) 7s ago 12m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (13s) 9s ago 11m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (10m) 7s ago 11m 87.9M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (6m) 7s ago 6m 15.8M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (6m) 7s ago 6m 90.4M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (6m) 9s ago 6m 143M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (6m) 9s ago 6m 17.1M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (2m) 7s ago 13m 499M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:41:30.764 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (106s) 9s ago 10m 450M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (88s) 7s ago 13m 43.5M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (64s) 9s ago 10m 32.2M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (12m) 7s ago 12m 16.0M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (10m) 9s ago 10m 16.2M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (10m) 7s ago 10m 230M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 7f26bfd264c7 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (9m) 7s ago 9m 245M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (9m) 7s ago 9m 209M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (8m) 9s ago 8m 292M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (8m) 9s ago 8m 260M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (7m) 9s ago 7m 233M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:41:30.765 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (117s) 7s ago 11m 53.5M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:41:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f304006e330 msgr2=0x7f30400707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f304006e330 0x7f30400707f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f304c009a10 tx=0x7f304c009190 comp rx=0 tx=0).stop 2024-05-09T06:41:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 msgr2=0x7f3058196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 0x7f3058196730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f304800bd70 tx=0x7f304800bda0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 shutdown_connections 2024-05-09T06:41:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f304006e330 0x7f30400707f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30580ffba0 0x7f3058196730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 --2- 172.21.15.63:0/2080509605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3058100da0 0x7f3058196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 >> 172.21.15.63:0/2080509605 conn(0x7f30580fb120 msgr2=0x7f30580685d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 shutdown_connections 2024-05-09T06:41:30.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.786+0000 7f305d511700 1 -- 172.21.15.63:0/2080509605 wait complete. 2024-05-09T06:41:30.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:30 smithi193 ceph-mon[81665]: Upgrade: Updating osd.0 2024-05-09T06:41:30.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:30 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:30.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:30 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T06:41:30.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:30 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:30.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:30 smithi193 ceph-mon[81665]: Deploying daemon osd.0 on smithi063 2024-05-09T06:41:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.896+0000 7fa4fe81b700 1 -- 172.21.15.63:0/2848513978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f8100da0 msgr2=0x7fa4f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.896+0000 7fa4fe81b700 1 --2- 172.21.15.63:0/2848513978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f8100da0 0x7fa4f8101200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009910 tx=0x7fa4ec009c20 comp rx=0 tx=0).stop 2024-05-09T06:41:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.896+0000 7fa4fe81b700 1 -- 172.21.15.63:0/2848513978 shutdown_connections 2024-05-09T06:41:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.896+0000 7fa4fe81b700 1 --2- 172.21.15.63:0/2848513978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f8100da0 0x7fa4f8101200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.896+0000 7fa4fe81b700 1 --2- 172.21.15.63:0/2848513978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f80ffba0 0x7fa4f80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.896+0000 7fa4fe81b700 1 -- 172.21.15.63:0/2848513978 >> 172.21.15.63:0/2848513978 conn(0x7fa4f80fb120 msgr2=0x7fa4f80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.896+0000 7fa4fe81b700 1 -- 172.21.15.63:0/2848513978 shutdown_connections 2024-05-09T06:41:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.896+0000 7fa4fe81b700 1 -- 172.21.15.63:0/2848513978 wait complete. 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.896+0000 7fa4fe81b700 1 Processor -- start 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4fe81b700 1 -- start start 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4fe81b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f80ffba0 0x7fa4f8103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4fe81b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100da0 0x7fa4f8102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4fe81b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f8104090 con 0x7fa4f80ffba0 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4fe81b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f81026c0 con 0x7fa4f8100da0 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f80ffba0 0x7fa4f8103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f80ffba0 0x7fa4f8103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49614/0 (socket says 172.21.15.63:49614) 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4f7fff700 1 -- 172.21.15.63:0/1695794739 learned_addr learned my addr 172.21.15.63:0/1695794739 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:41:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4f77fe700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100da0 0x7fa4f8102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4f7fff700 1 -- 172.21.15.63:0/1695794739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100da0 msgr2=0x7fa4f8102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4f7fff700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100da0 0x7fa4f8102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.897+0000 7fa4f7fff700 1 -- 172.21.15.63:0/1695794739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec0095f0 con 0x7fa4f80ffba0 2024-05-09T06:41:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.898+0000 7fa4f7fff700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f80ffba0 0x7fa4f8103a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800d900 tx=0x7fa4e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.898+0000 7fa4f57fa700 1 -- 172.21.15.63:0/1695794739 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4e80041f0 con 0x7fa4f80ffba0 2024-05-09T06:41:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.898+0000 7fa4f57fa700 1 -- 172.21.15.63:0/1695794739 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4e8004350 con 0x7fa4f80ffba0 2024-05-09T06:41:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.898+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f81029a0 con 0x7fa4f80ffba0 2024-05-09T06:41:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.898+0000 7fa4f57fa700 1 -- 172.21.15.63:0/1695794739 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4e8003cf0 con 0x7fa4f80ffba0 2024-05-09T06:41:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.898+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f8102ef0 con 0x7fa4f80ffba0 2024-05-09T06:41:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.899+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f804ebb0 con 0x7fa4f80ffba0 2024-05-09T06:41:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.899+0000 7fa4f57fa700 1 -- 172.21.15.63:0/1695794739 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa4e8009730 con 0x7fa4f80ffba0 2024-05-09T06:41:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.900+0000 7fa4f57fa700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa4e006e330 0x7fa4e00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.900+0000 7fa4f57fa700 1 -- 172.21.15.63:0/1695794739 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa4e808e1a0 con 0x7fa4f80ffba0 2024-05-09T06:41:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.900+0000 7fa4f77fe700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa4e006e330 0x7fa4e00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:30.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.900+0000 7fa4f77fe700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa4e006e330 0x7fa4e00707f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006010 tx=0x7fa4ec00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:30.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:30.904+0000 7fa4f57fa700 1 -- 172.21.15.63:0/1695794739 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa4e805ae60 con 0x7fa4f80ffba0 2024-05-09T06:41:31.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.205+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa4f8108970 con 0x7fa4f80ffba0 2024-05-09T06:41:31.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.206+0000 7fa4f57fa700 1 -- 172.21.15.63:0/1695794739 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7fa4e805a9f0 con 0x7fa4f80ffba0 2024-05-09T06:41:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:41:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:41:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:41:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:41:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:41:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:41:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:41:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 10, 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 4 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:41:31.187 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:41:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa4e006e330 msgr2=0x7fa4e00707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa4e006e330 0x7fa4e00707f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006010 tx=0x7fa4ec00b540 comp rx=0 tx=0).stop 2024-05-09T06:41:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f80ffba0 msgr2=0x7fa4f8103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f80ffba0 0x7fa4f8103a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800d900 tx=0x7fa4e800dc10 comp rx=0 tx=0).stop 2024-05-09T06:41:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 shutdown_connections 2024-05-09T06:41:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa4e006e330 0x7fa4e00707f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4f80ffba0 0x7fa4f8103a70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 --2- 172.21.15.63:0/1695794739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100da0 0x7fa4f8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 >> 172.21.15.63:0/1695794739 conn(0x7fa4f80fb120 msgr2=0x7fa4f8068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:31.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 shutdown_connections 2024-05-09T06:41:31.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.208+0000 7fa4fe81b700 1 -- 172.21.15.63:0/1695794739 wait complete. 2024-05-09T06:41:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.320+0000 7f1835010700 1 -- 172.21.15.63:0/663951457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 msgr2=0x7f1830101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.320+0000 7f1835010700 1 --2- 172.21.15.63:0/663951457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 0x7f1830101060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1820009910 tx=0x7f1820009c20 comp rx=0 tx=0).stop 2024-05-09T06:41:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.320+0000 7f1835010700 1 -- 172.21.15.63:0/663951457 shutdown_connections 2024-05-09T06:41:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.320+0000 7f1835010700 1 --2- 172.21.15.63:0/663951457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1830101e40 0x7f18301022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.320+0000 7f1835010700 1 --2- 172.21.15.63:0/663951457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 0x7f1830101060 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.320+0000 7f1835010700 1 -- 172.21.15.63:0/663951457 >> 172.21.15.63:0/663951457 conn(0x7f18300fc1e0 msgr2=0x7f18300fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.320+0000 7f1835010700 1 -- 172.21.15.63:0/663951457 shutdown_connections 2024-05-09T06:41:31.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.320+0000 7f1835010700 1 -- 172.21.15.63:0/663951457 wait complete. 2024-05-09T06:41:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.321+0000 7f1835010700 1 Processor -- start 2024-05-09T06:41:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.321+0000 7f1835010700 1 -- start start 2024-05-09T06:41:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.321+0000 7f1835010700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 0x7f1830198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.321+0000 7f1835010700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1830101e40 0x7f1830198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.321+0000 7f1835010700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830199330 con 0x7f1830100c40 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.321+0000 7f1835010700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830199470 con 0x7f1830101e40 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.321+0000 7f182ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 0x7f1830198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.321+0000 7f182ed9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 0x7f1830198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49628/0 (socket says 172.21.15.63:49628) 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.321+0000 7f182ed9d700 1 -- 172.21.15.63:0/1889640188 learned_addr learned my addr 172.21.15.63:0/1889640188 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f182e59c700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1830101e40 0x7f1830198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f182ed9d700 1 -- 172.21.15.63:0/1889640188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1830101e40 msgr2=0x7f1830198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f182ed9d700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1830101e40 0x7f1830198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f182ed9d700 1 -- 172.21.15.63:0/1889640188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18200095f0 con 0x7f1830100c40 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f182ed9d700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 0x7f1830198860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f182000bd70 tx=0x7f182000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f1817fff700 1 -- 172.21.15.63:0/1889640188 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f182001d070 con 0x7f1830100c40 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183019de80 con 0x7f1830100c40 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f1817fff700 1 -- 172.21.15.63:0/1889640188 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f182000f460 con 0x7f1830100c40 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f1817fff700 1 -- 172.21.15.63:0/1889640188 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1820021640 con 0x7f1830100c40 2024-05-09T06:41:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.322+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183019e320 con 0x7f1830100c40 2024-05-09T06:41:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.323+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1830106810 con 0x7f1830100c40 2024-05-09T06:41:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.324+0000 7f1817fff700 1 -- 172.21.15.63:0/1889640188 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f182000fad0 con 0x7f1830100c40 2024-05-09T06:41:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.324+0000 7f1817fff700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f181806e330 0x7f18180707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.325+0000 7f1817fff700 1 -- 172.21.15.63:0/1889640188 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f182008f7b0 con 0x7f1830100c40 2024-05-09T06:41:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.325+0000 7f182e59c700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f181806e330 0x7f18180707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.325+0000 7f182e59c700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f181806e330 0x7f18180707f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f182400ac50 tx=0x7f182400a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.329+0000 7f1817fff700 1 -- 172.21.15.63:0/1889640188 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f182005c3f0 con 0x7f1830100c40 2024-05-09T06:41:31.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.591+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f18301097a0 con 0x7f1830100c40 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.591+0000 7f1817fff700 1 -- 172.21.15.63:0/1889640188 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1905 (secure 0 0 0) 0x7f182005bf80 con 0x7f1830100c40 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:41:31.572 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:41:31.573 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:41:31.574 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:41:31.575 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:41:31.575 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:41:31.575 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:41:31.575 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:41:31.575 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:41:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f181806e330 msgr2=0x7f18180707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f181806e330 0x7f18180707f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f182400ac50 tx=0x7f182400a380 comp rx=0 tx=0).stop 2024-05-09T06:41:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 msgr2=0x7f1830198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 0x7f1830198860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f182000bd70 tx=0x7f182000bda0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 shutdown_connections 2024-05-09T06:41:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f181806e330 0x7f18180707f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1830100c40 0x7f1830198860 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 --2- 172.21.15.63:0/1889640188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1830101e40 0x7f1830198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 >> 172.21.15.63:0/1889640188 conn(0x7f18300fc1e0 msgr2=0x7f18300734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:31.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 shutdown_connections 2024-05-09T06:41:31.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.594+0000 7f1835010700 1 -- 172.21.15.63:0/1889640188 wait complete. 2024-05-09T06:41:31.577 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:41:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.707+0000 7f084d35d700 1 -- 172.21.15.63:0/3376770811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848101e50 msgr2=0x7f08481022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.707+0000 7f084d35d700 1 --2- 172.21.15.63:0/3376770811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848101e50 0x7f08481022b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0830009910 tx=0x7f0830009c20 comp rx=0 tx=0).stop 2024-05-09T06:41:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.707+0000 7f084d35d700 1 -- 172.21.15.63:0/3376770811 shutdown_connections 2024-05-09T06:41:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.707+0000 7f084d35d700 1 --2- 172.21.15.63:0/3376770811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848101e50 0x7f08481022b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.707+0000 7f084d35d700 1 --2- 172.21.15.63:0/3376770811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0848100c50 0x7f0848101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.707+0000 7f084d35d700 1 -- 172.21.15.63:0/3376770811 >> 172.21.15.63:0/3376770811 conn(0x7f08480fc210 msgr2=0x7f08480fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.707+0000 7f084d35d700 1 -- 172.21.15.63:0/3376770811 shutdown_connections 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.707+0000 7f084d35d700 1 -- 172.21.15.63:0/3376770811 wait complete. 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.707+0000 7f084d35d700 1 Processor -- start 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f084d35d700 1 -- start start 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f084d35d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848100c50 0x7f0848198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f084d35d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0848101e50 0x7f0848198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f084d35d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0848199390 con 0x7f0848100c50 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f084d35d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08481994d0 con 0x7f0848101e50 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f0846ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848100c50 0x7f0848198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f0846ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848100c50 0x7f0848198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49650/0 (socket says 172.21.15.63:49650) 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f0846ffd700 1 -- 172.21.15.63:0/243155298 learned_addr learned my addr 172.21.15.63:0/243155298 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f08467fc700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0848101e50 0x7f0848198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f0846ffd700 1 -- 172.21.15.63:0/243155298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0848101e50 msgr2=0x7f0848198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f0846ffd700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0848101e50 0x7f0848198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f0846ffd700 1 -- 172.21.15.63:0/243155298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08300095f0 con 0x7f0848100c50 2024-05-09T06:41:31.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.708+0000 7f0846ffd700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848100c50 0x7f0848198830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f083800c8f0 tx=0x7f083800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:31.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.709+0000 7f083ffff700 1 -- 172.21.15.63:0/243155298 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0838007a70 con 0x7f0848100c50 2024-05-09T06:41:31.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.709+0000 7f083ffff700 1 -- 172.21.15.63:0/243155298 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0838007bd0 con 0x7f0848100c50 2024-05-09T06:41:31.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.709+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084819df30 con 0x7f0848100c50 2024-05-09T06:41:31.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.709+0000 7f083ffff700 1 -- 172.21.15.63:0/243155298 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0838018630 con 0x7f0848100c50 2024-05-09T06:41:31.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.709+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084819e420 con 0x7f0848100c50 2024-05-09T06:41:31.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.710+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084804ebb0 con 0x7f0848100c50 2024-05-09T06:41:31.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.711+0000 7f083ffff700 1 -- 172.21.15.63:0/243155298 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f083800f450 con 0x7f0848100c50 2024-05-09T06:41:31.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.711+0000 7f083ffff700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f083406e340 0x7f0834070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:31.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.711+0000 7f083ffff700 1 -- 172.21.15.63:0/243155298 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f083808e3b0 con 0x7f0848100c50 2024-05-09T06:41:31.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.711+0000 7f08467fc700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f083406e340 0x7f0834070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:31.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.712+0000 7f08467fc700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f083406e340 0x7f0834070800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08300051c0 tx=0x7f0830005920 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:31.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:31.716+0000 7f083ffff700 1 -- 172.21.15.63:0/243155298 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f083805b120 con 0x7f0848100c50 2024-05-09T06:41:31.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:31 smithi063 ceph-mon[92765]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:31.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:31 smithi063 ceph-mon[92765]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:31.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:31 smithi063 ceph-mon[92765]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:31.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:31 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1695794739' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:31.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:31 smithi193 ceph-mon[81665]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:31.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:31 smithi193 ceph-mon[81665]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:31.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:31 smithi193 ceph-mon[81665]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:31.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:31 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1695794739' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.015+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0848074b10 con 0x7f083406e340 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.016+0000 7f083ffff700 1 -- 172.21.15.63:0/243155298 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0848074b10 con 0x7f083406e340 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:41:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "6/23 daemons upgraded", 2024-05-09T06:41:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-05-09T06:41:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:41:31.997 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:41:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.018+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f083406e340 msgr2=0x7f0834070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.018+0000 7f084d35d700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f083406e340 0x7f0834070800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08300051c0 tx=0x7f0830005920 comp rx=0 tx=0).stop 2024-05-09T06:41:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.018+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848100c50 msgr2=0x7f0848198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:31.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.018+0000 7f084d35d700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848100c50 0x7f0848198830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f083800c8f0 tx=0x7f083800ccb0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.018+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 shutdown_connections 2024-05-09T06:41:31.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.018+0000 7f084d35d700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f083406e340 0x7f0834070800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.018+0000 7f084d35d700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848100c50 0x7f0848198830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.018+0000 7f084d35d700 1 --2- 172.21.15.63:0/243155298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0848101e50 0x7f0848198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:31.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.018+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 >> 172.21.15.63:0/243155298 conn(0x7f08480fc210 msgr2=0x7f08480733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:31.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.019+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 shutdown_connections 2024-05-09T06:41:31.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.019+0000 7f084d35d700 1 -- 172.21.15.63:0/243155298 wait complete. 2024-05-09T06:41:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.128+0000 7f43f991c700 1 -- 172.21.15.63:0/1271578493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 msgr2=0x7f43f40fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.128+0000 7f43f991c700 1 --2- 172.21.15.63:0/1271578493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 0x7f43f40fff40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009910 tx=0x7f43e4009c20 comp rx=0 tx=0).stop 2024-05-09T06:41:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.128+0000 7f43f991c700 1 -- 172.21.15.63:0/1271578493 shutdown_connections 2024-05-09T06:41:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.128+0000 7f43f991c700 1 --2- 172.21.15.63:0/1271578493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f4100d20 0x7f43f4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.128+0000 7f43f991c700 1 --2- 172.21.15.63:0/1271578493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 0x7f43f40fff40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.128+0000 7f43f991c700 1 -- 172.21.15.63:0/1271578493 >> 172.21.15.63:0/1271578493 conn(0x7f43f40fb0c0 msgr2=0x7f43f40fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.128+0000 7f43f991c700 1 -- 172.21.15.63:0/1271578493 shutdown_connections 2024-05-09T06:41:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.128+0000 7f43f991c700 1 -- 172.21.15.63:0/1271578493 wait complete. 2024-05-09T06:41:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.128+0000 7f43f991c700 1 Processor -- start 2024-05-09T06:41:32.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f991c700 1 -- start start 2024-05-09T06:41:32.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f991c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 0x7f43f41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:32.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f991c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f4100d20 0x7f43f4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f991c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f4199500 con 0x7f43f40ffb20 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f991c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f4199640 con 0x7f43f4100d20 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 0x7f43f41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 0x7f43f41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49662/0 (socket says 172.21.15.63:49662) 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f2ffd700 1 -- 172.21.15.63:0/212798256 learned_addr learned my addr 172.21.15.63:0/212798256 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f27fc700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f4100d20 0x7f43f4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f2ffd700 1 -- 172.21.15.63:0/212798256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f4100d20 msgr2=0x7f43f4198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f2ffd700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f4100d20 0x7f43f4198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f2ffd700 1 -- 172.21.15.63:0/212798256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e40095f0 con 0x7f43f40ffb20 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.129+0000 7f43f27fc700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f4100d20 0x7f43f4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:41:32.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.130+0000 7f43f2ffd700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 0x7f43f41989a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43e400bd70 tx=0x7f43e400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.130+0000 7f43f891a700 1 -- 172.21.15.63:0/212798256 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43e401d070 con 0x7f43f40ffb20 2024-05-09T06:41:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.130+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f419dff0 con 0x7f43f40ffb20 2024-05-09T06:41:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.130+0000 7f43f891a700 1 -- 172.21.15.63:0/212798256 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43e400f460 con 0x7f43f40ffb20 2024-05-09T06:41:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.130+0000 7f43f891a700 1 -- 172.21.15.63:0/212798256 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43e4021600 con 0x7f43f40ffb20 2024-05-09T06:41:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.130+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f419e4e0 con 0x7f43f40ffb20 2024-05-09T06:41:32.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.131+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f404ebb0 con 0x7f43f40ffb20 2024-05-09T06:41:32.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.131+0000 7f43f891a700 1 -- 172.21.15.63:0/212798256 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f43e400fa50 con 0x7f43f40ffb20 2024-05-09T06:41:32.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.132+0000 7f43f891a700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f43dc06e330 0x7f43dc0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:41:32.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.132+0000 7f43f27fc700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f43dc06e330 0x7f43dc0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:41:32.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.132+0000 7f43f891a700 1 -- 172.21.15.63:0/212798256 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f43e408f730 con 0x7f43f40ffb20 2024-05-09T06:41:32.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.133+0000 7f43f27fc700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f43dc06e330 0x7f43dc0707f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f43e80099d0 tx=0x7f43e8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:41:32.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.137+0000 7f43f891a700 1 -- 172.21.15.63:0/212798256 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f43e405c370 con 0x7f43f40ffb20 2024-05-09T06:41:32.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.435+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f43f4108880 con 0x7f43f40ffb20 2024-05-09T06:41:32.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.436+0000 7f43f891a700 1 -- 172.21.15.63:0/212798256 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f43e405bf00 con 0x7f43f40ffb20 2024-05-09T06:41:32.416 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:41:32.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f43dc06e330 msgr2=0x7f43dc0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:32.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f43dc06e330 0x7f43dc0707f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f43e80099d0 tx=0x7f43e8008040 comp rx=0 tx=0).stop 2024-05-09T06:41:32.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 msgr2=0x7f43f41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:41:32.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 0x7f43f41989a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43e400bd70 tx=0x7f43e400be50 comp rx=0 tx=0).stop 2024-05-09T06:41:32.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 shutdown_connections 2024-05-09T06:41:32.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f43dc06e330 0x7f43dc0707f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:32.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f40ffb20 0x7f43f41989a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:32.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 --2- 172.21.15.63:0/212798256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f4100d20 0x7f43f4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:41:32.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 >> 172.21.15.63:0/212798256 conn(0x7f43f40fb0c0 msgr2=0x7f43f4068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:41:32.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 shutdown_connections 2024-05-09T06:41:32.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:41:32.438+0000 7f43f991c700 1 -- 172.21.15.63:0/212798256 wait complete. 2024-05-09T06:41:32.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:32 smithi063 ceph-mon[92765]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:32.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:32 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1889640188' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:41:32.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:32 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/212798256' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:41:32.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:32 smithi193 ceph-mon[81665]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:32.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:32 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1889640188' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:41:32.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:32 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/212798256' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:41:33.261 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:33 smithi063 systemd[1]: Stopping Ceph osd.0 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:41:33.538 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:33 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[45817]: 2024-05-09T06:41:33.281+0000 7eff6e806700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:41:33.539 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:33 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[45817]: 2024-05-09T06:41:33.281+0000 7eff6e806700 -1 osd.0 50 *** Got signal Terminated *** 2024-05-09T06:41:33.539 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:33 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[45817]: 2024-05-09T06:41:33.281+0000 7eff6e806700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:41:33.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:33 smithi063 ceph-mon[92765]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:33.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:33 smithi063 ceph-mon[92765]: pgmap v43: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:33.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:33 smithi063 ceph-mon[92765]: osd.0 marked itself down and dead 2024-05-09T06:41:33.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:33 smithi193 ceph-mon[81665]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:41:33.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:33 smithi193 ceph-mon[81665]: pgmap v43: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:41:33.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:33 smithi193 ceph-mon[81665]: osd.0 marked itself down and dead 2024-05-09T06:41:34.346 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:34 smithi063 podman[101369]: 2024-05-09 06:41:34.098923816 +0000 UTC m=+0.940166929 container died 7f26bfd264c704e206dbb4830c86f181ba40e72567d603b6828bbc82b6d42187 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0, org.label-schema.build-date=20240430, RELEASE=reef-1758e8e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T06:41:34.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:34 smithi063 ceph-mon[92765]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:41:34.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:34 smithi063 ceph-mon[92765]: osdmap e51: 6 total, 5 up, 6 in 2024-05-09T06:41:34.606 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:34 smithi063 podman[101369]: 2024-05-09 06:41:34.456045212 +0000 UTC m=+1.297288325 container cleanup 7f26bfd264c704e206dbb4830c86f181ba40e72567d603b6828bbc82b6d42187 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=reef-1758e8e, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:41:34.606 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:34 smithi063 bash[101369]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0 2024-05-09T06:41:34.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:34 smithi193 ceph-mon[81665]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:41:34.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:34 smithi193 ceph-mon[81665]: osdmap e51: 6 total, 5 up, 6 in 2024-05-09T06:41:35.079 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:34 smithi063 podman[101386]: 2024-05-09 06:41:34.774231123 +0000 UTC m=+0.676064768 container remove 7f26bfd264c704e206dbb4830c86f181ba40e72567d603b6828bbc82b6d42187 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=reef-1758e8e, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-09T06:41:35.355 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:35 smithi063 podman[101478]: 2024-05-09 06:41:35.100043645 +0000 UTC m=+0.114076007 container create 680b11dfe4bf9779aa800fd6a8bbc40d8686137c37aa60936d0891bdf785efe6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T06:41:35.356 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:35 smithi063 podman[101478]: 2024-05-09 06:41:35.004985453 +0000 UTC m=+0.019017832 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:41:35.356 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:35 smithi063 podman[101478]: 2024-05-09 06:41:35.327663459 +0000 UTC m=+0.341695819 container init 680b11dfe4bf9779aa800fd6a8bbc40d8686137c37aa60936d0891bdf785efe6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T06:41:35.356 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:35 smithi063 podman[101478]: 2024-05-09 06:41:35.33134208 +0000 UTC m=+0.345374439 container start 680b11dfe4bf9779aa800fd6a8bbc40d8686137c37aa60936d0891bdf785efe6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:41:35.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:35 smithi193 ceph-mon[81665]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-09T06:41:35.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:35 smithi193 ceph-mon[81665]: osdmap e52: 6 total, 5 up, 6 in 2024-05-09T06:41:35.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:35 smithi063 ceph-mon[92765]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-09T06:41:35.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:35 smithi063 ceph-mon[92765]: osdmap e52: 6 total, 5 up, 6 in 2024-05-09T06:41:35.855 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:35 smithi063 podman[101478]: 2024-05-09 06:41:35.419171164 +0000 UTC m=+0.433203532 container attach 680b11dfe4bf9779aa800fd6a8bbc40d8686137c37aa60936d0891bdf785efe6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T06:41:35.855 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:35 smithi063 podman[101478]: 2024-05-09 06:41:35.632668411 +0000 UTC m=+0.646700770 container died 680b11dfe4bf9779aa800fd6a8bbc40d8686137c37aa60936d0891bdf785efe6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T06:41:36.568 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:36 smithi063 podman[101504]: 2024-05-09 06:41:36.30077801 +0000 UTC m=+0.659417547 container remove 680b11dfe4bf9779aa800fd6a8bbc40d8686137c37aa60936d0891bdf785efe6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-09T06:41:36.569 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:36 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.0.service: Deactivated successfully. 2024-05-09T06:41:36.569 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:36 smithi063 systemd[1]: Stopped Ceph osd.0 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:41:36.569 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:36 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.0.service: Consumed 1min 3.987s CPU time. 2024-05-09T06:41:36.569 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:36 smithi063 systemd[1]: Starting Ceph osd.0 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:41:36.855 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:36 smithi063 podman[101606]: 2024-05-09 06:41:36.76605334 +0000 UTC m=+0.111602923 container create f28f66c2007ef0eabd7041781a4002e4ee7584f2e32e3a01961c59267e73c86d (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:41:36.855 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:36 smithi063 podman[101606]: 2024-05-09 06:41:36.673297863 +0000 UTC m=+0.018847450 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:41:37.355 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:36 smithi063 podman[101606]: 2024-05-09 06:41:36.977014645 +0000 UTC m=+0.322564232 container init f28f66c2007ef0eabd7041781a4002e4ee7584f2e32e3a01961c59267e73c86d (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS) 2024-05-09T06:41:37.355 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:36 smithi063 podman[101606]: 2024-05-09 06:41:36.980390266 +0000 UTC m=+0.325940039 container start f28f66c2007ef0eabd7041781a4002e4ee7584f2e32e3a01961c59267e73c86d (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T06:41:37.355 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 podman[101606]: 2024-05-09 06:41:37.043573931 +0000 UTC m=+0.389123516 container attach f28f66c2007ef0eabd7041781a4002e4ee7584f2e32e3a01961c59267e73c86d (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T06:41:37.810 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:37 smithi063 ceph-mon[92765]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-09T06:41:37.810 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate[101621]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T06:41:37.810 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate[101621]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-09T06:41:37.810 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 bash[101606]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T06:41:37.810 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 bash[101606]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-09T06:41:37.810 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 bash[101606]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 bash[101606]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 bash[101606]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 bash[101606]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 bash[101606]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate[101621]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate[101621]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate[101621]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate[101621]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate[101621]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-09T06:41:37.811 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:37 smithi063 podman[101606]: 2024-05-09 06:41:37.476335763 +0000 UTC m=+0.821885351 container died f28f66c2007ef0eabd7041781a4002e4ee7584f2e32e3a01961c59267e73c86d (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T06:41:37.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:37 smithi193 ceph-mon[81665]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-09T06:41:38.355 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:38 smithi063 podman[101754]: 2024-05-09 06:41:38.105946476 +0000 UTC m=+0.623686592 container remove f28f66c2007ef0eabd7041781a4002e4ee7584f2e32e3a01961c59267e73c86d (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T06:41:38.355 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:38 smithi063 podman[101817]: 2024-05-09 06:41:38.237467349 +0000 UTC m=+0.019272556 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:41:38.355 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:38 smithi063 podman[101817]: 2024-05-09 06:41:38.348453628 +0000 UTC m=+0.130258842 container create 6e387840e5b75ed5723b3b36033b5a0b25ac299c0ac54fb82a010a1e3fe5a5f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T06:41:38.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:38 smithi063 ceph-mon[92765]: Health check failed: Degraded data redundancy: 41/228 objects degraded (17.982%), 18 pgs degraded (PG_DEGRADED) 2024-05-09T06:41:38.855 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:38 smithi063 podman[101817]: 2024-05-09 06:41:38.551111476 +0000 UTC m=+0.332916683 container init 6e387840e5b75ed5723b3b36033b5a0b25ac299c0ac54fb82a010a1e3fe5a5f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True) 2024-05-09T06:41:38.855 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:38 smithi063 podman[101817]: 2024-05-09 06:41:38.55451549 +0000 UTC m=+0.336320696 container start 6e387840e5b75ed5723b3b36033b5a0b25ac299c0ac54fb82a010a1e3fe5a5f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:41:38.855 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:38 smithi063 ceph-osd[101836]: -- 172.21.15.63:0/1493129605 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561bbb7cd680 con 0x561bbb797000 2024-05-09T06:41:38.855 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:38 smithi063 bash[101817]: 6e387840e5b75ed5723b3b36033b5a0b25ac299c0ac54fb82a010a1e3fe5a5f4 2024-05-09T06:41:38.855 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:38 smithi063 systemd[1]: Started Ceph osd.0 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:41:38.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:38 smithi193 ceph-mon[81665]: Health check failed: Degraded data redundancy: 41/228 objects degraded (17.982%), 18 pgs degraded (PG_DEGRADED) 2024-05-09T06:41:39.355 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:39 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[101832]: 2024-05-09T06:41:39.109+0000 7f1c2b6c37c0 -1 Falling back to public interface 2024-05-09T06:41:39.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:39 smithi063 ceph-mon[92765]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:39.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:39 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:39.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:39 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:39.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:39 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:39.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:39 smithi193 ceph-mon[81665]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:39.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:39 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:39.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:39 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:39.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:39 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:41.868 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:41 smithi063 ceph-mon[92765]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:41.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:41 smithi193 ceph-mon[81665]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:43 smithi063 ceph-mon[92765]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 478 B/s rd, 0 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:41:43.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:43 smithi193 ceph-mon[81665]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 478 B/s rd, 0 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:43.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:43.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:41:44.605 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:44 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[101832]: 2024-05-09T06:41:44.288+0000 7f1c2b6c37c0 -1 osd.0 50 log_to_monitors true 2024-05-09T06:41:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:44 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:44 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:44 smithi063 ceph-mon[92765]: pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:44 smithi063 ceph-mon[92765]: from='osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T06:41:45.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:44 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:45.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:44 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:45.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:44 smithi193 ceph-mon[81665]: pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:45.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:44 smithi193 ceph-mon[81665]: from='osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T06:41:46.095 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:41:45 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[101832]: 2024-05-09T06:41:45.752+0000 7f1c18b12700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:41:46.095 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:45 smithi063 ceph-mon[92765]: from='osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T06:41:46.095 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:45 smithi063 ceph-mon[92765]: osdmap e53: 6 total, 5 up, 6 in 2024-05-09T06:41:46.095 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:45 smithi063 ceph-mon[92765]: mgrmap e39: smithi063.egfguy(active, since 92s), standbys: smithi193.mmmqzh 2024-05-09T06:41:46.095 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:45 smithi063 ceph-mon[92765]: from='osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:41:46.095 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:46.096 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:46.096 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:45 smithi063 ceph-mon[92765]: from='osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249]' entity='osd.0' 2024-05-09T06:41:46.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:45 smithi193 ceph-mon[81665]: from='osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T06:41:46.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:45 smithi193 ceph-mon[81665]: osdmap e53: 6 total, 5 up, 6 in 2024-05-09T06:41:46.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:45 smithi193 ceph-mon[81665]: mgrmap e39: smithi063.egfguy(active, since 92s), standbys: smithi193.mmmqzh 2024-05-09T06:41:46.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:45 smithi193 ceph-mon[81665]: from='osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:41:46.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:46.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:46.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:45 smithi193 ceph-mon[81665]: from='osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249]' entity='osd.0' 2024-05-09T06:41:46.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:46 smithi063 ceph-mon[92765]: pgmap v53: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:46.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:46 smithi063 ceph-mon[92765]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:41:46.972 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:46 smithi063 ceph-mon[92765]: osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249] boot 2024-05-09T06:41:46.973 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:46 smithi063 ceph-mon[92765]: osdmap e54: 6 total, 6 up, 6 in 2024-05-09T06:41:46.973 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:46 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:41:47.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:46 smithi193 ceph-mon[81665]: pgmap v53: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s; 41/228 objects degraded (17.982%) 2024-05-09T06:41:47.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:46 smithi193 ceph-mon[81665]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:41:47.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:46 smithi193 ceph-mon[81665]: osd.0 [v2:172.21.15.63:6802/1522382249,v1:172.21.15.63:6803/1522382249] boot 2024-05-09T06:41:47.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:46 smithi193 ceph-mon[81665]: osdmap e54: 6 total, 6 up, 6 in 2024-05-09T06:41:47.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:46 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T06:41:49.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:48 smithi063 ceph-mon[92765]: osdmap e55: 6 total, 6 up, 6 in 2024-05-09T06:41:49.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:48 smithi063 ceph-mon[92765]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s; 6/228 objects degraded (2.632%); 46 B/s, 0 objects/s recovering 2024-05-09T06:41:49.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:48 smithi193 ceph-mon[81665]: osdmap e55: 6 total, 6 up, 6 in 2024-05-09T06:41:49.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:48 smithi193 ceph-mon[81665]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s; 6/228 objects degraded (2.632%); 46 B/s, 0 objects/s recovering 2024-05-09T06:41:50.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: Health check update: Degraded data redundancy: 6/228 objects degraded (2.632%), 3 pgs degraded (PG_DEGRADED) 2024-05-09T06:41:50.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:50.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:50.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:50.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:41:50.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:50.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:50.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:50.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:50.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:50.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-09T06:41:50.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-09T06:41:50.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:49 smithi063 ceph-mon[92765]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-05-09T06:41:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: Health check update: Degraded data redundancy: 6/228 objects degraded (2.632%), 3 pgs degraded (PG_DEGRADED) 2024-05-09T06:41:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:41:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:41:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:41:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:50.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:41:50.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-09T06:41:50.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-09T06:41:50.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:49 smithi193 ceph-mon[81665]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-05-09T06:41:51.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:50 smithi063 ceph-mon[92765]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 46 B/s, 0 objects/s recovering 2024-05-09T06:41:51.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:50 smithi193 ceph-mon[81665]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 46 B/s, 0 objects/s recovering 2024-05-09T06:41:52.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:51 smithi063 ceph-mon[92765]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/228 objects degraded (2.632%), 3 pgs degraded) 2024-05-09T06:41:52.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:51 smithi063 ceph-mon[92765]: Cluster is now healthy 2024-05-09T06:41:52.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:51 smithi193 ceph-mon[81665]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/228 objects degraded (2.632%), 3 pgs degraded) 2024-05-09T06:41:52.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:51 smithi193 ceph-mon[81665]: Cluster is now healthy 2024-05-09T06:41:53.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:52 smithi063 ceph-mon[92765]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 702 B/s rd, 1 op/s; 38 B/s, 0 objects/s recovering 2024-05-09T06:41:53.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:52 smithi193 ceph-mon[81665]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 702 B/s rd, 1 op/s; 38 B/s, 0 objects/s recovering 2024-05-09T06:41:55.515 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:55 smithi193 ceph-mon[81665]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 35 B/s, 0 objects/s recovering 2024-05-09T06:41:55.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:55 smithi063 ceph-mon[92765]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 35 B/s, 0 objects/s recovering 2024-05-09T06:41:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:57 smithi063 ceph-mon[92765]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 30 B/s, 0 objects/s recovering 2024-05-09T06:41:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:41:57.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:57 smithi193 ceph-mon[81665]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 30 B/s, 0 objects/s recovering 2024-05-09T06:41:57.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:57.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:41:58.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:58 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:58.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:58 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:41:59.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:41:59 smithi063 ceph-mon[92765]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 27 B/s, 0 objects/s recovering 2024-05-09T06:41:59.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:41:59 smithi193 ceph-mon[81665]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 27 B/s, 0 objects/s recovering 2024-05-09T06:42:01.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:01 smithi063 ceph-mon[92765]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:01.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:01 smithi193 ceph-mon[81665]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:02.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.563+0000 7f2f672cc700 1 -- 172.21.15.63:0/3496668667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 msgr2=0x7f2f601042d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:02.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.563+0000 7f2f672cc700 1 --2- 172.21.15.63:0/3496668667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 0x7f2f601042d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009910 tx=0x7f2f54009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:02.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.563+0000 7f2f672cc700 1 -- 172.21.15.63:0/3496668667 shutdown_connections 2024-05-09T06:42:02.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.563+0000 7f2f672cc700 1 --2- 172.21.15.63:0/3496668667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 0x7f2f601042d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:02.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.563+0000 7f2f672cc700 1 --2- 172.21.15.63:0/3496668667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f600ff4d0 0x7f2f60101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:02.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.563+0000 7f2f672cc700 1 -- 172.21.15.63:0/3496668667 >> 172.21.15.63:0/3496668667 conn(0x7f2f600fb0f0 msgr2=0x7f2f600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:02.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.563+0000 7f2f672cc700 1 -- 172.21.15.63:0/3496668667 shutdown_connections 2024-05-09T06:42:02.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.563+0000 7f2f672cc700 1 -- 172.21.15.63:0/3496668667 wait complete. 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.564+0000 7f2f672cc700 1 Processor -- start 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.564+0000 7f2f672cc700 1 -- start start 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.564+0000 7f2f672cc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f600ff4d0 0x7f2f6019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.564+0000 7f2f672cc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 0x7f2f6019d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.564+0000 7f2f672cc700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6019d7c0 con 0x7f2f60101e40 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.564+0000 7f2f672cc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f601a2180 con 0x7f2f600ff4d0 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.564+0000 7f2f64867700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 0x7f2f6019d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.564+0000 7f2f64867700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 0x7f2f6019d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51908/0 (socket says 172.21.15.63:51908) 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.564+0000 7f2f64867700 1 -- 172.21.15.63:0/3014492637 learned_addr learned my addr 172.21.15.63:0/3014492637 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f65068700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f600ff4d0 0x7f2f6019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f64867700 1 -- 172.21.15.63:0/3014492637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f600ff4d0 msgr2=0x7f2f6019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f64867700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f600ff4d0 0x7f2f6019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:02.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f64867700 1 -- 172.21.15.63:0/3014492637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f540095f0 con 0x7f2f60101e40 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f64867700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 0x7f2f6019d1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400b650 tx=0x7f2f54005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f527fc700 1 -- 172.21.15.63:0/3014492637 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f5401d070 con 0x7f2f60101e40 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f601a2320 con 0x7f2f60101e40 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f527fc700 1 -- 172.21.15.63:0/3014492637 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f540052a0 con 0x7f2f60101e40 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f527fc700 1 -- 172.21.15.63:0/3014492637 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f54022470 con 0x7f2f60101e40 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.565+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f601a2860 con 0x7f2f60101e40 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.566+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6004ebb0 con 0x7f2f60101e40 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.567+0000 7f2f527fc700 1 -- 172.21.15.63:0/3014492637 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2f5400f830 con 0x7f2f60101e40 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.567+0000 7f2f527fc700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f4c06e200 0x7f2f4c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:02.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.567+0000 7f2f527fc700 1 -- 172.21.15.63:0/3014492637 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2f5408f710 con 0x7f2f60101e40 2024-05-09T06:42:02.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.567+0000 7f2f65068700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f4c06e200 0x7f2f4c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:02.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.568+0000 7f2f65068700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f4c06e200 0x7f2f4c0706c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2f6019e0f0 tx=0x7f2f5c009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:02.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.572+0000 7f2f527fc700 1 -- 172.21.15.63:0/3014492637 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2f54058a40 con 0x7f2f60101e40 2024-05-09T06:42:02.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.800+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f60102860 con 0x7f2f4c06e200 2024-05-09T06:42:02.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.801+0000 7f2f527fc700 1 -- 172.21.15.63:0/3014492637 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2f60102860 con 0x7f2f4c06e200 2024-05-09T06:42:02.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f4c06e200 msgr2=0x7f2f4c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:02.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f4c06e200 0x7f2f4c0706c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2f6019e0f0 tx=0x7f2f5c009260 comp rx=0 tx=0).stop 2024-05-09T06:42:02.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 msgr2=0x7f2f6019d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:02.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 0x7f2f6019d1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400b650 tx=0x7f2f54005e30 comp rx=0 tx=0).stop 2024-05-09T06:42:02.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 shutdown_connections 2024-05-09T06:42:02.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f4c06e200 0x7f2f4c0706c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:02.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f600ff4d0 0x7f2f6019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:02.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 --2- 172.21.15.63:0/3014492637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f60101e40 0x7f2f6019d1a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:02.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 >> 172.21.15.63:0/3014492637 conn(0x7f2f600fb0f0 msgr2=0x7f2f600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:02.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 shutdown_connections 2024-05-09T06:42:02.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.803+0000 7f2f672cc700 1 -- 172.21.15.63:0/3014492637 wait complete. 2024-05-09T06:42:02.794 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:42:02.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.914+0000 7ff40e132700 1 -- 172.21.15.63:0/1373493260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 msgr2=0x7ff408101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:02.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.914+0000 7ff40e132700 1 --2- 172.21.15.63:0/1373493260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 0x7ff408101070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009910 tx=0x7ff3f8009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:02.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.914+0000 7ff40e132700 1 -- 172.21.15.63:0/1373493260 shutdown_connections 2024-05-09T06:42:02.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.914+0000 7ff40e132700 1 --2- 172.21.15.63:0/1373493260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff408101e50 0x7ff4081022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:02.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.914+0000 7ff40e132700 1 --2- 172.21.15.63:0/1373493260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 0x7ff408101070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:02.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.914+0000 7ff40e132700 1 -- 172.21.15.63:0/1373493260 >> 172.21.15.63:0/1373493260 conn(0x7ff4080fc210 msgr2=0x7ff4080fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:02.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.914+0000 7ff40e132700 1 -- 172.21.15.63:0/1373493260 shutdown_connections 2024-05-09T06:42:02.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.914+0000 7ff40e132700 1 -- 172.21.15.63:0/1373493260 wait complete. 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff40e132700 1 Processor -- start 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff40e132700 1 -- start start 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff40e132700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 0x7ff408196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff40e132700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff408101e50 0x7ff408196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff40e132700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4081971c0 con 0x7ff408100c50 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff40e132700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408197300 con 0x7ff408101e50 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff4077fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 0x7ff408196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff4077fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 0x7ff408196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51918/0 (socket says 172.21.15.63:51918) 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff4077fe700 1 -- 172.21.15.63:0/292342448 learned_addr learned my addr 172.21.15.63:0/292342448 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff406ffd700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff408101e50 0x7ff408196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:02.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff4077fe700 1 -- 172.21.15.63:0/292342448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff408101e50 msgr2=0x7ff408196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:02.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff4077fe700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff408101e50 0x7ff408196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:02.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff4077fe700 1 -- 172.21.15.63:0/292342448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80095f0 con 0x7ff408100c50 2024-05-09T06:42:02.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.915+0000 7ff406ffd700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff408101e50 0x7ff408196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:42:02.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.916+0000 7ff4077fe700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 0x7ff408196660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80059d0 tx=0x7ff3f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:02.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.916+0000 7ff404ff9700 1 -- 172.21.15.63:0/292342448 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f801d070 con 0x7ff408100c50 2024-05-09T06:42:02.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.916+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40819bcb0 con 0x7ff408100c50 2024-05-09T06:42:02.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.916+0000 7ff404ff9700 1 -- 172.21.15.63:0/292342448 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3f8005dc0 con 0x7ff408100c50 2024-05-09T06:42:02.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.916+0000 7ff404ff9700 1 -- 172.21.15.63:0/292342448 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f8021410 con 0x7ff408100c50 2024-05-09T06:42:02.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.916+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40819c1a0 con 0x7ff408100c50 2024-05-09T06:42:02.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.917+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4081907d0 con 0x7ff408100c50 2024-05-09T06:42:02.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.917+0000 7ff404ff9700 1 -- 172.21.15.63:0/292342448 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff3f800f840 con 0x7ff408100c50 2024-05-09T06:42:02.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.918+0000 7ff404ff9700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff3f406e200 0x7ff3f40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:02.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.918+0000 7ff404ff9700 1 -- 172.21.15.63:0/292342448 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff3f808f390 con 0x7ff408100c50 2024-05-09T06:42:02.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.918+0000 7ff406ffd700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff3f406e200 0x7ff3f40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:02.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.918+0000 7ff406ffd700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff3f406e200 0x7ff3f40706c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0005fd0 tx=0x7ff3f0005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:02.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:02.923+0000 7ff404ff9700 1 -- 172.21.15.63:0/292342448 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff3f805c1f0 con 0x7ff408100c50 2024-05-09T06:42:03.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.149+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff40802cf00 con 0x7ff3f406e200 2024-05-09T06:42:03.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.150+0000 7ff404ff9700 1 -- 172.21.15.63:0/292342448 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff40802cf00 con 0x7ff3f406e200 2024-05-09T06:42:03.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.152+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff3f406e200 msgr2=0x7ff3f40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.152+0000 7ff40e132700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff3f406e200 0x7ff3f40706c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0005fd0 tx=0x7ff3f0005ee0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.152+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 msgr2=0x7ff408196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.153+0000 7ff40e132700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 0x7ff408196660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80059d0 tx=0x7ff3f800be20 comp rx=0 tx=0).stop 2024-05-09T06:42:03.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.153+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 shutdown_connections 2024-05-09T06:42:03.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.153+0000 7ff40e132700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff3f406e200 0x7ff3f40706c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.153+0000 7ff40e132700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff408100c50 0x7ff408196660 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.153+0000 7ff40e132700 1 --2- 172.21.15.63:0/292342448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff408101e50 0x7ff408196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.153+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 >> 172.21.15.63:0/292342448 conn(0x7ff4080fc210 msgr2=0x7ff408105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:03.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.153+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 shutdown_connections 2024-05-09T06:42:03.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.153+0000 7ff40e132700 1 -- 172.21.15.63:0/292342448 wait complete. 2024-05-09T06:42:03.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.262+0000 7fb80eba8700 1 -- 172.21.15.63:0/2217300440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 msgr2=0x7fb808101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.262+0000 7fb80eba8700 1 --2- 172.21.15.63:0/2217300440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 0x7fb808101070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009910 tx=0x7fb7fc009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:03.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.262+0000 7fb80eba8700 1 -- 172.21.15.63:0/2217300440 shutdown_connections 2024-05-09T06:42:03.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.262+0000 7fb80eba8700 1 --2- 172.21.15.63:0/2217300440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb808101e50 0x7fb8081022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.262+0000 7fb80eba8700 1 --2- 172.21.15.63:0/2217300440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 0x7fb808101070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.262+0000 7fb80eba8700 1 -- 172.21.15.63:0/2217300440 >> 172.21.15.63:0/2217300440 conn(0x7fb8080fc210 msgr2=0x7fb8080fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:03.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.262+0000 7fb80eba8700 1 -- 172.21.15.63:0/2217300440 shutdown_connections 2024-05-09T06:42:03.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.262+0000 7fb80eba8700 1 -- 172.21.15.63:0/2217300440 wait complete. 2024-05-09T06:42:03.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.263+0000 7fb80eba8700 1 Processor -- start 2024-05-09T06:42:03.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.263+0000 7fb80eba8700 1 -- start start 2024-05-09T06:42:03.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.263+0000 7fb80eba8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 0x7fb808198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:03.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.263+0000 7fb80eba8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb808101e50 0x7fb808198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:03.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.263+0000 7fb80eba8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb808199360 con 0x7fb808100c50 2024-05-09T06:42:03.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.263+0000 7fb80eba8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081994a0 con 0x7fb808101e50 2024-05-09T06:42:03.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.263+0000 7fb80c944700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 0x7fb808198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:03.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.263+0000 7fb80c944700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 0x7fb808198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51934/0 (socket says 172.21.15.63:51934) 2024-05-09T06:42:03.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.263+0000 7fb80c944700 1 -- 172.21.15.63:0/220273790 learned_addr learned my addr 172.21.15.63:0/220273790 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:03.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb807fff700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb808101e50 0x7fb808198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:03.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb80c944700 1 -- 172.21.15.63:0/220273790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb808101e50 msgr2=0x7fb808198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb80c944700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb808101e50 0x7fb808198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb80c944700 1 -- 172.21.15.63:0/220273790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc0095f0 con 0x7fb808100c50 2024-05-09T06:42:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb80c944700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 0x7fb808198800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00bd70 tx=0x7fb7fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb805ffb700 1 -- 172.21.15.63:0/220273790 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7fc01d070 con 0x7fb808100c50 2024-05-09T06:42:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb805ffb700 1 -- 172.21.15.63:0/220273790 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7fc00f460 con 0x7fb808100c50 2024-05-09T06:42:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80819dea0 con 0x7fb808100c50 2024-05-09T06:42:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb805ffb700 1 -- 172.21.15.63:0/220273790 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7fc021600 con 0x7fb808100c50 2024-05-09T06:42:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.264+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80819e390 con 0x7fb808100c50 2024-05-09T06:42:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.265+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb808106300 con 0x7fb808100c50 2024-05-09T06:42:03.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.266+0000 7fb805ffb700 1 -- 172.21.15.63:0/220273790 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb7fc00fa50 con 0x7fb808100c50 2024-05-09T06:42:03.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.266+0000 7fb805ffb700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb7f806e210 0x7fb7f80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:03.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.267+0000 7fb805ffb700 1 -- 172.21.15.63:0/220273790 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb7fc08f3c0 con 0x7fb808100c50 2024-05-09T06:42:03.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.267+0000 7fb807fff700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb7f806e210 0x7fb7f80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:03.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.267+0000 7fb807fff700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb7f806e210 0x7fb7f80706d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005fd0 tx=0x7fb7f400a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:03.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.271+0000 7fb805ffb700 1 -- 172.21.15.63:0/220273790 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb7fc05c2a0 con 0x7fb808100c50 2024-05-09T06:42:03.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.490+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb80802ce50 con 0x7fb7f806e210 2024-05-09T06:42:03.470 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:03 smithi063 ceph-mon[92765]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:03.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.502+0000 7fb805ffb700 1 -- 172.21.15.63:0/220273790 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb80802ce50 con 0x7fb7f806e210 2024-05-09T06:42:03.482 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:42:03.482 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (10m) 19s ago 12m 25.1M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:42:03.482 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (12m) 19s ago 12m 7960k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (11m) 42s ago 11m 7944k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (51s) 19s ago 12m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (46s) 42s ago 11m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (10m) 19s ago 12m 87.1M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (6m) 19s ago 6m 16.0M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (6m) 19s ago 6m 90.4M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (6m) 42s ago 6m 143M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (6m) 42s ago 6m 17.1M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (3m) 19s ago 13m 501M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (2m) 42s ago 11m 450M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (2m) 19s ago 14m 44.8M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (97s) 42s ago 11m 32.2M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (12m) 19s ago 12m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:42:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (11m) 42s ago 11m 16.2M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:42:03.484 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (25s) 19s ago 10m 14.8M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:42:03.484 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (10m) 19s ago 10m 245M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 3005e767d3a7 2024-05-09T06:42:03.484 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (9m) 19s ago 9m 210M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:42:03.484 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (9m) 42s ago 9m 292M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:42:03.484 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (8m) 42s ago 8m 260M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:42:03.484 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (8m) 42s ago 8m 233M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:42:03.484 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (2m) 19s ago 12m 55.9M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:42:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb7f806e210 msgr2=0x7fb7f80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb7f806e210 0x7fb7f80706d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005fd0 tx=0x7fb7f400a560 comp rx=0 tx=0).stop 2024-05-09T06:42:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 msgr2=0x7fb808198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 0x7fb808198800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00bd70 tx=0x7fb7fc00be50 comp rx=0 tx=0).stop 2024-05-09T06:42:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 shutdown_connections 2024-05-09T06:42:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb7f806e210 0x7fb7f80706d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb808100c50 0x7fb808198800 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 --2- 172.21.15.63:0/220273790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb808101e50 0x7fb808198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 >> 172.21.15.63:0/220273790 conn(0x7fb8080fc210 msgr2=0x7fb808103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:03.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 shutdown_connections 2024-05-09T06:42:03.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.505+0000 7fb80eba8700 1 -- 172.21.15.63:0/220273790 wait complete. 2024-05-09T06:42:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.613+0000 7feed6113700 1 -- 172.21.15.63:0/1215777214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0101760 msgr2=0x7feed0103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.613+0000 7feed6113700 1 --2- 172.21.15.63:0/1215777214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0101760 0x7feed0103b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feec0009910 tx=0x7feec0009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.613+0000 7feed6113700 1 -- 172.21.15.63:0/1215777214 shutdown_connections 2024-05-09T06:42:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.613+0000 7feed6113700 1 --2- 172.21.15.63:0/1215777214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed0104090 0x7feed0106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.613+0000 7feed6113700 1 --2- 172.21.15.63:0/1215777214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0101760 0x7feed0103b50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.613+0000 7feed6113700 1 -- 172.21.15.63:0/1215777214 >> 172.21.15.63:0/1215777214 conn(0x7feed00fb0f0 msgr2=0x7feed00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.613+0000 7feed6113700 1 -- 172.21.15.63:0/1215777214 shutdown_connections 2024-05-09T06:42:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.613+0000 7feed6113700 1 -- 172.21.15.63:0/1215777214 wait complete. 2024-05-09T06:42:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.613+0000 7feed6113700 1 Processor -- start 2024-05-09T06:42:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feed6113700 1 -- start start 2024-05-09T06:42:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feed6113700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed0101760 0x7feed0196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feed6113700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0104090 0x7feed0196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feed6113700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0197170 con 0x7feed0104090 2024-05-09T06:42:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feed6113700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01972b0 con 0x7feed0101760 2024-05-09T06:42:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feeceffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0104090 0x7feed0196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feeceffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0104090 0x7feed0196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51942/0 (socket says 172.21.15.63:51942) 2024-05-09T06:42:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feeceffd700 1 -- 172.21.15.63:0/3750031603 learned_addr learned my addr 172.21.15.63:0/3750031603 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feecf7fe700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed0101760 0x7feed0196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feeceffd700 1 -- 172.21.15.63:0/3750031603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed0101760 msgr2=0x7feed0196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feeceffd700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed0101760 0x7feed0196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feeceffd700 1 -- 172.21.15.63:0/3750031603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec00095f0 con 0x7feed0104090 2024-05-09T06:42:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.614+0000 7feeceffd700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0104090 0x7feed0196b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feec400d8d0 tx=0x7feec400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.615+0000 7feeccff9700 1 -- 172.21.15.63:0/3750031603 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec40098a0 con 0x7feed0104090 2024-05-09T06:42:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.615+0000 7feeccff9700 1 -- 172.21.15.63:0/3750031603 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feec4010460 con 0x7feed0104090 2024-05-09T06:42:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.615+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed019bd10 con 0x7feed0104090 2024-05-09T06:42:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.615+0000 7feeccff9700 1 -- 172.21.15.63:0/3750031603 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec400f5f0 con 0x7feed0104090 2024-05-09T06:42:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.615+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed019c2e0 con 0x7feed0104090 2024-05-09T06:42:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.616+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed004ebb0 con 0x7feed0104090 2024-05-09T06:42:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.616+0000 7feeccff9700 1 -- 172.21.15.63:0/3750031603 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feec4009a00 con 0x7feed0104090 2024-05-09T06:42:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.617+0000 7feeccff9700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feeb806e210 0x7feeb80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.617+0000 7feeccff9700 1 -- 172.21.15.63:0/3750031603 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7feec408e000 con 0x7feed0104090 2024-05-09T06:42:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.617+0000 7feecf7fe700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feeb806e210 0x7feeb80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.617+0000 7feecf7fe700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feeb806e210 0x7feeb80706d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feec0006010 tx=0x7feec000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.621+0000 7feeccff9700 1 -- 172.21.15.63:0/3750031603 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feec405ae60 con 0x7feed0104090 2024-05-09T06:42:03.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:03 smithi193 ceph-mon[81665]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:03.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.924+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feed019c6f0 con 0x7feed0104090 2024-05-09T06:42:03.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.925+0000 7feeccff9700 1 -- 172.21.15.63:0/3750031603 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7feec405a9f0 con 0x7feed0104090 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 5, 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 1 2024-05-09T06:42:03.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:42:03.907 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:42:03.907 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:42:03.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:42:03.907 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:42:03.907 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 9, 2024-05-09T06:42:03.907 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 5 2024-05-09T06:42:03.907 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:42:03.907 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feeb806e210 msgr2=0x7feeb80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feeb806e210 0x7feeb80706d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feec0006010 tx=0x7feec000b540 comp rx=0 tx=0).stop 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0104090 msgr2=0x7feed0196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0104090 0x7feed0196b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feec400d8d0 tx=0x7feec400dbe0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 shutdown_connections 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feeb806e210 0x7feeb80706d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feed0101760 0x7feed0196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 --2- 172.21.15.63:0/3750031603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0104090 0x7feed0196b50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 >> 172.21.15.63:0/3750031603 conn(0x7feed00fb0f0 msgr2=0x7feed00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 shutdown_connections 2024-05-09T06:42:03.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:03.927+0000 7feed6113700 1 -- 172.21.15.63:0/3750031603 wait complete. 2024-05-09T06:42:04.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.042+0000 7f8d5cada700 1 -- 172.21.15.63:0/1228307206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d58101790 msgr2=0x7f8d58103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.042+0000 7f8d5cada700 1 --2- 172.21.15.63:0/1228307206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d58101790 0x7f8d58103b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009910 tx=0x7f8d48009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:04.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.042+0000 7f8d5cada700 1 -- 172.21.15.63:0/1228307206 shutdown_connections 2024-05-09T06:42:04.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.042+0000 7f8d5cada700 1 --2- 172.21.15.63:0/1228307206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d581040c0 0x7f8d581064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.042+0000 7f8d5cada700 1 --2- 172.21.15.63:0/1228307206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d58101790 0x7f8d58103b80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.042+0000 7f8d5cada700 1 -- 172.21.15.63:0/1228307206 >> 172.21.15.63:0/1228307206 conn(0x7f8d580fb120 msgr2=0x7f8d580fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:04.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.042+0000 7f8d5cada700 1 -- 172.21.15.63:0/1228307206 shutdown_connections 2024-05-09T06:42:04.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.042+0000 7f8d5cada700 1 -- 172.21.15.63:0/1228307206 wait complete. 2024-05-09T06:42:04.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.043+0000 7f8d5cada700 1 Processor -- start 2024-05-09T06:42:04.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.043+0000 7f8d5cada700 1 -- start start 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.043+0000 7f8d5cada700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d58101790 0x7f8d58196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.043+0000 7f8d5cada700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d581040c0 0x7f8d58196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.043+0000 7f8d5cada700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d581971a0 con 0x7f8d581040c0 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.043+0000 7f8d5cada700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d581972e0 con 0x7f8d58101790 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.043+0000 7f8d55d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d581040c0 0x7f8d58196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.043+0000 7f8d55d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d581040c0 0x7f8d58196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51966/0 (socket says 172.21.15.63:51966) 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.043+0000 7f8d55d9b700 1 -- 172.21.15.63:0/2090877347 learned_addr learned my addr 172.21.15.63:0/2090877347 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d5659c700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d58101790 0x7f8d58196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d55d9b700 1 -- 172.21.15.63:0/2090877347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d58101790 msgr2=0x7f8d58196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d55d9b700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d58101790 0x7f8d58196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d55d9b700 1 -- 172.21.15.63:0/2090877347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d480095f0 con 0x7f8d581040c0 2024-05-09T06:42:04.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d55d9b700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d581040c0 0x7f8d58196b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00d8d0 tx=0x7f8d4c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:04.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d477fe700 1 -- 172.21.15.63:0/2090877347 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4c0098a0 con 0x7f8d581040c0 2024-05-09T06:42:04.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5819bd40 con 0x7f8d581040c0 2024-05-09T06:42:04.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d477fe700 1 -- 172.21.15.63:0/2090877347 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d4c010460 con 0x7f8d581040c0 2024-05-09T06:42:04.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d477fe700 1 -- 172.21.15.63:0/2090877347 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4c00f5f0 con 0x7f8d581040c0 2024-05-09T06:42:04.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.044+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5819c290 con 0x7f8d581040c0 2024-05-09T06:42:04.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.045+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5804ebb0 con 0x7f8d581040c0 2024-05-09T06:42:04.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.046+0000 7f8d477fe700 1 -- 172.21.15.63:0/2090877347 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8d4c009a00 con 0x7f8d581040c0 2024-05-09T06:42:04.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.047+0000 7f8d477fe700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8d4006e200 0x7f8d400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:04.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.047+0000 7f8d477fe700 1 -- 172.21.15.63:0/2090877347 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8d4c08e000 con 0x7f8d581040c0 2024-05-09T06:42:04.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.047+0000 7f8d5659c700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8d4006e200 0x7f8d400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:04.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.048+0000 7f8d5659c700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8d4006e200 0x7f8d400706c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d48005090 tx=0x7f8d48005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:04.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.056+0000 7f8d477fe700 1 -- 172.21.15.63:0/2090877347 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8d4c05ae60 con 0x7f8d581040c0 2024-05-09T06:42:04.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.308+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8d5819c6a0 con 0x7f8d581040c0 2024-05-09T06:42:04.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.309+0000 7f8d477fe700 1 -- 172.21.15.63:0/2090877347 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1905 (secure 0 0 0) 0x7f8d4c05a9f0 con 0x7f8d581040c0 2024-05-09T06:42:04.289 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:42:04.289 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:42:04.289 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:42:04.289 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:42:04.289 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:42:04.289 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:42:04.289 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:42:04.289 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:42:04.290 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:42:04.291 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:42:04.292 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:42:04.292 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:42:04.292 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:42:04.292 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:42:04.292 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:42:04.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8d4006e200 msgr2=0x7f8d400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8d4006e200 0x7f8d400706c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d48005090 tx=0x7f8d48005960 comp rx=0 tx=0).stop 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d581040c0 msgr2=0x7f8d58196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d581040c0 0x7f8d58196b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00d8d0 tx=0x7f8d4c00dbe0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 shutdown_connections 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8d4006e200 0x7f8d400706c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d58101790 0x7f8d58196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 --2- 172.21.15.63:0/2090877347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d581040c0 0x7f8d58196b80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 >> 172.21.15.63:0/2090877347 conn(0x7f8d580fb120 msgr2=0x7f8d580fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 shutdown_connections 2024-05-09T06:42:04.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.312+0000 7f8d5cada700 1 -- 172.21.15.63:0/2090877347 wait complete. 2024-05-09T06:42:04.294 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:42:04.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.425+0000 7f06e0c16700 1 -- 172.21.15.63:0/689555528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 msgr2=0x7f06dc0ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.425+0000 7f06e0c16700 1 --2- 172.21.15.63:0/689555528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 0x7f06dc0ff930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009910 tx=0x7f06cc009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:04.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.425+0000 7f06e0c16700 1 -- 172.21.15.63:0/689555528 shutdown_connections 2024-05-09T06:42:04.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.425+0000 7f06e0c16700 1 --2- 172.21.15.63:0/689555528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06dc0ffe70 0x7f06dc1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.425+0000 7f06e0c16700 1 --2- 172.21.15.63:0/689555528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 0x7f06dc0ff930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.425+0000 7f06e0c16700 1 -- 172.21.15.63:0/689555528 >> 172.21.15.63:0/689555528 conn(0x7f06dc0fb0d0 msgr2=0x7f06dc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:04.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.425+0000 7f06e0c16700 1 -- 172.21.15.63:0/689555528 shutdown_connections 2024-05-09T06:42:04.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.425+0000 7f06e0c16700 1 -- 172.21.15.63:0/689555528 wait complete. 2024-05-09T06:42:04.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06e0c16700 1 Processor -- start 2024-05-09T06:42:04.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06e0c16700 1 -- start start 2024-05-09T06:42:04.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06e0c16700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 0x7f06dc105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:04.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06e0c16700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06dc0ffe70 0x7f06dc105580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:04.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06e0c16700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc101b80 con 0x7f06dc0ff510 2024-05-09T06:42:04.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06e0c16700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc101cc0 con 0x7f06dc0ffe70 2024-05-09T06:42:04.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06da59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 0x7f06dc105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:04.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06da59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 0x7f06dc105040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51982/0 (socket says 172.21.15.63:51982) 2024-05-09T06:42:04.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06da59c700 1 -- 172.21.15.63:0/1227773150 learned_addr learned my addr 172.21.15.63:0/1227773150 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06d9d9b700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06dc0ffe70 0x7f06dc105580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06da59c700 1 -- 172.21.15.63:0/1227773150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06dc0ffe70 msgr2=0x7f06dc105580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.426+0000 7f06da59c700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06dc0ffe70 0x7f06dc105580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.427+0000 7f06da59c700 1 -- 172.21.15.63:0/1227773150 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc0095f0 con 0x7f06dc0ff510 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.427+0000 7f06da59c700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 0x7f06dc105040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005b40 tx=0x7f06cc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.427+0000 7f06cb7fe700 1 -- 172.21.15.63:0/1227773150 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06cc01d070 con 0x7f06dc0ff510 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.427+0000 7f06cb7fe700 1 -- 172.21.15.63:0/1227773150 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06cc0050d0 con 0x7f06dc0ff510 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.427+0000 7f06cb7fe700 1 -- 172.21.15.63:0/1227773150 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06cc022470 con 0x7f06dc0ff510 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.427+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc101f40 con 0x7f06dc0ff510 2024-05-09T06:42:04.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.427+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc1024b0 con 0x7f06dc0ff510 2024-05-09T06:42:04.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.428+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc1929b0 con 0x7f06dc0ff510 2024-05-09T06:42:04.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.429+0000 7f06cb7fe700 1 -- 172.21.15.63:0/1227773150 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f06cc0037f0 con 0x7f06dc0ff510 2024-05-09T06:42:04.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.430+0000 7f06cb7fe700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f06c406e210 0x7f06c40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:04.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.430+0000 7f06cb7fe700 1 -- 172.21.15.63:0/1227773150 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f06cc08f620 con 0x7f06dc0ff510 2024-05-09T06:42:04.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.430+0000 7f06d9d9b700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f06c406e210 0x7f06c40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:04.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.430+0000 7f06d9d9b700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f06c406e210 0x7f06c40706d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009c60 tx=0x7f06d0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.435+0000 7f06cb7fe700 1 -- 172.21.15.63:0/1227773150 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f06cc05c500 con 0x7f06dc0ff510 2024-05-09T06:42:04.581 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:04 smithi063 ceph-mon[92765]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:04.581 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:04 smithi063 ceph-mon[92765]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:04.581 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:04 smithi063 ceph-mon[92765]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:04.582 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:04 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3750031603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:04.582 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-09T06:42:04.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:04 smithi193 ceph-mon[81665]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:04.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:04 smithi193 ceph-mon[81665]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:04.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:04 smithi193 ceph-mon[81665]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:04.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:04 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3750031603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:04.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-09T06:42:04.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.675+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06dc061330 con 0x7f06c406e210 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.677+0000 7f06cb7fe700 1 -- 172.21.15.63:0/1227773150 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f06dc061330 con 0x7f06c406e210 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:42:04.657 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:42:04.658 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "7/23 daemons upgraded", 2024-05-09T06:42:04.658 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-05-09T06:42:04.658 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:42:04.658 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:42:04.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f06c406e210 msgr2=0x7f06c40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f06c406e210 0x7f06c40706d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009c60 tx=0x7f06d0009260 comp rx=0 tx=0).stop 2024-05-09T06:42:04.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 msgr2=0x7f06dc105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 0x7f06dc105040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005b40 tx=0x7f06cc00f690 comp rx=0 tx=0).stop 2024-05-09T06:42:04.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 shutdown_connections 2024-05-09T06:42:04.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f06c406e210 0x7f06c40706d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06dc0ff510 0x7f06dc105040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 --2- 172.21.15.63:0/1227773150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06dc0ffe70 0x7f06dc105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 >> 172.21.15.63:0/1227773150 conn(0x7f06dc0fb0d0 msgr2=0x7f06dc107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:04.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 shutdown_connections 2024-05-09T06:42:04.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.679+0000 7f06e0c16700 1 -- 172.21.15.63:0/1227773150 wait complete. 2024-05-09T06:42:04.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.790+0000 7fb3f7455700 1 -- 172.21.15.63:0/143739630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0104080 msgr2=0x7fb3f0106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.790+0000 7fb3f7455700 1 --2- 172.21.15.63:0/143739630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0104080 0x7fb3f0106470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009910 tx=0x7fb3e0009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:04.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.790+0000 7fb3f7455700 1 -- 172.21.15.63:0/143739630 shutdown_connections 2024-05-09T06:42:04.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.790+0000 7fb3f7455700 1 --2- 172.21.15.63:0/143739630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0104080 0x7fb3f0106470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.790+0000 7fb3f7455700 1 --2- 172.21.15.63:0/143739630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f0101750 0x7fb3f0103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.790+0000 7fb3f7455700 1 -- 172.21.15.63:0/143739630 >> 172.21.15.63:0/143739630 conn(0x7fb3f00fb0c0 msgr2=0x7fb3f00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:04.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.790+0000 7fb3f7455700 1 -- 172.21.15.63:0/143739630 shutdown_connections 2024-05-09T06:42:04.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.790+0000 7fb3f7455700 1 -- 172.21.15.63:0/143739630 wait complete. 2024-05-09T06:42:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f7455700 1 Processor -- start 2024-05-09T06:42:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f7455700 1 -- start start 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f7455700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0101750 0x7fb3f0196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f7455700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f0104080 0x7fb3f0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f7455700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0197160 con 0x7fb3f0101750 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f7455700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f01972a0 con 0x7fb3f0104080 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f51f1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0101750 0x7fb3f0196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f51f1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0101750 0x7fb3f0196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36866/0 (socket says 172.21.15.63:36866) 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f51f1700 1 -- 172.21.15.63:0/1991005576 learned_addr learned my addr 172.21.15.63:0/1991005576 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f49f0700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f0104080 0x7fb3f0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f51f1700 1 -- 172.21.15.63:0/1991005576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f0104080 msgr2=0x7fb3f0196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f51f1700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f0104080 0x7fb3f0196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.791+0000 7fb3f51f1700 1 -- 172.21.15.63:0/1991005576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00095f0 con 0x7fb3f0101750 2024-05-09T06:42:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.792+0000 7fb3f51f1700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0101750 0x7fb3f0196600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00e920 tx=0x7fb3ec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:04.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.792+0000 7fb3e67fc700 1 -- 172.21.15.63:0/1991005576 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3ec00c9d0 con 0x7fb3f0101750 2024-05-09T06:42:04.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.792+0000 7fb3e67fc700 1 -- 172.21.15.63:0/1991005576 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3ec010430 con 0x7fb3f0101750 2024-05-09T06:42:04.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.792+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f019bd00 con 0x7fb3f0101750 2024-05-09T06:42:04.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.792+0000 7fb3e67fc700 1 -- 172.21.15.63:0/1991005576 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3ec018630 con 0x7fb3f0101750 2024-05-09T06:42:04.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.792+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f019c270 con 0x7fb3f0101750 2024-05-09T06:42:04.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.793+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f004ebb0 con 0x7fb3f0101750 2024-05-09T06:42:04.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.793+0000 7fb3e67fc700 1 -- 172.21.15.63:0/1991005576 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb3ec0105a0 con 0x7fb3f0101750 2024-05-09T06:42:04.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.794+0000 7fb3e67fc700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3dc06e200 0x7fb3dc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:04.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.794+0000 7fb3e67fc700 1 -- 172.21.15.63:0/1991005576 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb3ec014070 con 0x7fb3f0101750 2024-05-09T06:42:04.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.794+0000 7fb3f49f0700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3dc06e200 0x7fb3dc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:04.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.794+0000 7fb3f49f0700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3dc06e200 0x7fb3dc0706c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000b800 tx=0x7fb3e0009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:04.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:04.798+0000 7fb3e67fc700 1 -- 172.21.15.63:0/1991005576 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb3ec053740 con 0x7fb3f0101750 2024-05-09T06:42:05.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.100+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb3f0109790 con 0x7fb3f0101750 2024-05-09T06:42:05.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.101+0000 7fb3e67fc700 1 -- 172.21.15.63:0/1991005576 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb3ec05b250 con 0x7fb3f0101750 2024-05-09T06:42:05.081 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3dc06e200 msgr2=0x7fb3dc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3dc06e200 0x7fb3dc0706c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000b800 tx=0x7fb3e0009f70 comp rx=0 tx=0).stop 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0101750 msgr2=0x7fb3f0196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0101750 0x7fb3f0196600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00e920 tx=0x7fb3ec00ec30 comp rx=0 tx=0).stop 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 shutdown_connections 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3dc06e200 0x7fb3dc0706c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3f0101750 0x7fb3f0196600 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 --2- 172.21.15.63:0/1991005576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f0104080 0x7fb3f0196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 >> 172.21.15.63:0/1991005576 conn(0x7fb3f00fb0c0 msgr2=0x7fb3f00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 shutdown_connections 2024-05-09T06:42:05.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:05.103+0000 7fb3f7455700 1 -- 172.21.15.63:0/1991005576 wait complete. 2024-05-09T06:42:05.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:05 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-09T06:42:05.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:05 smithi063 ceph-mon[92765]: Upgrade: osd.1 is safe to restart 2024-05-09T06:42:05.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:05 smithi063 ceph-mon[92765]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:42:05.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:05 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2090877347' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:42:05.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:05 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1991005576' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:42:05.617 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:05 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-09T06:42:05.617 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:05 smithi193 ceph-mon[81665]: Upgrade: osd.1 is safe to restart 2024-05-09T06:42:05.617 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:05 smithi193 ceph-mon[81665]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:42:05.617 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:05 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2090877347' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:42:05.617 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:05 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1991005576' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:42:06.484 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:06 smithi063 ceph-mon[92765]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:06.484 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:06 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:06.484 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:06 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T06:42:06.484 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:06 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:42:06.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:06 smithi193 ceph-mon[81665]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:06.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:06 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:06.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:06 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T06:42:06.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:06 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:42:07.510 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:07 smithi063 ceph-mon[92765]: Upgrade: Updating osd.1 2024-05-09T06:42:07.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:07 smithi063 ceph-mon[92765]: Deploying daemon osd.1 on smithi063 2024-05-09T06:42:07.511 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:07 smithi063 ceph-mon[92765]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:07.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:07 smithi193 ceph-mon[81665]: Upgrade: Updating osd.1 2024-05-09T06:42:07.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:07 smithi193 ceph-mon[81665]: Deploying daemon osd.1 on smithi063 2024-05-09T06:42:07.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:07 smithi193 ceph-mon[81665]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:09.195 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:08 smithi063 systemd[1]: Stopping Ceph osd.1 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:42:09.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:09 smithi063 ceph-mon[92765]: pgmap v66: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:42:09.605 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:09 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[53735]: 2024-05-09T06:42:09.220+0000 7f4b5cbf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:42:09.605 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:09 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[53735]: 2024-05-09T06:42:09.220+0000 7f4b5cbf4700 -1 osd.1 55 *** Got signal Terminated *** 2024-05-09T06:42:09.605 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:09 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[53735]: 2024-05-09T06:42:09.220+0000 7f4b5cbf4700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:42:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:09 smithi193 ceph-mon[81665]: pgmap v66: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:42:10.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:10 smithi063 ceph-mon[92765]: osd.1 marked itself down and dead 2024-05-09T06:42:10.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:10 smithi193 ceph-mon[81665]: osd.1 marked itself down and dead 2024-05-09T06:42:11.068 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:10 smithi063 podman[107049]: 2024-05-09 06:42:10.784201873 +0000 UTC m=+1.679770345 container stop 3005e767d3a71bba798bc70b99409453a5f8242f4021739284e0def502c3ad95 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=reef-1758e8e, maintainer=Guillaume Abrioux ) 2024-05-09T06:42:11.069 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:10 smithi063 podman[107049]: 2024-05-09 06:42:10.809901324 +0000 UTC m=+1.705469801 container died 3005e767d3a71bba798bc70b99409453a5f8242f4021739284e0def502c3ad95 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=reef-1758e8e, ceph=True) 2024-05-09T06:42:11.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:11 smithi063 ceph-mon[92765]: pgmap v67: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:11.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:11 smithi063 ceph-mon[92765]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:42:11.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:11 smithi063 ceph-mon[92765]: osdmap e56: 6 total, 5 up, 6 in 2024-05-09T06:42:11.334 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:11 smithi063 podman[107049]: 2024-05-09 06:42:11.178532954 +0000 UTC m=+2.074101412 container cleanup 3005e767d3a71bba798bc70b99409453a5f8242f4021739284e0def502c3ad95 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-1758e8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T06:42:11.334 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:11 smithi063 bash[107049]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1 2024-05-09T06:42:11.605 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:11 smithi063 podman[107066]: 2024-05-09 06:42:11.354865094 +0000 UTC m=+0.563145599 container remove 3005e767d3a71bba798bc70b99409453a5f8242f4021739284e0def502c3ad95 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-1758e8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T06:42:11.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:11 smithi193 ceph-mon[81665]: pgmap v67: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:11.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:11 smithi193 ceph-mon[81665]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:42:11.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:11 smithi193 ceph-mon[81665]: osdmap e56: 6 total, 5 up, 6 in 2024-05-09T06:42:12.105 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:11 smithi063 podman[107159]: 2024-05-09 06:42:11.716952949 +0000 UTC m=+0.019122618 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:42:12.105 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:11 smithi063 podman[107159]: 2024-05-09 06:42:11.831021115 +0000 UTC m=+0.133190825 container create f7c9f24eb1fa2a18d5cbb9c037123bee99f18f32e85bc959d3e986182091be11 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:42:12.105 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:12 smithi063 podman[107159]: 2024-05-09 06:42:12.050567573 +0000 UTC m=+0.352737246 container init f7c9f24eb1fa2a18d5cbb9c037123bee99f18f32e85bc959d3e986182091be11 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-09T06:42:12.105 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:12 smithi063 podman[107159]: 2024-05-09 06:42:12.054840901 +0000 UTC m=+0.357010570 container start f7c9f24eb1fa2a18d5cbb9c037123bee99f18f32e85bc959d3e986182091be11 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-09T06:42:12.591 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: osdmap e57: 6 total, 5 up, 6 in 2024-05-09T06:42:12.591 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]: dispatch 2024-05-09T06:42:12.591 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]: dispatch 2024-05-09T06:42:12.591 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [3, 1]}]: dispatch 2024-05-09T06:42:12.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.19", "id": [0, 1]}]: dispatch 2024-05-09T06:42:12.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-05-09T06:42:12.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 1]}]: dispatch 2024-05-09T06:42:12.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 0]}]: dispatch 2024-05-09T06:42:12.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.4", "id": [3, 0]}]: dispatch 2024-05-09T06:42:12.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [4, 5]}]: dispatch 2024-05-09T06:42:12.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [4, 2]}]: dispatch 2024-05-09T06:42:12.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:12.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:42:12.592 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:12 smithi063 podman[107159]: 2024-05-09 06:42:12.135179155 +0000 UTC m=+0.437348833 container attach f7c9f24eb1fa2a18d5cbb9c037123bee99f18f32e85bc959d3e986182091be11 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-09T06:42:12.592 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:12 smithi063 podman[107159]: 2024-05-09 06:42:12.377063425 +0000 UTC m=+0.679233096 container died f7c9f24eb1fa2a18d5cbb9c037123bee99f18f32e85bc959d3e986182091be11 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T06:42:12.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: osdmap e57: 6 total, 5 up, 6 in 2024-05-09T06:42:12.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]: dispatch 2024-05-09T06:42:12.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]: dispatch 2024-05-09T06:42:12.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [3, 1]}]: dispatch 2024-05-09T06:42:12.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.19", "id": [0, 1]}]: dispatch 2024-05-09T06:42:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-05-09T06:42:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 1]}]: dispatch 2024-05-09T06:42:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 0]}]: dispatch 2024-05-09T06:42:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.4", "id": [3, 0]}]: dispatch 2024-05-09T06:42:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [4, 5]}]: dispatch 2024-05-09T06:42:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [4, 2]}]: dispatch 2024-05-09T06:42:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:12.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:42:13.209 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:12 smithi063 podman[107185]: 2024-05-09 06:42:12.937106375 +0000 UTC m=+0.554135983 container remove f7c9f24eb1fa2a18d5cbb9c037123bee99f18f32e85bc959d3e986182091be11 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-09T06:42:13.209 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:12 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.1.service: Deactivated successfully. 2024-05-09T06:42:13.209 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:12 smithi063 systemd[1]: Stopped Ceph osd.1 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:42:13.209 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:12 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.1.service: Consumed 1min 11.188s CPU time. 2024-05-09T06:42:13.209 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:13 smithi063 systemd[1]: Starting Ceph osd.1 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:42:13.503 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:13 smithi063 podman[107285]: 2024-05-09 06:42:13.301919999 +0000 UTC m=+0.019160828 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:42:13.503 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:13 smithi063 podman[107285]: 2024-05-09 06:42:13.421571713 +0000 UTC m=+0.138812582 container create bc4e38396fc7abc74c8bcc827bf445cbd125644fd10710ce5395a0fae9e6b7f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T06:42:13.503 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-09T06:42:13.503 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]': finished 2024-05-09T06:42:13.503 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]': finished 2024-05-09T06:42:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [3, 1]}]': finished 2024-05-09T06:42:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.19", "id": [0, 1]}]': finished 2024-05-09T06:42:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-05-09T06:42:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 1]}]': finished 2024-05-09T06:42:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 0]}]': finished 2024-05-09T06:42:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.4", "id": [3, 0]}]': finished 2024-05-09T06:42:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [4, 5]}]': finished 2024-05-09T06:42:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [4, 2]}]': finished 2024-05-09T06:42:13.504 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: osdmap e58: 6 total, 5 up, 6 in 2024-05-09T06:42:13.505 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:13 smithi063 ceph-mon[92765]: osdmap e59: 6 total, 5 up, 6 in 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]': finished 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]': finished 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [3, 1]}]': finished 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.19", "id": [0, 1]}]': finished 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 1]}]': finished 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 0]}]': finished 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.4", "id": [3, 0]}]': finished 2024-05-09T06:42:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [4, 5]}]': finished 2024-05-09T06:42:13.523 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [4, 2]}]': finished 2024-05-09T06:42:13.523 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: osdmap e58: 6 total, 5 up, 6 in 2024-05-09T06:42:13.523 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:13 smithi193 ceph-mon[81665]: osdmap e59: 6 total, 5 up, 6 in 2024-05-09T06:42:13.855 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:13 smithi063 podman[107285]: 2024-05-09 06:42:13.632332392 +0000 UTC m=+0.349573221 container init bc4e38396fc7abc74c8bcc827bf445cbd125644fd10710ce5395a0fae9e6b7f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=) 2024-05-09T06:42:13.855 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:13 smithi063 podman[107285]: 2024-05-09 06:42:13.635817344 +0000 UTC m=+0.353058173 container start bc4e38396fc7abc74c8bcc827bf445cbd125644fd10710ce5395a0fae9e6b7f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T06:42:13.855 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:13 smithi063 podman[107285]: 2024-05-09 06:42:13.707151216 +0000 UTC m=+0.424392043 container attach bc4e38396fc7abc74c8bcc827bf445cbd125644fd10710ce5395a0fae9e6b7f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T06:42:14.355 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate[107300]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T06:42:14.355 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate[107300]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-09T06:42:14.355 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 bash[107285]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 bash[107285]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 bash[107285]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 bash[107285]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 bash[107285]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 bash[107285]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 bash[107285]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate[107300]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate[107300]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate[107300]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate[107300]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate[107300]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-09T06:42:14.356 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 podman[107285]: 2024-05-09 06:42:14.133158233 +0000 UTC m=+0.850399060 container died bc4e38396fc7abc74c8bcc827bf445cbd125644fd10710ce5395a0fae9e6b7f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T06:42:15.158 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:14 smithi063 podman[107439]: 2024-05-09 06:42:14.911924488 +0000 UTC m=+0.768303778 container remove bc4e38396fc7abc74c8bcc827bf445cbd125644fd10710ce5395a0fae9e6b7f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T06:42:15.159 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:15 smithi063 podman[107504]: 2024-05-09 06:42:15.053980406 +0000 UTC m=+0.022302546 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:42:15.159 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:15 smithi063 podman[107504]: 2024-05-09 06:42:15.17960264 +0000 UTC m=+0.147924767 container create f1a348429588844ad28718d0138f8222f8f127b34d3fa073a0d5aecc45b6f341 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-09T06:42:15.411 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:15 smithi063 ceph-mon[92765]: pgmap v73: 65 pgs: 6 unknown, 2 peering, 10 active+undersized+degraded, 2 stale+active+clean, 15 active+undersized, 30 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26/225 objects degraded (11.556%) 2024-05-09T06:42:15.411 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:15 smithi063 ceph-mon[92765]: osdmap e60: 6 total, 5 up, 6 in 2024-05-09T06:42:15.411 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:15 smithi063 ceph-mon[92765]: Health check failed: Degraded data redundancy: 26/225 objects degraded (11.556%), 10 pgs degraded (PG_DEGRADED) 2024-05-09T06:42:15.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:15 smithi193 ceph-mon[81665]: pgmap v73: 65 pgs: 6 unknown, 2 peering, 10 active+undersized+degraded, 2 stale+active+clean, 15 active+undersized, 30 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26/225 objects degraded (11.556%) 2024-05-09T06:42:15.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:15 smithi193 ceph-mon[81665]: osdmap e60: 6 total, 5 up, 6 in 2024-05-09T06:42:15.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:15 smithi193 ceph-mon[81665]: Health check failed: Degraded data redundancy: 26/225 objects degraded (11.556%), 10 pgs degraded (PG_DEGRADED) 2024-05-09T06:42:15.855 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:15 smithi063 podman[107504]: 2024-05-09 06:42:15.432350788 +0000 UTC m=+0.400672904 container init f1a348429588844ad28718d0138f8222f8f127b34d3fa073a0d5aecc45b6f341 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc) 2024-05-09T06:42:15.856 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:15 smithi063 podman[107504]: 2024-05-09 06:42:15.436539488 +0000 UTC m=+0.404861607 container start f1a348429588844ad28718d0138f8222f8f127b34d3fa073a0d5aecc45b6f341 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_BRANCH=HEAD) 2024-05-09T06:42:15.856 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:15 smithi063 ceph-osd[107523]: -- 172.21.15.63:0/1880284795 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564070c2e000 con 0x56406fe49000 2024-05-09T06:42:15.856 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:15 smithi063 bash[107504]: f1a348429588844ad28718d0138f8222f8f127b34d3fa073a0d5aecc45b6f341 2024-05-09T06:42:15.856 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:15 smithi063 systemd[1]: Started Ceph osd.1 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:42:16.217 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:15 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[107519]: 2024-05-09T06:42:15.980+0000 7f70cc67e7c0 -1 Falling back to public interface 2024-05-09T06:42:16.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:16 smithi063 ceph-mon[92765]: osdmap e61: 6 total, 5 up, 6 in 2024-05-09T06:42:16.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:16 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:16.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:16 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:16.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:16 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:42:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:16 smithi193 ceph-mon[81665]: osdmap e61: 6 total, 5 up, 6 in 2024-05-09T06:42:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:16 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:16 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:16 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:42:17.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:17 smithi063 ceph-mon[92765]: pgmap v76: 65 pgs: 28 active+clean, 1 active+recovery_wait+undersized+remapped, 18 active+undersized, 1 active+undersized+remapped, 10 active+undersized+degraded, 2 peering, 5 unknown; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26/228 objects degraded (11.404%); 711/228 objects misplaced (311.842%); 1024 KiB/s, 4 objects/s recovering 2024-05-09T06:42:17.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:17 smithi063 ceph-mon[92765]: osdmap e62: 6 total, 5 up, 6 in 2024-05-09T06:42:17.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:17 smithi193 ceph-mon[81665]: pgmap v76: 65 pgs: 28 active+clean, 1 active+recovery_wait+undersized+remapped, 18 active+undersized, 1 active+undersized+remapped, 10 active+undersized+degraded, 2 peering, 5 unknown; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26/228 objects degraded (11.404%); 711/228 objects misplaced (311.842%); 1024 KiB/s, 4 objects/s recovering 2024-05-09T06:42:17.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:17 smithi193 ceph-mon[81665]: osdmap e62: 6 total, 5 up, 6 in 2024-05-09T06:42:18.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:18 smithi063 ceph-mon[92765]: osdmap e63: 6 total, 5 up, 6 in 2024-05-09T06:42:18.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:18 smithi193 ceph-mon[81665]: osdmap e63: 6 total, 5 up, 6 in 2024-05-09T06:42:19.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:19 smithi063 ceph-mon[92765]: pgmap v79: 65 pgs: 1 remapped+peering, 8 active+clean+remapped, 27 active+clean, 1 active+recovery_wait+undersized+remapped, 18 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 91 op/s; 26/228 objects degraded (11.404%); 725/228 objects misplaced (317.982%); 2.0 MiB/s, 528 objects/s recovering 2024-05-09T06:42:19.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:19 smithi063 ceph-mon[92765]: osdmap e64: 6 total, 5 up, 6 in 2024-05-09T06:42:19.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:19 smithi193 ceph-mon[81665]: pgmap v79: 65 pgs: 1 remapped+peering, 8 active+clean+remapped, 27 active+clean, 1 active+recovery_wait+undersized+remapped, 18 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 91 op/s; 26/228 objects degraded (11.404%); 725/228 objects misplaced (317.982%); 2.0 MiB/s, 528 objects/s recovering 2024-05-09T06:42:19.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:19 smithi193 ceph-mon[81665]: osdmap e64: 6 total, 5 up, 6 in 2024-05-09T06:42:21.419 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:21 smithi063 ceph-mon[92765]: pgmap v81: 65 pgs: 1 remapped+peering, 9 active+clean+remapped, 27 active+clean, 18 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 26/228 objects degraded (11.404%); 15/228 objects misplaced (6.579%); 838 KiB/s, 574 objects/s recovering 2024-05-09T06:42:21.419 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:21 smithi063 ceph-mon[92765]: from='osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T06:42:21.420 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:21 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[107519]: 2024-05-09T06:42:21.124+0000 7f70cc67e7c0 -1 osd.1 55 log_to_monitors true 2024-05-09T06:42:21.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:21 smithi193 ceph-mon[81665]: pgmap v81: 65 pgs: 1 remapped+peering, 9 active+clean+remapped, 27 active+clean, 18 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 26/228 objects degraded (11.404%); 15/228 objects misplaced (6.579%); 838 KiB/s, 574 objects/s recovering 2024-05-09T06:42:21.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:21 smithi193 ceph-mon[81665]: from='osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T06:42:22.355 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:42:22 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[107519]: 2024-05-09T06:42:22.187+0000 7f70b9acd700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:42:22.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:22 smithi063 ceph-mon[92765]: from='osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T06:42:22.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:22 smithi063 ceph-mon[92765]: osdmap e65: 6 total, 5 up, 6 in 2024-05-09T06:42:22.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:22 smithi063 ceph-mon[92765]: from='osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:42:22.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:22.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:22.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:22 smithi063 ceph-mon[92765]: from='osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098]' entity='osd.1' 2024-05-09T06:42:22.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:22 smithi193 ceph-mon[81665]: from='osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T06:42:22.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:22 smithi193 ceph-mon[81665]: osdmap e65: 6 total, 5 up, 6 in 2024-05-09T06:42:22.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:22 smithi193 ceph-mon[81665]: from='osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:42:22.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:22 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:22.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:22 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:22.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:22 smithi193 ceph-mon[81665]: from='osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098]' entity='osd.1' 2024-05-09T06:42:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:23 smithi063 ceph-mon[92765]: pgmap v83: 65 pgs: 1 remapped+peering, 9 active+clean+remapped, 27 active+clean, 18 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 869 B/s rd, 1 op/s; 26/228 objects degraded (11.404%); 15/228 objects misplaced (6.579%); 696 KiB/s, 476 objects/s recovering 2024-05-09T06:42:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:23 smithi063 ceph-mon[92765]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:42:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:23 smithi063 ceph-mon[92765]: Health check update: Degraded data redundancy: 26/228 objects degraded (11.404%), 10 pgs degraded (PG_DEGRADED) 2024-05-09T06:42:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:23 smithi063 ceph-mon[92765]: osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098] boot 2024-05-09T06:42:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:23 smithi063 ceph-mon[92765]: osdmap e66: 6 total, 6 up, 6 in 2024-05-09T06:42:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:42:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:23.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:23.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:23 smithi193 ceph-mon[81665]: pgmap v83: 65 pgs: 1 remapped+peering, 9 active+clean+remapped, 27 active+clean, 18 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 869 B/s rd, 1 op/s; 26/228 objects degraded (11.404%); 15/228 objects misplaced (6.579%); 696 KiB/s, 476 objects/s recovering 2024-05-09T06:42:23.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:23 smithi193 ceph-mon[81665]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:42:23.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:23 smithi193 ceph-mon[81665]: Health check update: Degraded data redundancy: 26/228 objects degraded (11.404%), 10 pgs degraded (PG_DEGRADED) 2024-05-09T06:42:23.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:23 smithi193 ceph-mon[81665]: osd.1 [v2:172.21.15.63:6810/2256761098,v1:172.21.15.63:6811/2256761098] boot 2024-05-09T06:42:23.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:23 smithi193 ceph-mon[81665]: osdmap e66: 6 total, 6 up, 6 in 2024-05-09T06:42:23.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T06:42:23.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:23.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:25.512 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:25 smithi063 ceph-mon[92765]: pgmap v85: 65 pgs: 10 active+clean+remapped, 27 active+clean, 18 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 26/228 objects degraded (11.404%); 15/228 objects misplaced (6.579%); 0 B/s, 118 objects/s recovering 2024-05-09T06:42:25.513 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:25 smithi063 ceph-mon[92765]: osdmap e67: 6 total, 6 up, 6 in 2024-05-09T06:42:25.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:25 smithi193 ceph-mon[81665]: pgmap v85: 65 pgs: 10 active+clean+remapped, 27 active+clean, 18 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 26/228 objects degraded (11.404%); 15/228 objects misplaced (6.579%); 0 B/s, 118 objects/s recovering 2024-05-09T06:42:25.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:25 smithi193 ceph-mon[81665]: osdmap e67: 6 total, 6 up, 6 in 2024-05-09T06:42:26.557 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:26 smithi063 ceph-mon[92765]: osdmap e68: 6 total, 6 up, 6 in 2024-05-09T06:42:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:26 smithi193 ceph-mon[81665]: osdmap e68: 6 total, 6 up, 6 in 2024-05-09T06:42:27.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: pgmap v88: 65 pgs: 10 active+clean+remapped, 29 active+clean, 16 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 26/228 objects degraded (11.404%); 15/228 objects misplaced (6.579%) 2024-05-09T06:42:27.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: osdmap e69: 6 total, 6 up, 6 in 2024-05-09T06:42:27.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:27.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:27.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:42:27.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:42:27.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:27.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:42:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-09T06:42:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:27.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:42:27.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: pgmap v88: 65 pgs: 10 active+clean+remapped, 29 active+clean, 16 active+undersized, 10 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 26/228 objects degraded (11.404%); 15/228 objects misplaced (6.579%) 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: osdmap e69: 6 total, 6 up, 6 in 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:27.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:42:28.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:28 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-09T06:42:28.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:28 smithi063 ceph-mon[92765]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-05-09T06:42:28.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:28 smithi063 ceph-mon[92765]: osdmap e70: 6 total, 6 up, 6 in 2024-05-09T06:42:28.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:28 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-09T06:42:28.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:28 smithi193 ceph-mon[81665]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-05-09T06:42:28.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:28 smithi193 ceph-mon[81665]: osdmap e70: 6 total, 6 up, 6 in 2024-05-09T06:42:29.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:29 smithi063 ceph-mon[92765]: pgmap v91: 65 pgs: 3 peering, 62 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 8.0 MiB/s, 0 objects/s recovering 2024-05-09T06:42:29.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:29 smithi063 ceph-mon[92765]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-09T06:42:29.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:29 smithi063 ceph-mon[92765]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 26/228 objects degraded (11.404%), 10 pgs degraded) 2024-05-09T06:42:29.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:29 smithi193 ceph-mon[81665]: pgmap v91: 65 pgs: 3 peering, 62 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 8.0 MiB/s, 0 objects/s recovering 2024-05-09T06:42:29.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:29 smithi193 ceph-mon[81665]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-09T06:42:29.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:29 smithi193 ceph-mon[81665]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 26/228 objects degraded (11.404%), 10 pgs degraded) 2024-05-09T06:42:31.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:31 smithi063 ceph-mon[92765]: pgmap v92: 65 pgs: 3 peering, 62 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5.4 MiB/s, 0 objects/s recovering 2024-05-09T06:42:31.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:31 smithi193 ceph-mon[81665]: pgmap v92: 65 pgs: 3 peering, 62 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5.4 MiB/s, 0 objects/s recovering 2024-05-09T06:42:32.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:32 smithi063 ceph-mon[92765]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-09T06:42:32.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:32 smithi063 ceph-mon[92765]: Cluster is now healthy 2024-05-09T06:42:32.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:32 smithi193 ceph-mon[81665]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-09T06:42:32.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:32 smithi193 ceph-mon[81665]: Cluster is now healthy 2024-05-09T06:42:33.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:33 smithi063 ceph-mon[92765]: pgmap v93: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 738 B/s rd, 0 op/s; 4.6 MiB/s, 0 objects/s recovering 2024-05-09T06:42:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:33 smithi193 ceph-mon[81665]: pgmap v93: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 738 B/s rd, 0 op/s; 4.6 MiB/s, 0 objects/s recovering 2024-05-09T06:42:35.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.229+0000 7fdc15017700 1 -- 172.21.15.63:0/2489979365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc100ffb50 msgr2=0x7fdc100fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.229+0000 7fdc15017700 1 --2- 172.21.15.63:0/2489979365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc100ffb50 0x7fdc100fff70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009910 tx=0x7fdc00009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:35.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.229+0000 7fdc15017700 1 -- 172.21.15.63:0/2489979365 shutdown_connections 2024-05-09T06:42:35.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.229+0000 7fdc15017700 1 --2- 172.21.15.63:0/2489979365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc10100d50 0x7fdc101011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.229+0000 7fdc15017700 1 --2- 172.21.15.63:0/2489979365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc100ffb50 0x7fdc100fff70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.229+0000 7fdc15017700 1 -- 172.21.15.63:0/2489979365 >> 172.21.15.63:0/2489979365 conn(0x7fdc100fb0f0 msgr2=0x7fdc100fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:35.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.229+0000 7fdc15017700 1 -- 172.21.15.63:0/2489979365 shutdown_connections 2024-05-09T06:42:35.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.229+0000 7fdc15017700 1 -- 172.21.15.63:0/2489979365 wait complete. 2024-05-09T06:42:35.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.230+0000 7fdc15017700 1 Processor -- start 2024-05-09T06:42:35.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.230+0000 7fdc15017700 1 -- start start 2024-05-09T06:42:35.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.230+0000 7fdc15017700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc100ffb50 0x7fdc101039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:35.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.230+0000 7fdc15017700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc10100d50 0x7fdc10102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:35.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.230+0000 7fdc15017700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc10104010 con 0x7fdc10100d50 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.230+0000 7fdc15017700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc10102670 con 0x7fdc100ffb50 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.230+0000 7fdc0e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc10100d50 0x7fdc10102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.230+0000 7fdc0e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc10100d50 0x7fdc10102130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44372/0 (socket says 172.21.15.63:44372) 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.230+0000 7fdc0e59c700 1 -- 172.21.15.63:0/2325710353 learned_addr learned my addr 172.21.15.63:0/2325710353 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc0ed9d700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc100ffb50 0x7fdc101039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc0e59c700 1 -- 172.21.15.63:0/2325710353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc100ffb50 msgr2=0x7fdc101039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc0e59c700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc100ffb50 0x7fdc101039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc0e59c700 1 -- 172.21.15.63:0/2325710353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc000095f0 con 0x7fdc10100d50 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc0e59c700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc10100d50 0x7fdc10102130 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800d8d0 tx=0x7fdbf800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc07fff700 1 -- 172.21.15.63:0/2325710353 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf80098a0 con 0x7fdc10100d50 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc07fff700 1 -- 172.21.15.63:0/2325710353 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbf8010460 con 0x7fdc10100d50 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc10102920 con 0x7fdc10100d50 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc07fff700 1 -- 172.21.15.63:0/2325710353 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf800f5f0 con 0x7fdc10100d50 2024-05-09T06:42:35.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.231+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc10102ef0 con 0x7fdc10100d50 2024-05-09T06:42:35.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.232+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1004ebb0 con 0x7fdc10100d50 2024-05-09T06:42:35.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.233+0000 7fdc07fff700 1 -- 172.21.15.63:0/2325710353 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdbf8009a00 con 0x7fdc10100d50 2024-05-09T06:42:35.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.233+0000 7fdc07fff700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdbfc06e200 0x7fdbfc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:35.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.234+0000 7fdc07fff700 1 -- 172.21.15.63:0/2325710353 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fdbf808e040 con 0x7fdc10100d50 2024-05-09T06:42:35.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.234+0000 7fdc0ed9d700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdbfc06e200 0x7fdbfc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:35.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.234+0000 7fdc0ed9d700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdbfc06e200 0x7fdbfc0706c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000c00 tx=0x7fdc00005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:35.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.238+0000 7fdc07fff700 1 -- 172.21.15.63:0/2325710353 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdbf805ae00 con 0x7fdc10100d50 2024-05-09T06:42:35.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.469+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc100698e0 con 0x7fdbfc06e200 2024-05-09T06:42:35.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.470+0000 7fdc07fff700 1 -- 172.21.15.63:0/2325710353 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdc100698e0 con 0x7fdbfc06e200 2024-05-09T06:42:35.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.472+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdbfc06e200 msgr2=0x7fdbfc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.472+0000 7fdc15017700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdbfc06e200 0x7fdbfc0706c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000c00 tx=0x7fdc00005960 comp rx=0 tx=0).stop 2024-05-09T06:42:35.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.472+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc10100d50 msgr2=0x7fdc10102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.472+0000 7fdc15017700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc10100d50 0x7fdc10102130 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800d8d0 tx=0x7fdbf800dbe0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.472+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 shutdown_connections 2024-05-09T06:42:35.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.472+0000 7fdc15017700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdbfc06e200 0x7fdbfc0706c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.472+0000 7fdc15017700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc100ffb50 0x7fdc101039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.472+0000 7fdc15017700 1 --2- 172.21.15.63:0/2325710353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc10100d50 0x7fdc10102130 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.472+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 >> 172.21.15.63:0/2325710353 conn(0x7fdc100fb0f0 msgr2=0x7fdc100691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:35.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.473+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 shutdown_connections 2024-05-09T06:42:35.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.473+0000 7fdc15017700 1 -- 172.21.15.63:0/2325710353 wait complete. 2024-05-09T06:42:35.463 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:42:35.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:35 smithi063 ceph-mon[92765]: pgmap v94: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 4.0 MiB/s, 0 objects/s recovering 2024-05-09T06:42:35.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 -- 172.21.15.63:0/1492890905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 msgr2=0x7f1714102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 --2- 172.21.15.63:0/1492890905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 0x7f1714102350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1708009910 tx=0x7f1708009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:35.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 -- 172.21.15.63:0/1492890905 shutdown_connections 2024-05-09T06:42:35.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 --2- 172.21.15.63:0/1492890905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 0x7f1714102350 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 --2- 172.21.15.63:0/1492890905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17140fef10 0x7f17140ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 -- 172.21.15.63:0/1492890905 >> 172.21.15.63:0/1492890905 conn(0x7f17140fa870 msgr2=0x7f17140fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:35.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 -- 172.21.15.63:0/1492890905 shutdown_connections 2024-05-09T06:42:35.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 -- 172.21.15.63:0/1492890905 wait complete. 2024-05-09T06:42:35.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 Processor -- start 2024-05-09T06:42:35.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.582+0000 7f171a616700 1 -- start start 2024-05-09T06:42:35.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f171a616700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17140fef10 0x7f17141017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:35.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f171a616700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 0x7f17140ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:35.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f171a616700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141003e0 con 0x7f1714101ef0 2024-05-09T06:42:35.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f171a616700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1714100520 con 0x7f17140fef10 2024-05-09T06:42:35.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f17137fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 0x7f17140ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:35.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f17137fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 0x7f17140ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44376/0 (socket says 172.21.15.63:44376) 2024-05-09T06:42:35.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f17137fe700 1 -- 172.21.15.63:0/1066296788 learned_addr learned my addr 172.21.15.63:0/1066296788 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f1713fff700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17140fef10 0x7f17141017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f17137fe700 1 -- 172.21.15.63:0/1066296788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17140fef10 msgr2=0x7f17141017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f17137fe700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17140fef10 0x7f17141017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f17137fe700 1 -- 172.21.15.63:0/1066296788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080095f0 con 0x7f1714101ef0 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.583+0000 7f17137fe700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 0x7f17140ffea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1708006010 tx=0x7f170800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.584+0000 7f17117fa700 1 -- 172.21.15.63:0/1066296788 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f170801d070 con 0x7f1714101ef0 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.584+0000 7f17117fa700 1 -- 172.21.15.63:0/1066296788 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f170800f460 con 0x7f1714101ef0 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.584+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17141007a0 con 0x7f1714101ef0 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.584+0000 7f17117fa700 1 -- 172.21.15.63:0/1066296788 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1708021620 con 0x7f1714101ef0 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.584+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1714100d10 con 0x7f1714101ef0 2024-05-09T06:42:35.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.585+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17141907d0 con 0x7f1714101ef0 2024-05-09T06:42:35.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.585+0000 7f17117fa700 1 -- 172.21.15.63:0/1066296788 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f170800f5d0 con 0x7f1714101ef0 2024-05-09T06:42:35.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.586+0000 7f17117fa700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f16fc06e210 0x7f16fc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:35.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.586+0000 7f17117fa700 1 -- 172.21.15.63:0/1066296788 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f170808f3e0 con 0x7f1714101ef0 2024-05-09T06:42:35.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.586+0000 7f1713fff700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f16fc06e210 0x7f16fc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:35.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.586+0000 7f1713fff700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f16fc06e210 0x7f16fc0706d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1714068f60 tx=0x7f170400a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:35.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.590+0000 7f17117fa700 1 -- 172.21.15.63:0/1066296788 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f170805c120 con 0x7f1714101ef0 2024-05-09T06:42:35.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:35 smithi193 ceph-mon[81665]: pgmap v94: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 4.0 MiB/s, 0 objects/s recovering 2024-05-09T06:42:35.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.817+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1714100d10 con 0x7f16fc06e210 2024-05-09T06:42:35.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.819+0000 7f17117fa700 1 -- 172.21.15.63:0/1066296788 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1714100d10 con 0x7f16fc06e210 2024-05-09T06:42:35.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f16fc06e210 msgr2=0x7f16fc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f16fc06e210 0x7f16fc0706d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1714068f60 tx=0x7f170400a380 comp rx=0 tx=0).stop 2024-05-09T06:42:35.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 msgr2=0x7f17140ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 0x7f17140ffea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1708006010 tx=0x7f170800bd70 comp rx=0 tx=0).stop 2024-05-09T06:42:35.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 shutdown_connections 2024-05-09T06:42:35.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f16fc06e210 0x7f16fc0706d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17140fef10 0x7f17141017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 --2- 172.21.15.63:0/1066296788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1714101ef0 0x7f17140ffea0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 >> 172.21.15.63:0/1066296788 conn(0x7f17140fa870 msgr2=0x7f17141054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:35.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 shutdown_connections 2024-05-09T06:42:35.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.821+0000 7f171a616700 1 -- 172.21.15.63:0/1066296788 wait complete. 2024-05-09T06:42:35.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.929+0000 7feb54ad7700 1 -- 172.21.15.63:0/3990612496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 msgr2=0x7feb501022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.929+0000 7feb54ad7700 1 --2- 172.21.15.63:0/3990612496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 0x7feb501022b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb44009910 tx=0x7feb44009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:35.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.929+0000 7feb54ad7700 1 -- 172.21.15.63:0/3990612496 shutdown_connections 2024-05-09T06:42:35.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.929+0000 7feb54ad7700 1 --2- 172.21.15.63:0/3990612496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 0x7feb501022b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.929+0000 7feb54ad7700 1 --2- 172.21.15.63:0/3990612496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50100c50 0x7feb50101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.929+0000 7feb54ad7700 1 -- 172.21.15.63:0/3990612496 >> 172.21.15.63:0/3990612496 conn(0x7feb500fc210 msgr2=0x7feb500fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:35.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.929+0000 7feb54ad7700 1 -- 172.21.15.63:0/3990612496 shutdown_connections 2024-05-09T06:42:35.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.929+0000 7feb54ad7700 1 -- 172.21.15.63:0/3990612496 wait complete. 2024-05-09T06:42:35.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.930+0000 7feb54ad7700 1 Processor -- start 2024-05-09T06:42:35.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.930+0000 7feb54ad7700 1 -- start start 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.930+0000 7feb54ad7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50100c50 0x7feb50074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.930+0000 7feb54ad7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 0x7feb500732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.930+0000 7feb54ad7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb50073820 con 0x7feb50101e50 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.930+0000 7feb54ad7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb50073960 con 0x7feb50100c50 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb4dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 0x7feb500732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb4dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 0x7feb500732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44406/0 (socket says 172.21.15.63:44406) 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb4dd9b700 1 -- 172.21.15.63:0/2892726044 learned_addr learned my addr 172.21.15.63:0/2892726044 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb4e59c700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50100c50 0x7feb50074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb4dd9b700 1 -- 172.21.15.63:0/2892726044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50100c50 msgr2=0x7feb50074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb4dd9b700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50100c50 0x7feb50074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb4dd9b700 1 -- 172.21.15.63:0/2892726044 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440095f0 con 0x7feb50101e50 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb4dd9b700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 0x7feb500732e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feb4400bd70 tx=0x7feb4400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb3f7fe700 1 -- 172.21.15.63:0/2892726044 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4401d070 con 0x7feb50101e50 2024-05-09T06:42:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb50073be0 con 0x7feb50101e50 2024-05-09T06:42:35.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb3f7fe700 1 -- 172.21.15.63:0/2892726044 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb44005dc0 con 0x7feb50101e50 2024-05-09T06:42:35.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.931+0000 7feb3f7fe700 1 -- 172.21.15.63:0/2892726044 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb44021410 con 0x7feb50101e50 2024-05-09T06:42:35.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.932+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb50074150 con 0x7feb50101e50 2024-05-09T06:42:35.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.932+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb501907d0 con 0x7feb50101e50 2024-05-09T06:42:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.933+0000 7feb3f7fe700 1 -- 172.21.15.63:0/2892726044 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feb4400f810 con 0x7feb50101e50 2024-05-09T06:42:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.934+0000 7feb3f7fe700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feb3806e210 0x7feb380706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.934+0000 7feb3f7fe700 1 -- 172.21.15.63:0/2892726044 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6790+0+0 (secure 0 0 0) 0x7feb4408f2a0 con 0x7feb50101e50 2024-05-09T06:42:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.934+0000 7feb4e59c700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feb3806e210 0x7feb380706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.934+0000 7feb4e59c700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feb3806e210 0x7feb380706d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feb50101cb0 tx=0x7feb4000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:35.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:35.938+0000 7feb3f7fe700 1 -- 172.21.15.63:0/2892726044 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feb4405bfe0 con 0x7feb50101e50 2024-05-09T06:42:36.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.155+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feb50074150 con 0x7feb3806e210 2024-05-09T06:42:36.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.165+0000 7feb3f7fe700 1 -- 172.21.15.63:0/2892726044 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7feb50074150 con 0x7feb3806e210 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (11m) 14s ago 12m 25.2M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (13m) 14s ago 13m 8238k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (12m) 75s ago 12m 7944k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (84s) 14s ago 13m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (78s) 75s ago 12m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (11m) 14s ago 12m 87.2M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (7m) 14s ago 7m 16.2M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (7m) 14s ago 7m 90.4M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (7m) 75s ago 7m 143M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (7m) 75s ago 7m 17.1M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (3m) 14s ago 14m 507M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:42:36.146 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (2m) 75s ago 11m 450M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (2m) 14s ago 14m 47.2M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (2m) 75s ago 11m 32.2M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (13m) 14s ago 13m 16.0M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (12m) 75s ago 12m 16.2M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (57s) 14s ago 11m 143M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (21s) 14s ago 10m 12.4M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (10m) 14s ago 10m 268M 1216M 18.2.2-1297-g1758e8ea 196422f1b641 e54de486ad42 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (9m) 75s ago 9m 292M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (9m) 75s ago 9m 260M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (8m) 75s ago 8m 233M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:42:36.147 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (3m) 14s ago 12m 58.3M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:42:36.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feb3806e210 msgr2=0x7feb380706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feb3806e210 0x7feb380706d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feb50101cb0 tx=0x7feb4000b410 comp rx=0 tx=0).stop 2024-05-09T06:42:36.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 msgr2=0x7feb500732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 0x7feb500732e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feb4400bd70 tx=0x7feb4400bda0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 shutdown_connections 2024-05-09T06:42:36.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7feb3806e210 0x7feb380706d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50100c50 0x7feb50074bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 --2- 172.21.15.63:0/2892726044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb50101e50 0x7feb500732e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 >> 172.21.15.63:0/2892726044 conn(0x7feb500fc210 msgr2=0x7feb501053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:36.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 shutdown_connections 2024-05-09T06:42:36.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.168+0000 7feb54ad7700 1 -- 172.21.15.63:0/2892726044 wait complete. 2024-05-09T06:42:36.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.280+0000 7f42124b5700 1 -- 172.21.15.63:0/4084900507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 msgr2=0x7f420c102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.280+0000 7f42124b5700 1 --2- 172.21.15.63:0/4084900507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 0x7f420c102680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009910 tx=0x7f41fc009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.280+0000 7f42124b5700 1 -- 172.21.15.63:0/4084900507 shutdown_connections 2024-05-09T06:42:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.280+0000 7f42124b5700 1 --2- 172.21.15.63:0/4084900507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f420c0fece0 0x7f420c0ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.280+0000 7f42124b5700 1 --2- 172.21.15.63:0/4084900507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 0x7f420c102680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.280+0000 7f42124b5700 1 -- 172.21.15.63:0/4084900507 >> 172.21.15.63:0/4084900507 conn(0x7f420c0fa860 msgr2=0x7f420c0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.280+0000 7f42124b5700 1 -- 172.21.15.63:0/4084900507 shutdown_connections 2024-05-09T06:42:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.280+0000 7f42124b5700 1 -- 172.21.15.63:0/4084900507 wait complete. 2024-05-09T06:42:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.281+0000 7f42124b5700 1 Processor -- start 2024-05-09T06:42:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.281+0000 7f42124b5700 1 -- start start 2024-05-09T06:42:36.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.281+0000 7f42124b5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f420c0fece0 0x7f420c1018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.281+0000 7f42124b5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 0x7f420c0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.281+0000 7f42124b5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c1004f0 con 0x7f420c102260 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.281+0000 7f42124b5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c100630 con 0x7f420c0fece0 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.281+0000 7f420b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 0x7f420c0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.281+0000 7f420b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 0x7f420c0fffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44424/0 (socket says 172.21.15.63:44424) 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.281+0000 7f420b7fe700 1 -- 172.21.15.63:0/986926839 learned_addr learned my addr 172.21.15.63:0/986926839 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f420bfff700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f420c0fece0 0x7f420c1018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f420b7fe700 1 -- 172.21.15.63:0/986926839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f420c0fece0 msgr2=0x7f420c1018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f420b7fe700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f420c0fece0 0x7f420c1018c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f420b7fe700 1 -- 172.21.15.63:0/986926839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc0095f0 con 0x7f420c102260 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f420b7fe700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 0x7f420c0fffb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f420000d900 tx=0x7f420000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f42097fa700 1 -- 172.21.15.63:0/986926839 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42000041f0 con 0x7f420c102260 2024-05-09T06:42:36.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f42097fa700 1 -- 172.21.15.63:0/986926839 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4200004cf0 con 0x7f420c102260 2024-05-09T06:42:36.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f42097fa700 1 -- 172.21.15.63:0/986926839 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420000b750 con 0x7f420c102260 2024-05-09T06:42:36.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420c100910 con 0x7f420c102260 2024-05-09T06:42:36.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.282+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420c19e6b0 con 0x7f420c102260 2024-05-09T06:42:36.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.284+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420c1907c0 con 0x7f420c102260 2024-05-09T06:42:36.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.284+0000 7f42097fa700 1 -- 172.21.15.63:0/986926839 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4200004350 con 0x7f420c102260 2024-05-09T06:42:36.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.285+0000 7f42097fa700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f41f406e210 0x7f41f40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.285+0000 7f42097fa700 1 -- 172.21.15.63:0/986926839 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f420008e320 con 0x7f420c102260 2024-05-09T06:42:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.285+0000 7f420bfff700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f41f406e210 0x7f41f40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.285+0000 7f420bfff700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f41f406e210 0x7f41f40706d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00b5c0 tx=0x7f41fc005160 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:36.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.289+0000 7f42097fa700 1 -- 172.21.15.63:0/986926839 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f420005b0e0 con 0x7f420c102260 2024-05-09T06:42:36.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.588+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f420c04ebb0 con 0x7f420c102260 2024-05-09T06:42:36.569 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:36 smithi063 ceph-mon[92765]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:36.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.589+0000 7f42097fa700 1 -- 172.21.15.63:0/986926839 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f4200016360 con 0x7f420c102260 2024-05-09T06:42:36.570 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:42:36.570 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:42:36.570 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:42:36.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:42:36.570 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:42:36.570 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:42:36.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:42:36.570 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4, 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 8, 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 6 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:42:36.571 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:42:36.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f41f406e210 msgr2=0x7f41f40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f41f406e210 0x7f41f40706d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00b5c0 tx=0x7f41fc005160 comp rx=0 tx=0).stop 2024-05-09T06:42:36.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 msgr2=0x7f420c0fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 0x7f420c0fffb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f420000d900 tx=0x7f420000dcc0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 shutdown_connections 2024-05-09T06:42:36.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f41f406e210 0x7f41f40706d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f420c0fece0 0x7f420c1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 --2- 172.21.15.63:0/986926839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c102260 0x7f420c0fffb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 >> 172.21.15.63:0/986926839 conn(0x7f420c0fa860 msgr2=0x7f420c105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:36.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 shutdown_connections 2024-05-09T06:42:36.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.591+0000 7f42124b5700 1 -- 172.21.15.63:0/986926839 wait complete. 2024-05-09T06:42:36.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:36 smithi193 ceph-mon[81665]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.700+0000 7f33b4741700 1 -- 172.21.15.63:0/298696656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 msgr2=0x7f33ac1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.700+0000 7f33b4741700 1 --2- 172.21.15.63:0/298696656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 0x7f33ac1022a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009910 tx=0x7f33a0009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.700+0000 7f33b4741700 1 -- 172.21.15.63:0/298696656 shutdown_connections 2024-05-09T06:42:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.700+0000 7f33b4741700 1 --2- 172.21.15.63:0/298696656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 0x7f33ac1022a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.700+0000 7f33b4741700 1 --2- 172.21.15.63:0/298696656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac100c40 0x7f33ac101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.700+0000 7f33b4741700 1 -- 172.21.15.63:0/298696656 >> 172.21.15.63:0/298696656 conn(0x7f33ac0fc1e0 msgr2=0x7f33ac0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.700+0000 7f33b4741700 1 -- 172.21.15.63:0/298696656 shutdown_connections 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.700+0000 7f33b4741700 1 -- 172.21.15.63:0/298696656 wait complete. 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b4741700 1 Processor -- start 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b4741700 1 -- start start 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b4741700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac100c40 0x7f33ac196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b4741700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 0x7f33ac196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b4741700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1971b0 con 0x7f33ac101e40 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b4741700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1972f0 con 0x7f33ac100c40 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b1cdc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 0x7f33ac196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b1cdc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 0x7f33ac196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44438/0 (socket says 172.21.15.63:44438) 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b1cdc700 1 -- 172.21.15.63:0/370633809 learned_addr learned my addr 172.21.15.63:0/370633809 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.701+0000 7f33b24dd700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac100c40 0x7f33ac196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.702+0000 7f33b1cdc700 1 -- 172.21.15.63:0/370633809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac100c40 msgr2=0x7f33ac196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.702+0000 7f33b1cdc700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac100c40 0x7f33ac196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.702+0000 7f33b1cdc700 1 -- 172.21.15.63:0/370633809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a00095f0 con 0x7f33ac101e40 2024-05-09T06:42:36.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.702+0000 7f33b1cdc700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 0x7f33ac196b90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33a00092e0 tx=0x7f33a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:36.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.702+0000 7f339f7fe700 1 -- 172.21.15.63:0/370633809 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33a001d070 con 0x7f33ac101e40 2024-05-09T06:42:36.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.702+0000 7f339f7fe700 1 -- 172.21.15.63:0/370633809 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33a000f460 con 0x7f33ac101e40 2024-05-09T06:42:36.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.702+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac19bca0 con 0x7f33ac101e40 2024-05-09T06:42:36.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.702+0000 7f339f7fe700 1 -- 172.21.15.63:0/370633809 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33a0021620 con 0x7f33ac101e40 2024-05-09T06:42:36.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.702+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac19c210 con 0x7f33ac101e40 2024-05-09T06:42:36.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.703+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac1907c0 con 0x7f33ac101e40 2024-05-09T06:42:36.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.704+0000 7f339f7fe700 1 -- 172.21.15.63:0/370633809 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f33a000f5d0 con 0x7f33ac101e40 2024-05-09T06:42:36.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.704+0000 7f339f7fe700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f339806e200 0x7f33980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:36.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.704+0000 7f339f7fe700 1 -- 172.21.15.63:0/370633809 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f33a008f420 con 0x7f33ac101e40 2024-05-09T06:42:36.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.704+0000 7f33b24dd700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f339806e200 0x7f33980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:36.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.705+0000 7f33b24dd700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f339806e200 0x7f33980706c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33ac101ca0 tx=0x7f33a800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:36.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.709+0000 7f339f7fe700 1 -- 172.21.15.63:0/370633809 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f33a005c1e0 con 0x7f33ac101e40 2024-05-09T06:42:36.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.960+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f33ac04ebb0 con 0x7f33ac101e40 2024-05-09T06:42:36.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.960+0000 7f339f7fe700 1 -- 172.21.15.63:0/370633809 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1905 (secure 0 0 0) 0x7f33a005bd70 con 0x7f33ac101e40 2024-05-09T06:42:36.940 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:42:36.940 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:42:36.940 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:42:36.941 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:42:36.942 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:42:36.943 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:42:36.944 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:42:36.944 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:42:36.944 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:42:36.944 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:42:36.944 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:42:36.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.962+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f339806e200 msgr2=0x7f33980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.962+0000 7f33b4741700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f339806e200 0x7f33980706c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33ac101ca0 tx=0x7f33a800b410 comp rx=0 tx=0).stop 2024-05-09T06:42:36.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.962+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 msgr2=0x7f33ac196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:36.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.962+0000 7f33b4741700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 0x7f33ac196b90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33a00092e0 tx=0x7f33a000bd70 comp rx=0 tx=0).stop 2024-05-09T06:42:36.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.962+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 shutdown_connections 2024-05-09T06:42:36.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.962+0000 7f33b4741700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f339806e200 0x7f33980706c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.962+0000 7f33b4741700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac100c40 0x7f33ac196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.962+0000 7f33b4741700 1 --2- 172.21.15.63:0/370633809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33ac101e40 0x7f33ac196b90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:36.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.962+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 >> 172.21.15.63:0/370633809 conn(0x7f33ac0fc1e0 msgr2=0x7f33ac1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:36.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.963+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 shutdown_connections 2024-05-09T06:42:36.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:36.963+0000 7f33b4741700 1 -- 172.21.15.63:0/370633809 wait complete. 2024-05-09T06:42:36.946 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:42:37.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.071+0000 7f2b7fb96700 1 -- 172.21.15.63:0/4287035540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 msgr2=0x7f2b78100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.071+0000 7f2b7fb96700 1 --2- 172.21.15.63:0/4287035540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 0x7f2b78100270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009910 tx=0x7f2b74009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.071+0000 7f2b7fb96700 1 -- 172.21.15.63:0/4287035540 shutdown_connections 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.071+0000 7f2b7fb96700 1 --2- 172.21.15.63:0/4287035540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 0x7f2b78100270 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.071+0000 7f2b7fb96700 1 --2- 172.21.15.63:0/4287035540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b78101140 0x7f2b78101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.071+0000 7f2b7fb96700 1 -- 172.21.15.63:0/4287035540 >> 172.21.15.63:0/4287035540 conn(0x7f2b780fa860 msgr2=0x7f2b780fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.071+0000 7f2b7fb96700 1 -- 172.21.15.63:0/4287035540 shutdown_connections 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.071+0000 7f2b7fb96700 1 -- 172.21.15.63:0/4287035540 wait complete. 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7fb96700 1 Processor -- start 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7fb96700 1 -- start start 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7fb96700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 0x7f2b78103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7fb96700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b78101140 0x7f2b781021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7fb96700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b78102720 con 0x7f2b780ffdf0 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7fb96700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b78102860 con 0x7f2b78101140 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7d932700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 0x7f2b78103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7d932700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 0x7f2b78103b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44456/0 (socket says 172.21.15.63:44456) 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7d932700 1 -- 172.21.15.63:0/2997875131 learned_addr learned my addr 172.21.15.63:0/2997875131 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.072+0000 7f2b7d131700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b78101140 0x7f2b781021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.073+0000 7f2b7d932700 1 -- 172.21.15.63:0/2997875131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b78101140 msgr2=0x7f2b781021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.073+0000 7f2b7d932700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b78101140 0x7f2b781021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.073+0000 7f2b7d932700 1 -- 172.21.15.63:0/2997875131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b740095f0 con 0x7f2b780ffdf0 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.073+0000 7f2b7d932700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 0x7f2b78103b40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e8f0 tx=0x7f2b6c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.073+0000 7f2b6affd700 1 -- 172.21.15.63:0/2997875131 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b6c009790 con 0x7f2b780ffdf0 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.073+0000 7f2b6affd700 1 -- 172.21.15.63:0/2997875131 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b6c00c810 con 0x7f2b780ffdf0 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.073+0000 7f2b6affd700 1 -- 172.21.15.63:0/2997875131 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b6c003d20 con 0x7f2b780ffdf0 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.073+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b78102b40 con 0x7f2b780ffdf0 2024-05-09T06:42:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.073+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7819e720 con 0x7f2b780ffdf0 2024-05-09T06:42:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.074+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7804ebb0 con 0x7f2b780ffdf0 2024-05-09T06:42:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.075+0000 7f2b6affd700 1 -- 172.21.15.63:0/2997875131 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2b6c0041f0 con 0x7f2b780ffdf0 2024-05-09T06:42:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.075+0000 7f2b6affd700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2b6406e200 0x7f2b640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.075+0000 7f2b7d131700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2b6406e200 0x7f2b640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.075+0000 7f2b6affd700 1 -- 172.21.15.63:0/2997875131 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f2b6c014070 con 0x7f2b780ffdf0 2024-05-09T06:42:37.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.076+0000 7f2b7d131700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2b6406e200 0x7f2b640706c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400b800 tx=0x7f2b74009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:37.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.080+0000 7f2b6affd700 1 -- 172.21.15.63:0/2997875131 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2b6c05b270 con 0x7f2b780ffdf0 2024-05-09T06:42:37.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.310+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b780ff480 con 0x7f2b6406e200 2024-05-09T06:42:37.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.311+0000 7f2b6affd700 1 -- 172.21.15.63:0/2997875131 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2b780ff480 con 0x7f2b6406e200 2024-05-09T06:42:37.291 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:42:37.291 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:42:37.291 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:42:37.291 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:42:37.291 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:42:37.291 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:42:37.292 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:42:37.292 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:42:37.292 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:42:37.292 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "8/23 daemons upgraded", 2024-05-09T06:42:37.292 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-05-09T06:42:37.292 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:42:37.292 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:42:37.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.313+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2b6406e200 msgr2=0x7f2b640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:37.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.313+0000 7f2b7fb96700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2b6406e200 0x7f2b640706c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400b800 tx=0x7f2b74009f70 comp rx=0 tx=0).stop 2024-05-09T06:42:37.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.313+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 msgr2=0x7f2b78103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:37.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.313+0000 7f2b7fb96700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 0x7f2b78103b40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e8f0 tx=0x7f2b6c00ec00 comp rx=0 tx=0).stop 2024-05-09T06:42:37.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.313+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 shutdown_connections 2024-05-09T06:42:37.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.314+0000 7f2b7fb96700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2b6406e200 0x7f2b640706c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.314+0000 7f2b7fb96700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b780ffdf0 0x7f2b78103b40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.314+0000 7f2b7fb96700 1 --2- 172.21.15.63:0/2997875131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b78101140 0x7f2b781021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.314+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 >> 172.21.15.63:0/2997875131 conn(0x7f2b780fa860 msgr2=0x7f2b780fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:37.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.314+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 shutdown_connections 2024-05-09T06:42:37.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.314+0000 7f2b7fb96700 1 -- 172.21.15.63:0/2997875131 wait complete. 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.422+0000 7fa3600c6700 1 -- 172.21.15.63:0/3656653903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 msgr2=0x7fa3580ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.422+0000 7fa3600c6700 1 --2- 172.21.15.63:0/3656653903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 0x7fa3580ff330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa354009910 tx=0x7fa354009c20 comp rx=0 tx=0).stop 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.422+0000 7fa3600c6700 1 -- 172.21.15.63:0/3656653903 shutdown_connections 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.422+0000 7fa3600c6700 1 --2- 172.21.15.63:0/3656653903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa358101ef0 0x7fa358102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.422+0000 7fa3600c6700 1 --2- 172.21.15.63:0/3656653903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 0x7fa3580ff330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.422+0000 7fa3600c6700 1 -- 172.21.15.63:0/3656653903 >> 172.21.15.63:0/3656653903 conn(0x7fa3580fa870 msgr2=0x7fa3580fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.422+0000 7fa3600c6700 1 -- 172.21.15.63:0/3656653903 shutdown_connections 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.422+0000 7fa3600c6700 1 -- 172.21.15.63:0/3656653903 wait complete. 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.423+0000 7fa3600c6700 1 Processor -- start 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.423+0000 7fa3600c6700 1 -- start start 2024-05-09T06:42:37.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.423+0000 7fa3600c6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 0x7fa358196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.423+0000 7fa3600c6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa358101ef0 0x7fa358196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.423+0000 7fa3600c6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581971d0 con 0x7fa3580fef10 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.423+0000 7fa3600c6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa358197310 con 0x7fa358101ef0 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.423+0000 7fa35de62700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 0x7fa358196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.423+0000 7fa35de62700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 0x7fa358196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44466/0 (socket says 172.21.15.63:44466) 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.423+0000 7fa35de62700 1 -- 172.21.15.63:0/2095917317 learned_addr learned my addr 172.21.15.63:0/2095917317 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa35d661700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa358101ef0 0x7fa358196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa35de62700 1 -- 172.21.15.63:0/2095917317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa358101ef0 msgr2=0x7fa358196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa35de62700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa358101ef0 0x7fa358196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa35de62700 1 -- 172.21.15.63:0/2095917317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3540095f0 con 0x7fa3580fef10 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa35de62700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 0x7fa358196670 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa35400bd70 tx=0x7fa35400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa34effd700 1 -- 172.21.15.63:0/2095917317 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35401d070 con 0x7fa3580fef10 2024-05-09T06:42:37.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa34effd700 1 -- 172.21.15.63:0/2095917317 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa35400f460 con 0x7fa3580fef10 2024-05-09T06:42:37.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35819bcc0 con 0x7fa3580fef10 2024-05-09T06:42:37.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa34effd700 1 -- 172.21.15.63:0/2095917317 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa354021620 con 0x7fa3580fef10 2024-05-09T06:42:37.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.424+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35819c1b0 con 0x7fa3580fef10 2024-05-09T06:42:37.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.425+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3581907d0 con 0x7fa3580fef10 2024-05-09T06:42:37.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.426+0000 7fa34effd700 1 -- 172.21.15.63:0/2095917317 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa35400fa70 con 0x7fa3580fef10 2024-05-09T06:42:37.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.426+0000 7fa34effd700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa34406e210 0x7fa3440706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:42:37.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.426+0000 7fa34effd700 1 -- 172.21.15.63:0/2095917317 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fa35408f510 con 0x7fa3580fef10 2024-05-09T06:42:37.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.426+0000 7fa35d661700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa34406e210 0x7fa3440706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:42:37.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.427+0000 7fa35d661700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa34406e210 0x7fa3440706d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa348005fd0 tx=0x7fa34800a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:42:37.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.431+0000 7fa34effd700 1 -- 172.21.15.63:0/2095917317 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa35405c2d0 con 0x7fa3580fef10 2024-05-09T06:42:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:37 smithi063 ceph-mon[92765]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:37 smithi063 ceph-mon[92765]: pgmap v95: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 op/s; 3.2 MiB/s, 0 objects/s recovering 2024-05-09T06:42:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:37 smithi063 ceph-mon[92765]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:37 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/986926839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:37 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/370633809' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:42:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:37 smithi193 ceph-mon[81665]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:37 smithi193 ceph-mon[81665]: pgmap v95: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 op/s; 3.2 MiB/s, 0 objects/s recovering 2024-05-09T06:42:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:37 smithi193 ceph-mon[81665]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:37 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/986926839' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:42:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:37 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/370633809' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:42:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:37.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.726+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa35804ebb0 con 0x7fa3580fef10 2024-05-09T06:42:37.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.727+0000 7fa34effd700 1 -- 172.21.15.63:0/2095917317 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa35405be60 con 0x7fa3580fef10 2024-05-09T06:42:37.707 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:42:37.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa34406e210 msgr2=0x7fa3440706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:37.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa34406e210 0x7fa3440706d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa348005fd0 tx=0x7fa34800a560 comp rx=0 tx=0).stop 2024-05-09T06:42:37.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 msgr2=0x7fa358196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:42:37.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 0x7fa358196670 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa35400bd70 tx=0x7fa35400be50 comp rx=0 tx=0).stop 2024-05-09T06:42:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 shutdown_connections 2024-05-09T06:42:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa34406e210 0x7fa3440706d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3580fef10 0x7fa358196670 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 --2- 172.21.15.63:0/2095917317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa358101ef0 0x7fa358196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:42:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 >> 172.21.15.63:0/2095917317 conn(0x7fa3580fa870 msgr2=0x7fa3581054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:42:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 shutdown_connections 2024-05-09T06:42:37.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:42:37.729+0000 7fa3600c6700 1 -- 172.21.15.63:0/2095917317 wait complete. 2024-05-09T06:42:38.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:38 smithi063 ceph-mon[92765]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:38.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:38 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2095917317' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:42:38.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:38 smithi193 ceph-mon[81665]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:42:38.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:38 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2095917317' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:42:39.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:39 smithi063 ceph-mon[92765]: pgmap v96: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 1.8 MiB/s, 2 objects/s recovering 2024-05-09T06:42:39.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:39 smithi193 ceph-mon[81665]: pgmap v96: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 1.8 MiB/s, 2 objects/s recovering 2024-05-09T06:42:41.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:41 smithi063 ceph-mon[92765]: pgmap v97: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:41.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:41 smithi193 ceph-mon[81665]: pgmap v97: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:42.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-09T06:42:42.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:42.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:42:42.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-09T06:42:42.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:42.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:42:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:43 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-09T06:42:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:43 smithi063 ceph-mon[92765]: Upgrade: osd.2 is safe to restart 2024-05-09T06:42:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:43 smithi063 ceph-mon[92765]: pgmap v98: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:42:43.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:43 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-09T06:42:43.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:43 smithi193 ceph-mon[81665]: Upgrade: osd.2 is safe to restart 2024-05-09T06:42:43.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:43 smithi193 ceph-mon[81665]: pgmap v98: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:42:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:44 smithi063 ceph-mon[92765]: Upgrade: Updating osd.2 2024-05-09T06:42:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:44 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:44 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T06:42:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:44 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:42:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:44 smithi063 ceph-mon[92765]: Deploying daemon osd.2 on smithi063 2024-05-09T06:42:45.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:44 smithi063 ceph-mon[92765]: pgmap v99: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:42:45.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:44 smithi193 ceph-mon[81665]: Upgrade: Updating osd.2 2024-05-09T06:42:45.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:44 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:45.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:44 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T06:42:45.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:44 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:42:45.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:44 smithi193 ceph-mon[81665]: Deploying daemon osd.2 on smithi063 2024-05-09T06:42:45.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:44 smithi193 ceph-mon[81665]: pgmap v99: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:42:47.416 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:47 smithi063 ceph-mon[92765]: pgmap v100: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:47.417 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:47 smithi063 systemd[1]: Stopping Ceph osd.2 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:42:47.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:47 smithi193 ceph-mon[81665]: pgmap v100: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:42:47.855 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:47 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[61576]: 2024-05-09T06:42:47.436+0000 7ff1c9564700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:42:47.855 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:47 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[61576]: 2024-05-09T06:42:47.436+0000 7ff1c9564700 -1 osd.2 70 *** Got signal Terminated *** 2024-05-09T06:42:47.855 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:47 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[61576]: 2024-05-09T06:42:47.436+0000 7ff1c9564700 -1 osd.2 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:42:48.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:48 smithi063 ceph-mon[92765]: osd.2 marked itself down and dead 2024-05-09T06:42:48.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:48 smithi193 ceph-mon[81665]: osd.2 marked itself down and dead 2024-05-09T06:42:48.910 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:48 smithi063 podman[112901]: 2024-05-09 06:42:48.675922008 +0000 UTC m=+1.353715403 container died e54de486ad4221fd50bbb5c60d9172953cf433803a04a0aa8dce8eb5a2a0d534 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2, RELEASE=reef-1758e8e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:42:49.163 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:49 smithi063 podman[112901]: 2024-05-09 06:42:49.020306707 +0000 UTC m=+1.698100089 container cleanup e54de486ad4221fd50bbb5c60d9172953cf433803a04a0aa8dce8eb5a2a0d534 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-1758e8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux ) 2024-05-09T06:42:49.164 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:49 smithi063 bash[112901]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2 2024-05-09T06:42:49.605 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:49 smithi063 podman[112920]: 2024-05-09 06:42:49.187876784 +0000 UTC m=+0.505092694 container remove e54de486ad4221fd50bbb5c60d9172953cf433803a04a0aa8dce8eb5a2a0d534 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=reef-1758e8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T06:42:49.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:49 smithi063 ceph-mon[92765]: pgmap v101: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:42:49.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:49 smithi063 ceph-mon[92765]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:42:49.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:49 smithi063 ceph-mon[92765]: osdmap e71: 6 total, 5 up, 6 in 2024-05-09T06:42:49.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:49 smithi193 ceph-mon[81665]: pgmap v101: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:42:49.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:49 smithi193 ceph-mon[81665]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:42:49.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:49 smithi193 ceph-mon[81665]: osdmap e71: 6 total, 5 up, 6 in 2024-05-09T06:42:49.882 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:49 smithi063 podman[113015]: 2024-05-09 06:42:49.554006316 +0000 UTC m=+0.018865808 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:42:49.882 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:49 smithi063 podman[113015]: 2024-05-09 06:42:49.655396738 +0000 UTC m=+0.120256208 container create d7178b474b7c71a51c9cd6d2dec8f9393182c06ea8fe8173dcb76ca96e6d5435 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T06:42:49.882 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:49 smithi063 podman[113015]: 2024-05-09 06:42:49.899688231 +0000 UTC m=+0.364547712 container init d7178b474b7c71a51c9cd6d2dec8f9393182c06ea8fe8173dcb76ca96e6d5435 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T06:42:49.882 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:49 smithi063 podman[113015]: 2024-05-09 06:42:49.903201698 +0000 UTC m=+0.368061177 container start d7178b474b7c71a51c9cd6d2dec8f9393182c06ea8fe8173dcb76ca96e6d5435 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-05-09T06:42:50.175 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:49 smithi063 podman[113015]: 2024-05-09 06:42:49.991395551 +0000 UTC m=+0.456255030 container attach d7178b474b7c71a51c9cd6d2dec8f9393182c06ea8fe8173dcb76ca96e6d5435 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-09T06:42:50.458 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:50 smithi063 ceph-mon[92765]: osdmap e72: 6 total, 5 up, 6 in 2024-05-09T06:42:50.458 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:50 smithi063 podman[113015]: 2024-05-09 06:42:50.203680275 +0000 UTC m=+0.668539754 container died d7178b474b7c71a51c9cd6d2dec8f9393182c06ea8fe8173dcb76ca96e6d5435 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=) 2024-05-09T06:42:50.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:50 smithi193 ceph-mon[81665]: osdmap e72: 6 total, 5 up, 6 in 2024-05-09T06:42:51.105 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:50 smithi063 podman[113041]: 2024-05-09 06:42:50.795281654 +0000 UTC m=+0.585172907 container remove d7178b474b7c71a51c9cd6d2dec8f9393182c06ea8fe8173dcb76ca96e6d5435 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-09T06:42:51.105 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:50 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.2.service: Deactivated successfully. 2024-05-09T06:42:51.105 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:50 smithi063 systemd[1]: Stopped Ceph osd.2 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:42:51.105 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:50 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.2.service: Consumed 1min 3.043s CPU time. 2024-05-09T06:42:51.105 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 systemd[1]: Starting Ceph osd.2 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:42:51.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:51 smithi193 ceph-mon[81665]: pgmap v104: 65 pgs: 4 active+undersized, 3 active+undersized+degraded, 6 stale+active+clean, 52 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 7/228 objects degraded (3.070%) 2024-05-09T06:42:51.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:51 smithi193 ceph-mon[81665]: Health check failed: Degraded data redundancy: 7/228 objects degraded (3.070%), 3 pgs degraded (PG_DEGRADED) 2024-05-09T06:42:51.477 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:51 smithi063 ceph-mon[92765]: pgmap v104: 65 pgs: 4 active+undersized, 3 active+undersized+degraded, 6 stale+active+clean, 52 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 7/228 objects degraded (3.070%) 2024-05-09T06:42:51.478 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:51 smithi063 ceph-mon[92765]: Health check failed: Degraded data redundancy: 7/228 objects degraded (3.070%), 3 pgs degraded (PG_DEGRADED) 2024-05-09T06:42:51.478 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 podman[113159]: 2024-05-09 06:42:51.166460367 +0000 UTC m=+0.019562995 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:42:51.478 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 podman[113159]: 2024-05-09 06:42:51.287982101 +0000 UTC m=+0.141084715 container create 49ad4a3faddab98fdc1e183413e03acbdaf8a6e52948d6fb4b431c63646a0f00 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T06:42:51.855 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 podman[113159]: 2024-05-09 06:42:51.498813418 +0000 UTC m=+0.351916043 container init 49ad4a3faddab98fdc1e183413e03acbdaf8a6e52948d6fb4b431c63646a0f00 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux ) 2024-05-09T06:42:51.855 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 podman[113159]: 2024-05-09 06:42:51.504066502 +0000 UTC m=+0.357169124 container start 49ad4a3faddab98fdc1e183413e03acbdaf8a6e52948d6fb4b431c63646a0f00 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T06:42:51.855 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 podman[113159]: 2024-05-09 06:42:51.632182831 +0000 UTC m=+0.485285447 container attach 49ad4a3faddab98fdc1e183413e03acbdaf8a6e52948d6fb4b431c63646a0f00 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T06:42:52.282 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate[113174]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T06:42:52.282 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate[113174]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-09T06:42:52.282 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 bash[113159]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T06:42:52.282 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 bash[113159]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-09T06:42:52.282 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 bash[113159]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 bash[113159]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 bash[113159]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 bash[113159]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 bash[113159]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate[113174]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate[113174]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate[113174]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate[113174]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate[113174]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-09T06:42:52.283 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:51 smithi063 podman[113159]: 2024-05-09 06:42:51.996372972 +0000 UTC m=+0.849475597 container died 49ad4a3faddab98fdc1e183413e03acbdaf8a6e52948d6fb4b431c63646a0f00 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T06:42:52.866 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:52 smithi063 podman[113299]: 2024-05-09 06:42:52.632561217 +0000 UTC m=+0.626622725 container remove 49ad4a3faddab98fdc1e183413e03acbdaf8a6e52948d6fb4b431c63646a0f00 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T06:42:52.867 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:52 smithi063 podman[113363]: 2024-05-09 06:42:52.756419674 +0000 UTC m=+0.018831477 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:42:53.179 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:52 smithi063 podman[113363]: 2024-05-09 06:42:52.887325688 +0000 UTC m=+0.149737521 container create 313cdd3a98673ebd1a2d175ff13db803878435517149edf48b8c5cfd3032aca4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:42:53.179 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:53 smithi063 podman[113363]: 2024-05-09 06:42:53.106661432 +0000 UTC m=+0.369073232 container init 313cdd3a98673ebd1a2d175ff13db803878435517149edf48b8c5cfd3032aca4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:42:53.179 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:53 smithi063 podman[113363]: 2024-05-09 06:42:53.110214072 +0000 UTC m=+0.372625864 container start 313cdd3a98673ebd1a2d175ff13db803878435517149edf48b8c5cfd3032aca4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:42:53.179 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:53 smithi063 ceph-osd[113381]: -- 172.21.15.63:0/2860104230 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564caa31b680 con 0x564caa2e4c00 2024-05-09T06:42:53.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:53 smithi063 ceph-mon[92765]: pgmap v105: 65 pgs: 9 active+undersized, 9 active+undersized+degraded, 2 stale+active+clean, 45 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 21/228 objects degraded (9.211%) 2024-05-09T06:42:53.605 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:53 smithi063 bash[113363]: 313cdd3a98673ebd1a2d175ff13db803878435517149edf48b8c5cfd3032aca4 2024-05-09T06:42:53.605 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:53 smithi063 systemd[1]: Started Ceph osd.2 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:42:53.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:53 smithi193 ceph-mon[81665]: pgmap v105: 65 pgs: 9 active+undersized, 9 active+undersized+degraded, 2 stale+active+clean, 45 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 21/228 objects degraded (9.211%) 2024-05-09T06:42:54.090 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:53 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[113377]: 2024-05-09T06:42:53.649+0000 7f919c2997c0 -1 Falling back to public interface 2024-05-09T06:42:54.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:54.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:54.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:42:54.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:54.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:54.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:42:55.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:55 smithi063 ceph-mon[92765]: pgmap v106: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 31/228 objects degraded (13.596%) 2024-05-09T06:42:55.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:55 smithi193 ceph-mon[81665]: pgmap v106: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 31/228 objects degraded (13.596%) 2024-05-09T06:42:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:57 smithi063 ceph-mon[92765]: pgmap v107: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31/228 objects degraded (13.596%) 2024-05-09T06:42:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:42:57.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:57 smithi193 ceph-mon[81665]: pgmap v107: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31/228 objects degraded (13.596%) 2024-05-09T06:42:57.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:57.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:42:58.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:58 smithi063 ceph-mon[92765]: Health check update: Degraded data redundancy: 31/228 objects degraded (13.596%), 14 pgs degraded (PG_DEGRADED) 2024-05-09T06:42:58.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:58 smithi193 ceph-mon[81665]: Health check update: Degraded data redundancy: 31/228 objects degraded (13.596%), 14 pgs degraded (PG_DEGRADED) 2024-05-09T06:42:59.105 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:58 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[113377]: 2024-05-09T06:42:58.643+0000 7f919c2997c0 -1 osd.2 70 log_to_monitors true 2024-05-09T06:42:59.605 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:42:59 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[113377]: 2024-05-09T06:42:59.306+0000 7f918d6f0700 -1 osd.2 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:42:59.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:59 smithi063 ceph-mon[92765]: pgmap v108: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 31/228 objects degraded (13.596%) 2024-05-09T06:42:59.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:59 smithi063 ceph-mon[92765]: from='osd.2 [v2:172.21.15.63:6818/616478,v1:172.21.15.63:6819/616478]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T06:42:59.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:59.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:42:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:59.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:59 smithi193 ceph-mon[81665]: pgmap v108: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 31/228 objects degraded (13.596%) 2024-05-09T06:42:59.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:59 smithi193 ceph-mon[81665]: from='osd.2 [v2:172.21.15.63:6818/616478,v1:172.21.15.63:6819/616478]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T06:42:59.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:42:59.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:42:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:00.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:00 smithi063 ceph-mon[92765]: from='osd.2 [v2:172.21.15.63:6818/616478,v1:172.21.15.63:6819/616478]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T06:43:00.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:00 smithi063 ceph-mon[92765]: osdmap e73: 6 total, 5 up, 6 in 2024-05-09T06:43:00.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:00 smithi063 ceph-mon[92765]: from='osd.2 [v2:172.21.15.63:6818/616478,v1:172.21.15.63:6819/616478]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:43:00.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:00 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:00.553 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:00 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:00 smithi193 ceph-mon[81665]: from='osd.2 [v2:172.21.15.63:6818/616478,v1:172.21.15.63:6819/616478]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T06:43:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:00 smithi193 ceph-mon[81665]: osdmap e73: 6 total, 5 up, 6 in 2024-05-09T06:43:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:00 smithi193 ceph-mon[81665]: from='osd.2 [v2:172.21.15.63:6818/616478,v1:172.21.15.63:6819/616478]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-09T06:43:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:00 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:00.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:00 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:01.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:01 smithi063 ceph-mon[92765]: pgmap v110: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 819 B/s rd, 1 op/s; 31/228 objects degraded (13.596%) 2024-05-09T06:43:01.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:01 smithi063 ceph-mon[92765]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:43:01.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:01 smithi063 ceph-mon[92765]: osd.2 [v2:172.21.15.63:6818/616478,v1:172.21.15.63:6819/616478] boot 2024-05-09T06:43:01.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:01 smithi063 ceph-mon[92765]: osdmap e74: 6 total, 6 up, 6 in 2024-05-09T06:43:01.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:01 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:43:01.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:01 smithi063 ceph-mon[92765]: osdmap e75: 6 total, 6 up, 6 in 2024-05-09T06:43:01.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:01 smithi193 ceph-mon[81665]: pgmap v110: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 819 B/s rd, 1 op/s; 31/228 objects degraded (13.596%) 2024-05-09T06:43:01.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:01 smithi193 ceph-mon[81665]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:43:01.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:01 smithi193 ceph-mon[81665]: osd.2 [v2:172.21.15.63:6818/616478,v1:172.21.15.63:6819/616478] boot 2024-05-09T06:43:01.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:01 smithi193 ceph-mon[81665]: osdmap e74: 6 total, 6 up, 6 in 2024-05-09T06:43:01.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:01 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T06:43:01.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:01 smithi193 ceph-mon[81665]: osdmap e75: 6 total, 6 up, 6 in 2024-05-09T06:43:03.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:03 smithi063 ceph-mon[92765]: pgmap v113: 65 pgs: 7 peering, 13 active+undersized, 10 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 25/228 objects degraded (10.965%) 2024-05-09T06:43:03.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:03 smithi063 ceph-mon[92765]: Health check update: Degraded data redundancy: 25/228 objects degraded (10.965%), 10 pgs degraded (PG_DEGRADED) 2024-05-09T06:43:03.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:03 smithi193 ceph-mon[81665]: pgmap v113: 65 pgs: 7 peering, 13 active+undersized, 10 active+undersized+degraded, 35 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 25/228 objects degraded (10.965%) 2024-05-09T06:43:03.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:03 smithi193 ceph-mon[81665]: Health check update: Degraded data redundancy: 25/228 objects degraded (10.965%), 10 pgs degraded (PG_DEGRADED) 2024-05-09T06:43:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:43:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:43:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:43:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:04.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-09T06:43:04.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-09T06:43:04.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:04 smithi063 ceph-mon[92765]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-05-09T06:43:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:43:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:43:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:43:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:04.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-09T06:43:04.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-09T06:43:04.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:04 smithi193 ceph-mon[81665]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-05-09T06:43:05.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:05 smithi063 ceph-mon[92765]: pgmap v114: 65 pgs: 7 peering, 7 active+undersized, 6 active+undersized+degraded, 45 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 16/228 objects degraded (7.018%) 2024-05-09T06:43:05.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:05 smithi193 ceph-mon[81665]: pgmap v114: 65 pgs: 7 peering, 7 active+undersized, 6 active+undersized+degraded, 45 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 16/228 objects degraded (7.018%) 2024-05-09T06:43:06.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:06 smithi063 ceph-mon[92765]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/228 objects degraded (7.018%), 6 pgs degraded) 2024-05-09T06:43:06.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:06 smithi063 ceph-mon[92765]: Cluster is now healthy 2024-05-09T06:43:06.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:06 smithi193 ceph-mon[81665]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/228 objects degraded (7.018%), 6 pgs degraded) 2024-05-09T06:43:06.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:06 smithi193 ceph-mon[81665]: Cluster is now healthy 2024-05-09T06:43:07.829 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:07 smithi063 ceph-mon[92765]: pgmap v115: 65 pgs: 7 peering, 58 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 747 B/s rd, 1 op/s 2024-05-09T06:43:07.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.849+0000 7f77b4928700 1 -- 172.21.15.63:0/1471009708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b0101ef0 msgr2=0x7f77b0102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:07.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.849+0000 7f77b4928700 1 --2- 172.21.15.63:0/1471009708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b0101ef0 0x7f77b0102350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7798009910 tx=0x7f7798009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:07.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.849+0000 7f77b4928700 1 -- 172.21.15.63:0/1471009708 shutdown_connections 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.849+0000 7f77b4928700 1 --2- 172.21.15.63:0/1471009708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b0101ef0 0x7f77b0102350 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.849+0000 7f77b4928700 1 --2- 172.21.15.63:0/1471009708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b00fef10 0x7f77b00ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.849+0000 7f77b4928700 1 -- 172.21.15.63:0/1471009708 >> 172.21.15.63:0/1471009708 conn(0x7f77b00fa870 msgr2=0x7f77b00fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.849+0000 7f77b4928700 1 -- 172.21.15.63:0/1471009708 shutdown_connections 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.849+0000 7f77b4928700 1 -- 172.21.15.63:0/1471009708 wait complete. 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77b4928700 1 Processor -- start 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77b4928700 1 -- start start 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77b4928700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b00fef10 0x7f77b01017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77b4928700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0101ef0 0x7f77b00ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77b4928700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b01003e0 con 0x7f77b00fef10 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77b4928700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b0100520 con 0x7f77b0101ef0 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77ae59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b00fef10 0x7f77b01017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:07.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77ae59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b00fef10 0x7f77b01017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50362/0 (socket says 172.21.15.63:50362) 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77ae59c700 1 -- 172.21.15.63:0/4034907892 learned_addr learned my addr 172.21.15.63:0/4034907892 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.850+0000 7f77a7fff700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0101ef0 0x7f77b00ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77ae59c700 1 -- 172.21.15.63:0/4034907892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0101ef0 msgr2=0x7f77b00ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77ae59c700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0101ef0 0x7f77b00ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77ae59c700 1 -- 172.21.15.63:0/4034907892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980095f0 con 0x7f77b00fef10 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77a7fff700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0101ef0 0x7f77b00ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77ae59c700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b00fef10 0x7f77b01017b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77a000d900 tx=0x7f77a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77a77fe700 1 -- 172.21.15.63:0/4034907892 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a00041f0 con 0x7f77b00fef10 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b0100800 con 0x7f77b00fef10 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77a77fe700 1 -- 172.21.15.63:0/4034907892 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77a0004350 con 0x7f77b00fef10 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77a77fe700 1 -- 172.21.15.63:0/4034907892 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a0003dc0 con 0x7f77b00fef10 2024-05-09T06:43:07.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.851+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b019e6d0 con 0x7f77b00fef10 2024-05-09T06:43:07.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.852+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b01907d0 con 0x7f77b00fef10 2024-05-09T06:43:07.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.853+0000 7f77a77fe700 1 -- 172.21.15.63:0/4034907892 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f77a0009730 con 0x7f77b00fef10 2024-05-09T06:43:07.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.853+0000 7f77a77fe700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f779c06e200 0x7f779c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:07.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.853+0000 7f77a77fe700 1 -- 172.21.15.63:0/4034907892 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f77a008e430 con 0x7f77b00fef10 2024-05-09T06:43:07.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.853+0000 7f77a7fff700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f779c06e200 0x7f779c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:07.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.854+0000 7f77a7fff700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f779c06e200 0x7f779c0706c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f77980051c0 tx=0x7f7798005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:07.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:07.858+0000 7f77a77fe700 1 -- 172.21.15.63:0/4034907892 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f77a005b170 con 0x7f77b00fef10 2024-05-09T06:43:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:07 smithi193 ceph-mon[81665]: pgmap v115: 65 pgs: 7 peering, 58 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 747 B/s rd, 1 op/s 2024-05-09T06:43:08.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.085+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77b002cdb0 con 0x7f779c06e200 2024-05-09T06:43:08.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.087+0000 7f77a77fe700 1 -- 172.21.15.63:0/4034907892 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f77b002cdb0 con 0x7f779c06e200 2024-05-09T06:43:08.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f779c06e200 msgr2=0x7f779c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f779c06e200 0x7f779c0706c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f77980051c0 tx=0x7f7798005a90 comp rx=0 tx=0).stop 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b00fef10 msgr2=0x7f77b01017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b00fef10 0x7f77b01017b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77a000d900 tx=0x7f77a000dcc0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 shutdown_connections 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f779c06e200 0x7f779c0706c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77b00fef10 0x7f77b01017b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 --2- 172.21.15.63:0/4034907892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77b0101ef0 0x7f77b00ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 >> 172.21.15.63:0/4034907892 conn(0x7f77b00fa870 msgr2=0x7f77b01054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 shutdown_connections 2024-05-09T06:43:08.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.089+0000 7f77b4928700 1 -- 172.21.15.63:0/4034907892 wait complete. 2024-05-09T06:43:08.079 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:43:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.199+0000 7f036ff49700 1 -- 172.21.15.63:0/1269144515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 msgr2=0x7f0368101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.199+0000 7f036ff49700 1 --2- 172.21.15.63:0/1269144515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 0x7f0368101070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0364009910 tx=0x7f0364009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.199+0000 7f036ff49700 1 -- 172.21.15.63:0/1269144515 shutdown_connections 2024-05-09T06:43:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.199+0000 7f036ff49700 1 --2- 172.21.15.63:0/1269144515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0368101e50 0x7f03681022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.199+0000 7f036ff49700 1 --2- 172.21.15.63:0/1269144515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 0x7f0368101070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.199+0000 7f036ff49700 1 -- 172.21.15.63:0/1269144515 >> 172.21.15.63:0/1269144515 conn(0x7f03680fc210 msgr2=0x7f03680fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:08.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.199+0000 7f036ff49700 1 -- 172.21.15.63:0/1269144515 shutdown_connections 2024-05-09T06:43:08.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.199+0000 7f036ff49700 1 -- 172.21.15.63:0/1269144515 wait complete. 2024-05-09T06:43:08.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.200+0000 7f036ff49700 1 Processor -- start 2024-05-09T06:43:08.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.200+0000 7f036ff49700 1 -- start start 2024-05-09T06:43:08.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.200+0000 7f036ff49700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 0x7f0368196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:08.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.200+0000 7f036ff49700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0368101e50 0x7f0368196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:08.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.200+0000 7f036ff49700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0368197140 con 0x7f0368100c50 2024-05-09T06:43:08.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.200+0000 7f036ff49700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0368197280 con 0x7f0368101e50 2024-05-09T06:43:08.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.200+0000 7f036dce5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 0x7f0368196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.200+0000 7f036dce5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 0x7f0368196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50376/0 (socket says 172.21.15.63:50376) 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.200+0000 7f036dce5700 1 -- 172.21.15.63:0/1267358246 learned_addr learned my addr 172.21.15.63:0/1267358246 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.201+0000 7f036dce5700 1 -- 172.21.15.63:0/1267358246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0368101e50 msgr2=0x7f0368196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.201+0000 7f036dce5700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0368101e50 0x7f0368196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.201+0000 7f036dce5700 1 -- 172.21.15.63:0/1267358246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03640095f0 con 0x7f0368100c50 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.201+0000 7f036dce5700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 0x7f0368196670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f03640059d0 tx=0x7f036400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.201+0000 7f035effd700 1 -- 172.21.15.63:0/1267358246 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f036401d070 con 0x7f0368100c50 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.201+0000 7f035effd700 1 -- 172.21.15.63:0/1267358246 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f036400f460 con 0x7f0368100c50 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.201+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036819bc40 con 0x7f0368100c50 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.201+0000 7f035effd700 1 -- 172.21.15.63:0/1267358246 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0364021640 con 0x7f0368100c50 2024-05-09T06:43:08.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.201+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036819c180 con 0x7f0368100c50 2024-05-09T06:43:08.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.202+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03681907d0 con 0x7f0368100c50 2024-05-09T06:43:08.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.203+0000 7f035effd700 1 -- 172.21.15.63:0/1267358246 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f036400f5d0 con 0x7f0368100c50 2024-05-09T06:43:08.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.203+0000 7f035effd700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f035406e200 0x7f03540706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:08.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.203+0000 7f035effd700 1 -- 172.21.15.63:0/1267358246 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f036408f630 con 0x7f0368100c50 2024-05-09T06:43:08.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.203+0000 7f036d4e4700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f035406e200 0x7f03540706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:08.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.204+0000 7f036d4e4700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f035406e200 0x7f03540706c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0358005fd0 tx=0x7f0358005d00 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:08.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.208+0000 7f035effd700 1 -- 172.21.15.63:0/1267358246 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f036405c370 con 0x7f0368100c50 2024-05-09T06:43:08.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.435+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f036802ce50 con 0x7f035406e200 2024-05-09T06:43:08.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.437+0000 7f035effd700 1 -- 172.21.15.63:0/1267358246 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f036802ce50 con 0x7f035406e200 2024-05-09T06:43:08.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f035406e200 msgr2=0x7f03540706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f035406e200 0x7f03540706c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0358005fd0 tx=0x7f0358005d00 comp rx=0 tx=0).stop 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 msgr2=0x7f0368196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 0x7f0368196670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f03640059d0 tx=0x7f036400be20 comp rx=0 tx=0).stop 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 shutdown_connections 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f035406e200 0x7f03540706c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0368100c50 0x7f0368196670 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 --2- 172.21.15.63:0/1267358246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0368101e50 0x7f0368196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 >> 172.21.15.63:0/1267358246 conn(0x7f03680fc210 msgr2=0x7f0368105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 shutdown_connections 2024-05-09T06:43:08.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.439+0000 7f036ff49700 1 -- 172.21.15.63:0/1267358246 wait complete. 2024-05-09T06:43:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.547+0000 7f37f02b0700 1 -- 172.21.15.63:0/1434302835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 msgr2=0x7f37e8101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.547+0000 7f37f02b0700 1 --2- 172.21.15.63:0/1434302835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 0x7f37e8101070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009910 tx=0x7f37e4009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.547+0000 7f37f02b0700 1 -- 172.21.15.63:0/1434302835 shutdown_connections 2024-05-09T06:43:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.547+0000 7f37f02b0700 1 --2- 172.21.15.63:0/1434302835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e8101e50 0x7f37e81022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.547+0000 7f37f02b0700 1 --2- 172.21.15.63:0/1434302835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 0x7f37e8101070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.547+0000 7f37f02b0700 1 -- 172.21.15.63:0/1434302835 >> 172.21.15.63:0/1434302835 conn(0x7f37e80fc210 msgr2=0x7f37e80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.547+0000 7f37f02b0700 1 -- 172.21.15.63:0/1434302835 shutdown_connections 2024-05-09T06:43:08.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.547+0000 7f37f02b0700 1 -- 172.21.15.63:0/1434302835 wait complete. 2024-05-09T06:43:08.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37f02b0700 1 Processor -- start 2024-05-09T06:43:08.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37f02b0700 1 -- start start 2024-05-09T06:43:08.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37f02b0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 0x7f37e8196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:08.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37f02b0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e8101e50 0x7f37e8196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:08.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37f02b0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e81971c0 con 0x7f37e8100c50 2024-05-09T06:43:08.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37f02b0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e8197300 con 0x7f37e8101e50 2024-05-09T06:43:08.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37ee04c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 0x7f37e8196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:08.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37ee04c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 0x7f37e8196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50380/0 (socket says 172.21.15.63:50380) 2024-05-09T06:43:08.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37ee04c700 1 -- 172.21.15.63:0/2902444830 learned_addr learned my addr 172.21.15.63:0/2902444830 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37ed84b700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e8101e50 0x7f37e8196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37ee04c700 1 -- 172.21.15.63:0/2902444830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e8101e50 msgr2=0x7f37e8196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37ee04c700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e8101e50 0x7f37e8196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37ee04c700 1 -- 172.21.15.63:0/2902444830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e40095f0 con 0x7f37e8100c50 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.548+0000 7f37ee04c700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 0x7f37e8196660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37e40059d0 tx=0x7f37e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.549+0000 7f37df7fe700 1 -- 172.21.15.63:0/2902444830 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e401d070 con 0x7f37e8100c50 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.549+0000 7f37df7fe700 1 -- 172.21.15.63:0/2902444830 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37e400f460 con 0x7f37e8100c50 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.549+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e819bcb0 con 0x7f37e8100c50 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.549+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e819c1a0 con 0x7f37e8100c50 2024-05-09T06:43:08.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.549+0000 7f37df7fe700 1 -- 172.21.15.63:0/2902444830 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e4021640 con 0x7f37e8100c50 2024-05-09T06:43:08.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.550+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc005320 con 0x7f37e8100c50 2024-05-09T06:43:08.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.552+0000 7f37df7fe700 1 -- 172.21.15.63:0/2902444830 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f37e4003a60 con 0x7f37e8100c50 2024-05-09T06:43:08.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.553+0000 7f37df7fe700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f37d406e200 0x7f37d40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:08.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.553+0000 7f37df7fe700 1 -- 172.21.15.63:0/2902444830 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f37e408f8e0 con 0x7f37e8100c50 2024-05-09T06:43:08.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.553+0000 7f37ed84b700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f37d406e200 0x7f37d40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:08.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.553+0000 7f37ed84b700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f37d406e200 0x7f37d40706c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f37d8005950 tx=0x7f37d80058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:08.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.557+0000 7f37df7fe700 1 -- 172.21.15.63:0/2902444830 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f37e405c7c0 con 0x7f37e8100c50 2024-05-09T06:43:08.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.774+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f37cc000bf0 con 0x7f37d406e200 2024-05-09T06:43:08.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.785+0000 7f37df7fe700 1 -- 172.21.15.63:0/2902444830 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f37cc000bf0 con 0x7f37d406e200 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (12m) 10s ago 13m 25.4M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (13m) 10s ago 13m 8472k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (12m) 107s ago 12m 7944k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (116s) 10s ago 13m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (111s) 107s ago 12m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (11m) 10s ago 13m 85.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (7m) 10s ago 7m 16.5M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (7m) 10s ago 7m 90.4M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:43:08.767 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (7m) 107s ago 7m 143M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (7m) 107s ago 7m 17.1M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (4m) 10s ago 14m 508M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (3m) 107s ago 12m 450M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (3m) 10s ago 15m 48.4M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (2m) 107s ago 12m 32.2M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (13m) 10s ago 13m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (12m) 107s ago 12m 16.2M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (90s) 10s ago 11m 144M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (53s) 10s ago 11m 135M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (16s) 10s ago 10m 11.6M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (10m) 107s ago 10m 292M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 738a82f116dd 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (9m) 107s ago 9m 260M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (9m) 107s ago 9m 233M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:43:08.768 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (3m) 10s ago 13m 58.4M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:43:08.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f37d406e200 msgr2=0x7f37d40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f37d406e200 0x7f37d40706c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f37d8005950 tx=0x7f37d80058e0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 msgr2=0x7f37e8196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 0x7f37e8196660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37e40059d0 tx=0x7f37e400be20 comp rx=0 tx=0).stop 2024-05-09T06:43:08.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 shutdown_connections 2024-05-09T06:43:08.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f37d406e200 0x7f37d40706c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37e8100c50 0x7f37e8196660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 --2- 172.21.15.63:0/2902444830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e8101e50 0x7f37e8196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 >> 172.21.15.63:0/2902444830 conn(0x7f37e80fc210 msgr2=0x7f37e8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:08.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 shutdown_connections 2024-05-09T06:43:08.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.787+0000 7f37f02b0700 1 -- 172.21.15.63:0/2902444830 wait complete. 2024-05-09T06:43:08.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.894+0000 7fe344eb0700 1 -- 172.21.15.63:0/2818770775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe340104010 msgr2=0x7fe340106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.894+0000 7fe344eb0700 1 --2- 172.21.15.63:0/2818770775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe340104010 0x7fe340106400 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe334009910 tx=0x7fe334009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:08.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.894+0000 7fe344eb0700 1 -- 172.21.15.63:0/2818770775 shutdown_connections 2024-05-09T06:43:08.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.894+0000 7fe344eb0700 1 --2- 172.21.15.63:0/2818770775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe340104010 0x7fe340106400 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.894+0000 7fe344eb0700 1 --2- 172.21.15.63:0/2818770775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3401016e0 0x7fe340103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.894+0000 7fe344eb0700 1 -- 172.21.15.63:0/2818770775 >> 172.21.15.63:0/2818770775 conn(0x7fe3400fb0f0 msgr2=0x7fe3400fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:08.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.894+0000 7fe344eb0700 1 -- 172.21.15.63:0/2818770775 shutdown_connections 2024-05-09T06:43:08.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.894+0000 7fe344eb0700 1 -- 172.21.15.63:0/2818770775 wait complete. 2024-05-09T06:43:08.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe344eb0700 1 Processor -- start 2024-05-09T06:43:08.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe344eb0700 1 -- start start 2024-05-09T06:43:08.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe344eb0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3401016e0 0x7fe340196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:08.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe344eb0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe340104010 0x7fe340196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:08.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe344eb0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe340197180 con 0x7fe3401016e0 2024-05-09T06:43:08.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe344eb0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401972c0 con 0x7fe340104010 2024-05-09T06:43:08.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe33e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3401016e0 0x7fe340196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:08.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe33e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3401016e0 0x7fe340196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50398/0 (socket says 172.21.15.63:50398) 2024-05-09T06:43:08.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe33e59c700 1 -- 172.21.15.63:0/2323073283 learned_addr learned my addr 172.21.15.63:0/2323073283 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:08.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe33dd9b700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe340104010 0x7fe340196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:08.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe33e59c700 1 -- 172.21.15.63:0/2323073283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe340104010 msgr2=0x7fe340196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:08.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe33e59c700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe340104010 0x7fe340196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:08.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe33e59c700 1 -- 172.21.15.63:0/2323073283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3340095f0 con 0x7fe3401016e0 2024-05-09T06:43:08.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.895+0000 7fe33dd9b700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe340104010 0x7fe340196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:43:08.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.896+0000 7fe33e59c700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3401016e0 0x7fe340196620 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe33000e920 tx=0x7fe33000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:08.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.896+0000 7fe32f7fe700 1 -- 172.21.15.63:0/2323073283 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe330009790 con 0x7fe3401016e0 2024-05-09T06:43:08.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.896+0000 7fe32f7fe700 1 -- 172.21.15.63:0/2323073283 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe33000cba0 con 0x7fe3401016e0 2024-05-09T06:43:08.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.896+0000 7fe32f7fe700 1 -- 172.21.15.63:0/2323073283 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe330018650 con 0x7fe3401016e0 2024-05-09T06:43:08.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.896+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34019bd20 con 0x7fe3401016e0 2024-05-09T06:43:08.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.896+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34019c290 con 0x7fe3401016e0 2024-05-09T06:43:08.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.897+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34004ec20 con 0x7fe3401016e0 2024-05-09T06:43:08.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.897+0000 7fe32f7fe700 1 -- 172.21.15.63:0/2323073283 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe33000cd10 con 0x7fe3401016e0 2024-05-09T06:43:08.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.898+0000 7fe32f7fe700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe32806e210 0x7fe3280706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:08.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.898+0000 7fe32f7fe700 1 -- 172.21.15.63:0/2323073283 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fe330014070 con 0x7fe3401016e0 2024-05-09T06:43:08.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.898+0000 7fe33dd9b700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe32806e210 0x7fe3280706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:08.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.898+0000 7fe33dd9b700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe32806e210 0x7fe3280706d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe334005f50 tx=0x7fe334005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:08.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:08.902+0000 7fe32f7fe700 1 -- 172.21.15.63:0/2323073283 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe330053740 con 0x7fe3401016e0 2024-05-09T06:43:09.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.196+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe3401097b0 con 0x7fe3401016e0 2024-05-09T06:43:09.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.196+0000 7fe32f7fe700 1 -- 172.21.15.63:0/2323073283 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7fe34004ec20 con 0x7fe3401016e0 2024-05-09T06:43:09.176 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:43:09.176 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:43:09.176 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:43:09.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 3, 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 3 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 7, 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 7 2024-05-09T06:43:09.177 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:43:09.178 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:43:09.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe32806e210 msgr2=0x7fe3280706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe32806e210 0x7fe3280706d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe334005f50 tx=0x7fe334005dc0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3401016e0 msgr2=0x7fe340196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3401016e0 0x7fe340196620 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe33000e920 tx=0x7fe33000ece0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 shutdown_connections 2024-05-09T06:43:09.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe32806e210 0x7fe3280706d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3401016e0 0x7fe340196620 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 --2- 172.21.15.63:0/2323073283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe340104010 0x7fe340196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 >> 172.21.15.63:0/2323073283 conn(0x7fe3400fb0f0 msgr2=0x7fe3400fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:09.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 shutdown_connections 2024-05-09T06:43:09.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.199+0000 7fe344eb0700 1 -- 172.21.15.63:0/2323073283 wait complete. 2024-05-09T06:43:09.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.309+0000 7fd330ea6700 1 -- 172.21.15.63:0/3024522136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 msgr2=0x7fd32c101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.309+0000 7fd330ea6700 1 --2- 172.21.15.63:0/3024522136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 0x7fd32c101060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009910 tx=0x7fd31c009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:09.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.309+0000 7fd330ea6700 1 -- 172.21.15.63:0/3024522136 shutdown_connections 2024-05-09T06:43:09.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.309+0000 7fd330ea6700 1 --2- 172.21.15.63:0/3024522136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd32c101e40 0x7fd32c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.309+0000 7fd330ea6700 1 --2- 172.21.15.63:0/3024522136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 0x7fd32c101060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.309+0000 7fd330ea6700 1 -- 172.21.15.63:0/3024522136 >> 172.21.15.63:0/3024522136 conn(0x7fd32c0fc1e0 msgr2=0x7fd32c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:09.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.309+0000 7fd330ea6700 1 -- 172.21.15.63:0/3024522136 shutdown_connections 2024-05-09T06:43:09.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.309+0000 7fd330ea6700 1 -- 172.21.15.63:0/3024522136 wait complete. 2024-05-09T06:43:09.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.310+0000 7fd330ea6700 1 Processor -- start 2024-05-09T06:43:09.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.310+0000 7fd330ea6700 1 -- start start 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd330ea6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 0x7fd32c196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd330ea6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd32c101e40 0x7fd32c196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd330ea6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c197190 con 0x7fd32c100c40 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd330ea6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c1972d0 con 0x7fd32c101e40 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd32a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 0x7fd32c196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd32a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 0x7fd32c196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50410/0 (socket says 172.21.15.63:50410) 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd32a59c700 1 -- 172.21.15.63:0/3177879791 learned_addr learned my addr 172.21.15.63:0/3177879791 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd329d9b700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd32c101e40 0x7fd32c196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd32a59c700 1 -- 172.21.15.63:0/3177879791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd32c101e40 msgr2=0x7fd32c196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd32a59c700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd32c101e40 0x7fd32c196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd32a59c700 1 -- 172.21.15.63:0/3177879791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c0095f0 con 0x7fd32c100c40 2024-05-09T06:43:09.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd32a59c700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 0x7fd32c196630 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00b620 tx=0x7fd31c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:09.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd31b7fe700 1 -- 172.21.15.63:0/3177879791 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd31c01d070 con 0x7fd32c100c40 2024-05-09T06:43:09.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.311+0000 7fd31b7fe700 1 -- 172.21.15.63:0/3177879791 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd31c0052a0 con 0x7fd32c100c40 2024-05-09T06:43:09.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.312+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32c19bcd0 con 0x7fd32c100c40 2024-05-09T06:43:09.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.312+0000 7fd31b7fe700 1 -- 172.21.15.63:0/3177879791 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd31c017400 con 0x7fd32c100c40 2024-05-09T06:43:09.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.312+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c19c1c0 con 0x7fd32c100c40 2024-05-09T06:43:09.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.313+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32c1907c0 con 0x7fd32c100c40 2024-05-09T06:43:09.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.313+0000 7fd31b7fe700 1 -- 172.21.15.63:0/3177879791 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd31c003680 con 0x7fd32c100c40 2024-05-09T06:43:09.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.314+0000 7fd31b7fe700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd31406e210 0x7fd3140706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:09.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.314+0000 7fd31b7fe700 1 -- 172.21.15.63:0/3177879791 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fd31c08f5f0 con 0x7fd32c100c40 2024-05-09T06:43:09.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.314+0000 7fd329d9b700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd31406e210 0x7fd3140706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:09.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.314+0000 7fd329d9b700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd31406e210 0x7fd3140706d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd320009c60 tx=0x7fd320009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.318+0000 7fd31b7fe700 1 -- 172.21.15.63:0/3177879791 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd31c05c330 con 0x7fd32c100c40 2024-05-09T06:43:09.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.576+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd32c04ebb0 con 0x7fd32c100c40 2024-05-09T06:43:09.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.577+0000 7fd31b7fe700 1 -- 172.21.15.63:0/3177879791 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1905 (secure 0 0 0) 0x7fd31c05bec0 con 0x7fd32c100c40 2024-05-09T06:43:09.557 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:43:09.557 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:43:09.558 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:43:09.558 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:43:09.558 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:43:09.558 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:43:09.558 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:43:09.558 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:43:09.559 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:43:09.559 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:43:09.559 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:43:09.559 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:43:09.559 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:43:09.559 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:43:09.560 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:43:09.560 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:43:09.560 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:43:09.560 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:43:09.560 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:43:09.560 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:43:09.561 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:43:09.561 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:43:09.561 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:43:09.561 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:43:09.561 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:43:09.561 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:43:09.562 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:43:09.562 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:43:09.562 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:43:09.562 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:43:09.562 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:43:09.562 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:43:09.563 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:43:09.563 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-05-09T06:43:09.563 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:43:09.563 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:43:09.563 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:43:09.563 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:43:09.564 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:43:09.564 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:43:09.564 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:43:09.564 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:43:09.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd31406e210 msgr2=0x7fd3140706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd31406e210 0x7fd3140706d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd320009c60 tx=0x7fd320009260 comp rx=0 tx=0).stop 2024-05-09T06:43:09.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 msgr2=0x7fd32c196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 0x7fd32c196630 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00b620 tx=0x7fd31c005f00 comp rx=0 tx=0).stop 2024-05-09T06:43:09.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 shutdown_connections 2024-05-09T06:43:09.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd31406e210 0x7fd3140706d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c100c40 0x7fd32c196630 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 --2- 172.21.15.63:0/3177879791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd32c101e40 0x7fd32c196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 >> 172.21.15.63:0/3177879791 conn(0x7fd32c0fc1e0 msgr2=0x7fd32c105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:09.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 shutdown_connections 2024-05-09T06:43:09.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.579+0000 7fd330ea6700 1 -- 172.21.15.63:0/3177879791 wait complete. 2024-05-09T06:43:09.568 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:43:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.688+0000 7f7b900f6700 1 -- 172.21.15.63:0/1407246653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88101710 msgr2=0x7f7b88103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.688+0000 7f7b900f6700 1 --2- 172.21.15.63:0/1407246653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88101710 0x7f7b88103b00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009910 tx=0x7f7b84009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.688+0000 7f7b900f6700 1 -- 172.21.15.63:0/1407246653 shutdown_connections 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.688+0000 7f7b900f6700 1 --2- 172.21.15.63:0/1407246653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b88104040 0x7f7b88106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.688+0000 7f7b900f6700 1 --2- 172.21.15.63:0/1407246653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88101710 0x7f7b88103b00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.688+0000 7f7b900f6700 1 -- 172.21.15.63:0/1407246653 >> 172.21.15.63:0/1407246653 conn(0x7f7b880fb0c0 msgr2=0x7f7b880fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.688+0000 7f7b900f6700 1 -- 172.21.15.63:0/1407246653 shutdown_connections 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.688+0000 7f7b900f6700 1 -- 172.21.15.63:0/1407246653 wait complete. 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.689+0000 7f7b900f6700 1 Processor -- start 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.689+0000 7f7b900f6700 1 -- start start 2024-05-09T06:43:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.689+0000 7f7b900f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b88101710 0x7f7b881965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:09.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.689+0000 7f7b900f6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88104040 0x7f7b88196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:09.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.689+0000 7f7b900f6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b88197140 con 0x7f7b88104040 2024-05-09T06:43:09.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.689+0000 7f7b900f6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b88197280 con 0x7f7b88101710 2024-05-09T06:43:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.689+0000 7f7b8d691700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88104040 0x7f7b88196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.689+0000 7f7b8d691700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88104040 0x7f7b88196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50426/0 (socket says 172.21.15.63:50426) 2024-05-09T06:43:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.689+0000 7f7b8d691700 1 -- 172.21.15.63:0/770323580 learned_addr learned my addr 172.21.15.63:0/770323580 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b8de92700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b88101710 0x7f7b881965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b8d691700 1 -- 172.21.15.63:0/770323580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b88101710 msgr2=0x7f7b881965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b8d691700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b88101710 0x7f7b881965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b8d691700 1 -- 172.21.15.63:0/770323580 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840095f0 con 0x7f7b88104040 2024-05-09T06:43:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b8d691700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88104040 0x7f7b88196b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800d8d0 tx=0x7f7b7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b7effd700 1 -- 172.21.15.63:0/770323580 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b78009940 con 0x7f7b88104040 2024-05-09T06:43:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8819bce0 con 0x7f7b88104040 2024-05-09T06:43:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b7effd700 1 -- 172.21.15.63:0/770323580 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b78010460 con 0x7f7b88104040 2024-05-09T06:43:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b7effd700 1 -- 172.21.15.63:0/770323580 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b7800f5f0 con 0x7f7b88104040 2024-05-09T06:43:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.690+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8819c2b0 con 0x7f7b88104040 2024-05-09T06:43:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.691+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8804ebb0 con 0x7f7b88104040 2024-05-09T06:43:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.692+0000 7f7b7effd700 1 -- 172.21.15.63:0/770323580 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7b780105d0 con 0x7f7b88104040 2024-05-09T06:43:09.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.692+0000 7f7b7effd700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7b7406e200 0x7f7b740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:09.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.692+0000 7f7b7effd700 1 -- 172.21.15.63:0/770323580 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f7b7808e160 con 0x7f7b88104040 2024-05-09T06:43:09.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.692+0000 7f7b8de92700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7b7406e200 0x7f7b740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:09.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.693+0000 7f7b8de92700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7b7406e200 0x7f7b740706c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b840051c0 tx=0x7f7b84005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:09.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.697+0000 7f7b7effd700 1 -- 172.21.15.63:0/770323580 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7b7805af20 con 0x7f7b88104040 2024-05-09T06:43:09.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:09 smithi063 ceph-mon[92765]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:09.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:09 smithi063 ceph-mon[92765]: pgmap v116: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:43:09.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:09 smithi063 ceph-mon[92765]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:09.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:09 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2323073283' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:09 smithi193 ceph-mon[81665]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:09 smithi193 ceph-mon[81665]: pgmap v116: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:43:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:09 smithi193 ceph-mon[81665]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:09.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:09 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2323073283' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.921+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b88105f20 con 0x7f7b7406e200 2024-05-09T06:43:09.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.923+0000 7f7b7effd700 1 -- 172.21.15.63:0/770323580 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7b88105f20 con 0x7f7b7406e200 2024-05-09T06:43:09.902 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:43:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:43:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:43:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:43:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:43:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:43:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:43:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:43:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:43:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "9/23 daemons upgraded", 2024-05-09T06:43:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-05-09T06:43:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:43:09.903 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:43:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7b7406e200 msgr2=0x7f7b740706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7b7406e200 0x7f7b740706c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7b840051c0 tx=0x7f7b84005a90 comp rx=0 tx=0).stop 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88104040 msgr2=0x7f7b88196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88104040 0x7f7b88196b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800d8d0 tx=0x7f7b7800dc90 comp rx=0 tx=0).stop 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 shutdown_connections 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7b7406e200 0x7f7b740706c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b88101710 0x7f7b881965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 --2- 172.21.15.63:0/770323580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b88104040 0x7f7b88196b20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 >> 172.21.15.63:0/770323580 conn(0x7f7b880fb0c0 msgr2=0x7f7b880fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 shutdown_connections 2024-05-09T06:43:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:09.925+0000 7f7b900f6700 1 -- 172.21.15.63:0/770323580 wait complete. 2024-05-09T06:43:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.033+0000 7fc2bba40700 1 -- 172.21.15.63:0/4003859859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 msgr2=0x7fc2b4101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.033+0000 7fc2bba40700 1 --2- 172.21.15.63:0/4003859859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 0x7fc2b4101070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009910 tx=0x7fc2b0009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.033+0000 7fc2bba40700 1 -- 172.21.15.63:0/4003859859 shutdown_connections 2024-05-09T06:43:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.033+0000 7fc2bba40700 1 --2- 172.21.15.63:0/4003859859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b4101e50 0x7fc2b41022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.033+0000 7fc2bba40700 1 --2- 172.21.15.63:0/4003859859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 0x7fc2b4101070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.033+0000 7fc2bba40700 1 -- 172.21.15.63:0/4003859859 >> 172.21.15.63:0/4003859859 conn(0x7fc2b40fc210 msgr2=0x7fc2b40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.033+0000 7fc2bba40700 1 -- 172.21.15.63:0/4003859859 shutdown_connections 2024-05-09T06:43:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.033+0000 7fc2bba40700 1 -- 172.21.15.63:0/4003859859 wait complete. 2024-05-09T06:43:10.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2bba40700 1 Processor -- start 2024-05-09T06:43:10.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2bba40700 1 -- start start 2024-05-09T06:43:10.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2bba40700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 0x7fc2b4074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:10.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2bba40700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b4101e50 0x7fc2b40732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:10.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2bba40700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4073820 con 0x7fc2b4100c50 2024-05-09T06:43:10.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2bba40700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4073960 con 0x7fc2b4101e50 2024-05-09T06:43:10.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2b97dc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 0x7fc2b4074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:10.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2b97dc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 0x7fc2b4074bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50452/0 (socket says 172.21.15.63:50452) 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2b97dc700 1 -- 172.21.15.63:0/3781791761 learned_addr learned my addr 172.21.15.63:0/3781791761 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2b8fdb700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b4101e50 0x7fc2b40732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2b97dc700 1 -- 172.21.15.63:0/3781791761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b4101e50 msgr2=0x7fc2b40732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2b97dc700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b4101e50 0x7fc2b40732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.034+0000 7fc2b97dc700 1 -- 172.21.15.63:0/3781791761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b00095f0 con 0x7fc2b4100c50 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.035+0000 7fc2b97dc700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 0x7fc2b4074bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0005f00 tx=0x7fc2b000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.035+0000 7fc2aa7fc700 1 -- 172.21.15.63:0/3781791761 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2b001d070 con 0x7fc2b4100c50 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.035+0000 7fc2aa7fc700 1 -- 172.21.15.63:0/3781791761 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2b000fe30 con 0x7fc2b4100c50 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.035+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b4073be0 con 0x7fc2b4100c50 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.035+0000 7fc2aa7fc700 1 -- 172.21.15.63:0/3781791761 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2b0017400 con 0x7fc2b4100c50 2024-05-09T06:43:10.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.035+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b4074150 con 0x7fc2b4100c50 2024-05-09T06:43:10.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.036+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b41907d0 con 0x7fc2b4100c50 2024-05-09T06:43:10.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.036+0000 7fc2aa7fc700 1 -- 172.21.15.63:0/3781791761 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc2b00038a0 con 0x7fc2b4100c50 2024-05-09T06:43:10.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.037+0000 7fc2aa7fc700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc2a006e200 0x7fc2a00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:10.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.037+0000 7fc2aa7fc700 1 -- 172.21.15.63:0/3781791761 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fc2b008f5a0 con 0x7fc2b4100c50 2024-05-09T06:43:10.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.037+0000 7fc2b8fdb700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc2a006e200 0x7fc2a00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:10.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.037+0000 7fc2b8fdb700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc2a006e200 0x7fc2a00706c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009bf0 tx=0x7fc2a4009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:10.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.042+0000 7fc2aa7fc700 1 -- 172.21.15.63:0/3781791761 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc2b005c2e0 con 0x7fc2b4100c50 2024-05-09T06:43:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.334+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc2b4074150 con 0x7fc2b4100c50 2024-05-09T06:43:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.334+0000 7fc2aa7fc700 1 -- 172.21.15.63:0/3781791761 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc2b4074150 con 0x7fc2b4100c50 2024-05-09T06:43:10.314 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:43:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc2a006e200 msgr2=0x7fc2a00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc2a006e200 0x7fc2a00706c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009bf0 tx=0x7fc2a4009260 comp rx=0 tx=0).stop 2024-05-09T06:43:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 msgr2=0x7fc2b4074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 0x7fc2b4074bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0005f00 tx=0x7fc2b000f510 comp rx=0 tx=0).stop 2024-05-09T06:43:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 shutdown_connections 2024-05-09T06:43:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc2a006e200 0x7fc2a00706c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b4100c50 0x7fc2b4074bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 --2- 172.21.15.63:0/3781791761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b4101e50 0x7fc2b40732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 >> 172.21.15.63:0/3781791761 conn(0x7fc2b40fc210 msgr2=0x7fc2b4105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 shutdown_connections 2024-05-09T06:43:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:10.337+0000 7fc2bba40700 1 -- 172.21.15.63:0/3781791761 wait complete. 2024-05-09T06:43:10.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:10 smithi063 ceph-mon[92765]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:10.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:10 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3177879791' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:43:10.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:10 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3781791761' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:43:10.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:10 smithi193 ceph-mon[81665]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:10.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:10 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3177879791' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:43:10.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:10 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3781791761' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:43:11.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:11 smithi063 ceph-mon[92765]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:11.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:11 smithi063 ceph-mon[92765]: pgmap v117: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-05-09T06:43:11.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:11 smithi193 ceph-mon[81665]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:11.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:11 smithi193 ceph-mon[81665]: pgmap v117: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-05-09T06:43:13.516 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:13 smithi063 ceph-mon[92765]: pgmap v118: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 944 B/s rd, 1 op/s 2024-05-09T06:43:13.516 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:13.516 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:43:13.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:13 smithi193 ceph-mon[81665]: pgmap v118: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 944 B/s rd, 1 op/s 2024-05-09T06:43:13.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:13.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:43:15.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:15 smithi063 ceph-mon[92765]: pgmap v119: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-09T06:43:15.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:15 smithi193 ceph-mon[81665]: pgmap v119: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-09T06:43:17.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:17 smithi063 ceph-mon[92765]: pgmap v120: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:43:17.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:17 smithi193 ceph-mon[81665]: pgmap v120: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:43:18.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:18 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:18.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:18 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:19.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:19 smithi193 ceph-mon[81665]: pgmap v121: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:43:19.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:19 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-09T06:43:19.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:19 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-09T06:43:19.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:19 smithi193 ceph-mon[81665]: Upgrade: osd.3 is safe to restart 2024-05-09T06:43:19.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:19 smithi063 ceph-mon[92765]: pgmap v121: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:43:19.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:19 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-09T06:43:19.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:19 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-09T06:43:19.756 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:19 smithi063 ceph-mon[92765]: Upgrade: osd.3 is safe to restart 2024-05-09T06:43:21.586 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:21 smithi193 ceph-mon[81665]: pgmap v122: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:43:21.586 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:21 smithi193 ceph-mon[81665]: Upgrade: Updating osd.3 2024-05-09T06:43:21.586 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:21.586 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T06:43:21.586 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:43:21.586 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:21 smithi193 ceph-mon[81665]: Deploying daemon osd.3 on smithi193 2024-05-09T06:43:21.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:21 smithi063 ceph-mon[92765]: pgmap v122: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:43:21.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:21 smithi063 ceph-mon[92765]: Upgrade: Updating osd.3 2024-05-09T06:43:21.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:21.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T06:43:21.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:43:21.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:21 smithi063 ceph-mon[92765]: Deploying daemon osd.3 on smithi193 2024-05-09T06:43:23.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:23 smithi193 ceph-mon[81665]: pgmap v123: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:43:23.765 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:23 smithi193 systemd[1]: Stopping Ceph osd.3 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:43:23.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:23 smithi063 ceph-mon[92765]: pgmap v123: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:43:24.121 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:23 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[41190]: 2024-05-09T06:43:23.784+0000 7f520e1c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:43:24.122 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:23 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[41190]: 2024-05-09T06:43:23.784+0000 7f520e1c1700 -1 osd.3 75 *** Got signal Terminated *** 2024-05-09T06:43:24.122 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:23 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[41190]: 2024-05-09T06:43:23.784+0000 7f520e1c1700 -1 osd.3 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:43:24.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:24 smithi193 ceph-mon[81665]: osd.3 marked itself down and dead 2024-05-09T06:43:24.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:24 smithi063 ceph-mon[92765]: osd.3 marked itself down and dead 2024-05-09T06:43:25.091 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:24 smithi193 podman[90968]: 2024-05-09 06:43:24.801885834 +0000 UTC m=+1.114318459 container died 738a82f116dd7801fcca304d1f5684c67b9ec53fa62b426bdf1f61afbb4922ab (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=reef-1758e8e) 2024-05-09T06:43:25.371 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:25 smithi193 podman[90968]: 2024-05-09 06:43:25.209773918 +0000 UTC m=+1.522206547 container cleanup 738a82f116dd7801fcca304d1f5684c67b9ec53fa62b426bdf1f61afbb4922ab (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240430, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=reef-1758e8e, org.label-schema.schema-version=1.0) 2024-05-09T06:43:25.371 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:25 smithi193 bash[90968]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3 2024-05-09T06:43:25.813 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:25 smithi193 ceph-mon[81665]: pgmap v124: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:43:25.813 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:25 smithi193 ceph-mon[81665]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:43:25.813 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:25 smithi193 ceph-mon[81665]: osdmap e76: 6 total, 5 up, 6 in 2024-05-09T06:43:25.813 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:25 smithi193 podman[90997]: 2024-05-09 06:43:25.419705585 +0000 UTC m=+0.617889553 container remove 738a82f116dd7801fcca304d1f5684c67b9ec53fa62b426bdf1f61afbb4922ab (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-1758e8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430) 2024-05-09T06:43:25.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:25 smithi063 ceph-mon[92765]: pgmap v124: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:43:25.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:25 smithi063 ceph-mon[92765]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:43:25.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:25 smithi063 ceph-mon[92765]: osdmap e76: 6 total, 5 up, 6 in 2024-05-09T06:43:26.085 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:25 smithi193 podman[91089]: 2024-05-09 06:43:25.737197669 +0000 UTC m=+0.017969344 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:43:26.085 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:25 smithi193 podman[91089]: 2024-05-09 06:43:25.854052744 +0000 UTC m=+0.134824454 container create f6c1c16320256988f91f026e30500c07c5b8553c6d420e3b1551d1338519ef14 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:43:26.086 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:26 smithi193 podman[91089]: 2024-05-09 06:43:26.106544253 +0000 UTC m=+0.387315928 container init f6c1c16320256988f91f026e30500c07c5b8553c6d420e3b1551d1338519ef14 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T06:43:26.371 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:26 smithi193 podman[91089]: 2024-05-09 06:43:26.109936249 +0000 UTC m=+0.390707923 container start f6c1c16320256988f91f026e30500c07c5b8553c6d420e3b1551d1338519ef14 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:43:26.371 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:26 smithi193 podman[91089]: 2024-05-09 06:43:26.181382378 +0000 UTC m=+0.462154054 container attach f6c1c16320256988f91f026e30500c07c5b8553c6d420e3b1551d1338519ef14 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc) 2024-05-09T06:43:26.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:26 smithi193 ceph-mon[81665]: osdmap e77: 6 total, 5 up, 6 in 2024-05-09T06:43:26.672 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:26 smithi193 podman[91089]: 2024-05-09 06:43:26.40995872 +0000 UTC m=+0.690730390 container died f6c1c16320256988f91f026e30500c07c5b8553c6d420e3b1551d1338519ef14 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-09T06:43:26.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:26 smithi063 ceph-mon[92765]: osdmap e77: 6 total, 5 up, 6 in 2024-05-09T06:43:27.374 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:26 smithi193 podman[91115]: 2024-05-09 06:43:26.976504623 +0000 UTC m=+0.560642851 container remove f6c1c16320256988f91f026e30500c07c5b8553c6d420e3b1551d1338519ef14 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:43:27.374 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:26 smithi193 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.3.service: Deactivated successfully. 2024-05-09T06:43:27.374 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:26 smithi193 systemd[1]: Stopped Ceph osd.3 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:43:27.374 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:26 smithi193 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.3.service: Consumed 1min 21.865s CPU time. 2024-05-09T06:43:27.374 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:27 smithi193 systemd[1]: Starting Ceph osd.3 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:43:27.676 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:27 smithi193 ceph-mon[81665]: pgmap v127: 65 pgs: 7 peering, 11 stale+active+clean, 47 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:43:27.676 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:27 smithi193 ceph-mon[81665]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-05-09T06:43:27.676 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:27.677 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:43:27.677 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:27 smithi193 podman[91216]: 2024-05-09 06:43:27.343209728 +0000 UTC m=+0.017419904 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:43:27.677 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:27 smithi193 podman[91216]: 2024-05-09 06:43:27.454813791 +0000 UTC m=+0.129023951 container create 06f146403c51dca65d0f9d232798f025374daeb3d915a27870df590e1d3be20e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-09T06:43:27.677 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:27 smithi193 podman[91216]: 2024-05-09 06:43:27.697753112 +0000 UTC m=+0.371963269 container init 06f146403c51dca65d0f9d232798f025374daeb3d915a27870df590e1d3be20e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:43:27.769 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:27 smithi063 ceph-mon[92765]: pgmap v127: 65 pgs: 7 peering, 11 stale+active+clean, 47 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-05-09T06:43:27.770 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:27 smithi063 ceph-mon[92765]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-05-09T06:43:27.770 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:27.770 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:43:28.121 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:27 smithi193 podman[91216]: 2024-05-09 06:43:27.701218165 +0000 UTC m=+0.375428322 container start 06f146403c51dca65d0f9d232798f025374daeb3d915a27870df590e1d3be20e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-09T06:43:28.121 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:27 smithi193 podman[91216]: 2024-05-09 06:43:27.780287986 +0000 UTC m=+0.454498151 container attach 06f146403c51dca65d0f9d232798f025374daeb3d915a27870df590e1d3be20e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:43:28.433 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate[91232]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T06:43:28.433 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate[91232]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 bash[91216]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 bash[91216]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 bash[91216]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 bash[91216]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 bash[91216]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 bash[91216]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 bash[91216]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate[91232]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate[91232]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate[91232]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate[91232]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate[91232]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-09T06:43:28.434 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 podman[91216]: 2024-05-09 06:43:28.180160023 +0000 UTC m=+0.854370173 container died 06f146403c51dca65d0f9d232798f025374daeb3d915a27870df590e1d3be20e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:43:28.737 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:28 smithi193 ceph-mon[81665]: Health check failed: Degraded data redundancy: 31/228 objects degraded (13.596%), 10 pgs degraded (PG_DEGRADED) 2024-05-09T06:43:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:28 smithi063 ceph-mon[92765]: Health check failed: Degraded data redundancy: 31/228 objects degraded (13.596%), 10 pgs degraded (PG_DEGRADED) 2024-05-09T06:43:29.112 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 podman[91367]: 2024-05-09 06:43:28.758795321 +0000 UTC m=+0.572586483 container remove 06f146403c51dca65d0f9d232798f025374daeb3d915a27870df590e1d3be20e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True) 2024-05-09T06:43:29.112 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:28 smithi193 podman[91432]: 2024-05-09 06:43:28.896355821 +0000 UTC m=+0.018216822 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:43:29.112 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:29 smithi193 podman[91432]: 2024-05-09 06:43:29.001302439 +0000 UTC m=+0.123163435 container create 96afdd638dd5254f20b49cfbe743bbd54eb68f9a6d9d0e21b476f31d2cacf69b (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T06:43:29.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:29 smithi193 ceph-mon[81665]: pgmap v128: 65 pgs: 12 active+undersized, 10 active+undersized+degraded, 7 peering, 1 stale+active+clean, 35 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/228 objects degraded (13.596%) 2024-05-09T06:43:29.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:29.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:29.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:43:29.622 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:29 smithi193 podman[91432]: 2024-05-09 06:43:29.228833742 +0000 UTC m=+0.350694745 container init 96afdd638dd5254f20b49cfbe743bbd54eb68f9a6d9d0e21b476f31d2cacf69b (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T06:43:29.622 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:29 smithi193 podman[91432]: 2024-05-09 06:43:29.232247952 +0000 UTC m=+0.354108955 container start 96afdd638dd5254f20b49cfbe743bbd54eb68f9a6d9d0e21b476f31d2cacf69b (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-96805dc) 2024-05-09T06:43:29.622 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:29 smithi193 ceph-osd[91452]: -- 172.21.15.193:0/2356339182 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe8b023680 con 0x55fe8afed000 2024-05-09T06:43:29.622 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:29 smithi193 bash[91432]: 96afdd638dd5254f20b49cfbe743bbd54eb68f9a6d9d0e21b476f31d2cacf69b 2024-05-09T06:43:29.622 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:29 smithi193 systemd[1]: Started Ceph osd.3 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:43:29.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:29 smithi063 ceph-mon[92765]: pgmap v128: 65 pgs: 12 active+undersized, 10 active+undersized+degraded, 7 peering, 1 stale+active+clean, 35 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/228 objects degraded (13.596%) 2024-05-09T06:43:29.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:29.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:29.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:43:30.121 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:29 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[91448]: 2024-05-09T06:43:29.778+0000 7f42b6d327c0 -1 Falling back to public interface 2024-05-09T06:43:30.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:30 smithi063 ceph-mon[92765]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-05-09T06:43:30.859 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:30 smithi193 ceph-mon[81665]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-05-09T06:43:31.750 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:31 smithi193 ceph-mon[81665]: pgmap v129: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/228 objects degraded (18.860%) 2024-05-09T06:43:31.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:31 smithi063 ceph-mon[92765]: pgmap v129: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/228 objects degraded (18.860%) 2024-05-09T06:43:33.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:33 smithi063 ceph-mon[92765]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/228 objects degraded (18.860%) 2024-05-09T06:43:33.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:33.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:33 smithi193 ceph-mon[81665]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/228 objects degraded (18.860%) 2024-05-09T06:43:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:34.779 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:34 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:34.779 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:34 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:34.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:34 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:34.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:34 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:35.403 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:35 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[91448]: 2024-05-09T06:43:35.158+0000 7f42b6d327c0 -1 osd.3 75 log_to_monitors true 2024-05-09T06:43:35.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:35 smithi193 ceph-mon[81665]: pgmap v131: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/228 objects degraded (18.860%) 2024-05-09T06:43:35.671 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:35 smithi193 ceph-mon[81665]: from='osd.3 [v2:172.21.15.193:6800/2657331774,v1:172.21.15.193:6801/2657331774]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T06:43:35.672 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:35 smithi193 ceph-mon[81665]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T06:43:35.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:35 smithi063 ceph-mon[92765]: pgmap v131: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/228 objects degraded (18.860%) 2024-05-09T06:43:35.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:35 smithi063 ceph-mon[92765]: from='osd.3 [v2:172.21.15.193:6800/2657331774,v1:172.21.15.193:6801/2657331774]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T06:43:35.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:35 smithi063 ceph-mon[92765]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T06:43:36.621 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:43:36 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[91448]: 2024-05-09T06:43:36.350+0000 7f42a4181700 -1 osd.3 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:43:36.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:36 smithi193 ceph-mon[81665]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T06:43:36.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:36 smithi193 ceph-mon[81665]: from='osd.3 [v2:172.21.15.193:6800/2657331774,v1:172.21.15.193:6801/2657331774]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:43:36.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:36 smithi193 ceph-mon[81665]: osdmap e78: 6 total, 5 up, 6 in 2024-05-09T06:43:36.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:36 smithi193 ceph-mon[81665]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:43:36.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:36 smithi193 ceph-mon[81665]: from='osd.3 ' entity='osd.3' 2024-05-09T06:43:36.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:36 smithi063 ceph-mon[92765]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T06:43:36.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:36 smithi063 ceph-mon[92765]: from='osd.3 [v2:172.21.15.193:6800/2657331774,v1:172.21.15.193:6801/2657331774]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:43:36.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:36 smithi063 ceph-mon[92765]: osdmap e78: 6 total, 5 up, 6 in 2024-05-09T06:43:36.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:36 smithi063 ceph-mon[92765]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:43:36.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:36 smithi063 ceph-mon[92765]: from='osd.3 ' entity='osd.3' 2024-05-09T06:43:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: pgmap v133: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1023 MiB used, 535 GiB / 536 GiB avail; 43/228 objects degraded (18.860%) 2024-05-09T06:43:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:43:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:43:37.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:43:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-09T06:43:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:43:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: osd.3 [v2:172.21.15.193:6800/2657331774,v1:172.21.15.193:6801/2657331774] boot 2024-05-09T06:43:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: osdmap e79: 6 total, 6 up, 6 in 2024-05-09T06:43:37.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:43:37.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: pgmap v133: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1023 MiB used, 535 GiB / 536 GiB avail; 43/228 objects degraded (18.860%) 2024-05-09T06:43:37.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:37.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:37.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: osd.3 [v2:172.21.15.193:6800/2657331774,v1:172.21.15.193:6801/2657331774] boot 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: osdmap e79: 6 total, 6 up, 6 in 2024-05-09T06:43:37.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T06:43:38.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:38 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-09T06:43:38.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:38 smithi063 ceph-mon[92765]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-05-09T06:43:38.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:38 smithi063 ceph-mon[92765]: Health check update: Degraded data redundancy: 43/228 objects degraded (18.860%), 15 pgs degraded (PG_DEGRADED) 2024-05-09T06:43:38.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:38 smithi063 ceph-mon[92765]: osdmap e80: 6 total, 6 up, 6 in 2024-05-09T06:43:38.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:38 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-09T06:43:38.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:38 smithi193 ceph-mon[81665]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-05-09T06:43:38.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:38 smithi193 ceph-mon[81665]: Health check update: Degraded data redundancy: 43/228 objects degraded (18.860%), 15 pgs degraded (PG_DEGRADED) 2024-05-09T06:43:38.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:38 smithi193 ceph-mon[81665]: osdmap e80: 6 total, 6 up, 6 in 2024-05-09T06:43:39.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:39 smithi063 ceph-mon[92765]: pgmap v135: 65 pgs: 7 peering, 12 active+undersized, 14 active+undersized+degraded, 32 active+clean; 205 MiB data, 1023 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 39/228 objects degraded (17.105%) 2024-05-09T06:43:39.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:39 smithi193 ceph-mon[81665]: pgmap v135: 65 pgs: 7 peering, 12 active+undersized, 14 active+undersized+degraded, 32 active+clean; 205 MiB data, 1023 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 39/228 objects degraded (17.105%) 2024-05-09T06:43:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.461+0000 7fcba6920700 1 -- 172.21.15.63:0/1058882119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 msgr2=0x7fcba0103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.461+0000 7fcba6920700 1 --2- 172.21.15.63:0/1058882119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 0x7fcba0103b80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009910 tx=0x7fcb90009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.461+0000 7fcba6920700 1 -- 172.21.15.63:0/1058882119 shutdown_connections 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.461+0000 7fcba6920700 1 --2- 172.21.15.63:0/1058882119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcba01040c0 0x7fcba01064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.461+0000 7fcba6920700 1 --2- 172.21.15.63:0/1058882119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 0x7fcba0103b80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.461+0000 7fcba6920700 1 -- 172.21.15.63:0/1058882119 >> 172.21.15.63:0/1058882119 conn(0x7fcba00fb120 msgr2=0x7fcba00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.461+0000 7fcba6920700 1 -- 172.21.15.63:0/1058882119 shutdown_connections 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.461+0000 7fcba6920700 1 -- 172.21.15.63:0/1058882119 wait complete. 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.462+0000 7fcba6920700 1 Processor -- start 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.462+0000 7fcba6920700 1 -- start start 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.462+0000 7fcba6920700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 0x7fcba01010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.462+0000 7fcba6920700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcba01040c0 0x7fcba00ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.462+0000 7fcba6920700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00ffc80 con 0x7fcba0101790 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.462+0000 7fcba6920700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00ffdc0 con 0x7fcba01040c0 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.462+0000 7fcb9ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 0x7fcba01010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.462+0000 7fcb9ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 0x7fcba01010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36140/0 (socket says 172.21.15.63:36140) 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.462+0000 7fcb9ffff700 1 -- 172.21.15.63:0/2490914791 learned_addr learned my addr 172.21.15.63:0/2490914791 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcb97fff700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcba01040c0 0x7fcba00ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcb9ffff700 1 -- 172.21.15.63:0/2490914791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcba01040c0 msgr2=0x7fcba00ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcb9ffff700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcba01040c0 0x7fcba00ff740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcb9ffff700 1 -- 172.21.15.63:0/2490914791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb900095f0 con 0x7fcba0101790 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcb9ffff700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 0x7fcba01010a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb90006010 tx=0x7fcb9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcb9dffb700 1 -- 172.21.15.63:0/2490914791 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb9001d070 con 0x7fcba0101790 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcb9dffb700 1 -- 172.21.15.63:0/2490914791 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb90005dc0 con 0x7fcba0101790 2024-05-09T06:43:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba0100040 con 0x7fcba0101790 2024-05-09T06:43:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcb9dffb700 1 -- 172.21.15.63:0/2490914791 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb90021640 con 0x7fcba0101790 2024-05-09T06:43:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.463+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba0100530 con 0x7fcba0101790 2024-05-09T06:43:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.464+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba004ebb0 con 0x7fcba0101790 2024-05-09T06:43:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.465+0000 7fcb9dffb700 1 -- 172.21.15.63:0/2490914791 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcb9000f880 con 0x7fcba0101790 2024-05-09T06:43:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.466+0000 7fcb9dffb700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcb8c06e210 0x7fcb8c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.466+0000 7fcb97fff700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcb8c06e210 0x7fcb8c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.466+0000 7fcb9dffb700 1 -- 172.21.15.63:0/2490914791 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fcb9008f450 con 0x7fcba0101790 2024-05-09T06:43:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.466+0000 7fcb97fff700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcb8c06e210 0x7fcb8c0706d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcb88005950 tx=0x7fcb880058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.471+0000 7fcb9dffb700 1 -- 172.21.15.63:0/2490914791 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcb9005c210 con 0x7fcba0101790 2024-05-09T06:43:40.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.703+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcba0105f80 con 0x7fcb8c06e210 2024-05-09T06:43:40.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.705+0000 7fcb9dffb700 1 -- 172.21.15.63:0/2490914791 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fcba0105f80 con 0x7fcb8c06e210 2024-05-09T06:43:40.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.706+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcb8c06e210 msgr2=0x7fcb8c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:40.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.706+0000 7fcba6920700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcb8c06e210 0x7fcb8c0706d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcb88005950 tx=0x7fcb880058e0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.706+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 msgr2=0x7fcba01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:40.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.706+0000 7fcba6920700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 0x7fcba01010a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb90006010 tx=0x7fcb9000be20 comp rx=0 tx=0).stop 2024-05-09T06:43:40.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.707+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 shutdown_connections 2024-05-09T06:43:40.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.707+0000 7fcba6920700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcb8c06e210 0x7fcb8c0706d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.707+0000 7fcba6920700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba0101790 0x7fcba01010a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.707+0000 7fcba6920700 1 --2- 172.21.15.63:0/2490914791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcba01040c0 0x7fcba00ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.707+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 >> 172.21.15.63:0/2490914791 conn(0x7fcba00fb120 msgr2=0x7fcba00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:40.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.707+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 shutdown_connections 2024-05-09T06:43:40.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.707+0000 7fcba6920700 1 -- 172.21.15.63:0/2490914791 wait complete. 2024-05-09T06:43:40.697 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:43:40.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.817+0000 7fc2b4902700 1 -- 172.21.15.63:0/3087402766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b0100d50 msgr2=0x7fc2b01011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:40.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.817+0000 7fc2b4902700 1 --2- 172.21.15.63:0/3087402766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b0100d50 0x7fc2b01011b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009910 tx=0x7fc2a4009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:40.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.817+0000 7fc2b4902700 1 -- 172.21.15.63:0/3087402766 shutdown_connections 2024-05-09T06:43:40.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.817+0000 7fc2b4902700 1 --2- 172.21.15.63:0/3087402766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b0100d50 0x7fc2b01011b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.817+0000 7fc2b4902700 1 --2- 172.21.15.63:0/3087402766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b00ffb50 0x7fc2b00fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.817+0000 7fc2b4902700 1 -- 172.21.15.63:0/3087402766 >> 172.21.15.63:0/3087402766 conn(0x7fc2b00fb0f0 msgr2=0x7fc2b00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:40.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.817+0000 7fc2b4902700 1 -- 172.21.15.63:0/3087402766 shutdown_connections 2024-05-09T06:43:40.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.817+0000 7fc2b4902700 1 -- 172.21.15.63:0/3087402766 wait complete. 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2b4902700 1 Processor -- start 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2b4902700 1 -- start start 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2b4902700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b00ffb50 0x7fc2b0196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2b4902700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b0100d50 0x7fc2b0196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2b4902700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b01971a0 con 0x7fc2b00ffb50 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2b4902700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b01972e0 con 0x7fc2b0100d50 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2ae59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b00ffb50 0x7fc2b0196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2ae59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b00ffb50 0x7fc2b0196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36156/0 (socket says 172.21.15.63:36156) 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2ae59c700 1 -- 172.21.15.63:0/3135389532 learned_addr learned my addr 172.21.15.63:0/3135389532 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:40.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2add9b700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b0100d50 0x7fc2b0196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2ae59c700 1 -- 172.21.15.63:0/3135389532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b0100d50 msgr2=0x7fc2b0196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2ae59c700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b0100d50 0x7fc2b0196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2ae59c700 1 -- 172.21.15.63:0/3135389532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40095f0 con 0x7fc2b00ffb50 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.818+0000 7fc2ae59c700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b00ffb50 0x7fc2b0196640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000d900 tx=0x7fc2a000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.819+0000 7fc29f7fe700 1 -- 172.21.15.63:0/3135389532 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a00041f0 con 0x7fc2b00ffb50 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.819+0000 7fc29f7fe700 1 -- 172.21.15.63:0/3135389532 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2a0004350 con 0x7fc2b00ffb50 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.819+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b019bd40 con 0x7fc2b00ffb50 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.819+0000 7fc29f7fe700 1 -- 172.21.15.63:0/3135389532 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a0003cf0 con 0x7fc2b00ffb50 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.819+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b019c2b0 con 0x7fc2b00ffb50 2024-05-09T06:43:40.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.820+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b004ec20 con 0x7fc2b00ffb50 2024-05-09T06:43:40.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.820+0000 7fc29f7fe700 1 -- 172.21.15.63:0/3135389532 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc2a0009730 con 0x7fc2b00ffb50 2024-05-09T06:43:40.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.821+0000 7fc29f7fe700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc29806e200 0x7fc2980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:40.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.821+0000 7fc29f7fe700 1 -- 172.21.15.63:0/3135389532 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fc2a008e0e0 con 0x7fc2b00ffb50 2024-05-09T06:43:40.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.821+0000 7fc2add9b700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc29806e200 0x7fc2980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:40.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.821+0000 7fc2add9b700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc29806e200 0x7fc2980706c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4005090 tx=0x7fc2a4005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:40.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:40.825+0000 7fc29f7fe700 1 -- 172.21.15.63:0/3135389532 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc2b004ec20 con 0x7fc2b00ffb50 2024-05-09T06:43:41.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.050+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2b004ebb0 con 0x7fc29806e200 2024-05-09T06:43:41.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.051+0000 7fc29f7fe700 1 -- 172.21.15.63:0/3135389532 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc2b004ebb0 con 0x7fc29806e200 2024-05-09T06:43:41.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc29806e200 msgr2=0x7fc2980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc29806e200 0x7fc2980706c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4005090 tx=0x7fc2a4005960 comp rx=0 tx=0).stop 2024-05-09T06:43:41.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b00ffb50 msgr2=0x7fc2b0196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b00ffb50 0x7fc2b0196640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000d900 tx=0x7fc2a000dc10 comp rx=0 tx=0).stop 2024-05-09T06:43:41.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 shutdown_connections 2024-05-09T06:43:41.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc29806e200 0x7fc2980706c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2b00ffb50 0x7fc2b0196640 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 --2- 172.21.15.63:0/3135389532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2b0100d50 0x7fc2b0196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 >> 172.21.15.63:0/3135389532 conn(0x7fc2b00fb0f0 msgr2=0x7fc2b0069210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:41.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 shutdown_connections 2024-05-09T06:43:41.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.054+0000 7fc2b4902700 1 -- 172.21.15.63:0/3135389532 wait complete. 2024-05-09T06:43:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.162+0000 7f4057cba700 1 -- 172.21.15.63:0/3540174089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 msgr2=0x7f40500ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.162+0000 7f4057cba700 1 --2- 172.21.15.63:0/3540174089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 0x7f40500ff330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4044009910 tx=0x7f4044009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 -- 172.21.15.63:0/3540174089 shutdown_connections 2024-05-09T06:43:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 --2- 172.21.15.63:0/3540174089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4050101ef0 0x7f4050102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 --2- 172.21.15.63:0/3540174089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 0x7f40500ff330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 -- 172.21.15.63:0/3540174089 >> 172.21.15.63:0/3540174089 conn(0x7f40500fa870 msgr2=0x7f40500fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 -- 172.21.15.63:0/3540174089 shutdown_connections 2024-05-09T06:43:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 -- 172.21.15.63:0/3540174089 wait complete. 2024-05-09T06:43:41.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 Processor -- start 2024-05-09T06:43:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 -- start start 2024-05-09T06:43:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 0x7f4050196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4050101ef0 0x7f4050196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501971d0 con 0x7f40500fef10 2024-05-09T06:43:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.163+0000 7f4057cba700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4050197310 con 0x7f4050101ef0 2024-05-09T06:43:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4055a56700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 0x7f4050196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4055a56700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 0x7f4050196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36170/0 (socket says 172.21.15.63:36170) 2024-05-09T06:43:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4055a56700 1 -- 172.21.15.63:0/1790345718 learned_addr learned my addr 172.21.15.63:0/1790345718 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:41.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4055255700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4050101ef0 0x7f4050196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4055a56700 1 -- 172.21.15.63:0/1790345718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4050101ef0 msgr2=0x7f4050196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4055a56700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4050101ef0 0x7f4050196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4055a56700 1 -- 172.21.15.63:0/1790345718 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440095f0 con 0x7f40500fef10 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4055255700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4050101ef0 0x7f4050196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4055a56700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 0x7f4050196670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40440059d0 tx=0x7f404400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4042ffd700 1 -- 172.21.15.63:0/1790345718 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f404401d070 con 0x7f40500fef10 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4042ffd700 1 -- 172.21.15.63:0/1790345718 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4044005dc0 con 0x7f40500fef10 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4042ffd700 1 -- 172.21.15.63:0/1790345718 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4044021410 con 0x7f40500fef10 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405019bcc0 con 0x7f40500fef10 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.164+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405019c160 con 0x7f40500fef10 2024-05-09T06:43:41.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.165+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40501907d0 con 0x7f40500fef10 2024-05-09T06:43:41.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.166+0000 7f4042ffd700 1 -- 172.21.15.63:0/1790345718 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f404400f840 con 0x7f40500fef10 2024-05-09T06:43:41.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.166+0000 7f4042ffd700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f403c06e200 0x7f403c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:41.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.166+0000 7f4042ffd700 1 -- 172.21.15.63:0/1790345718 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f404408f4b0 con 0x7f40500fef10 2024-05-09T06:43:41.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.166+0000 7f4055255700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f403c06e200 0x7f403c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:41.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.167+0000 7f4055255700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f403c06e200 0x7f403c0706c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f404c005950 tx=0x7f404c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:41.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.171+0000 7f4042ffd700 1 -- 172.21.15.63:0/1790345718 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f404405c1f0 con 0x7f40500fef10 2024-05-09T06:43:41.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.388+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f405002cdb0 con 0x7f403c06e200 2024-05-09T06:43:41.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.399+0000 7f4042ffd700 1 -- 172.21.15.63:0/1790345718 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f405002cdb0 con 0x7f403c06e200 2024-05-09T06:43:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:43:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (12m) 42s ago 14m 25.4M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:43:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (14m) 42s ago 14m 8472k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:43:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (13m) 9s ago 13m 8116k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:43:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (2m) 42s ago 14m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:43:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (2m) 9s ago 13m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:43:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (12m) 42s ago 13m 85.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:43:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (8m) 42s ago 8m 16.5M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (8m) 42s ago 8m 90.4M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (8m) 9s ago 8m 141M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (8m) 9s ago 8m 18.0M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (4m) 42s ago 15m 508M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (3m) 9s ago 13m 452M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (3m) 42s ago 15m 48.4M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (3m) 9s ago 12m 37.8M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (14m) 42s ago 14m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (13m) 9s ago 13m 16.7M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (2m) 42s ago 12m 144M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (86s) 42s ago 11m 135M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (48s) 42s ago 11m 11.6M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (12s) 9s ago 10m 15.0M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (10m) 9s ago 10m 265M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 65036dc98b18 2024-05-09T06:43:41.380 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (10m) 9s ago 10m 267M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:43:41.381 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (4m) 42s ago 13m 58.4M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.401+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f403c06e200 msgr2=0x7f403c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.401+0000 7f4057cba700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f403c06e200 0x7f403c0706c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f404c005950 tx=0x7f404c0058e0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.401+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 msgr2=0x7f4050196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.401+0000 7f4057cba700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 0x7f4050196670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40440059d0 tx=0x7f404400be20 comp rx=0 tx=0).stop 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.401+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 shutdown_connections 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.401+0000 7f4057cba700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f403c06e200 0x7f403c0706c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.401+0000 7f4057cba700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40500fef10 0x7f4050196670 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.401+0000 7f4057cba700 1 --2- 172.21.15.63:0/1790345718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4050101ef0 0x7f4050196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.401+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 >> 172.21.15.63:0/1790345718 conn(0x7f40500fa870 msgr2=0x7f40501054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.402+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 shutdown_connections 2024-05-09T06:43:41.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.402+0000 7f4057cba700 1 -- 172.21.15.63:0/1790345718 wait complete. 2024-05-09T06:43:41.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.509+0000 7f9fef505700 1 -- 172.21.15.63:0/1396201328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe8100da0 msgr2=0x7f9fe8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.509+0000 7f9fef505700 1 --2- 172.21.15.63:0/1396201328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe8100da0 0x7f9fe8101200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009910 tx=0x7f9fd8009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:41.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.509+0000 7f9fef505700 1 -- 172.21.15.63:0/1396201328 shutdown_connections 2024-05-09T06:43:41.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.509+0000 7f9fef505700 1 --2- 172.21.15.63:0/1396201328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe8100da0 0x7f9fe8101200 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.509+0000 7f9fef505700 1 --2- 172.21.15.63:0/1396201328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fe80ffba0 0x7f9fe80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.509+0000 7f9fef505700 1 -- 172.21.15.63:0/1396201328 >> 172.21.15.63:0/1396201328 conn(0x7f9fe80fb120 msgr2=0x7f9fe80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:41.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.509+0000 7f9fef505700 1 -- 172.21.15.63:0/1396201328 shutdown_connections 2024-05-09T06:43:41.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.509+0000 7f9fef505700 1 -- 172.21.15.63:0/1396201328 wait complete. 2024-05-09T06:43:41.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.510+0000 7f9fef505700 1 Processor -- start 2024-05-09T06:43:41.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.510+0000 7f9fef505700 1 -- start start 2024-05-09T06:43:41.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.510+0000 7f9fef505700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe80ffba0 0x7f9fe8103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:41.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.510+0000 7f9fef505700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fe8100da0 0x7f9fe8102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:41.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.510+0000 7f9fef505700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81026c0 con 0x7f9fe80ffba0 2024-05-09T06:43:41.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.510+0000 7f9fef505700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8102800 con 0x7f9fe8100da0 2024-05-09T06:43:41.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.510+0000 7f9fed2a1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe80ffba0 0x7f9fe8103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.510+0000 7f9fed2a1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe80ffba0 0x7f9fe8103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36190/0 (socket says 172.21.15.63:36190) 2024-05-09T06:43:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.510+0000 7f9fed2a1700 1 -- 172.21.15.63:0/3680999059 learned_addr learned my addr 172.21.15.63:0/3680999059 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fecaa0700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fe8100da0 0x7f9fe8102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fed2a1700 1 -- 172.21.15.63:0/3680999059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fe8100da0 msgr2=0x7f9fe8102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fed2a1700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fe8100da0 0x7f9fe8102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fed2a1700 1 -- 172.21.15.63:0/3680999059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd80095f0 con 0x7f9fe80ffba0 2024-05-09T06:43:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fed2a1700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe80ffba0 0x7f9fe8103ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c8f0 tx=0x7f9fe400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fde7fc700 1 -- 172.21.15.63:0/3680999059 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe4007a70 con 0x7f9fe80ffba0 2024-05-09T06:43:41.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe8102ae0 con 0x7f9fe80ffba0 2024-05-09T06:43:41.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fde7fc700 1 -- 172.21.15.63:0/3680999059 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9fe4007bd0 con 0x7f9fe80ffba0 2024-05-09T06:43:41.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe819e710 con 0x7f9fe80ffba0 2024-05-09T06:43:41.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.511+0000 7f9fde7fc700 1 -- 172.21.15.63:0/3680999059 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe4018630 con 0x7f9fe80ffba0 2024-05-09T06:43:41.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.512+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe804ebb0 con 0x7f9fe80ffba0 2024-05-09T06:43:41.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.513+0000 7f9fde7fc700 1 -- 172.21.15.63:0/3680999059 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9fe400f450 con 0x7f9fe80ffba0 2024-05-09T06:43:41.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.513+0000 7f9fde7fc700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f9fd406e200 0x7f9fd40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:41.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.513+0000 7f9fde7fc700 1 -- 172.21.15.63:0/3680999059 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f9fe408e280 con 0x7f9fe80ffba0 2024-05-09T06:43:41.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.513+0000 7f9fecaa0700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f9fd406e200 0x7f9fd40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:41.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.514+0000 7f9fecaa0700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f9fd406e200 0x7f9fd40706c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800b5c0 tx=0x7f9fd8005160 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:41.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.518+0000 7f9fde7fc700 1 -- 172.21.15.63:0/3680999059 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9fe405b070 con 0x7f9fe80ffba0 2024-05-09T06:43:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.814+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9fe819ea20 con 0x7f9fe80ffba0 2024-05-09T06:43:41.794 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:41 smithi063 ceph-mon[92765]: pgmap v137: 65 pgs: 7 peering, 8 active+undersized, 11 active+undersized+degraded, 39 active+clean; 205 MiB data, 1023 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 32/228 objects degraded (14.035%) 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.815+0000 7f9fde7fc700 1 -- 172.21.15.63:0/3680999059 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f9fe405ac00 con 0x7f9fe80ffba0 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:43:41.795 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2, 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 4 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 6, 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 8 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:43:41.796 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f9fd406e200 msgr2=0x7f9fd40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f9fd406e200 0x7f9fd40706c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800b5c0 tx=0x7f9fd8005160 comp rx=0 tx=0).stop 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe80ffba0 msgr2=0x7f9fe8103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe80ffba0 0x7f9fe8103ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c8f0 tx=0x7f9fe400ccb0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 shutdown_connections 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f9fd406e200 0x7f9fd40706c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fe80ffba0 0x7f9fe8103ae0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 --2- 172.21.15.63:0/3680999059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fe8100da0 0x7f9fe8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 >> 172.21.15.63:0/3680999059 conn(0x7f9fe80fb120 msgr2=0x7f9fe8068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 shutdown_connections 2024-05-09T06:43:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.818+0000 7f9fef505700 1 -- 172.21.15.63:0/3680999059 wait complete. 2024-05-09T06:43:41.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:41 smithi193 ceph-mon[81665]: pgmap v137: 65 pgs: 7 peering, 8 active+undersized, 11 active+undersized+degraded, 39 active+clean; 205 MiB data, 1023 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 32/228 objects degraded (14.035%) 2024-05-09T06:43:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.927+0000 7f49a9ccf700 1 -- 172.21.15.63:0/2782646934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 msgr2=0x7f49a41022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.927+0000 7f49a9ccf700 1 --2- 172.21.15.63:0/2782646934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 0x7f49a41022b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f498c009910 tx=0x7f498c009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.927+0000 7f49a9ccf700 1 -- 172.21.15.63:0/2782646934 shutdown_connections 2024-05-09T06:43:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.927+0000 7f49a9ccf700 1 --2- 172.21.15.63:0/2782646934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 0x7f49a41022b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.927+0000 7f49a9ccf700 1 --2- 172.21.15.63:0/2782646934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a4100c50 0x7f49a4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.927+0000 7f49a9ccf700 1 -- 172.21.15.63:0/2782646934 >> 172.21.15.63:0/2782646934 conn(0x7f49a40fc210 msgr2=0x7f49a40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.927+0000 7f49a9ccf700 1 -- 172.21.15.63:0/2782646934 shutdown_connections 2024-05-09T06:43:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.927+0000 7f49a9ccf700 1 -- 172.21.15.63:0/2782646934 wait complete. 2024-05-09T06:43:41.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f49a9ccf700 1 Processor -- start 2024-05-09T06:43:41.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f49a9ccf700 1 -- start start 2024-05-09T06:43:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f49a9ccf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a4100c50 0x7f49a4196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f49a9ccf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 0x7f49a4196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f49a9ccf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a4197170 con 0x7f49a4101e50 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f49a9ccf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a41972b0 con 0x7f49a4100c50 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f499affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 0x7f49a4196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f499affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 0x7f49a4196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36202/0 (socket says 172.21.15.63:36202) 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f499affd700 1 -- 172.21.15.63:0/1307561394 learned_addr learned my addr 172.21.15.63:0/1307561394 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f49a37fe700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a4100c50 0x7f49a4196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f499affd700 1 -- 172.21.15.63:0/1307561394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a4100c50 msgr2=0x7f49a4196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f499affd700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a4100c50 0x7f49a4196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.928+0000 7f499affd700 1 -- 172.21.15.63:0/1307561394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0095f0 con 0x7f49a4101e50 2024-05-09T06:43:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.929+0000 7f499affd700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 0x7f49a4196b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f498c006010 tx=0x7f498c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.929+0000 7f49a17fa700 1 -- 172.21.15.63:0/1307561394 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f498c01d070 con 0x7f49a4101e50 2024-05-09T06:43:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.929+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a419bc60 con 0x7f49a4101e50 2024-05-09T06:43:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.929+0000 7f49a17fa700 1 -- 172.21.15.63:0/1307561394 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f498c00f460 con 0x7f49a4101e50 2024-05-09T06:43:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.929+0000 7f49a17fa700 1 -- 172.21.15.63:0/1307561394 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f498c021620 con 0x7f49a4101e50 2024-05-09T06:43:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.929+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a419c150 con 0x7f49a4101e50 2024-05-09T06:43:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.930+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a41907d0 con 0x7f49a4101e50 2024-05-09T06:43:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.930+0000 7f49a17fa700 1 -- 172.21.15.63:0/1307561394 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f498c00f5d0 con 0x7f49a4101e50 2024-05-09T06:43:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.931+0000 7f49a17fa700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f499006e200 0x7f49900706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.931+0000 7f49a17fa700 1 -- 172.21.15.63:0/1307561394 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f498c08f510 con 0x7f49a4101e50 2024-05-09T06:43:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.931+0000 7f49a37fe700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f499006e200 0x7f49900706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.931+0000 7f49a37fe700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f499006e200 0x7f49900706c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49a4101cb0 tx=0x7f4994007400 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:41.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:41.935+0000 7f49a17fa700 1 -- 172.21.15.63:0/1307561394 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f498c05c250 con 0x7f49a4101e50 2024-05-09T06:43:42.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.191+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f49a404ebb0 con 0x7f49a4101e50 2024-05-09T06:43:42.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.191+0000 7f49a17fa700 1 -- 172.21.15.63:0/1307561394 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1905 (secure 0 0 0) 0x7f498c05bde0 con 0x7f49a4101e50 2024-05-09T06:43:42.171 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:43:42.171 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:43:42.171 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:43:42.171 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:43:42.171 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:43:42.171 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:43:42.171 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:43:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:43:42.173 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:43:42.174 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:43:42.174 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:43:42.174 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:43:42.174 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:43:42.174 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f499006e200 msgr2=0x7f49900706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f499006e200 0x7f49900706c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49a4101cb0 tx=0x7f4994007400 comp rx=0 tx=0).stop 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 msgr2=0x7f49a4196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 0x7f49a4196b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f498c006010 tx=0x7f498c00bd70 comp rx=0 tx=0).stop 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 shutdown_connections 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f499006e200 0x7f49900706c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a4100c50 0x7f49a4196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 --2- 172.21.15.63:0/1307561394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49a4101e50 0x7f49a4196b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 >> 172.21.15.63:0/1307561394 conn(0x7f49a40fc210 msgr2=0x7f49a41053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 shutdown_connections 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.193+0000 7f49a9ccf700 1 -- 172.21.15.63:0/1307561394 wait complete. 2024-05-09T06:43:42.175 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:43:42.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 -- 172.21.15.63:0/279916941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 msgr2=0x7f87c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 --2- 172.21.15.63:0/279916941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 0x7f87c0101200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009910 tx=0x7f87a8009c20 comp rx=0 tx=0).stop 2024-05-09T06:43:42.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 -- 172.21.15.63:0/279916941 shutdown_connections 2024-05-09T06:43:42.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 --2- 172.21.15.63:0/279916941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 0x7f87c0101200 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 --2- 172.21.15.63:0/279916941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c00ffba0 0x7f87c00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 -- 172.21.15.63:0/279916941 >> 172.21.15.63:0/279916941 conn(0x7f87c00fb120 msgr2=0x7f87c00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 -- 172.21.15.63:0/279916941 shutdown_connections 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 -- 172.21.15.63:0/279916941 wait complete. 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 Processor -- start 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.304+0000 7f87c68cb700 1 -- start start 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87c68cb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c00ffba0 0x7f87c01966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87c68cb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 0x7f87c0196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87c68cb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0197210 con 0x7f87c0100da0 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87c68cb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0197350 con 0x7f87c00ffba0 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87b7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 0x7f87c0196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87b7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 0x7f87c0196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36230/0 (socket says 172.21.15.63:36230) 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87b7fff700 1 -- 172.21.15.63:0/864691455 learned_addr learned my addr 172.21.15.63:0/864691455 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:42.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87bffff700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c00ffba0 0x7f87c01966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87b7fff700 1 -- 172.21.15.63:0/864691455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c00ffba0 msgr2=0x7f87c01966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87b7fff700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c00ffba0 0x7f87c01966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87b7fff700 1 -- 172.21.15.63:0/864691455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80095f0 con 0x7f87c0100da0 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87b7fff700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 0x7f87c0196bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87bdffb700 1 -- 172.21.15.63:0/864691455 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a801d070 con 0x7f87c0100da0 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.305+0000 7f87bdffb700 1 -- 172.21.15.63:0/864691455 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87a8005dc0 con 0x7f87c0100da0 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.306+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c019bd50 con 0x7f87c0100da0 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.306+0000 7f87bdffb700 1 -- 172.21.15.63:0/864691455 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a8021410 con 0x7f87c0100da0 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.306+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c019c240 con 0x7f87c0100da0 2024-05-09T06:43:42.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.306+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c004ebb0 con 0x7f87c0100da0 2024-05-09T06:43:42.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.307+0000 7f87bdffb700 1 -- 172.21.15.63:0/864691455 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f87a800f460 con 0x7f87c0100da0 2024-05-09T06:43:42.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.308+0000 7f87bdffb700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f87ac06e210 0x7f87ac0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:42.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.308+0000 7f87bdffb700 1 -- 172.21.15.63:0/864691455 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f87a808f3d0 con 0x7f87c0100da0 2024-05-09T06:43:42.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.308+0000 7f87bffff700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f87ac06e210 0x7f87ac0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:42.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.308+0000 7f87bffff700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f87ac06e210 0x7f87ac0706d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87c0100c00 tx=0x7f87b000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:42.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.312+0000 7f87bdffb700 1 -- 172.21.15.63:0/864691455 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f87a805c190 con 0x7f87c0100da0 2024-05-09T06:43:42.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.543+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87c0068d10 con 0x7f87ac06e210 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.544+0000 7f87bdffb700 1 -- 172.21.15.63:0/864691455 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f87c0068d10 con 0x7f87ac06e210 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "10/23 daemons upgraded", 2024-05-09T06:43:42.524 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-05-09T06:43:42.525 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:43:42.525 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:43:42.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f87ac06e210 msgr2=0x7f87ac0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f87ac06e210 0x7f87ac0706d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87c0100c00 tx=0x7f87b000b410 comp rx=0 tx=0).stop 2024-05-09T06:43:42.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 msgr2=0x7f87c0196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 0x7f87c0196bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a800bd70 comp rx=0 tx=0).stop 2024-05-09T06:43:42.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 shutdown_connections 2024-05-09T06:43:42.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f87ac06e210 0x7f87ac0706d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c00ffba0 0x7f87c01966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 --2- 172.21.15.63:0/864691455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c0100da0 0x7f87c0196bf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 >> 172.21.15.63:0/864691455 conn(0x7f87c00fb120 msgr2=0x7f87c0068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:42.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 shutdown_connections 2024-05-09T06:43:42.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.546+0000 7f87c68cb700 1 -- 172.21.15.63:0/864691455 wait complete. 2024-05-09T06:43:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.654+0000 7fb418132700 1 -- 172.21.15.63:0/2769699964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 msgr2=0x7fb4100fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.654+0000 7fb418132700 1 --2- 172.21.15.63:0/2769699964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 0x7fb4100fffc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb40000b0a0 tx=0x7fb40000b3b0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.654+0000 7fb418132700 1 -- 172.21.15.63:0/2769699964 shutdown_connections 2024-05-09T06:43:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.654+0000 7fb418132700 1 --2- 172.21.15.63:0/2769699964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb410100da0 0x7fb410101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.654+0000 7fb418132700 1 --2- 172.21.15.63:0/2769699964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 0x7fb4100fffc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.654+0000 7fb418132700 1 -- 172.21.15.63:0/2769699964 >> 172.21.15.63:0/2769699964 conn(0x7fb4100fb120 msgr2=0x7fb4100fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.654+0000 7fb418132700 1 -- 172.21.15.63:0/2769699964 shutdown_connections 2024-05-09T06:43:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.654+0000 7fb418132700 1 -- 172.21.15.63:0/2769699964 wait complete. 2024-05-09T06:43:42.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.655+0000 7fb418132700 1 Processor -- start 2024-05-09T06:43:42.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.655+0000 7fb418132700 1 -- start start 2024-05-09T06:43:42.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.655+0000 7fb418132700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 0x7fb410069090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.655+0000 7fb418132700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb410100da0 0x7fb41019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.655+0000 7fb418132700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41019b9b0 con 0x7fb4100ffba0 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.655+0000 7fb418132700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41019baf0 con 0x7fb410100da0 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.655+0000 7fb415ece700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 0x7fb410069090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.655+0000 7fb415ece700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 0x7fb410069090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36250/0 (socket says 172.21.15.63:36250) 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.655+0000 7fb415ece700 1 -- 172.21.15.63:0/891154109 learned_addr learned my addr 172.21.15.63:0/891154109 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb4156cd700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb410100da0 0x7fb41019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb415ece700 1 -- 172.21.15.63:0/891154109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb410100da0 msgr2=0x7fb41019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb415ece700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb410100da0 0x7fb41019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb415ece700 1 -- 172.21.15.63:0/891154109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400009d00 con 0x7fb4100ffba0 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb415ece700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 0x7fb410069090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb400015040 tx=0x7fb400003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb40effd700 1 -- 172.21.15.63:0/891154109 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb400021070 con 0x7fb4100ffba0 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41019bd40 con 0x7fb4100ffba0 2024-05-09T06:43:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb40effd700 1 -- 172.21.15.63:0/891154109 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb40000ce60 con 0x7fb4100ffba0 2024-05-09T06:43:42.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb40effd700 1 -- 172.21.15.63:0/891154109 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb40000e040 con 0x7fb4100ffba0 2024-05-09T06:43:42.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.656+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41019c1d0 con 0x7fb4100ffba0 2024-05-09T06:43:42.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.657+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41004ec20 con 0x7fb4100ffba0 2024-05-09T06:43:42.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.658+0000 7fb40effd700 1 -- 172.21.15.63:0/891154109 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb4000074e0 con 0x7fb4100ffba0 2024-05-09T06:43:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.658+0000 7fb40effd700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3f806e200 0x7fb3f80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:43:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.658+0000 7fb40effd700 1 -- 172.21.15.63:0/891154109 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fb40008f3b0 con 0x7fb4100ffba0 2024-05-09T06:43:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.658+0000 7fb4156cd700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3f806e200 0x7fb3f80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:43:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.659+0000 7fb4156cd700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3f806e200 0x7fb3f80706c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb404005950 tx=0x7fb4040058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:43:42.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.663+0000 7fb40effd700 1 -- 172.21.15.63:0/891154109 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb41004ec20 con 0x7fb4100ffba0 2024-05-09T06:43:42.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:42 smithi063 ceph-mon[92765]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:42.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:42 smithi063 ceph-mon[92765]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:42.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:42 smithi063 ceph-mon[92765]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:42.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:42 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3680999059' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:42.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:42.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:43:42.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:42 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1307561394' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:43:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:42 smithi193 ceph-mon[81665]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:42 smithi193 ceph-mon[81665]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:42 smithi193 ceph-mon[81665]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:42 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3680999059' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:43:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:43:42.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:42 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1307561394' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:43:42.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.956+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb41018f720 con 0x7fb4100ffba0 2024-05-09T06:43:42.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.956+0000 7fb40effd700 1 -- 172.21.15.63:0/891154109 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb40005c0f0 con 0x7fb4100ffba0 2024-05-09T06:43:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:43:42.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.958+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3f806e200 msgr2=0x7fb3f80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.958+0000 7fb418132700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3f806e200 0x7fb3f80706c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb404005950 tx=0x7fb4040058e0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.958+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 msgr2=0x7fb410069090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:43:42.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.958+0000 7fb418132700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 0x7fb410069090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb400015040 tx=0x7fb400003ce0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.958+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 shutdown_connections 2024-05-09T06:43:42.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.958+0000 7fb418132700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb3f806e200 0x7fb3f80706c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.958+0000 7fb418132700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4100ffba0 0x7fb410069090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.958+0000 7fb418132700 1 --2- 172.21.15.63:0/891154109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb410100da0 0x7fb41019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:43:42.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.958+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 >> 172.21.15.63:0/891154109 conn(0x7fb4100fb120 msgr2=0x7fb4101043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:43:42.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.959+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 shutdown_connections 2024-05-09T06:43:42.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:43:42.959+0000 7fb418132700 1 -- 172.21.15.63:0/891154109 wait complete. 2024-05-09T06:43:43.772 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:43 smithi193 ceph-mon[81665]: pgmap v138: 65 pgs: 7 peering, 58 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 926 B/s rd, 2 op/s 2024-05-09T06:43:43.880 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:43 smithi193 ceph-mon[81665]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/228 objects degraded (14.035%), 11 pgs degraded) 2024-05-09T06:43:43.880 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:43 smithi193 ceph-mon[81665]: Cluster is now healthy 2024-05-09T06:43:43.880 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:43 smithi193 ceph-mon[81665]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:43.880 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:43 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/891154109' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:43:43.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:43 smithi063 ceph-mon[92765]: pgmap v138: 65 pgs: 7 peering, 58 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 926 B/s rd, 2 op/s 2024-05-09T06:43:43.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:43 smithi063 ceph-mon[92765]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/228 objects degraded (14.035%), 11 pgs degraded) 2024-05-09T06:43:43.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:43 smithi063 ceph-mon[92765]: Cluster is now healthy 2024-05-09T06:43:43.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:43 smithi063 ceph-mon[92765]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:43:43.881 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:43 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/891154109' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:43:45.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:45 smithi063 ceph-mon[92765]: pgmap v139: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-09T06:43:45.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:45 smithi193 ceph-mon[81665]: pgmap v139: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-09T06:43:47.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:47 smithi063 ceph-mon[92765]: pgmap v140: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-09T06:43:47.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:47 smithi193 ceph-mon[81665]: pgmap v140: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-09T06:43:49.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:49 smithi063 ceph-mon[92765]: pgmap v141: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-09T06:43:49.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:49 smithi193 ceph-mon[81665]: pgmap v141: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-09T06:43:51.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:51 smithi063 ceph-mon[92765]: pgmap v142: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 954 B/s rd, 1 op/s 2024-05-09T06:43:51.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:51 smithi193 ceph-mon[81665]: pgmap v142: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 954 B/s rd, 1 op/s 2024-05-09T06:43:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-09T06:43:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:52.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-09T06:43:52.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:53.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:53 smithi193 ceph-mon[81665]: pgmap v143: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-09T06:43:53.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:53 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-09T06:43:53.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:53 smithi193 ceph-mon[81665]: Upgrade: osd.4 is safe to restart 2024-05-09T06:43:53.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:53 smithi063 ceph-mon[92765]: pgmap v143: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-09T06:43:53.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:53 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-09T06:43:53.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:53 smithi063 ceph-mon[92765]: Upgrade: osd.4 is safe to restart 2024-05-09T06:43:55.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:55 smithi193 ceph-mon[81665]: pgmap v144: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:43:55.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:55 smithi193 ceph-mon[81665]: Upgrade: Updating osd.4 2024-05-09T06:43:55.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:55.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T06:43:55.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:43:55.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:55 smithi193 ceph-mon[81665]: Deploying daemon osd.4 on smithi193 2024-05-09T06:43:55.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:55 smithi063 ceph-mon[92765]: pgmap v144: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:43:55.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:55 smithi063 ceph-mon[92765]: Upgrade: Updating osd.4 2024-05-09T06:43:55.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:55.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T06:43:55.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:43:55.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:55 smithi063 ceph-mon[92765]: Deploying daemon osd.4 on smithi193 2024-05-09T06:43:57.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:57 smithi193 ceph-mon[81665]: pgmap v145: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:43:57.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:57.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:43:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:57 smithi063 ceph-mon[92765]: pgmap v145: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:43:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:43:57.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:43:57.717 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:57 smithi193 systemd[1]: Stopping Ceph osd.4 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:43:58.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:57 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[48066]: 2024-05-09T06:43:57.737+0000 7f484baf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:43:58.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:57 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[48066]: 2024-05-09T06:43:57.737+0000 7f484baf4700 -1 osd.4 80 *** Got signal Terminated *** 2024-05-09T06:43:58.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:57 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[48066]: 2024-05-09T06:43:57.737+0000 7f484baf4700 -1 osd.4 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:43:58.564 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:58 smithi193 ceph-mon[81665]: osd.4 marked itself down and dead 2024-05-09T06:43:58.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:58 smithi063 ceph-mon[92765]: osd.4 marked itself down and dead 2024-05-09T06:43:58.825 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:58 smithi193 podman[95892]: 2024-05-09 06:43:58.58617619 +0000 UTC m=+0.976413038 container died 65036dc98b18d407979e76f7337bd83573ea96598c999f6af638ebacff237150 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, RELEASE=reef-1758e8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:43:59.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:58 smithi193 podman[95892]: 2024-05-09 06:43:58.946089291 +0000 UTC m=+1.336326130 container cleanup 65036dc98b18d407979e76f7337bd83573ea96598c999f6af638ebacff237150 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=reef-1758e8e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T06:43:59.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:58 smithi193 bash[95892]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4 2024-05-09T06:43:59.520 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:59 smithi193 ceph-mon[81665]: pgmap v146: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:43:59.520 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:59 smithi193 ceph-mon[81665]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:43:59.520 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:43:59 smithi193 ceph-mon[81665]: osdmap e81: 6 total, 5 up, 6 in 2024-05-09T06:43:59.520 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:59 smithi193 podman[95910]: 2024-05-09 06:43:59.239066032 +0000 UTC m=+0.655411848 container remove 65036dc98b18d407979e76f7337bd83573ea96598c999f6af638ebacff237150 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4, CEPH_POINT_RELEASE=, RELEASE=reef-1758e8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:43:59.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:59 smithi063 ceph-mon[92765]: pgmap v146: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:43:59.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:59 smithi063 ceph-mon[92765]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:43:59.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:43:59 smithi063 ceph-mon[92765]: osdmap e81: 6 total, 5 up, 6 in 2024-05-09T06:43:59.822 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:59 smithi193 podman[96002]: 2024-05-09 06:43:59.444582973 +0000 UTC m=+0.018384797 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:43:59.822 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:59 smithi193 podman[96002]: 2024-05-09 06:43:59.573302812 +0000 UTC m=+0.147104627 container create 3f195e17fee0d7519fd1509759ac4252d89d03a3c4237a30d3fd85002c5c02d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T06:43:59.822 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:59 smithi193 podman[96002]: 2024-05-09 06:43:59.842325623 +0000 UTC m=+0.416127435 container init 3f195e17fee0d7519fd1509759ac4252d89d03a3c4237a30d3fd85002c5c02d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-09T06:44:00.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:59 smithi193 podman[96002]: 2024-05-09 06:43:59.845675392 +0000 UTC m=+0.419477204 container start 3f195e17fee0d7519fd1509759ac4252d89d03a3c4237a30d3fd85002c5c02d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-09T06:44:00.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:43:59 smithi193 podman[96002]: 2024-05-09 06:43:59.917259246 +0000 UTC m=+0.491061076 container attach 3f195e17fee0d7519fd1509759ac4252d89d03a3c4237a30d3fd85002c5c02d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T06:44:00.408 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:00 smithi193 ceph-mon[81665]: osdmap e82: 6 total, 5 up, 6 in 2024-05-09T06:44:00.408 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:00 smithi193 podman[96002]: 2024-05-09 06:44:00.144182823 +0000 UTC m=+0.717984638 container died 3f195e17fee0d7519fd1509759ac4252d89d03a3c4237a30d3fd85002c5c02d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T06:44:00.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:00 smithi063 ceph-mon[92765]: osdmap e82: 6 total, 5 up, 6 in 2024-05-09T06:44:01.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:00 smithi193 podman[96028]: 2024-05-09 06:44:00.696040374 +0000 UTC m=+0.545353749 container remove 3f195e17fee0d7519fd1509759ac4252d89d03a3c4237a30d3fd85002c5c02d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T06:44:01.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:00 smithi193 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-09T06:44:01.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:00 smithi193 systemd[1]: Stopped Ceph osd.4 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:44:01.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:00 smithi193 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.4.service: Consumed 1min 8.484s CPU time. 2024-05-09T06:44:01.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:00 smithi193 systemd[1]: Starting Ceph osd.4 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:44:01.378 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:01 smithi193 ceph-mon[81665]: pgmap v149: 65 pgs: 5 active+undersized, 3 active+undersized+degraded, 3 stale+active+clean, 54 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 7/228 objects degraded (3.070%) 2024-05-09T06:44:01.379 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:01 smithi193 ceph-mon[81665]: Health check failed: Degraded data redundancy: 7/228 objects degraded (3.070%), 3 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:01.379 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 podman[96130]: 2024-05-09 06:44:01.046413595 +0000 UTC m=+0.018067051 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:44:01.379 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 podman[96130]: 2024-05-09 06:44:01.147130052 +0000 UTC m=+0.118783492 container create 13757876513b055ff7bb2b0a27154362243453d7dab796727e5da952451bc11e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:44:01.379 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 podman[96130]: 2024-05-09 06:44:01.399699436 +0000 UTC m=+0.371352877 container init 13757876513b055ff7bb2b0a27154362243453d7dab796727e5da952451bc11e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:44:01.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:01 smithi063 ceph-mon[92765]: pgmap v149: 65 pgs: 5 active+undersized, 3 active+undersized+degraded, 3 stale+active+clean, 54 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 7/228 objects degraded (3.070%) 2024-05-09T06:44:01.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:01 smithi063 ceph-mon[92765]: Health check failed: Degraded data redundancy: 7/228 objects degraded (3.070%), 3 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:01.779 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 podman[96130]: 2024-05-09 06:44:01.403166923 +0000 UTC m=+0.374820373 container start 13757876513b055ff7bb2b0a27154362243453d7dab796727e5da952451bc11e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:44:01.780 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 podman[96130]: 2024-05-09 06:44:01.499615269 +0000 UTC m=+0.471268727 container attach 13757876513b055ff7bb2b0a27154362243453d7dab796727e5da952451bc11e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-05-09T06:44:02.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate[96146]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T06:44:02.121 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate[96146]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-09T06:44:02.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 bash[96130]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T06:44:02.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 bash[96130]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-09T06:44:02.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 bash[96130]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-09T06:44:02.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 bash[96130]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-09T06:44:02.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 bash[96130]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-09T06:44:02.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 bash[96130]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T06:44:02.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 bash[96130]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-09T06:44:02.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate[96146]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-09T06:44:02.122 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate[96146]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-09T06:44:02.123 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate[96146]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-09T06:44:02.123 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate[96146]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T06:44:02.123 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate[96146]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-09T06:44:02.123 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:01 smithi193 podman[96130]: 2024-05-09 06:44:01.874968021 +0000 UTC m=+0.846621463 container died 13757876513b055ff7bb2b0a27154362243453d7dab796727e5da952451bc11e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T06:44:02.840 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:02 smithi193 podman[96276]: 2024-05-09 06:44:02.512112894 +0000 UTC m=+0.630423374 container remove 13757876513b055ff7bb2b0a27154362243453d7dab796727e5da952451bc11e (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-activate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T06:44:02.841 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:02 smithi193 podman[96342]: 2024-05-09 06:44:02.63064655 +0000 UTC m=+0.017893229 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:44:02.841 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:02 smithi193 podman[96342]: 2024-05-09 06:44:02.73813961 +0000 UTC m=+0.125386293 container create 14c9b4920f5a3effa6e414b14a412559f4d5072be4ad4006cbc95c05d2a3c723 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T06:44:03.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:03 smithi193 ceph-mon[81665]: pgmap v150: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 1 stale+active+clean, 37 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 33/228 objects degraded (14.474%) 2024-05-09T06:44:03.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:03.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:03.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:03.372 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:02 smithi193 podman[96342]: 2024-05-09 06:44:02.981934759 +0000 UTC m=+0.369181447 container init 14c9b4920f5a3effa6e414b14a412559f4d5072be4ad4006cbc95c05d2a3c723 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:44:03.372 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:02 smithi193 podman[96342]: 2024-05-09 06:44:02.985159777 +0000 UTC m=+0.372406457 container start 14c9b4920f5a3effa6e414b14a412559f4d5072be4ad4006cbc95c05d2a3c723 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T06:44:03.372 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:03 smithi193 ceph-osd[96361]: -- 172.21.15.193:0/3413785778 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c06bc1680 con 0x557c06b8ac00 2024-05-09T06:44:03.372 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:03 smithi193 bash[96342]: 14c9b4920f5a3effa6e414b14a412559f4d5072be4ad4006cbc95c05d2a3c723 2024-05-09T06:44:03.372 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:03 smithi193 systemd[1]: Started Ceph osd.4 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:44:03.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:03 smithi063 ceph-mon[92765]: pgmap v150: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 1 stale+active+clean, 37 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 33/228 objects degraded (14.474%) 2024-05-09T06:44:03.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:03.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:03.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:03.871 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:03 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:44:03.523+0000 7f0a66d8c7c0 -1 Falling back to public interface 2024-05-09T06:44:05.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:05 smithi063 ceph-mon[92765]: pgmap v151: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 40/228 objects degraded (17.544%) 2024-05-09T06:44:05.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:05 smithi193 ceph-mon[81665]: pgmap v151: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 40/228 objects degraded (17.544%) 2024-05-09T06:44:07.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:06 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:07.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:06 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:07.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:06 smithi193 ceph-mon[81665]: pgmap v152: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 40/228 objects degraded (17.544%) 2024-05-09T06:44:07.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:06 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:07.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:06 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:07.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:06 smithi063 ceph-mon[92765]: pgmap v152: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 40/228 objects degraded (17.544%) 2024-05-09T06:44:08.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:08 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:08.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:08 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:08.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:08 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:08.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:08 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:08.740 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:08 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:44:08.612+0000 7f0a66d8c7c0 -1 osd.4 80 log_to_monitors true 2024-05-09T06:44:09.336 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:44:09 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:44:09.118+0000 7f0a541db700 -1 osd.4 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:44:09.337 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:09 smithi193 ceph-mon[81665]: pgmap v153: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 514 B/s rd, 1 op/s; 40/228 objects degraded (17.544%) 2024-05-09T06:44:09.337 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:09 smithi193 ceph-mon[81665]: Health check update: Degraded data redundancy: 40/228 objects degraded (17.544%), 15 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:09.337 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:09 smithi193 ceph-mon[81665]: from='osd.4 [v2:172.21.15.193:6808/909880686,v1:172.21.15.193:6809/909880686]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T06:44:09.337 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:09 smithi193 ceph-mon[81665]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T06:44:09.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:09 smithi063 ceph-mon[92765]: pgmap v153: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 514 B/s rd, 1 op/s; 40/228 objects degraded (17.544%) 2024-05-09T06:44:09.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:09 smithi063 ceph-mon[92765]: Health check update: Degraded data redundancy: 40/228 objects degraded (17.544%), 15 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:09.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:09 smithi063 ceph-mon[92765]: from='osd.4 [v2:172.21.15.193:6808/909880686,v1:172.21.15.193:6809/909880686]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T06:44:09.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:09 smithi063 ceph-mon[92765]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T06:44:10.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:10 smithi063 ceph-mon[92765]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T06:44:10.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:10 smithi063 ceph-mon[92765]: from='osd.4 [v2:172.21.15.193:6808/909880686,v1:172.21.15.193:6809/909880686]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:44:10.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:10 smithi063 ceph-mon[92765]: osdmap e83: 6 total, 5 up, 6 in 2024-05-09T06:44:10.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:10 smithi063 ceph-mon[92765]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:44:10.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:10 smithi193 ceph-mon[81665]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T06:44:10.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:10 smithi193 ceph-mon[81665]: from='osd.4 [v2:172.21.15.193:6808/909880686,v1:172.21.15.193:6809/909880686]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:44:10.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:10 smithi193 ceph-mon[81665]: osdmap e83: 6 total, 5 up, 6 in 2024-05-09T06:44:10.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:10 smithi193 ceph-mon[81665]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:44:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:44:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: osd.4 [v2:172.21.15.193:6808/909880686,v1:172.21.15.193:6809/909880686] boot 2024-05-09T06:44:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: osdmap e84: 6 total, 6 up, 6 in 2024-05-09T06:44:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:44:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: pgmap v156: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 40/228 objects degraded (17.544%) 2024-05-09T06:44:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:44:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-09T06:44:11.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:11 smithi063 ceph-mon[92765]: osdmap e85: 6 total, 6 up, 6 in 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: osd.4 [v2:172.21.15.193:6808/909880686,v1:172.21.15.193:6809/909880686] boot 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: osdmap e84: 6 total, 6 up, 6 in 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: pgmap v156: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 40/228 objects degraded (17.544%) 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:44:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-09T06:44:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:11 smithi193 ceph-mon[81665]: osdmap e85: 6 total, 6 up, 6 in 2024-05-09T06:44:12.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:12 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-09T06:44:12.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:12 smithi063 ceph-mon[92765]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-09T06:44:12.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:12 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-09T06:44:12.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:12 smithi193 ceph-mon[81665]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-09T06:44:13.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.084+0000 7f566308a700 1 -- 172.21.15.63:0/3487346132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 msgr2=0x7f565c1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.084+0000 7f566308a700 1 --2- 172.21.15.63:0/3487346132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 0x7f565c1022a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f564c009910 tx=0x7f564c009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:13.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.084+0000 7f566308a700 1 -- 172.21.15.63:0/3487346132 shutdown_connections 2024-05-09T06:44:13.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.084+0000 7f566308a700 1 --2- 172.21.15.63:0/3487346132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 0x7f565c1022a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.084+0000 7f566308a700 1 --2- 172.21.15.63:0/3487346132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c100c40 0x7f565c101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.084+0000 7f566308a700 1 -- 172.21.15.63:0/3487346132 >> 172.21.15.63:0/3487346132 conn(0x7f565c0fc1e0 msgr2=0x7f565c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:13.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.084+0000 7f566308a700 1 -- 172.21.15.63:0/3487346132 shutdown_connections 2024-05-09T06:44:13.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.084+0000 7f566308a700 1 -- 172.21.15.63:0/3487346132 wait complete. 2024-05-09T06:44:13.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f566308a700 1 Processor -- start 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f566308a700 1 -- start start 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f566308a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c100c40 0x7f565c196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f566308a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 0x7f565c196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f566308a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c197120 con 0x7f565c101e40 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f566308a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c197260 con 0x7f565c100c40 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f5653fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 0x7f565c196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f5653fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 0x7f565c196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55782/0 (socket says 172.21.15.63:55782) 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f5653fff700 1 -- 172.21.15.63:0/1354246781 learned_addr learned my addr 172.21.15.63:0/1354246781 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.085+0000 7f5660e26700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c100c40 0x7f565c196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.086+0000 7f5653fff700 1 -- 172.21.15.63:0/1354246781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c100c40 msgr2=0x7f565c196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.086+0000 7f5653fff700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c100c40 0x7f565c196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.086+0000 7f5653fff700 1 -- 172.21.15.63:0/1354246781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c0095f0 con 0x7f565c101e40 2024-05-09T06:44:13.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.086+0000 7f5653fff700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 0x7f565c196b90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f564c006010 tx=0x7f564c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:13.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.086+0000 7f5651ffb700 1 -- 172.21.15.63:0/1354246781 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f564c01d070 con 0x7f565c101e40 2024-05-09T06:44:13.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.086+0000 7f5651ffb700 1 -- 172.21.15.63:0/1354246781 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f564c00f460 con 0x7f565c101e40 2024-05-09T06:44:13.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.086+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565c19bc20 con 0x7f565c101e40 2024-05-09T06:44:13.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.086+0000 7f5651ffb700 1 -- 172.21.15.63:0/1354246781 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f564c021620 con 0x7f565c101e40 2024-05-09T06:44:13.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.086+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c19c160 con 0x7f565c101e40 2024-05-09T06:44:13.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.087+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565c1907c0 con 0x7f565c101e40 2024-05-09T06:44:13.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.088+0000 7f5651ffb700 1 -- 172.21.15.63:0/1354246781 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f564c00f5d0 con 0x7f565c101e40 2024-05-09T06:44:13.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.088+0000 7f5651ffb700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f564406e200 0x7f56440706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:13.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.089+0000 7f5651ffb700 1 -- 172.21.15.63:0/1354246781 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f564c08f4e0 con 0x7f565c101e40 2024-05-09T06:44:13.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.089+0000 7f5660e26700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f564406e200 0x7f56440706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:13.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.089+0000 7f5660e26700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f564406e200 0x7f56440706c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f565c101ca0 tx=0x7f565800a3b0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:13.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.093+0000 7f5651ffb700 1 -- 172.21.15.63:0/1354246781 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f564c05c2a0 con 0x7f565c101e40 2024-05-09T06:44:13.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.327+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f565c02ce50 con 0x7f564406e200 2024-05-09T06:44:13.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.329+0000 7f5651ffb700 1 -- 172.21.15.63:0/1354246781 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f565c02ce50 con 0x7f564406e200 2024-05-09T06:44:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f564406e200 msgr2=0x7f56440706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f564406e200 0x7f56440706c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f565c101ca0 tx=0x7f565800a3b0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 msgr2=0x7f565c196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 0x7f565c196b90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f564c006010 tx=0x7f564c00bd70 comp rx=0 tx=0).stop 2024-05-09T06:44:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 shutdown_connections 2024-05-09T06:44:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f564406e200 0x7f56440706c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c100c40 0x7f565c196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 --2- 172.21.15.63:0/1354246781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f565c101e40 0x7f565c196b90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 >> 172.21.15.63:0/1354246781 conn(0x7f565c0fc1e0 msgr2=0x7f565c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:13.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 shutdown_connections 2024-05-09T06:44:13.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.331+0000 7f566308a700 1 -- 172.21.15.63:0/1354246781 wait complete. 2024-05-09T06:44:13.322 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:44:13.322 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:13 smithi063 ceph-mon[92765]: pgmap v158: 65 pgs: 9 peering, 10 active+undersized, 5 active+undersized+degraded, 41 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 14/228 objects degraded (6.140%) 2024-05-09T06:44:13.323 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:13.323 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:44:13.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.443+0000 7f0b52a74700 1 -- 172.21.15.63:0/1690463416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c0ffb50 msgr2=0x7f0b4c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.443+0000 7f0b52a74700 1 --2- 172.21.15.63:0/1690463416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c0ffb50 0x7f0b4c0fff70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00b0a0 tx=0x7f0b3c00b3b0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.443+0000 7f0b52a74700 1 -- 172.21.15.63:0/1690463416 shutdown_connections 2024-05-09T06:44:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.443+0000 7f0b52a74700 1 --2- 172.21.15.63:0/1690463416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b4c100d50 0x7f0b4c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.443+0000 7f0b52a74700 1 --2- 172.21.15.63:0/1690463416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c0ffb50 0x7f0b4c0fff70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.443+0000 7f0b52a74700 1 -- 172.21.15.63:0/1690463416 >> 172.21.15.63:0/1690463416 conn(0x7f0b4c0fb0d0 msgr2=0x7f0b4c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.443+0000 7f0b52a74700 1 -- 172.21.15.63:0/1690463416 shutdown_connections 2024-05-09T06:44:13.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.443+0000 7f0b52a74700 1 -- 172.21.15.63:0/1690463416 wait complete. 2024-05-09T06:44:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b52a74700 1 Processor -- start 2024-05-09T06:44:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b52a74700 1 -- start start 2024-05-09T06:44:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b52a74700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b4c0ffb50 0x7f0b4c195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b52a74700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c100d50 0x7f0b4c195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b52a74700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c196090 con 0x7f0b4c100d50 2024-05-09T06:44:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b52a74700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c1961d0 con 0x7f0b4c0ffb50 2024-05-09T06:44:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b4bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c100d50 0x7f0b4c195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b4bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c100d50 0x7f0b4c195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55798/0 (socket says 172.21.15.63:55798) 2024-05-09T06:44:13.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b4bfff700 1 -- 172.21.15.63:0/1753118115 learned_addr learned my addr 172.21.15.63:0/1753118115 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:13.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b50810700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b4c0ffb50 0x7f0b4c195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:13.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b4bfff700 1 -- 172.21.15.63:0/1753118115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b4c0ffb50 msgr2=0x7f0b4c195530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b4bfff700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b4c0ffb50 0x7f0b4c195530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b4bfff700 1 -- 172.21.15.63:0/1753118115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c009d00 con 0x7f0b4c100d50 2024-05-09T06:44:13.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.444+0000 7f0b50810700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b4c0ffb50 0x7f0b4c195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:44:13.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.445+0000 7f0b4bfff700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c100d50 0x7f0b4c195a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400eab0 tx=0x7f0b3400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:13.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.445+0000 7f0b49ffb700 1 -- 172.21.15.63:0/1753118115 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b34013070 con 0x7f0b4c100d50 2024-05-09T06:44:13.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.445+0000 7f0b49ffb700 1 -- 172.21.15.63:0/1753118115 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b3400bbd0 con 0x7f0b4c100d50 2024-05-09T06:44:13.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.445+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c069140 con 0x7f0b4c100d50 2024-05-09T06:44:13.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.445+0000 7f0b49ffb700 1 -- 172.21.15.63:0/1753118115 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b34016670 con 0x7f0b4c100d50 2024-05-09T06:44:13.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.445+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c069710 con 0x7f0b4c100d50 2024-05-09T06:44:13.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.446+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c04ebb0 con 0x7f0b4c100d50 2024-05-09T06:44:13.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.447+0000 7f0b49ffb700 1 -- 172.21.15.63:0/1753118115 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0b3400bd40 con 0x7f0b4c100d50 2024-05-09T06:44:13.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.447+0000 7f0b49ffb700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0b3806e200 0x7f0b380706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:13.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.447+0000 7f0b49ffb700 1 -- 172.21.15.63:0/1753118115 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f0b3408fb40 con 0x7f0b4c100d50 2024-05-09T06:44:13.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.447+0000 7f0b50810700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0b3806e200 0x7f0b380706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:13.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.448+0000 7f0b50810700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0b3806e200 0x7f0b380706c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000f80 tx=0x7f0b3c00b720 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:13.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.452+0000 7f0b49ffb700 1 -- 172.21.15.63:0/1753118115 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0b3405c930 con 0x7f0b4c100d50 2024-05-09T06:44:13.539 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:13 smithi193 ceph-mon[81665]: pgmap v158: 65 pgs: 9 peering, 10 active+undersized, 5 active+undersized+degraded, 41 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 14/228 objects degraded (6.140%) 2024-05-09T06:44:13.539 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:13.539 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:44:13.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.678+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b4c18f270 con 0x7f0b3806e200 2024-05-09T06:44:13.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.679+0000 7f0b49ffb700 1 -- 172.21.15.63:0/1753118115 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f0b4c18f270 con 0x7f0b3806e200 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0b3806e200 msgr2=0x7f0b380706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0b3806e200 0x7f0b380706c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000f80 tx=0x7f0b3c00b720 comp rx=0 tx=0).stop 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c100d50 msgr2=0x7f0b4c195a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c100d50 0x7f0b4c195a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400eab0 tx=0x7f0b3400ee70 comp rx=0 tx=0).stop 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 shutdown_connections 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0b3806e200 0x7f0b380706c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b4c0ffb50 0x7f0b4c195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 --2- 172.21.15.63:0/1753118115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4c100d50 0x7f0b4c195a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 >> 172.21.15.63:0/1753118115 conn(0x7f0b4c0fb0d0 msgr2=0x7f0b4c1042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 shutdown_connections 2024-05-09T06:44:13.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.681+0000 7f0b52a74700 1 -- 172.21.15.63:0/1753118115 wait complete. 2024-05-09T06:44:13.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.790+0000 7f138a9a1700 1 -- 172.21.15.63:0/1166090068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 msgr2=0x7f13841020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.790+0000 7f138a9a1700 1 --2- 172.21.15.63:0/1166090068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 0x7f13841020e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1374009910 tx=0x7f1374009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:13.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.790+0000 7f138a9a1700 1 -- 172.21.15.63:0/1166090068 shutdown_connections 2024-05-09T06:44:13.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.790+0000 7f138a9a1700 1 --2- 172.21.15.63:0/1166090068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 0x7f13841020e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.790+0000 7f138a9a1700 1 --2- 172.21.15.63:0/1166090068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1384100a80 0x7f1384100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.790+0000 7f138a9a1700 1 -- 172.21.15.63:0/1166090068 >> 172.21.15.63:0/1166090068 conn(0x7f13840fc020 msgr2=0x7f13840fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:13.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.791+0000 7f138a9a1700 1 -- 172.21.15.63:0/1166090068 shutdown_connections 2024-05-09T06:44:13.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.791+0000 7f138a9a1700 1 -- 172.21.15.63:0/1166090068 wait complete. 2024-05-09T06:44:13.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.791+0000 7f138a9a1700 1 Processor -- start 2024-05-09T06:44:13.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.791+0000 7f138a9a1700 1 -- start start 2024-05-09T06:44:13.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.791+0000 7f138a9a1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1384100a80 0x7f1384196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:13.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.791+0000 7f138a9a1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 0x7f13841969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:13.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.791+0000 7f138a9a1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1384196ff0 con 0x7f1384101c80 2024-05-09T06:44:13.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.791+0000 7f138a9a1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1384197130 con 0x7f1384100a80 2024-05-09T06:44:13.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.791+0000 7f138919e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 0x7f13841969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138999f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1384100a80 0x7f1384196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138919e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 0x7f13841969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55816/0 (socket says 172.21.15.63:55816) 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138919e700 1 -- 172.21.15.63:0/2733214633 learned_addr learned my addr 172.21.15.63:0/2733214633 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138919e700 1 -- 172.21.15.63:0/2733214633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1384100a80 msgr2=0x7f1384196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138919e700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1384100a80 0x7f1384196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138919e700 1 -- 172.21.15.63:0/2733214633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13740095f0 con 0x7f1384101c80 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138999f700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1384100a80 0x7f1384196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138919e700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 0x7f13841969d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1374005b40 tx=0x7f137400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f137affd700 1 -- 172.21.15.63:0/2733214633 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f137401d070 con 0x7f1384101c80 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f137affd700 1 -- 172.21.15.63:0/2733214633 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13740050d0 con 0x7f1384101c80 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138419bae0 con 0x7f1384101c80 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f137affd700 1 -- 172.21.15.63:0/2733214633 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1374022470 con 0x7f1384101c80 2024-05-09T06:44:13.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.792+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138419c050 con 0x7f1384101c80 2024-05-09T06:44:13.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.793+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1384190600 con 0x7f1384101c80 2024-05-09T06:44:13.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.794+0000 7f137affd700 1 -- 172.21.15.63:0/2733214633 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f13740037f0 con 0x7f1384101c80 2024-05-09T06:44:13.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.794+0000 7f137affd700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f137006e210 0x7f13700706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:13.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.794+0000 7f137affd700 1 -- 172.21.15.63:0/2733214633 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f137408f740 con 0x7f1384101c80 2024-05-09T06:44:13.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.794+0000 7f138999f700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f137006e210 0x7f13700706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:13.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.795+0000 7f138999f700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f137006e210 0x7f13700706d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1384101ae0 tx=0x7f1380009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:13.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:13.799+0000 7f137affd700 1 -- 172.21.15.63:0/2733214633 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f137405c500 con 0x7f1384101c80 2024-05-09T06:44:13.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.019+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f138402ce50 con 0x7f137006e210 2024-05-09T06:44:14.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.030+0000 7f137affd700 1 -- 172.21.15.63:0/2733214633 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f138402ce50 con 0x7f137006e210 2024-05-09T06:44:14.010 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:44:14.010 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (13m) 75s ago 14m 25.4M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:44:14.010 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (14m) 75s ago 14m 8472k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:44:14.010 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (13m) 8s ago 13m 8388k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (3m) 75s ago 14m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (2m) 8s ago 13m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (12m) 75s ago 14m 85.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (9m) 75s ago 9m 16.5M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (8m) 75s ago 8m 90.4M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (9m) 8s ago 9m 141M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (8m) 8s ago 8m 18.2M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (5m) 75s ago 16m 508M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (4m) 8s ago 13m 452M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (4m) 75s ago 16m 48.4M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (3m) 8s ago 13m 39.0M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (14m) 75s ago 14m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (13m) 8s ago 13m 16.3M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (2m) 75s ago 12m 144M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:44:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (118s) 75s ago 12m 135M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:44:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (81s) 75s ago 11m 11.6M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:44:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (45s) 8s ago 11m 107M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:44:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (11s) 8s ago 11m 14.8M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 14c9b4920f5a 2024-05-09T06:44:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (10m) 8s ago 10m 266M 2240M 18.2.2-1297-g1758e8ea 196422f1b641 0c0e514a2d04 2024-05-09T06:44:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (4m) 75s ago 14m 58.4M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:44:14.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.032+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f137006e210 msgr2=0x7f13700706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.032+0000 7f138a9a1700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f137006e210 0x7f13700706d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1384101ae0 tx=0x7f1380009260 comp rx=0 tx=0).stop 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.032+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 msgr2=0x7f13841969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.032+0000 7f138a9a1700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 0x7f13841969d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1374005b40 tx=0x7f137400f690 comp rx=0 tx=0).stop 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.033+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 shutdown_connections 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.033+0000 7f138a9a1700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f137006e210 0x7f13700706d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.033+0000 7f138a9a1700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1384100a80 0x7f1384196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.033+0000 7f138a9a1700 1 --2- 172.21.15.63:0/2733214633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1384101c80 0x7f13841969d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.033+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 >> 172.21.15.63:0/2733214633 conn(0x7f13840fc020 msgr2=0x7f1384105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.033+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 shutdown_connections 2024-05-09T06:44:14.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.033+0000 7f138a9a1700 1 -- 172.21.15.63:0/2733214633 wait complete. 2024-05-09T06:44:14.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.140+0000 7fe4def7e700 1 -- 172.21.15.63:0/2758340370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4d80ffb30 msgr2=0x7fe4d80fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.140+0000 7fe4def7e700 1 --2- 172.21.15.63:0/2758340370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4d80ffb30 0x7fe4d80fff50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800b0a0 tx=0x7fe4c800b3b0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.140+0000 7fe4def7e700 1 -- 172.21.15.63:0/2758340370 shutdown_connections 2024-05-09T06:44:14.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.140+0000 7fe4def7e700 1 --2- 172.21.15.63:0/2758340370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8100d30 0x7fe4d8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.140+0000 7fe4def7e700 1 --2- 172.21.15.63:0/2758340370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4d80ffb30 0x7fe4d80fff50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.140+0000 7fe4def7e700 1 -- 172.21.15.63:0/2758340370 >> 172.21.15.63:0/2758340370 conn(0x7fe4d80fb0d0 msgr2=0x7fe4d80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:14.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.140+0000 7fe4def7e700 1 -- 172.21.15.63:0/2758340370 shutdown_connections 2024-05-09T06:44:14.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.140+0000 7fe4def7e700 1 -- 172.21.15.63:0/2758340370 wait complete. 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4def7e700 1 Processor -- start 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4def7e700 1 -- start start 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4def7e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4d80ffb30 0x7fe4d81966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4def7e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8100d30 0x7fe4d8196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4def7e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8197240 con 0x7fe4d8100d30 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4def7e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8197380 con 0x7fe4d80ffb30 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4d7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8100d30 0x7fe4d8196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4d7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8100d30 0x7fe4d8196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55836/0 (socket says 172.21.15.63:55836) 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4d7fff700 1 -- 172.21.15.63:0/3047156147 learned_addr learned my addr 172.21.15.63:0/3047156147 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4dcd1a700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4d80ffb30 0x7fe4d81966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:14.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4d7fff700 1 -- 172.21.15.63:0/3047156147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4d80ffb30 msgr2=0x7fe4d81966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4d7fff700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4d80ffb30 0x7fe4d81966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.141+0000 7fe4d7fff700 1 -- 172.21.15.63:0/3047156147 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c0009520 con 0x7fe4d8100d30 2024-05-09T06:44:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.142+0000 7fe4d7fff700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8100d30 0x7fe4d8196c20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000eab0 tx=0x7fe4c000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.142+0000 7fe4d5ffb700 1 -- 172.21.15.63:0/3047156147 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c000cba0 con 0x7fe4d8100d30 2024-05-09T06:44:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.142+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c8009d00 con 0x7fe4d8100d30 2024-05-09T06:44:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.142+0000 7fe4d5ffb700 1 -- 172.21.15.63:0/3047156147 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4c00044e0 con 0x7fe4d8100d30 2024-05-09T06:44:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.142+0000 7fe4d5ffb700 1 -- 172.21.15.63:0/3047156147 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c0005040 con 0x7fe4d8100d30 2024-05-09T06:44:14.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.142+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d819c220 con 0x7fe4d8100d30 2024-05-09T06:44:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.143+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d804ebb0 con 0x7fe4d8100d30 2024-05-09T06:44:14.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.143+0000 7fe4d5ffb700 1 -- 172.21.15.63:0/3047156147 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe4c0004750 con 0x7fe4d8100d30 2024-05-09T06:44:14.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.144+0000 7fe4d5ffb700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe4c406e200 0x7fe4c40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:14.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.144+0000 7fe4d5ffb700 1 -- 172.21.15.63:0/3047156147 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fe4c0014070 con 0x7fe4d8100d30 2024-05-09T06:44:14.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.144+0000 7fe4dcd1a700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe4c406e200 0x7fe4c40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:14.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.144+0000 7fe4dcd1a700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe4c406e200 0x7fe4c40706c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009fd0 tx=0x7fe4c8006000 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:14.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.149+0000 7fe4d5ffb700 1 -- 172.21.15.63:0/3047156147 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe4c005b020 con 0x7fe4d8100d30 2024-05-09T06:44:14.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:14 smithi063 ceph-mon[92765]: Health check update: Degraded data redundancy: 14/228 objects degraded (6.140%), 5 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:14.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:14 smithi063 ceph-mon[92765]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:14.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:14 smithi063 ceph-mon[92765]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.444+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe4d819c6e0 con 0x7fe4d8100d30 2024-05-09T06:44:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.444+0000 7fe4d5ffb700 1 -- 172.21.15.63:0/3047156147 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7fe4c005abb0 con 0x7fe4d8100d30 2024-05-09T06:44:14.424 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:44:14.425 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:44:14.425 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:44:14.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:44:14.425 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:44:14.425 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:44:14.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:44:14.426 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:44:14.426 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 1, 2024-05-09T06:44:14.426 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 5 2024-05-09T06:44:14.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:44:14.426 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:44:14.427 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 4 2024-05-09T06:44:14.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:44:14.427 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:44:14.427 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 5, 2024-05-09T06:44:14.427 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 9 2024-05-09T06:44:14.427 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:44:14.428 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:44:14.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe4c406e200 msgr2=0x7fe4c40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe4c406e200 0x7fe4c40706c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009fd0 tx=0x7fe4c8006000 comp rx=0 tx=0).stop 2024-05-09T06:44:14.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8100d30 msgr2=0x7fe4d8196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8100d30 0x7fe4d8196c20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000eab0 tx=0x7fe4c000edc0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 shutdown_connections 2024-05-09T06:44:14.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe4c406e200 0x7fe4c40706c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4d80ffb30 0x7fe4d81966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 --2- 172.21.15.63:0/3047156147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8100d30 0x7fe4d8196c20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 >> 172.21.15.63:0/3047156147 conn(0x7fe4d80fb0d0 msgr2=0x7fe4d8069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 shutdown_connections 2024-05-09T06:44:14.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.447+0000 7fe4def7e700 1 -- 172.21.15.63:0/3047156147 wait complete. 2024-05-09T06:44:14.463 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:14 smithi193 ceph-mon[81665]: Health check update: Degraded data redundancy: 14/228 objects degraded (6.140%), 5 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:14.463 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:14 smithi193 ceph-mon[81665]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:14.463 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:14 smithi193 ceph-mon[81665]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.556+0000 7f850aa72700 1 -- 172.21.15.63:0/52005780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 msgr2=0x7f85040fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.556+0000 7f850aa72700 1 --2- 172.21.15.63:0/52005780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 0x7f85040fff50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009910 tx=0x7f84f8009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.556+0000 7f850aa72700 1 -- 172.21.15.63:0/52005780 shutdown_connections 2024-05-09T06:44:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.556+0000 7f850aa72700 1 --2- 172.21.15.63:0/52005780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8504100d30 0x7f8504101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.556+0000 7f850aa72700 1 --2- 172.21.15.63:0/52005780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 0x7f85040fff50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.556+0000 7f850aa72700 1 -- 172.21.15.63:0/52005780 >> 172.21.15.63:0/52005780 conn(0x7f85040fb0d0 msgr2=0x7f85040fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.556+0000 7f850aa72700 1 -- 172.21.15.63:0/52005780 shutdown_connections 2024-05-09T06:44:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.556+0000 7f850aa72700 1 -- 172.21.15.63:0/52005780 wait complete. 2024-05-09T06:44:14.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850aa72700 1 Processor -- start 2024-05-09T06:44:14.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850aa72700 1 -- start start 2024-05-09T06:44:14.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850aa72700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 0x7f8504103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:14.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850aa72700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8504100d30 0x7f8504102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:14.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850aa72700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8504102650 con 0x7f85040ffb30 2024-05-09T06:44:14.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850aa72700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8504102790 con 0x7f8504100d30 2024-05-09T06:44:14.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850880e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 0x7f8504103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850880e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 0x7f8504103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55862/0 (socket says 172.21.15.63:55862) 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850880e700 1 -- 172.21.15.63:0/1382475908 learned_addr learned my addr 172.21.15.63:0/1382475908 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f8503fff700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8504100d30 0x7f8504102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850880e700 1 -- 172.21.15.63:0/1382475908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8504100d30 msgr2=0x7f8504102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850880e700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8504100d30 0x7f8504102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850880e700 1 -- 172.21.15.63:0/1382475908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f80095f0 con 0x7f85040ffb30 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f8503fff700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8504100d30 0x7f8504102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.557+0000 7f850880e700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 0x7f8504103a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84f800b620 tx=0x7f84f800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.558+0000 7f8501ffb700 1 -- 172.21.15.63:0/1382475908 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f801d070 con 0x7f85040ffb30 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.558+0000 7f8501ffb700 1 -- 172.21.15.63:0/1382475908 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84f80052a0 con 0x7f85040ffb30 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.558+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8504102a10 con 0x7f85040ffb30 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.558+0000 7f8501ffb700 1 -- 172.21.15.63:0/1382475908 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f8017400 con 0x7f85040ffb30 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.558+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8504102f80 con 0x7f85040ffb30 2024-05-09T06:44:14.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.559+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850404ebb0 con 0x7f85040ffb30 2024-05-09T06:44:14.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.560+0000 7f8501ffb700 1 -- 172.21.15.63:0/1382475908 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f84f8003810 con 0x7f85040ffb30 2024-05-09T06:44:14.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.560+0000 7f8501ffb700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f84f406e210 0x7f84f40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:14.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.560+0000 7f8501ffb700 1 -- 172.21.15.63:0/1382475908 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f84f808f530 con 0x7f85040ffb30 2024-05-09T06:44:14.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.560+0000 7f8503fff700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f84f406e210 0x7f84f40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:14.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.561+0000 7f8503fff700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f84f406e210 0x7f84f40706d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009be0 tx=0x7f84f0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:14.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.565+0000 7f8501ffb700 1 -- 172.21.15.63:0/1382475908 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f84f805c270 con 0x7f85040ffb30 2024-05-09T06:44:14.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.819+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f85041088d0 con 0x7f85040ffb30 2024-05-09T06:44:14.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.819+0000 7f8501ffb700 1 -- 172.21.15.63:0/1382475908 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1905 (secure 0 0 0) 0x7f84f805be00 con 0x7f85040ffb30 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:e15 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 15 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:35:24.143344+0000 2024-05-09T06:44:14.799 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:44:14.800 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:14492} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.63:6828/1815819553,v1:172.21.15.63:6829/1815819553] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:44:14.801 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:44:14.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f84f406e210 msgr2=0x7f84f40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f84f406e210 0x7f84f40706d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009be0 tx=0x7f84f0009260 comp rx=0 tx=0).stop 2024-05-09T06:44:14.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 msgr2=0x7f8504103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 0x7f8504103a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84f800b620 tx=0x7f84f800f510 comp rx=0 tx=0).stop 2024-05-09T06:44:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 shutdown_connections 2024-05-09T06:44:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f84f406e210 0x7f84f40706d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85040ffb30 0x7f8504103a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 --2- 172.21.15.63:0/1382475908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8504100d30 0x7f8504102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 >> 172.21.15.63:0/1382475908 conn(0x7f85040fb0d0 msgr2=0x7f85040691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 shutdown_connections 2024-05-09T06:44:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.822+0000 7f850aa72700 1 -- 172.21.15.63:0/1382475908 wait complete. 2024-05-09T06:44:14.803 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 15 2024-05-09T06:44:14.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.932+0000 7f44e996e700 1 -- 172.21.15.63:0/2809449971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 msgr2=0x7f44e4103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.932+0000 7f44e996e700 1 --2- 172.21.15.63:0/2809449971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 0x7f44e4103b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009910 tx=0x7f44d4009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:14.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.932+0000 7f44e996e700 1 -- 172.21.15.63:0/2809449971 shutdown_connections 2024-05-09T06:44:14.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.932+0000 7f44e996e700 1 --2- 172.21.15.63:0/2809449971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e4104090 0x7f44e4106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.932+0000 7f44e996e700 1 --2- 172.21.15.63:0/2809449971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 0x7f44e4103b50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.932+0000 7f44e996e700 1 -- 172.21.15.63:0/2809449971 >> 172.21.15.63:0/2809449971 conn(0x7f44e40fb0d0 msgr2=0x7f44e40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:14.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.932+0000 7f44e996e700 1 -- 172.21.15.63:0/2809449971 shutdown_connections 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.932+0000 7f44e996e700 1 -- 172.21.15.63:0/2809449971 wait complete. 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.932+0000 7f44e996e700 1 Processor -- start 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e996e700 1 -- start start 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e996e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 0x7f44e4101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e996e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e4104090 0x7f44e40ff710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e996e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40ffc50 con 0x7f44e4101760 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e996e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40ffd90 con 0x7f44e4104090 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 0x7f44e4101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 0x7f44e4101070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33080/0 (socket says 172.21.15.63:33080) 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e2ffd700 1 -- 172.21.15.63:0/2564529112 learned_addr learned my addr 172.21.15.63:0/2564529112 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e27fc700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e4104090 0x7f44e40ff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e2ffd700 1 -- 172.21.15.63:0/2564529112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e4104090 msgr2=0x7f44e40ff710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e2ffd700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e4104090 0x7f44e40ff710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e2ffd700 1 -- 172.21.15.63:0/2564529112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d40095f0 con 0x7f44e4101760 2024-05-09T06:44:14.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.933+0000 7f44e2ffd700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 0x7f44e4101070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44d40092e0 tx=0x7f44d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.934+0000 7f44e896c700 1 -- 172.21.15.63:0/2564529112 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44d401d070 con 0x7f44e4101760 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.934+0000 7f44e896c700 1 -- 172.21.15.63:0/2564529112 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44d400f460 con 0x7f44e4101760 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.934+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e4100010 con 0x7f44e4101760 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.934+0000 7f44e896c700 1 -- 172.21.15.63:0/2564529112 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44d4021620 con 0x7f44e4101760 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.934+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e4100500 con 0x7f44e4101760 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.935+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e404ebb0 con 0x7f44e4101760 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.935+0000 7f44e896c700 1 -- 172.21.15.63:0/2564529112 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f44d400f5d0 con 0x7f44e4101760 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.936+0000 7f44e896c700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f44cc06e200 0x7f44cc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.936+0000 7f44e896c700 1 -- 172.21.15.63:0/2564529112 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f44d408f4d0 con 0x7f44e4101760 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.936+0000 7f44e27fc700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f44cc06e200 0x7f44cc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.936+0000 7f44e27fc700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f44cc06e200 0x7f44cc0706c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44d8005950 tx=0x7f44d800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:14.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:14.940+0000 7f44e896c700 1 -- 172.21.15.63:0/2564529112 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f44d405c210 con 0x7f44e4101760 2024-05-09T06:44:15.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.167+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44e4106060 con 0x7f44cc06e200 2024-05-09T06:44:15.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.168+0000 7f44e896c700 1 -- 172.21.15.63:0/2564529112 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f44e4106060 con 0x7f44cc06e200 2024-05-09T06:44:15.147 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "11/23 daemons upgraded", 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:44:15.148 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f44cc06e200 msgr2=0x7f44cc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f44cc06e200 0x7f44cc0706c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44d8005950 tx=0x7f44d800b410 comp rx=0 tx=0).stop 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 msgr2=0x7f44e4101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 0x7f44e4101070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44d40092e0 tx=0x7f44d400bd70 comp rx=0 tx=0).stop 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 shutdown_connections 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f44cc06e200 0x7f44cc0706c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44e4101760 0x7f44e4101070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 --2- 172.21.15.63:0/2564529112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e4104090 0x7f44e40ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 >> 172.21.15.63:0/2564529112 conn(0x7f44e40fb0d0 msgr2=0x7f44e40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 shutdown_connections 2024-05-09T06:44:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.170+0000 7f44e996e700 1 -- 172.21.15.63:0/2564529112 wait complete. 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.279+0000 7f6d9af89700 1 -- 172.21.15.63:0/2778807455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d941017f0 msgr2=0x7f6d94103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.279+0000 7f6d9af89700 1 --2- 172.21.15.63:0/2778807455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d941017f0 0x7f6d94103be0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009910 tx=0x7f6d84009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.279+0000 7f6d9af89700 1 -- 172.21.15.63:0/2778807455 shutdown_connections 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.279+0000 7f6d9af89700 1 --2- 172.21.15.63:0/2778807455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d94104120 0x7f6d94106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.279+0000 7f6d9af89700 1 --2- 172.21.15.63:0/2778807455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d941017f0 0x7f6d94103be0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.279+0000 7f6d9af89700 1 -- 172.21.15.63:0/2778807455 >> 172.21.15.63:0/2778807455 conn(0x7f6d940fb120 msgr2=0x7f6d940fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.279+0000 7f6d9af89700 1 -- 172.21.15.63:0/2778807455 shutdown_connections 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.279+0000 7f6d9af89700 1 -- 172.21.15.63:0/2778807455 wait complete. 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.279+0000 7f6d9af89700 1 Processor -- start 2024-05-09T06:44:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d9af89700 1 -- start start 2024-05-09T06:44:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d9af89700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d941017f0 0x7f6d94196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d9af89700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d94104120 0x7f6d94196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d9af89700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d94197280 con 0x7f6d94104120 2024-05-09T06:44:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d9af89700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d941973c0 con 0x7f6d941017f0 2024-05-09T06:44:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d93fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d94104120 0x7f6d94196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d93fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d94104120 0x7f6d94196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33104/0 (socket says 172.21.15.63:33104) 2024-05-09T06:44:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d93fff700 1 -- 172.21.15.63:0/2713453267 learned_addr learned my addr 172.21.15.63:0/2713453267 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d98d25700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d941017f0 0x7f6d94196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d93fff700 1 -- 172.21.15.63:0/2713453267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d941017f0 msgr2=0x7f6d94196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d93fff700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d941017f0 0x7f6d94196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d93fff700 1 -- 172.21.15.63:0/2713453267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d840095f0 con 0x7f6d94104120 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.280+0000 7f6d93fff700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d94104120 0x7f6d94196c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b4b0 tx=0x7f6d8800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.281+0000 7f6d91ffb700 1 -- 172.21.15.63:0/2713453267 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d88011800 con 0x7f6d94104120 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.281+0000 7f6d91ffb700 1 -- 172.21.15.63:0/2713453267 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d88011e20 con 0x7f6d94104120 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.281+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9419be20 con 0x7f6d94104120 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.281+0000 7f6d91ffb700 1 -- 172.21.15.63:0/2713453267 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d8800f2e0 con 0x7f6d94104120 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.281+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9419c3c0 con 0x7f6d94104120 2024-05-09T06:44:15.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.282+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9404ebb0 con 0x7f6d94104120 2024-05-09T06:44:15.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.282+0000 7f6d91ffb700 1 -- 172.21.15.63:0/2713453267 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6d88010380 con 0x7f6d94104120 2024-05-09T06:44:15.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.283+0000 7f6d91ffb700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6d7c06e210 0x7f6d7c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:15.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.283+0000 7f6d98d25700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6d7c06e210 0x7f6d7c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:15.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.283+0000 7f6d91ffb700 1 -- 172.21.15.63:0/2713453267 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f6d8808dc80 con 0x7f6d94104120 2024-05-09T06:44:15.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.283+0000 7f6d98d25700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6d7c06e210 0x7f6d7c0706d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d84006010 tx=0x7f6d8400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:15.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.288+0000 7f6d91ffb700 1 -- 172.21.15.63:0/2713453267 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6d8805aa40 con 0x7f6d94104120 2024-05-09T06:44:15.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.580+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6d94109880 con 0x7f6d94104120 2024-05-09T06:44:15.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:15 smithi063 ceph-mon[92765]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:15.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:15 smithi063 ceph-mon[92765]: pgmap v159: 65 pgs: 9 peering, 3 active+undersized, 2 active+undersized+degraded, 51 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 5/228 objects degraded (2.193%) 2024-05-09T06:44:15.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:15 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3047156147' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:15.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:15 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1382475908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:44:15.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.580+0000 7f6d91ffb700 1 -- 172.21.15.63:0/2713453267 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+291 (secure 0 0 0) 0x7f6d88015140 con 0x7f6d94104120 2024-05-09T06:44:15.561 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN Degraded data redundancy: 5/228 objects degraded (2.193%), 2 pgs degraded 2024-05-09T06:44:15.561 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 5/228 objects degraded (2.193%), 2 pgs degraded 2024-05-09T06:44:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-05-09T06:44:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-05-09T06:44:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.582+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6d7c06e210 msgr2=0x7f6d7c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.582+0000 7f6d9af89700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6d7c06e210 0x7f6d7c0706d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d84006010 tx=0x7f6d8400b540 comp rx=0 tx=0).stop 2024-05-09T06:44:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.582+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d94104120 msgr2=0x7f6d94196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.582+0000 7f6d9af89700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d94104120 0x7f6d94196c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b4b0 tx=0x7f6d8800b7c0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.583+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 shutdown_connections 2024-05-09T06:44:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.583+0000 7f6d9af89700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6d7c06e210 0x7f6d7c0706d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.583+0000 7f6d9af89700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d941017f0 0x7f6d94196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.583+0000 7f6d9af89700 1 --2- 172.21.15.63:0/2713453267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d94104120 0x7f6d94196c60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.583+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 >> 172.21.15.63:0/2713453267 conn(0x7f6d940fb120 msgr2=0x7f6d940fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.583+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 shutdown_connections 2024-05-09T06:44:15.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:15.583+0000 7f6d9af89700 1 -- 172.21.15.63:0/2713453267 wait complete. 2024-05-09T06:44:15.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:15 smithi193 ceph-mon[81665]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:15.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:15 smithi193 ceph-mon[81665]: pgmap v159: 65 pgs: 9 peering, 3 active+undersized, 2 active+undersized+degraded, 51 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 5/228 objects degraded (2.193%) 2024-05-09T06:44:15.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:15 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3047156147' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:15.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:15 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1382475908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:44:16.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:16 smithi063 ceph-mon[92765]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:16.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:16 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2713453267' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:44:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:16 smithi193 ceph-mon[81665]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:16.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:16 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2713453267' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:44:17.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:17 smithi193 ceph-mon[81665]: pgmap v160: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-09T06:44:17.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:17 smithi193 ceph-mon[81665]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/228 objects degraded (2.193%), 2 pgs degraded) 2024-05-09T06:44:17.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:17 smithi193 ceph-mon[81665]: Cluster is now healthy 2024-05-09T06:44:17.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:17 smithi063 ceph-mon[92765]: pgmap v160: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-09T06:44:17.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:17 smithi063 ceph-mon[92765]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/228 objects degraded (2.193%), 2 pgs degraded) 2024-05-09T06:44:17.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:17 smithi063 ceph-mon[92765]: Cluster is now healthy 2024-05-09T06:44:19.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:19 smithi063 ceph-mon[92765]: pgmap v161: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 1017 B/s rd, 2 op/s 2024-05-09T06:44:19.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:19 smithi193 ceph-mon[81665]: pgmap v161: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 1017 B/s rd, 2 op/s 2024-05-09T06:44:21.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:21 smithi063 ceph-mon[92765]: pgmap v162: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-05-09T06:44:21.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:21 smithi193 ceph-mon[81665]: pgmap v162: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-05-09T06:44:23.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:23 smithi063 ceph-mon[92765]: pgmap v163: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 926 B/s rd, 2 op/s 2024-05-09T06:44:23.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:23.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:23 smithi193 ceph-mon[81665]: pgmap v163: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 926 B/s rd, 2 op/s 2024-05-09T06:44:23.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:25.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:25 smithi063 ceph-mon[92765]: pgmap v164: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:44:25.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:25 smithi193 ceph-mon[81665]: pgmap v164: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:44:26.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-09T06:44:26.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-09T06:44:27.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:27 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-09T06:44:27.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:27 smithi063 ceph-mon[92765]: Upgrade: osd.5 is safe to restart 2024-05-09T06:44:27.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:27 smithi063 ceph-mon[92765]: pgmap v165: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:44:27.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:27.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:44:27.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:27 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-09T06:44:27.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:27 smithi193 ceph-mon[81665]: Upgrade: osd.5 is safe to restart 2024-05-09T06:44:27.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:27 smithi193 ceph-mon[81665]: pgmap v165: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:44:27.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:27.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:44:28.835 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:28 smithi193 ceph-mon[81665]: Upgrade: Updating osd.5 2024-05-09T06:44:28.835 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:28.835 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T06:44:28.835 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:28.835 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:28 smithi193 ceph-mon[81665]: Deploying daemon osd.5 on smithi193 2024-05-09T06:44:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:28 smithi063 ceph-mon[92765]: Upgrade: Updating osd.5 2024-05-09T06:44:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T06:44:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:28.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:28 smithi063 ceph-mon[92765]: Deploying daemon osd.5 on smithi193 2024-05-09T06:44:29.694 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:29 smithi193 ceph-mon[81665]: pgmap v166: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:44:29.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:29 smithi063 ceph-mon[92765]: pgmap v166: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:44:31.121 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:30 smithi193 systemd[1]: Stopping Ceph osd.5 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:44:31.568 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:31 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[54915]: 2024-05-09T06:44:31.184+0000 7fca98a81700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:44:31.568 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:31 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[54915]: 2024-05-09T06:44:31.184+0000 7fca98a81700 -1 osd.5 85 *** Got signal Terminated *** 2024-05-09T06:44:31.568 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:31 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[54915]: 2024-05-09T06:44:31.184+0000 7fca98a81700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:44:31.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:31 smithi063 ceph-mon[92765]: pgmap v167: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:44:31.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:31 smithi063 ceph-mon[92765]: osd.5 marked itself down and dead 2024-05-09T06:44:31.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:31 smithi193 ceph-mon[81665]: pgmap v167: 65 pgs: 65 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:44:31.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:31 smithi193 ceph-mon[81665]: osd.5 marked itself down and dead 2024-05-09T06:44:32.339 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:32 smithi193 podman[100791]: 2024-05-09 06:44:32.090096193 +0000 UTC m=+1.022895811 container died 0c0e514a2d047243a89bd557f56acb512cf600d661dc21616b825b3852d35de0 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=reef-1758e8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:44:32.607 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:32 smithi193 ceph-mon[81665]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:44:32.607 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:32 smithi193 ceph-mon[81665]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-09T06:44:32.607 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:32 smithi193 ceph-mon[81665]: osdmap e86: 6 total, 5 up, 6 in 2024-05-09T06:44:32.607 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:32 smithi193 podman[100791]: 2024-05-09 06:44:32.443448241 +0000 UTC m=+1.376247859 container cleanup 0c0e514a2d047243a89bd557f56acb512cf600d661dc21616b825b3852d35de0 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5, RELEASE=reef-1758e8e, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T06:44:32.607 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:32 smithi193 bash[100791]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5 2024-05-09T06:44:32.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:32 smithi063 ceph-mon[92765]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-09T06:44:32.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:32 smithi063 ceph-mon[92765]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-09T06:44:32.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:32 smithi063 ceph-mon[92765]: osdmap e86: 6 total, 5 up, 6 in 2024-05-09T06:44:32.871 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:32 smithi193 podman[100818]: 2024-05-09 06:44:32.628075895 +0000 UTC m=+0.532352491 container remove 0c0e514a2d047243a89bd557f56acb512cf600d661dc21616b825b3852d35de0 (image=quay.ceph.io/ceph-ci/ceph@sha256:c9cd3d28c4c53174bf97f95fda1347a328396b687271f92f98e7a35573027e7f, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=reef-1758e8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:44:33.294 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:33 smithi193 podman[100912]: 2024-05-09 06:44:32.95704065 +0000 UTC m=+0.018134827 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:44:33.294 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:33 smithi193 podman[100912]: 2024-05-09 06:44:33.087423824 +0000 UTC m=+0.148517990 container create 6ee15f9f4d4e54251e35a937a8a59277a5863ae62245c2da414ffb2daff38e34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T06:44:33.294 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:33 smithi193 podman[100912]: 2024-05-09 06:44:33.315356676 +0000 UTC m=+0.376450862 container init 6ee15f9f4d4e54251e35a937a8a59277a5863ae62245c2da414ffb2daff38e34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T06:44:33.580 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:33 smithi193 podman[100912]: 2024-05-09 06:44:33.318947226 +0000 UTC m=+0.380041396 container start 6ee15f9f4d4e54251e35a937a8a59277a5863ae62245c2da414ffb2daff38e34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T06:44:33.580 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:33 smithi193 podman[100912]: 2024-05-09 06:44:33.415057323 +0000 UTC m=+0.476151518 container attach 6ee15f9f4d4e54251e35a937a8a59277a5863ae62245c2da414ffb2daff38e34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-09T06:44:33.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:33 smithi063 ceph-mon[92765]: pgmap v169: 65 pgs: 13 stale+active+clean, 52 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-09T06:44:33.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:33 smithi063 ceph-mon[92765]: osdmap e87: 6 total, 5 up, 6 in 2024-05-09T06:44:33.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:33 smithi193 ceph-mon[81665]: pgmap v169: 65 pgs: 13 stale+active+clean, 52 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-09T06:44:33.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:33 smithi193 ceph-mon[81665]: osdmap e87: 6 total, 5 up, 6 in 2024-05-09T06:44:33.871 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:33 smithi193 podman[100912]: 2024-05-09 06:44:33.620317521 +0000 UTC m=+0.681411690 container died 6ee15f9f4d4e54251e35a937a8a59277a5863ae62245c2da414ffb2daff38e34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T06:44:34.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:34 smithi193 podman[100938]: 2024-05-09 06:44:34.252122972 +0000 UTC m=+0.623436374 container remove 6ee15f9f4d4e54251e35a937a8a59277a5863ae62245c2da414ffb2daff38e34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:44:34.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:34 smithi193 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.5.service: Deactivated successfully. 2024-05-09T06:44:34.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:34 smithi193 systemd[1]: Stopped Ceph osd.5 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:44:34.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:34 smithi193 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.5.service: Consumed 1min 8.888s CPU time. 2024-05-09T06:44:34.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:34 smithi193 systemd[1]: Starting Ceph osd.5 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:44:34.909 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:34 smithi193 ceph-mon[81665]: Health check failed: Degraded data redundancy: 9/228 objects degraded (3.947%), 3 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:34.910 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:34 smithi193 podman[101042]: 2024-05-09 06:44:34.603739625 +0000 UTC m=+0.018635312 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:44:34.910 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:34 smithi193 podman[101042]: 2024-05-09 06:44:34.720464515 +0000 UTC m=+0.135360258 container create edf40d7d53eed07bd36960c182041630ac028796553afbcb908756301dc575ac (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-96805dc) 2024-05-09T06:44:34.910 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:34 smithi193 podman[101042]: 2024-05-09 06:44:34.931095166 +0000 UTC m=+0.345990843 container init edf40d7d53eed07bd36960c182041630ac028796553afbcb908756301dc575ac (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T06:44:35.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:34 smithi063 ceph-mon[92765]: Health check failed: Degraded data redundancy: 9/228 objects degraded (3.947%), 3 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:35.364 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:34 smithi193 podman[101042]: 2024-05-09 06:44:34.934548777 +0000 UTC m=+0.349444463 container start edf40d7d53eed07bd36960c182041630ac028796553afbcb908756301dc575ac (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T06:44:35.364 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 podman[101042]: 2024-05-09 06:44:35.031153107 +0000 UTC m=+0.446048802 container attach edf40d7d53eed07bd36960c182041630ac028796553afbcb908756301dc575ac (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:44:35.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate[101057]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T06:44:35.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate[101057]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-09T06:44:35.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 bash[101042]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T06:44:35.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 bash[101042]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-09T06:44:35.621 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 bash[101042]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 bash[101042]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 bash[101042]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 bash[101042]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 bash[101042]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate[101057]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate[101057]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate[101057]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate[101057]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate[101057]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-09T06:44:35.622 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 podman[101042]: 2024-05-09 06:44:35.410528683 +0000 UTC m=+0.825424360 container died edf40d7d53eed07bd36960c182041630ac028796553afbcb908756301dc575ac (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T06:44:35.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:35 smithi193 ceph-mon[81665]: pgmap v171: 65 pgs: 4 active+undersized, 3 active+undersized+degraded, 9 stale+active+clean, 49 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 9/228 objects degraded (3.947%) 2024-05-09T06:44:36.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:35 smithi063 ceph-mon[92765]: pgmap v171: 65 pgs: 4 active+undersized, 3 active+undersized+degraded, 9 stale+active+clean, 49 active+clean; 205 MiB data, 922 MiB used, 536 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 9/228 objects degraded (3.947%) 2024-05-09T06:44:36.198 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:35 smithi193 podman[101192]: 2024-05-09 06:44:35.968281811 +0000 UTC m=+0.547787133 container remove edf40d7d53eed07bd36960c182041630ac028796553afbcb908756301dc575ac (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T06:44:36.199 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:36 smithi193 podman[101256]: 2024-05-09 06:44:36.085835504 +0000 UTC m=+0.018339849 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:44:36.199 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:36 smithi193 podman[101256]: 2024-05-09 06:44:36.219560937 +0000 UTC m=+0.152065266 container create d219b4bb599f002375395018a5a108c1c485b68d9d4ac3d63ba42ebdbd8668f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T06:44:36.509 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:36 smithi193 podman[101256]: 2024-05-09 06:44:36.438531958 +0000 UTC m=+0.371036288 container init d219b4bb599f002375395018a5a108c1c485b68d9d4ac3d63ba42ebdbd8668f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-09T06:44:36.509 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:36 smithi193 podman[101256]: 2024-05-09 06:44:36.441715169 +0000 UTC m=+0.374219507 container start d219b4bb599f002375395018a5a108c1c485b68d9d4ac3d63ba42ebdbd8668f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:44:36.509 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:36 smithi193 ceph-osd[101275]: -- 172.21.15.193:0/1771462511 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557074d8d680 con 0x557074d57000 2024-05-09T06:44:36.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:36 smithi193 bash[101256]: d219b4bb599f002375395018a5a108c1c485b68d9d4ac3d63ba42ebdbd8668f0 2024-05-09T06:44:36.871 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:36 smithi193 systemd[1]: Started Ceph osd.5 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:44:37.370 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:36 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:44:36.986+0000 7f2ec84c47c0 -1 Falling back to public interface 2024-05-09T06:44:37.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:37 smithi063 ceph-mon[92765]: pgmap v172: 65 pgs: 15 active+undersized, 9 active+undersized+degraded, 1 stale+active+clean, 40 active+clean; 205 MiB data, 926 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 27/228 objects degraded (11.842%) 2024-05-09T06:44:37.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:37.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:37.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:37.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:37 smithi193 ceph-mon[81665]: pgmap v172: 65 pgs: 15 active+undersized, 9 active+undersized+degraded, 1 stale+active+clean, 40 active+clean; 205 MiB data, 926 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 27/228 objects degraded (11.842%) 2024-05-09T06:44:37.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:37.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:37.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:39.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:39 smithi063 ceph-mon[92765]: pgmap v173: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 205 MiB data, 926 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 35/228 objects degraded (15.351%) 2024-05-09T06:44:39.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:39 smithi193 ceph-mon[81665]: pgmap v173: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 205 MiB data, 926 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 35/228 objects degraded (15.351%) 2024-05-09T06:44:41.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:40 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:41.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:40 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:41.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:40 smithi063 ceph-mon[92765]: pgmap v174: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 205 MiB data, 926 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 35/228 objects degraded (15.351%) 2024-05-09T06:44:41.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:40 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:41.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:40 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:41.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:40 smithi193 ceph-mon[81665]: pgmap v174: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 205 MiB data, 926 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 35/228 objects degraded (15.351%) 2024-05-09T06:44:42.160 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:42.161 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:42.161 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:42 smithi193 ceph-mon[81665]: from='osd.5 [v2:172.21.15.193:6816/2078959605,v1:172.21.15.193:6817/2078959605]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T06:44:42.161 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:42 smithi193 ceph-mon[81665]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T06:44:42.161 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:41 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:44:41.922+0000 7f2ec84c47c0 -1 osd.5 85 log_to_monitors true 2024-05-09T06:44:42.161 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:44:42 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:44:42.077+0000 7f2eb991b700 -1 osd.5 85 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:44:42.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:42.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:42.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:42 smithi063 ceph-mon[92765]: from='osd.5 [v2:172.21.15.193:6816/2078959605,v1:172.21.15.193:6817/2078959605]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T06:44:42.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:42 smithi063 ceph-mon[92765]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T06:44:43.315 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:43 smithi193 ceph-mon[81665]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T06:44:43.315 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:43 smithi193 ceph-mon[81665]: osdmap e88: 6 total, 5 up, 6 in 2024-05-09T06:44:43.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:43 smithi193 ceph-mon[81665]: from='osd.5 [v2:172.21.15.193:6816/2078959605,v1:172.21.15.193:6817/2078959605]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:44:43.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:43 smithi193 ceph-mon[81665]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:44:43.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:43 smithi193 ceph-mon[81665]: pgmap v176: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 428 B/s rd, 1 op/s; 35/228 objects degraded (15.351%) 2024-05-09T06:44:43.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:43.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:44:43.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:43 smithi063 ceph-mon[92765]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T06:44:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:43 smithi063 ceph-mon[92765]: osdmap e88: 6 total, 5 up, 6 in 2024-05-09T06:44:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:43 smithi063 ceph-mon[92765]: from='osd.5 [v2:172.21.15.193:6816/2078959605,v1:172.21.15.193:6817/2078959605]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:44:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:43 smithi063 ceph-mon[92765]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-09T06:44:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:43 smithi063 ceph-mon[92765]: pgmap v176: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 428 B/s rd, 1 op/s; 35/228 objects degraded (15.351%) 2024-05-09T06:44:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:43.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:44:44.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:44 smithi063 ceph-mon[92765]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:44:44.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:44 smithi063 ceph-mon[92765]: osd.5 [v2:172.21.15.193:6816/2078959605,v1:172.21.15.193:6817/2078959605] boot 2024-05-09T06:44:44.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:44 smithi063 ceph-mon[92765]: osdmap e89: 6 total, 6 up, 6 in 2024-05-09T06:44:44.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:44 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:44:44.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:44 smithi063 ceph-mon[92765]: Health check update: Degraded data redundancy: 35/228 objects degraded (15.351%), 12 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:44.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:44 smithi063 ceph-mon[92765]: osdmap e90: 6 total, 6 up, 6 in 2024-05-09T06:44:44.365 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:44 smithi193 ceph-mon[81665]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-09T06:44:44.365 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:44 smithi193 ceph-mon[81665]: osd.5 [v2:172.21.15.193:6816/2078959605,v1:172.21.15.193:6817/2078959605] boot 2024-05-09T06:44:44.365 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:44 smithi193 ceph-mon[81665]: osdmap e89: 6 total, 6 up, 6 in 2024-05-09T06:44:44.365 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:44 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T06:44:44.365 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:44 smithi193 ceph-mon[81665]: Health check update: Degraded data redundancy: 35/228 objects degraded (15.351%), 12 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:44.365 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:44 smithi193 ceph-mon[81665]: osdmap e90: 6 total, 6 up, 6 in 2024-05-09T06:44:45.315 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: pgmap v179: 65 pgs: 16 active+undersized, 10 active+undersized+degraded, 39 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 31/228 objects degraded (13.596%) 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:45.316 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-09T06:44:45.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: pgmap v179: 65 pgs: 16 active+undersized, 10 active+undersized+degraded, 39 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 31/228 objects degraded (13.596%) 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:45.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-09T06:44:45.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.707+0000 7f3242b69700 1 -- 172.21.15.63:0/1229399733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 msgr2=0x7f323c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.707+0000 7f3242b69700 1 --2- 172.21.15.63:0/1229399733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 0x7f323c0fff50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3230009910 tx=0x7f3230009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.707+0000 7f3242b69700 1 -- 172.21.15.63:0/1229399733 shutdown_connections 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.707+0000 7f3242b69700 1 --2- 172.21.15.63:0/1229399733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c100d30 0x7f323c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.707+0000 7f3242b69700 1 --2- 172.21.15.63:0/1229399733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 0x7f323c0fff50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.707+0000 7f3242b69700 1 -- 172.21.15.63:0/1229399733 >> 172.21.15.63:0/1229399733 conn(0x7f323c0fb0d0 msgr2=0x7f323c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.707+0000 7f3242b69700 1 -- 172.21.15.63:0/1229399733 shutdown_connections 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.707+0000 7f3242b69700 1 -- 172.21.15.63:0/1229399733 wait complete. 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.708+0000 7f3242b69700 1 Processor -- start 2024-05-09T06:44:45.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.708+0000 7f3242b69700 1 -- start start 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.708+0000 7f3242b69700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 0x7f323c069b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.708+0000 7f3242b69700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c100d30 0x7f323c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.708+0000 7f3242b69700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c1971b0 con 0x7f323c0ffb30 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.708+0000 7f3242b69700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c19dd10 con 0x7f323c100d30 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.708+0000 7f3240905700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 0x7f323c069b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.708+0000 7f3240905700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 0x7f323c069b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50492/0 (socket says 172.21.15.63:50492) 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.708+0000 7f3240905700 1 -- 172.21.15.63:0/2049885006 learned_addr learned my addr 172.21.15.63:0/2049885006 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f323bfff700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c100d30 0x7f323c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f3240905700 1 -- 172.21.15.63:0/2049885006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c100d30 msgr2=0x7f323c19d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f3240905700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c100d30 0x7f323c19d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f3240905700 1 -- 172.21.15.63:0/2049885006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32300095f0 con 0x7f323c0ffb30 2024-05-09T06:44:45.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f3240905700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 0x7f323c069b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f323000bd70 tx=0x7f323000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:45.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f3239ffb700 1 -- 172.21.15.63:0/2049885006 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f323001d070 con 0x7f323c0ffb30 2024-05-09T06:44:45.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c19deb0 con 0x7f323c0ffb30 2024-05-09T06:44:45.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f3239ffb700 1 -- 172.21.15.63:0/2049885006 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3230005dc0 con 0x7f323c0ffb30 2024-05-09T06:44:45.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f3239ffb700 1 -- 172.21.15.63:0/2049885006 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3230021410 con 0x7f323c0ffb30 2024-05-09T06:44:45.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.709+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c19e3a0 con 0x7f323c0ffb30 2024-05-09T06:44:45.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.711+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c18f6b0 con 0x7f323c0ffb30 2024-05-09T06:44:45.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.711+0000 7f3239ffb700 1 -- 172.21.15.63:0/2049885006 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f323000f460 con 0x7f323c0ffb30 2024-05-09T06:44:45.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.712+0000 7f3239ffb700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f322c06e210 0x7f322c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:45.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.712+0000 7f3239ffb700 1 -- 172.21.15.63:0/2049885006 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f323008f3c0 con 0x7f323c0ffb30 2024-05-09T06:44:45.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.712+0000 7f323bfff700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f322c06e210 0x7f322c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:45.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.712+0000 7f323bfff700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f322c06e210 0x7f322c0706d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3228005950 tx=0x7f322800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:45.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.717+0000 7f3239ffb700 1 -- 172.21.15.63:0/2049885006 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f323005c100 con 0x7f323c0ffb30 2024-05-09T06:44:45.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.945+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f323c02ced0 con 0x7f322c06e210 2024-05-09T06:44:45.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.946+0000 7f3239ffb700 1 -- 172.21.15.63:0/2049885006 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f323c02ced0 con 0x7f322c06e210 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f322c06e210 msgr2=0x7f322c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f322c06e210 0x7f322c0706d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3228005950 tx=0x7f322800b410 comp rx=0 tx=0).stop 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 msgr2=0x7f323c069b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 0x7f323c069b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f323000bd70 tx=0x7f323000bda0 comp rx=0 tx=0).stop 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 shutdown_connections 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f322c06e210 0x7f322c0706d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323c0ffb30 0x7f323c069b80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 --2- 172.21.15.63:0/2049885006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f323c100d30 0x7f323c19d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 >> 172.21.15.63:0/2049885006 conn(0x7f323c0fb0d0 msgr2=0x7f323c104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 shutdown_connections 2024-05-09T06:44:45.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:45.948+0000 7f3242b69700 1 -- 172.21.15.63:0/2049885006 wait complete. 2024-05-09T06:44:45.939 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:44:46.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.063+0000 7f0cbfba5700 1 -- 172.21.15.63:0/191656560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb8100d30 msgr2=0x7f0cb8101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.063+0000 7f0cbfba5700 1 --2- 172.21.15.63:0/191656560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb8100d30 0x7f0cb8101190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009910 tx=0x7f0ca8009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:46.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.063+0000 7f0cbfba5700 1 -- 172.21.15.63:0/191656560 shutdown_connections 2024-05-09T06:44:46.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.063+0000 7f0cbfba5700 1 --2- 172.21.15.63:0/191656560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb8100d30 0x7f0cb8101190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.063+0000 7f0cbfba5700 1 --2- 172.21.15.63:0/191656560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb80ffb30 0x7f0cb80fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.063+0000 7f0cbfba5700 1 -- 172.21.15.63:0/191656560 >> 172.21.15.63:0/191656560 conn(0x7f0cb80fb0d0 msgr2=0x7f0cb80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:46.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.063+0000 7f0cbfba5700 1 -- 172.21.15.63:0/191656560 shutdown_connections 2024-05-09T06:44:46.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.063+0000 7f0cbfba5700 1 -- 172.21.15.63:0/191656560 wait complete. 2024-05-09T06:44:46.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbfba5700 1 Processor -- start 2024-05-09T06:44:46.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbfba5700 1 -- start start 2024-05-09T06:44:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbfba5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb80ffb30 0x7f0cb8196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbfba5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb8100d30 0x7f0cb8196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbfba5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb81971a0 con 0x7f0cb80ffb30 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbfba5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb81972e0 con 0x7f0cb8100d30 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbd941700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb80ffb30 0x7f0cb8196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbd941700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb80ffb30 0x7f0cb8196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50522/0 (socket says 172.21.15.63:50522) 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbd941700 1 -- 172.21.15.63:0/1450511992 learned_addr learned my addr 172.21.15.63:0/1450511992 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.064+0000 7f0cbd140700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb8100d30 0x7f0cb8196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.065+0000 7f0cbd941700 1 -- 172.21.15.63:0/1450511992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb8100d30 msgr2=0x7f0cb8196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.065+0000 7f0cbd941700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb8100d30 0x7f0cb8196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.065+0000 7f0cbd941700 1 -- 172.21.15.63:0/1450511992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80095f0 con 0x7f0cb80ffb30 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.065+0000 7f0cbd941700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb80ffb30 0x7f0cb8196640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c8f0 tx=0x7f0cb400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.065+0000 7f0caeffd700 1 -- 172.21.15.63:0/1450511992 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb40078e0 con 0x7f0cb80ffb30 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.065+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb819bd40 con 0x7f0cb80ffb30 2024-05-09T06:44:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.065+0000 7f0caeffd700 1 -- 172.21.15.63:0/1450511992 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cb400f450 con 0x7f0cb80ffb30 2024-05-09T06:44:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.065+0000 7f0caeffd700 1 -- 172.21.15.63:0/1450511992 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb4018610 con 0x7f0cb80ffb30 2024-05-09T06:44:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.065+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb819c2b0 con 0x7f0cb80ffb30 2024-05-09T06:44:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.066+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb804ec20 con 0x7f0cb80ffb30 2024-05-09T06:44:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.067+0000 7f0caeffd700 1 -- 172.21.15.63:0/1450511992 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0cb400f5c0 con 0x7f0cb80ffb30 2024-05-09T06:44:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.067+0000 7f0caeffd700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0ca406e200 0x7f0ca40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.067+0000 7f0caeffd700 1 -- 172.21.15.63:0/1450511992 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f0cb408e040 con 0x7f0cb80ffb30 2024-05-09T06:44:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.067+0000 7f0cbd140700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0ca406e200 0x7f0ca40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.068+0000 7f0cbd140700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0ca406e200 0x7f0ca40706c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8005090 tx=0x7f0ca8005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.072+0000 7f0caeffd700 1 -- 172.21.15.63:0/1450511992 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0cb804ec20 con 0x7f0cb80ffb30 2024-05-09T06:44:46.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.303+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cb804ebb0 con 0x7f0ca406e200 2024-05-09T06:44:46.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:46 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all osd 2024-05-09T06:44:46.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:46 smithi063 ceph-mon[92765]: Upgrade: Setting require_osd_release to 19 squid 2024-05-09T06:44:46.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:46 smithi063 ceph-mon[92765]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-09T06:44:46.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:46 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-09T06:44:46.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:46 smithi063 ceph-mon[92765]: osdmap e91: 6 total, 6 up, 6 in 2024-05-09T06:44:46.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:46 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:46.283 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:46 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-09T06:44:46.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.304+0000 7f0caeffd700 1 -- 172.21.15.63:0/1450511992 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0cb804ebb0 con 0x7f0ca406e200 2024-05-09T06:44:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0ca406e200 msgr2=0x7f0ca40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0ca406e200 0x7f0ca40706c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8005090 tx=0x7f0ca8005960 comp rx=0 tx=0).stop 2024-05-09T06:44:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb80ffb30 msgr2=0x7f0cb8196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb80ffb30 0x7f0cb8196640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c8f0 tx=0x7f0cb400cc00 comp rx=0 tx=0).stop 2024-05-09T06:44:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 shutdown_connections 2024-05-09T06:44:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0ca406e200 0x7f0ca40706c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb80ffb30 0x7f0cb8196640 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 --2- 172.21.15.63:0/1450511992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb8100d30 0x7f0cb8196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 >> 172.21.15.63:0/1450511992 conn(0x7f0cb80fb0d0 msgr2=0x7f0cb8069170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:46.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 shutdown_connections 2024-05-09T06:44:46.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.307+0000 7f0cbfba5700 1 -- 172.21.15.63:0/1450511992 wait complete. 2024-05-09T06:44:46.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:46 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all osd 2024-05-09T06:44:46.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:46 smithi193 ceph-mon[81665]: Upgrade: Setting require_osd_release to 19 squid 2024-05-09T06:44:46.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:46 smithi193 ceph-mon[81665]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-09T06:44:46.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:46 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-09T06:44:46.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:46 smithi193 ceph-mon[81665]: osdmap e91: 6 total, 6 up, 6 in 2024-05-09T06:44:46.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:46 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:46.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:46 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-09T06:44:46.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.417+0000 7fd23d877700 1 -- 172.21.15.63:0/4292870185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2380ffba0 msgr2=0x7fd2380fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.417+0000 7fd23d877700 1 --2- 172.21.15.63:0/4292870185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2380ffba0 0x7fd2380fffc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd228009910 tx=0x7fd228009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:46.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.417+0000 7fd23d877700 1 -- 172.21.15.63:0/4292870185 shutdown_connections 2024-05-09T06:44:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.417+0000 7fd23d877700 1 --2- 172.21.15.63:0/4292870185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd238100da0 0x7fd238101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.417+0000 7fd23d877700 1 --2- 172.21.15.63:0/4292870185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2380ffba0 0x7fd2380fffc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.417+0000 7fd23d877700 1 -- 172.21.15.63:0/4292870185 >> 172.21.15.63:0/4292870185 conn(0x7fd2380fb120 msgr2=0x7fd2380fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.417+0000 7fd23d877700 1 -- 172.21.15.63:0/4292870185 shutdown_connections 2024-05-09T06:44:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.417+0000 7fd23d877700 1 -- 172.21.15.63:0/4292870185 wait complete. 2024-05-09T06:44:46.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd23d877700 1 Processor -- start 2024-05-09T06:44:46.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd23d877700 1 -- start start 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd23d877700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2380ffba0 0x7fd238103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd23d877700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd238100da0 0x7fd238102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd23d877700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2381026c0 con 0x7fd238100da0 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd23d877700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd238102800 con 0x7fd2380ffba0 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd22e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd238100da0 0x7fd238102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd22e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd238100da0 0x7fd238102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50540/0 (socket says 172.21.15.63:50540) 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd22e7fc700 1 -- 172.21.15.63:0/2648994781 learned_addr learned my addr 172.21.15.63:0/2648994781 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd236ffd700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2380ffba0 0x7fd238103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd22e7fc700 1 -- 172.21.15.63:0/2648994781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2380ffba0 msgr2=0x7fd238103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd22e7fc700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2380ffba0 0x7fd238103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.418+0000 7fd22e7fc700 1 -- 172.21.15.63:0/2648994781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2280095f0 con 0x7fd238100da0 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.419+0000 7fd22e7fc700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd238100da0 0x7fd238102180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd22000c8a0 tx=0x7fd22000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.419+0000 7fd23c875700 1 -- 172.21.15.63:0/2648994781 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd22000cea0 con 0x7fd238100da0 2024-05-09T06:44:46.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.419+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd238102ae0 con 0x7fd238100da0 2024-05-09T06:44:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.419+0000 7fd23c875700 1 -- 172.21.15.63:0/2648994781 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd220004830 con 0x7fd238100da0 2024-05-09T06:44:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.419+0000 7fd23c875700 1 -- 172.21.15.63:0/2648994781 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2200055b0 con 0x7fd238100da0 2024-05-09T06:44:46.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.419+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23819e750 con 0x7fd238100da0 2024-05-09T06:44:46.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.420+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23804ebb0 con 0x7fd238100da0 2024-05-09T06:44:46.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.420+0000 7fd23c875700 1 -- 172.21.15.63:0/2648994781 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd2200049a0 con 0x7fd238100da0 2024-05-09T06:44:46.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.421+0000 7fd23c875700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd22406e200 0x7fd2240706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:46.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.421+0000 7fd23c875700 1 -- 172.21.15.63:0/2648994781 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fd22008df10 con 0x7fd238100da0 2024-05-09T06:44:46.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.421+0000 7fd236ffd700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd22406e200 0x7fd2240706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:46.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.421+0000 7fd236ffd700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd22406e200 0x7fd2240706c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd228006010 tx=0x7fd22800b580 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:46.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.426+0000 7fd23c875700 1 -- 172.21.15.63:0/2648994781 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd22005ac50 con 0x7fd238100da0 2024-05-09T06:44:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.644+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd238068d30 con 0x7fd22406e200 2024-05-09T06:44:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.654+0000 7fd23c875700 1 -- 172.21.15.63:0/2648994781 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd238068d30 con 0x7fd22406e200 2024-05-09T06:44:46.634 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:44:46.634 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (13m) 107s ago 15m 25.4M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:44:46.634 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (15m) 107s ago 15m 8472k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:44:46.634 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (14m) 6s ago 14m 8594k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:44:46.634 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (3m) 107s ago 15m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (3m) 6s ago 14m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (13m) 107s ago 14m 85.5M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (9m) 107s ago 9m 16.5M - 18.2.2-1297-g1758e8ea 196422f1b641 f51179a33764 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (9m) 107s ago 9m 90.4M - 18.2.2-1297-g1758e8ea 196422f1b641 4f73967cfcfc 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (9m) 6s ago 9m 141M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (9m) 6s ago 9m 18.5M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (5m) 107s ago 16m 508M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (5m) 6s ago 14m 452M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (4m) 107s ago 16m 48.4M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (4m) 6s ago 14m 36.5M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (15m) 107s ago 15m 15.8M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (14m) 6s ago 14m 16.0M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (3m) 107s ago 13m 144M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (2m) 107s ago 12m 135M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:44:46.635 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (113s) 107s ago 12m 11.6M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:44:46.636 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (77s) 6s ago 11m 108M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:44:46.636 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (44s) 6s ago 11m 96.2M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 14c9b4920f5a 2024-05-09T06:44:46.636 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (10s) 6s ago 11m 12.7M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 d219b4bb599f 2024-05-09T06:44:46.636 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 107s ago 14m 58.4M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:44:46.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd22406e200 msgr2=0x7fd2240706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd22406e200 0x7fd2240706c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd228006010 tx=0x7fd22800b580 comp rx=0 tx=0).stop 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd238100da0 msgr2=0x7fd238102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd238100da0 0x7fd238102180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd22000c8a0 tx=0x7fd22000cc60 comp rx=0 tx=0).stop 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 shutdown_connections 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd22406e200 0x7fd2240706c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2380ffba0 0x7fd238103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 --2- 172.21.15.63:0/2648994781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd238100da0 0x7fd238102180 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 >> 172.21.15.63:0/2648994781 conn(0x7fd2380fb120 msgr2=0x7fd238068620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 shutdown_connections 2024-05-09T06:44:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.657+0000 7fd23d877700 1 -- 172.21.15.63:0/2648994781 wait complete. 2024-05-09T06:44:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.764+0000 7f3907a9b700 1 -- 172.21.15.63:0/747171067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 msgr2=0x7f3900101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.764+0000 7f3907a9b700 1 --2- 172.21.15.63:0/747171067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 0x7f3900101070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009910 tx=0x7f38fc009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.764+0000 7f3907a9b700 1 -- 172.21.15.63:0/747171067 shutdown_connections 2024-05-09T06:44:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.764+0000 7f3907a9b700 1 --2- 172.21.15.63:0/747171067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3900101e50 0x7f39001022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.764+0000 7f3907a9b700 1 --2- 172.21.15.63:0/747171067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 0x7f3900101070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.764+0000 7f3907a9b700 1 -- 172.21.15.63:0/747171067 >> 172.21.15.63:0/747171067 conn(0x7f39000fc210 msgr2=0x7f39000fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.765+0000 7f3907a9b700 1 -- 172.21.15.63:0/747171067 shutdown_connections 2024-05-09T06:44:46.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.765+0000 7f3907a9b700 1 -- 172.21.15.63:0/747171067 wait complete. 2024-05-09T06:44:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.765+0000 7f3907a9b700 1 Processor -- start 2024-05-09T06:44:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.765+0000 7f3907a9b700 1 -- start start 2024-05-09T06:44:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.765+0000 7f3907a9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 0x7f3900196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.765+0000 7f3907a9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3900101e50 0x7f3900196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.765+0000 7f3907a9b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39001971c0 con 0x7f3900100c50 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.765+0000 7f3907a9b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900197300 con 0x7f3900101e50 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.765+0000 7f3905837700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 0x7f3900196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3905837700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 0x7f3900196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50558/0 (socket says 172.21.15.63:50558) 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3905837700 1 -- 172.21.15.63:0/2800600243 learned_addr learned my addr 172.21.15.63:0/2800600243 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3905036700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3900101e50 0x7f3900196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3905837700 1 -- 172.21.15.63:0/2800600243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3900101e50 msgr2=0x7f3900196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3905837700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3900101e50 0x7f3900196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3905837700 1 -- 172.21.15.63:0/2800600243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0095f0 con 0x7f3900100c50 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3905036700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3900101e50 0x7f3900196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3905837700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 0x7f3900196660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00bd70 tx=0x7f38fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f38f6ffd700 1 -- 172.21.15.63:0/2800600243 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38fc01d070 con 0x7f3900100c50 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390019bcb0 con 0x7f3900100c50 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f38f6ffd700 1 -- 172.21.15.63:0/2800600243 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38fc005dc0 con 0x7f3900100c50 2024-05-09T06:44:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f38f6ffd700 1 -- 172.21.15.63:0/2800600243 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38fc021410 con 0x7f3900100c50 2024-05-09T06:44:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.766+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390019c1a0 con 0x7f3900100c50 2024-05-09T06:44:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.767+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39001907d0 con 0x7f3900100c50 2024-05-09T06:44:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.768+0000 7f38f6ffd700 1 -- 172.21.15.63:0/2800600243 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f38fc00f820 con 0x7f3900100c50 2024-05-09T06:44:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.768+0000 7f38f6ffd700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f38ec06e210 0x7f38ec0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.768+0000 7f38f6ffd700 1 -- 172.21.15.63:0/2800600243 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f38fc08f4b0 con 0x7f3900100c50 2024-05-09T06:44:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.768+0000 7f3905036700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f38ec06e210 0x7f38ec0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:46.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.769+0000 7f3905036700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f38ec06e210 0x7f38ec0706d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005fd0 tx=0x7f38f0005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:46.773+0000 7f38f6ffd700 1 -- 172.21.15.63:0/2800600243 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f38fc05c1f0 con 0x7f3900100c50 2024-05-09T06:44:47.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.088+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f390004ebb0 con 0x7f3900100c50 2024-05-09T06:44:47.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.089+0000 7f38f6ffd700 1 -- 172.21.15.63:0/2800600243 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f38fc05bd80 con 0x7f3900100c50 2024-05-09T06:44:47.068 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:44:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:44:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:44:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:44:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 6 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 3 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 3, 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 10 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:44:47.069 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:44:47.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f38ec06e210 msgr2=0x7f38ec0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f38ec06e210 0x7f38ec0706d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005fd0 tx=0x7f38f0005ee0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 msgr2=0x7f3900196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 0x7f3900196660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00bd70 tx=0x7f38fc00be50 comp rx=0 tx=0).stop 2024-05-09T06:44:47.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 shutdown_connections 2024-05-09T06:44:47.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f38ec06e210 0x7f38ec0706d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3900100c50 0x7f3900196660 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 --2- 172.21.15.63:0/2800600243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3900101e50 0x7f3900196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 >> 172.21.15.63:0/2800600243 conn(0x7f39000fc210 msgr2=0x7f3900105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:47.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 shutdown_connections 2024-05-09T06:44:47.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.091+0000 7f3907a9b700 1 -- 172.21.15.63:0/2800600243 wait complete. 2024-05-09T06:44:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.203+0000 7f8f49e51700 1 -- 172.21.15.63:0/3364901789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f44100da0 msgr2=0x7f8f44101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.203+0000 7f8f49e51700 1 --2- 172.21.15.63:0/3364901789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f44100da0 0x7f8f44101200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009910 tx=0x7f8f38009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.203+0000 7f8f49e51700 1 -- 172.21.15.63:0/3364901789 shutdown_connections 2024-05-09T06:44:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.203+0000 7f8f49e51700 1 --2- 172.21.15.63:0/3364901789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f44100da0 0x7f8f44101200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.203+0000 7f8f49e51700 1 --2- 172.21.15.63:0/3364901789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f440ffba0 0x7f8f440fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.203+0000 7f8f49e51700 1 -- 172.21.15.63:0/3364901789 >> 172.21.15.63:0/3364901789 conn(0x7f8f440fb120 msgr2=0x7f8f440fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.203+0000 7f8f49e51700 1 -- 172.21.15.63:0/3364901789 shutdown_connections 2024-05-09T06:44:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.203+0000 7f8f49e51700 1 -- 172.21.15.63:0/3364901789 wait complete. 2024-05-09T06:44:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.204+0000 7f8f49e51700 1 Processor -- start 2024-05-09T06:44:47.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.204+0000 7f8f49e51700 1 -- start start 2024-05-09T06:44:47.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.204+0000 7f8f49e51700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f440ffba0 0x7f8f441988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.204+0000 7f8f49e51700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f44100da0 0x7f8f44198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.204+0000 7f8f49e51700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44199440 con 0x7f8f440ffba0 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.204+0000 7f8f49e51700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44199580 con 0x7f8f44100da0 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f437fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f440ffba0 0x7f8f441988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f437fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f440ffba0 0x7f8f441988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50608/0 (socket says 172.21.15.63:50608) 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f437fe700 1 -- 172.21.15.63:0/2502189421 learned_addr learned my addr 172.21.15.63:0/2502189421 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f42ffd700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f44100da0 0x7f8f44198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f437fe700 1 -- 172.21.15.63:0/2502189421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f44100da0 msgr2=0x7f8f44198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f437fe700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f44100da0 0x7f8f44198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f437fe700 1 -- 172.21.15.63:0/2502189421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380095f0 con 0x7f8f440ffba0 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f42ffd700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f44100da0 0x7f8f44198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f437fe700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f440ffba0 0x7f8f441988e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400d900 tx=0x7f8f3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f40ff9700 1 -- 172.21.15.63:0/2502189421 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f340041f0 con 0x7f8f440ffba0 2024-05-09T06:44:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f40ff9700 1 -- 172.21.15.63:0/2502189421 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f34004350 con 0x7f8f440ffba0 2024-05-09T06:44:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4419df90 con 0x7f8f440ffba0 2024-05-09T06:44:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f40ff9700 1 -- 172.21.15.63:0/2502189421 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f34003dc0 con 0x7f8f440ffba0 2024-05-09T06:44:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.205+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4419e450 con 0x7f8f440ffba0 2024-05-09T06:44:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.206+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4404ebb0 con 0x7f8f440ffba0 2024-05-09T06:44:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.208+0000 7f8f40ff9700 1 -- 172.21.15.63:0/2502189421 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8f34009730 con 0x7f8f440ffba0 2024-05-09T06:44:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.208+0000 7f8f40ff9700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8f2c06e200 0x7f8f2c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.208+0000 7f8f40ff9700 1 -- 172.21.15.63:0/2502189421 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f8f3408e410 con 0x7f8f440ffba0 2024-05-09T06:44:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.208+0000 7f8f42ffd700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8f2c06e200 0x7f8f2c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.209+0000 7f8f42ffd700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8f2c06e200 0x7f8f2c0706c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f380051c0 tx=0x7f8f38005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:47.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.213+0000 7f8f40ff9700 1 -- 172.21.15.63:0/2502189421 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8f3405b150 con 0x7f8f440ffba0 2024-05-09T06:44:47.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:47.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-09T06:44:47.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: osdmap e92: 6 total, 6 up, 6 in 2024-05-09T06:44:47.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: pgmap v182: 65 pgs: 11 active+undersized, 6 active+undersized+degraded, 48 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 18/228 objects degraded (7.895%) 2024-05-09T06:44:47.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:47.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:47.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-09T06:44:47.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 2 up:standby 2024-05-09T06:44:47.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:47.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:47.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:44:47.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:47.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:47.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-09T06:44:47.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: osdmap e92: 6 total, 6 up, 6 in 2024-05-09T06:44:47.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: pgmap v182: 65 pgs: 11 active+undersized, 6 active+undersized+degraded, 48 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 18/228 objects degraded (7.895%) 2024-05-09T06:44:47.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:47.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:47.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-09T06:44:47.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 2 up:standby 2024-05-09T06:44:47.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:44:47.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:47.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.471+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8f44108990 con 0x7f8f440ffba0 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.471+0000 7f8f40ff9700 1 -- 172.21.15.63:0/2502189421 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1710 (secure 0 0 0) 0x7f8f3405ace0 con 0x7f8f440ffba0 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:e16 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 16 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:44:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:44:47.035052+0000 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 0 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:44:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 24277 members: 24277 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1277475441,v1:172.21.15.63:6827/1277475441] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:44:47.453 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:44:47.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8f2c06e200 msgr2=0x7f8f2c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8f2c06e200 0x7f8f2c0706c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f380051c0 tx=0x7f8f38005a90 comp rx=0 tx=0).stop 2024-05-09T06:44:47.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f440ffba0 msgr2=0x7f8f441988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f440ffba0 0x7f8f441988e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400d900 tx=0x7f8f3400dcc0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 shutdown_connections 2024-05-09T06:44:47.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f8f2c06e200 0x7f8f2c0706c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f440ffba0 0x7f8f441988e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 --2- 172.21.15.63:0/2502189421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f44100da0 0x7f8f44198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 >> 172.21.15.63:0/2502189421 conn(0x7f8f440fb120 msgr2=0x7f8f44068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:47.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 shutdown_connections 2024-05-09T06:44:47.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.474+0000 7f8f49e51700 1 -- 172.21.15.63:0/2502189421 wait complete. 2024-05-09T06:44:47.455 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 16 2024-05-09T06:44:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.586+0000 7f7534c23700 1 -- 172.21.15.63:0/1221286715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530101e20 msgr2=0x7f7530102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.586+0000 7f7534c23700 1 --2- 172.21.15.63:0/1221286715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530101e20 0x7f7530102280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7518009910 tx=0x7f7518009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.586+0000 7f7534c23700 1 -- 172.21.15.63:0/1221286715 shutdown_connections 2024-05-09T06:44:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.586+0000 7f7534c23700 1 --2- 172.21.15.63:0/1221286715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530101e20 0x7f7530102280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.586+0000 7f7534c23700 1 --2- 172.21.15.63:0/1221286715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7530100c20 0x7f7530101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.586+0000 7f7534c23700 1 -- 172.21.15.63:0/1221286715 >> 172.21.15.63:0/1221286715 conn(0x7f75300fc200 msgr2=0x7f75300fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.586+0000 7f7534c23700 1 -- 172.21.15.63:0/1221286715 shutdown_connections 2024-05-09T06:44:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.586+0000 7f7534c23700 1 -- 172.21.15.63:0/1221286715 wait complete. 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.587+0000 7f7534c23700 1 Processor -- start 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.587+0000 7f7534c23700 1 -- start start 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.587+0000 7f7534c23700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530100c20 0x7f7530196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.587+0000 7f7534c23700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7530101e20 0x7f7530196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.587+0000 7f7534c23700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7530197170 con 0x7f7530100c20 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.587+0000 7f7534c23700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75301972b0 con 0x7f7530101e20 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.587+0000 7f752e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530100c20 0x7f7530196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.587+0000 7f752e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530100c20 0x7f7530196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50622/0 (socket says 172.21.15.63:50622) 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.587+0000 7f752e59c700 1 -- 172.21.15.63:0/577411275 learned_addr learned my addr 172.21.15.63:0/577411275 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f752dd9b700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7530101e20 0x7f7530196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f752e59c700 1 -- 172.21.15.63:0/577411275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7530101e20 msgr2=0x7f7530196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f752e59c700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7530101e20 0x7f7530196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f752e59c700 1 -- 172.21.15.63:0/577411275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180095f0 con 0x7f7530100c20 2024-05-09T06:44:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f752e59c700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530100c20 0x7f7530196610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f752000e8f0 tx=0x7f752000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f75277fe700 1 -- 172.21.15.63:0/577411275 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7520009790 con 0x7f7530100c20 2024-05-09T06:44:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f75277fe700 1 -- 172.21.15.63:0/577411275 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75200044e0 con 0x7f7530100c20 2024-05-09T06:44:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753019bcc0 con 0x7f7530100c20 2024-05-09T06:44:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753019c240 con 0x7f7530100c20 2024-05-09T06:44:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.588+0000 7f75277fe700 1 -- 172.21.15.63:0/577411275 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7520003e00 con 0x7f7530100c20 2024-05-09T06:44:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.589+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75301907a0 con 0x7f7530100c20 2024-05-09T06:44:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.590+0000 7f75277fe700 1 -- 172.21.15.63:0/577411275 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f752000cba0 con 0x7f7530100c20 2024-05-09T06:44:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.590+0000 7f75277fe700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f751c06e200 0x7f751c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.590+0000 7f75277fe700 1 -- 172.21.15.63:0/577411275 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f7520014070 con 0x7f7530100c20 2024-05-09T06:44:47.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.590+0000 7f752dd9b700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f751c06e200 0x7f751c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:47.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.591+0000 7f752dd9b700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f751c06e200 0x7f751c0706c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f751800b800 tx=0x7f7518009f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:47.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.595+0000 7f75277fe700 1 -- 172.21.15.63:0/577411275 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f752005b330 con 0x7f7530100c20 2024-05-09T06:44:47.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.846+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f753002ceb0 con 0x7f751c06e200 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.848+0000 7f75277fe700 1 -- 172.21.15.63:0/577411275 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f753002ceb0 con 0x7f751c06e200 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:44:47.827 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:44:47.828 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:44:47.828 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "12/23 daemons upgraded", 2024-05-09T06:44:47.828 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mds daemons", 2024-05-09T06:44:47.828 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:44:47.828 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:44:47.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f751c06e200 msgr2=0x7f751c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f751c06e200 0x7f751c0706c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f751800b800 tx=0x7f7518009f90 comp rx=0 tx=0).stop 2024-05-09T06:44:47.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530100c20 msgr2=0x7f7530196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530100c20 0x7f7530196610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f752000e8f0 tx=0x7f752000ecb0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 shutdown_connections 2024-05-09T06:44:47.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f751c06e200 0x7f751c0706c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7530100c20 0x7f7530196610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 --2- 172.21.15.63:0/577411275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7530101e20 0x7f7530196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 >> 172.21.15.63:0/577411275 conn(0x7f75300fc200 msgr2=0x7f7530105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:47.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 shutdown_connections 2024-05-09T06:44:47.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.850+0000 7f7534c23700 1 -- 172.21.15.63:0/577411275 wait complete. 2024-05-09T06:44:47.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.961+0000 7f406805b700 1 -- 172.21.15.63:0/1912155151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060100c20 msgr2=0x7f4060101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.961+0000 7f406805b700 1 --2- 172.21.15.63:0/1912155151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060100c20 0x7f4060101040 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f405c009910 tx=0x7f405c009c20 comp rx=0 tx=0).stop 2024-05-09T06:44:47.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.961+0000 7f406805b700 1 -- 172.21.15.63:0/1912155151 shutdown_connections 2024-05-09T06:44:47.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.961+0000 7f406805b700 1 --2- 172.21.15.63:0/1912155151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4060101e20 0x7f4060102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.961+0000 7f406805b700 1 --2- 172.21.15.63:0/1912155151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060100c20 0x7f4060101040 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.961+0000 7f406805b700 1 -- 172.21.15.63:0/1912155151 >> 172.21.15.63:0/1912155151 conn(0x7f40600fc200 msgr2=0x7f40600fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:47.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.961+0000 7f406805b700 1 -- 172.21.15.63:0/1912155151 shutdown_connections 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.961+0000 7f406805b700 1 -- 172.21.15.63:0/1912155151 wait complete. 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f406805b700 1 Processor -- start 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f406805b700 1 -- start start 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f406805b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4060100c20 0x7f4060196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f406805b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060101e20 0x7f4060196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f406805b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601971a0 con 0x7f4060101e20 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f406805b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601972e0 con 0x7f4060100c20 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f40655f6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060101e20 0x7f4060196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f40655f6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060101e20 0x7f4060196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50640/0 (socket says 172.21.15.63:50640) 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f40655f6700 1 -- 172.21.15.63:0/1612548918 learned_addr learned my addr 172.21.15.63:0/1612548918 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f4065df7700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4060100c20 0x7f4060196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f40655f6700 1 -- 172.21.15.63:0/1612548918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4060100c20 msgr2=0x7f4060196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f40655f6700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4060100c20 0x7f4060196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f40655f6700 1 -- 172.21.15.63:0/1612548918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c0095f0 con 0x7f4060101e20 2024-05-09T06:44:47.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f4065df7700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4060100c20 0x7f4060196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:44:47.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.962+0000 7f40655f6700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060101e20 0x7f4060196b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f405000d900 tx=0x7f405000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:47.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.963+0000 7f4056ffd700 1 -- 172.21.15.63:0/1612548918 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40500041f0 con 0x7f4060101e20 2024-05-09T06:44:47.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.963+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406019bd40 con 0x7f4060101e20 2024-05-09T06:44:47.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.963+0000 7f4056ffd700 1 -- 172.21.15.63:0/1612548918 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4050004cf0 con 0x7f4060101e20 2024-05-09T06:44:47.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.963+0000 7f4056ffd700 1 -- 172.21.15.63:0/1612548918 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f405000b750 con 0x7f4060101e20 2024-05-09T06:44:47.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.963+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406019c290 con 0x7f4060101e20 2024-05-09T06:44:47.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.964+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40601907a0 con 0x7f4060101e20 2024-05-09T06:44:47.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.964+0000 7f4056ffd700 1 -- 172.21.15.63:0/1612548918 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4050004350 con 0x7f4060101e20 2024-05-09T06:44:47.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.965+0000 7f4056ffd700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f404c06e210 0x7f404c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:44:47.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.965+0000 7f4056ffd700 1 -- 172.21.15.63:0/1612548918 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f405008e230 con 0x7f4060101e20 2024-05-09T06:44:47.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.965+0000 7f4065df7700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f404c06e210 0x7f404c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:44:47.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.965+0000 7f4065df7700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f404c06e210 0x7f404c0706d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f405c006010 tx=0x7f405c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:44:47.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:47.969+0000 7f4056ffd700 1 -- 172.21.15.63:0/1612548918 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f405005af70 con 0x7f4060101e20 2024-05-09T06:44:48.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.270+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f406004ebb0 con 0x7f4060101e20 2024-05-09T06:44:48.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.270+0000 7f4056ffd700 1 -- 172.21.15.63:0/1612548918 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+515 (secure 0 0 0) 0x7f405005ab00 con 0x7f4060101e20 2024-05-09T06:44:48.251 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN Degraded data redundancy: 18/228 objects degraded (7.895%), 6 pgs degraded 2024-05-09T06:44:48.251 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 18/228 objects degraded (7.895%), 6 pgs degraded 2024-05-09T06:44:48.251 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-05-09T06:44:48.251 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-05-09T06:44:48.252 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-05-09T06:44:48.252 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.18 is active+undersized+degraded, acting [4,1] 2024-05-09T06:44:48.252 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1a is active+undersized+degraded, acting [1,4] 2024-05-09T06:44:48.252 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1d is active+undersized+degraded, acting [1,0] 2024-05-09T06:44:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f404c06e210 msgr2=0x7f404c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f404c06e210 0x7f404c0706d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f405c006010 tx=0x7f405c00b540 comp rx=0 tx=0).stop 2024-05-09T06:44:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060101e20 msgr2=0x7f4060196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:44:48.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060101e20 0x7f4060196b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f405000d900 tx=0x7f405000dcc0 comp rx=0 tx=0).stop 2024-05-09T06:44:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 shutdown_connections 2024-05-09T06:44:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f404c06e210 0x7f404c0706d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4060100c20 0x7f4060196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 --2- 172.21.15.63:0/1612548918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4060101e20 0x7f4060196b80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:44:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 >> 172.21.15.63:0/1612548918 conn(0x7f40600fc200 msgr2=0x7f4060105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:44:48.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 shutdown_connections 2024-05-09T06:44:48.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:44:48.273+0000 7f406805b700 1 -- 172.21.15.63:0/1612548918 wait complete. 2024-05-09T06:44:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:48 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2800600243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi063.aoiibw"]}]: dispatch 2024-05-09T06:44:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:48 smithi063 ceph-mon[92765]: Upgrade: It appears safe to stop mds.cephfs.smithi063.aoiibw 2024-05-09T06:44:48.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:48 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2502189421' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:44:48.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:48 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2800600243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:48.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:44:48.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi063.aoiibw"]}]: dispatch 2024-05-09T06:44:48.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:48 smithi193 ceph-mon[81665]: Upgrade: It appears safe to stop mds.cephfs.smithi063.aoiibw 2024-05-09T06:44:48.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:48 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2502189421' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:44:49.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:49 smithi063 ceph-mon[92765]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:49.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:49 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.63:6828/311216115,v1:172.21.15.63:6829/311216115] up:boot 2024-05-09T06:44:49.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:49 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 3 up:standby 2024-05-09T06:44:49.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:44:49.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:49 smithi063 ceph-mon[92765]: pgmap v183: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 803 B/s rd, 1 op/s 2024-05-09T06:44:49.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:49 smithi063 ceph-mon[92765]: Health check update: Degraded data redundancy: 18/228 objects degraded (7.895%), 6 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:49.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:49 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1612548918' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:44:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:49 smithi193 ceph-mon[81665]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:44:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:49 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.63:6828/311216115,v1:172.21.15.63:6829/311216115] up:boot 2024-05-09T06:44:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:49 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 3 up:standby 2024-05-09T06:44:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:44:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:49 smithi193 ceph-mon[81665]: pgmap v183: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 803 B/s rd, 1 op/s 2024-05-09T06:44:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:49 smithi193 ceph-mon[81665]: Health check update: Degraded data redundancy: 18/228 objects degraded (7.895%), 6 pgs degraded (PG_DEGRADED) 2024-05-09T06:44:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:49 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1612548918' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:44:50.348 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:50 smithi063 ceph-mon[92765]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/228 objects degraded (7.895%), 6 pgs degraded) 2024-05-09T06:44:50.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:50 smithi063 ceph-mon[92765]: Cluster is now healthy 2024-05-09T06:44:50.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:50 smithi063 ceph-mon[92765]: Upgrade: Updating mds.cephfs.smithi063.aoiibw 2024-05-09T06:44:50.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:50 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:50.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:50 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.aoiibw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:44:50.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:50 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:50.349 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:50 smithi063 ceph-mon[92765]: Deploying daemon mds.cephfs.smithi063.aoiibw on smithi063 2024-05-09T06:44:50.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:50 smithi193 ceph-mon[81665]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/228 objects degraded (7.895%), 6 pgs degraded) 2024-05-09T06:44:50.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:50 smithi193 ceph-mon[81665]: Cluster is now healthy 2024-05-09T06:44:50.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:50 smithi193 ceph-mon[81665]: Upgrade: Updating mds.cephfs.smithi063.aoiibw 2024-05-09T06:44:50.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:50 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:50.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:50 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.aoiibw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:44:50.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:50 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:44:50.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:50 smithi193 ceph-mon[81665]: Deploying daemon mds.cephfs.smithi063.aoiibw on smithi063 2024-05-09T06:44:51.347 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:51 smithi063 ceph-mon[92765]: pgmap v184: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 671 B/s rd, 1 op/s 2024-05-09T06:44:51.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:51 smithi193 ceph-mon[81665]: pgmap v184: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 671 B/s rd, 1 op/s 2024-05-09T06:44:53.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:53 smithi063 ceph-mon[92765]: osdmap e93: 6 total, 6 up, 6 in 2024-05-09T06:44:53.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:53 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 2 up:standby 2024-05-09T06:44:53.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:53 smithi063 ceph-mon[92765]: pgmap v186: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 167 B/s rd, 0 op/s 2024-05-09T06:44:53.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:53 smithi193 ceph-mon[81665]: osdmap e93: 6 total, 6 up, 6 in 2024-05-09T06:44:53.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:53 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 2 up:standby 2024-05-09T06:44:53.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:53 smithi193 ceph-mon[81665]: pgmap v186: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 167 B/s rd, 0 op/s 2024-05-09T06:44:54.350 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:44:54.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:44:55.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:55 smithi063 ceph-mon[92765]: pgmap v187: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 126 B/s rd, 0 op/s 2024-05-09T06:44:55.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:55.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:55.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:55.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:55 smithi193 ceph-mon[81665]: pgmap v187: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 126 B/s rd, 0 op/s 2024-05-09T06:44:55.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:55.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:55.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:44:56.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:56 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] up:boot 2024-05-09T06:44:56.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:56 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 3 up:standby 2024-05-09T06:44:56.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:56 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:44:56.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:56 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] up:boot 2024-05-09T06:44:56.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:56 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 3 up:standby 2024-05-09T06:44:56.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:56 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:44:57.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:57 smithi063 ceph-mon[92765]: pgmap v188: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-05-09T06:44:57.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:57 smithi193 ceph-mon[81665]: pgmap v188: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-05-09T06:44:58.400 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:58 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:58.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:58 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:44:58.401 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:58 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:58.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:58 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:58.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:58 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:44:58.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:58 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:44:59.396 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:44:59 smithi063 ceph-mon[92765]: pgmap v189: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-05-09T06:44:59.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:44:59 smithi193 ceph-mon[81665]: pgmap v189: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-05-09T06:45:01.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:00 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:01.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:00 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:01.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:00 smithi063 ceph-mon[92765]: pgmap v190: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-05-09T06:45:01.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:00 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:01.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:00 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:01.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:00 smithi193 ceph-mon[81665]: pgmap v190: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-05-09T06:45:02.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:02 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:02.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:02 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:02.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:02 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:02.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:02 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:03.275 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:03 smithi063 ceph-mon[92765]: pgmap v191: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:03.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:03 smithi193 ceph-mon[81665]: pgmap v191: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:05.447 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: pgmap v192: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:05.447 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:05.447 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:05.447 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:05.447 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:45:05.448 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:05.448 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:05.448 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:05.448 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:05.448 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:05.448 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:05.448 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi063.fbzxbf"]}]: dispatch 2024-05-09T06:45:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: pgmap v192: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:45:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:05.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:05.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:05.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi063.fbzxbf"]}]: dispatch 2024-05-09T06:45:06.528 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:06 smithi063 ceph-mon[92765]: Upgrade: It appears safe to stop mds.cephfs.smithi063.fbzxbf 2024-05-09T06:45:06.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:06 smithi193 ceph-mon[81665]: Upgrade: It appears safe to stop mds.cephfs.smithi063.fbzxbf 2024-05-09T06:45:07.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:07 smithi063 ceph-mon[92765]: pgmap v193: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:07.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:07 smithi063 ceph-mon[92765]: Upgrade: Updating mds.cephfs.smithi063.fbzxbf 2024-05-09T06:45:07.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:07.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.fbzxbf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:45:07.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:07.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:07 smithi063 ceph-mon[92765]: Deploying daemon mds.cephfs.smithi063.fbzxbf on smithi063 2024-05-09T06:45:07.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:07 smithi193 ceph-mon[81665]: pgmap v193: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:07.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:07 smithi193 ceph-mon[81665]: Upgrade: Updating mds.cephfs.smithi063.fbzxbf 2024-05-09T06:45:07.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:07.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.fbzxbf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:45:07.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:07.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:07 smithi193 ceph-mon[81665]: Deploying daemon mds.cephfs.smithi063.fbzxbf on smithi063 2024-05-09T06:45:09.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:09 smithi193 ceph-mon[81665]: pgmap v194: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:10.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:09 smithi063 ceph-mon[92765]: pgmap v194: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:10.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:10 smithi193 ceph-mon[81665]: osdmap e94: 6 total, 6 up, 6 in 2024-05-09T06:45:10.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:10 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 2 up:standby 2024-05-09T06:45:11.087 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:10 smithi063 ceph-mon[92765]: osdmap e94: 6 total, 6 up, 6 in 2024-05-09T06:45:11.087 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:10 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 2 up:standby 2024-05-09T06:45:11.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:11 smithi063 ceph-mon[92765]: pgmap v196: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:11.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:11 smithi193 ceph-mon[81665]: pgmap v196: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:12.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:45:13.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:45:13.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:13 smithi063 ceph-mon[92765]: pgmap v197: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:13.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:13 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.63:6828/2182150991,v1:172.21.15.63:6829/2182150991] up:boot 2024-05-09T06:45:13.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:13 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 3 up:standby 2024-05-09T06:45:13.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:45:13.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:13.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:13.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:14.039 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:13 smithi193 ceph-mon[81665]: pgmap v197: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:14.039 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:13 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.63:6828/2182150991,v1:172.21.15.63:6829/2182150991] up:boot 2024-05-09T06:45:14.039 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:13 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi193.ggtmyb=up:active} 3 up:standby 2024-05-09T06:45:14.040 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:45:14.040 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:14.040 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:14.040 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:15.931 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:15 smithi063 ceph-mon[92765]: pgmap v198: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:16.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:15 smithi193 ceph-mon[81665]: pgmap v198: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:17.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:17 smithi193 ceph-mon[81665]: pgmap v199: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:18.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:17 smithi063 ceph-mon[92765]: pgmap v199: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:18.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.387+0000 7f0934766700 1 -- 172.21.15.63:0/115230208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 msgr2=0x7f092c1063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:18.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.387+0000 7f0934766700 1 --2- 172.21.15.63:0/115230208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 0x7f092c1063d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f091c009910 tx=0x7f091c009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:18.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.388+0000 7f0934766700 1 -- 172.21.15.63:0/115230208 shutdown_connections 2024-05-09T06:45:18.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.388+0000 7f0934766700 1 --2- 172.21.15.63:0/115230208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 0x7f092c1063d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.388+0000 7f0934766700 1 --2- 172.21.15.63:0/115230208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f092c1016b0 0x7f092c103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.388+0000 7f0934766700 1 -- 172.21.15.63:0/115230208 >> 172.21.15.63:0/115230208 conn(0x7f092c0fb0c0 msgr2=0x7f092c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:18.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.388+0000 7f0934766700 1 -- 172.21.15.63:0/115230208 shutdown_connections 2024-05-09T06:45:18.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.388+0000 7f0934766700 1 -- 172.21.15.63:0/115230208 wait complete. 2024-05-09T06:45:18.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.388+0000 7f0934766700 1 Processor -- start 2024-05-09T06:45:18.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.388+0000 7f0934766700 1 -- start start 2024-05-09T06:45:18.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0934766700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f092c1016b0 0x7f092c196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:18.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0934766700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 0x7f092c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:18.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0934766700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c197260 con 0x7f092c103fe0 2024-05-09T06:45:18.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0934766700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c1973a0 con 0x7f092c1016b0 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0931d01700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 0x7f092c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0931d01700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 0x7f092c196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59342/0 (socket says 172.21.15.63:59342) 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0931d01700 1 -- 172.21.15.63:0/3196043787 learned_addr learned my addr 172.21.15.63:0/3196043787 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0932502700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f092c1016b0 0x7f092c196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0931d01700 1 -- 172.21.15.63:0/3196043787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f092c1016b0 msgr2=0x7f092c196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0931d01700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f092c1016b0 0x7f092c196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0931d01700 1 -- 172.21.15.63:0/3196043787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0095f0 con 0x7f092c103fe0 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.389+0000 7f0931d01700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 0x7f092c196c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f091c006010 tx=0x7f091c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.390+0000 7f09237fe700 1 -- 172.21.15.63:0/3196043787 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f091c01d070 con 0x7f092c103fe0 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.390+0000 7f09237fe700 1 -- 172.21.15.63:0/3196043787 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f091c00f460 con 0x7f092c103fe0 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.390+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c19bda0 con 0x7f092c103fe0 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.390+0000 7f09237fe700 1 -- 172.21.15.63:0/3196043787 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f091c021620 con 0x7f092c103fe0 2024-05-09T06:45:18.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.390+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c19c310 con 0x7f092c103fe0 2024-05-09T06:45:18.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.391+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c04ebb0 con 0x7f092c103fe0 2024-05-09T06:45:18.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.391+0000 7f09237fe700 1 -- 172.21.15.63:0/3196043787 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f091c02b440 con 0x7f092c103fe0 2024-05-09T06:45:18.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.392+0000 7f09237fe700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f091806e210 0x7f09180706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:18.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.392+0000 7f09237fe700 1 -- 172.21.15.63:0/3196043787 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f091c08f4f0 con 0x7f092c103fe0 2024-05-09T06:45:18.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.392+0000 7f0932502700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f091806e210 0x7f09180706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:18.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.393+0000 7f0932502700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f091806e210 0x7f09180706d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0928009a10 tx=0x7f0928009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:18.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.397+0000 7f09237fe700 1 -- 172.21.15.63:0/3196043787 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f091c05c2b0 con 0x7f092c103fe0 2024-05-09T06:45:18.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.628+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f092c104a40 con 0x7f091806e210 2024-05-09T06:45:18.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.629+0000 7f09237fe700 1 -- 172.21.15.63:0/3196043787 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f092c104a40 con 0x7f091806e210 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.631+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f091806e210 msgr2=0x7f09180706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.631+0000 7f0934766700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f091806e210 0x7f09180706d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0928009a10 tx=0x7f0928009190 comp rx=0 tx=0).stop 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.631+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 msgr2=0x7f092c196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.631+0000 7f0934766700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 0x7f092c196c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f091c006010 tx=0x7f091c00bd70 comp rx=0 tx=0).stop 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.631+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 shutdown_connections 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.631+0000 7f0934766700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f091806e210 0x7f09180706d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.631+0000 7f0934766700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f092c1016b0 0x7f092c196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.631+0000 7f0934766700 1 --2- 172.21.15.63:0/3196043787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c103fe0 0x7f092c196c40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.631+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 >> 172.21.15.63:0/3196043787 conn(0x7f092c0fb0c0 msgr2=0x7f092c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.632+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 shutdown_connections 2024-05-09T06:45:18.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.632+0000 7f0934766700 1 -- 172.21.15.63:0/3196043787 wait complete. 2024-05-09T06:45:18.622 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:45:18.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.745+0000 7f3645a59700 1 -- 172.21.15.63:0/1264254943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640101760 msgr2=0x7f3640103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:18.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.745+0000 7f3645a59700 1 --2- 172.21.15.63:0/1264254943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640101760 0x7f3640103b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3630009910 tx=0x7f3630009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:18.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.746+0000 7f3645a59700 1 -- 172.21.15.63:0/1264254943 shutdown_connections 2024-05-09T06:45:18.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.746+0000 7f3645a59700 1 --2- 172.21.15.63:0/1264254943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3640104090 0x7f3640106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.746+0000 7f3645a59700 1 --2- 172.21.15.63:0/1264254943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640101760 0x7f3640103b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.746+0000 7f3645a59700 1 -- 172.21.15.63:0/1264254943 >> 172.21.15.63:0/1264254943 conn(0x7f36400fb0d0 msgr2=0x7f36400fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:18.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.746+0000 7f3645a59700 1 -- 172.21.15.63:0/1264254943 shutdown_connections 2024-05-09T06:45:18.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.746+0000 7f3645a59700 1 -- 172.21.15.63:0/1264254943 wait complete. 2024-05-09T06:45:18.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.746+0000 7f3645a59700 1 Processor -- start 2024-05-09T06:45:18.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.746+0000 7f3645a59700 1 -- start start 2024-05-09T06:45:18.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f3645a59700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3640101760 0x7f3640196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:18.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f3645a59700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640104090 0x7f3640196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:18.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f3645a59700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36401971a0 con 0x7f3640104090 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f3645a59700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36401972e0 con 0x7f3640101760 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f363e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640104090 0x7f3640196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f363e7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640104090 0x7f3640196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59360/0 (socket says 172.21.15.63:59360) 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f363e7fc700 1 -- 172.21.15.63:0/2594371858 learned_addr learned my addr 172.21.15.63:0/2594371858 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f363effd700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3640101760 0x7f3640196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f363e7fc700 1 -- 172.21.15.63:0/2594371858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3640101760 msgr2=0x7f3640196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f363e7fc700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3640101760 0x7f3640196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f363e7fc700 1 -- 172.21.15.63:0/2594371858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300095f0 con 0x7f3640104090 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f363e7fc700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640104090 0x7f3640196b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f362800d8d0 tx=0x7f362800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f3644a57700 1 -- 172.21.15.63:0/2594371858 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3628009940 con 0x7f3640104090 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364019bd40 con 0x7f3640104090 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.747+0000 7f3644a57700 1 -- 172.21.15.63:0/2594371858 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3628010460 con 0x7f3640104090 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.748+0000 7f3644a57700 1 -- 172.21.15.63:0/2594371858 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f362800f5f0 con 0x7f3640104090 2024-05-09T06:45:18.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.748+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364019c310 con 0x7f3640104090 2024-05-09T06:45:18.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.749+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364004ebb0 con 0x7f3640104090 2024-05-09T06:45:18.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.749+0000 7f3644a57700 1 -- 172.21.15.63:0/2594371858 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f36280105d0 con 0x7f3640104090 2024-05-09T06:45:18.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.750+0000 7f3644a57700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f362c06e200 0x7f362c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:18.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.750+0000 7f3644a57700 1 -- 172.21.15.63:0/2594371858 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f362808e240 con 0x7f3640104090 2024-05-09T06:45:18.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.750+0000 7f363effd700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f362c06e200 0x7f362c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:18.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.750+0000 7f363effd700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f362c06e200 0x7f362c0706c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36300051c0 tx=0x7f3630005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:18.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.754+0000 7f3644a57700 1 -- 172.21.15.63:0/2594371858 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f362805af80 con 0x7f3640104090 2024-05-09T06:45:18.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.982+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3640104ab0 con 0x7f362c06e200 2024-05-09T06:45:18.962 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:18 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:18.962 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:18 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:18.962 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:18 smithi063 ceph-mon[92765]: pgmap v200: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:18.962 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:18 smithi063 ceph-mon[92765]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:18.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.984+0000 7f3644a57700 1 -- 172.21.15.63:0/2594371858 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3640104ab0 con 0x7f362c06e200 2024-05-09T06:45:18.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f362c06e200 msgr2=0x7f362c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:18.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f362c06e200 0x7f362c0706c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36300051c0 tx=0x7f3630005a90 comp rx=0 tx=0).stop 2024-05-09T06:45:18.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640104090 msgr2=0x7f3640196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:18.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640104090 0x7f3640196b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f362800d8d0 tx=0x7f362800dc90 comp rx=0 tx=0).stop 2024-05-09T06:45:18.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 shutdown_connections 2024-05-09T06:45:18.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f362c06e200 0x7f362c0706c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3640101760 0x7f3640196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 --2- 172.21.15.63:0/2594371858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3640104090 0x7f3640196b80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:18.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 >> 172.21.15.63:0/2594371858 conn(0x7f36400fb0d0 msgr2=0x7f36400fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:18.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 shutdown_connections 2024-05-09T06:45:18.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:18.987+0000 7f3645a59700 1 -- 172.21.15.63:0/2594371858 wait complete. 2024-05-09T06:45:19.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.114+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2680297611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c0feee0 msgr2=0x7f2f1c0ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.114+0000 7f2f21bc4700 1 --2- 172.21.15.63:0/2680297611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c0feee0 0x7f2f1c0ff300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009910 tx=0x7f2f0c009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:19.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.114+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2680297611 shutdown_connections 2024-05-09T06:45:19.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.114+0000 7f2f21bc4700 1 --2- 172.21.15.63:0/2680297611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f1c101ec0 0x7f2f1c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.114+0000 7f2f21bc4700 1 --2- 172.21.15.63:0/2680297611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c0feee0 0x7f2f1c0ff300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.114+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2680297611 >> 172.21.15.63:0/2680297611 conn(0x7f2f1c0fa840 msgr2=0x7f2f1c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:19.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.114+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2680297611 shutdown_connections 2024-05-09T06:45:19.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.114+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2680297611 wait complete. 2024-05-09T06:45:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f21bc4700 1 Processor -- start 2024-05-09T06:45:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f21bc4700 1 -- start start 2024-05-09T06:45:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f21bc4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f1c0feee0 0x7f2f1c1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f21bc4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c101ec0 0x7f2f1c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f21bc4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c197240 con 0x7f2f1c101ec0 2024-05-09T06:45:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f21bc4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c197380 con 0x7f2f1c0feee0 2024-05-09T06:45:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f1affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c101ec0 0x7f2f1c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:19.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f1affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c101ec0 0x7f2f1c196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59380/0 (socket says 172.21.15.63:59380) 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f1affd700 1 -- 172.21.15.63:0/2968208126 learned_addr learned my addr 172.21.15.63:0/2968208126 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f1b7fe700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f1c0feee0 0x7f2f1c1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f1affd700 1 -- 172.21.15.63:0/2968208126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f1c0feee0 msgr2=0x7f2f1c1966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.115+0000 7f2f1affd700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f1c0feee0 0x7f2f1c1966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.116+0000 7f2f1affd700 1 -- 172.21.15.63:0/2968208126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c0095f0 con 0x7f2f1c101ec0 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.116+0000 7f2f1affd700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c101ec0 0x7f2f1c196c20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000d900 tx=0x7f2f1000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.116+0000 7f2f18ff9700 1 -- 172.21.15.63:0/2968208126 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f10004a00 con 0x7f2f1c101ec0 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.116+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c19bd90 con 0x7f2f1c101ec0 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.116+0000 7f2f18ff9700 1 -- 172.21.15.63:0/2968208126 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f100054e0 con 0x7f2f1c101ec0 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.117+0000 7f2f18ff9700 1 -- 172.21.15.63:0/2968208126 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f10009d00 con 0x7f2f1c101ec0 2024-05-09T06:45:19.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.117+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c19c330 con 0x7f2f1c101ec0 2024-05-09T06:45:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.118+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c1907c0 con 0x7f2f1c101ec0 2024-05-09T06:45:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.118+0000 7f2f18ff9700 1 -- 172.21.15.63:0/2968208126 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2f10010460 con 0x7f2f1c101ec0 2024-05-09T06:45:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.118+0000 7f2f18ff9700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f0406e210 0x7f2f040706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.118+0000 7f2f18ff9700 1 -- 172.21.15.63:0/2968208126 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f2f1008e200 con 0x7f2f1c101ec0 2024-05-09T06:45:19.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.118+0000 7f2f1b7fe700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f0406e210 0x7f2f040706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:19.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.119+0000 7f2f1b7fe700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f0406e210 0x7f2f040706d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c005090 tx=0x7f2f0c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:19.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.123+0000 7f2f18ff9700 1 -- 172.21.15.63:0/2968208126 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2f1005afb0 con 0x7f2f1c101ec0 2024-05-09T06:45:19.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:18 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:19.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:18 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:19.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:18 smithi193 ceph-mon[81665]: pgmap v200: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:19.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:18 smithi193 ceph-mon[81665]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:19.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.343+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f1c02ce00 con 0x7f2f0406e210 2024-05-09T06:45:19.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.365+0000 7f2f18ff9700 1 -- 172.21.15.63:0/2968208126 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2f1c02ce00 con 0x7f2f0406e210 2024-05-09T06:45:19.345 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:45:19.345 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (14m) 1s ago 15m 25.5M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:45:19.345 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (16m) 1s ago 15m 8820k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:45:19.345 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (14m) 39s ago 14m 8594k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:45:19.345 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (4m) 1s ago 15m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:45:19.345 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (4m) 39s ago 14m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:45:19.345 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (14m) 1s ago 15m 87.3M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (25s) 1s ago 10m 15.8M - 19.0.0-2515-g96805dcf d7acac7d5657 c7cf68a1a3e2 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (7s) 1s ago 10m 13.8M - 19.0.0-2515-g96805dcf d7acac7d5657 c24cf6de2c0f 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (10m) 39s ago 10m 141M - 18.2.2-1297-g1758e8ea 196422f1b641 d03cfbb2fc5a 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (9m) 39s ago 9m 18.5M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (6m) 1s ago 17m 512M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (5m) 39s ago 14m 452M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (5m) 1s ago 17m 51.8M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (4m) 39s ago 14m 36.5M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (15m) 1s ago 15m 15.7M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (14m) 39s ago 14m 16.0M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (3m) 1s ago 13m 147M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (3m) 1s ago 13m 134M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (2m) 1s ago 12m 90.1M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:45:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (110s) 39s ago 12m 108M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:45:19.347 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (76s) 39s ago 12m 96.2M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 14c9b4920f5a 2024-05-09T06:45:19.347 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (43s) 39s ago 11m 12.7M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 d219b4bb599f 2024-05-09T06:45:19.347 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 1s ago 15m 63.6M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:45:19.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.367+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f0406e210 msgr2=0x7f2f040706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.367+0000 7f2f21bc4700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f0406e210 0x7f2f040706d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c005090 tx=0x7f2f0c00b540 comp rx=0 tx=0).stop 2024-05-09T06:45:19.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.367+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c101ec0 msgr2=0x7f2f1c196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.367+0000 7f2f21bc4700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c101ec0 0x7f2f1c196c20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000d900 tx=0x7f2f1000dc10 comp rx=0 tx=0).stop 2024-05-09T06:45:19.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.368+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 shutdown_connections 2024-05-09T06:45:19.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.368+0000 7f2f21bc4700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2f0406e210 0x7f2f040706d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.368+0000 7f2f21bc4700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f1c0feee0 0x7f2f1c1966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.368+0000 7f2f21bc4700 1 --2- 172.21.15.63:0/2968208126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f1c101ec0 0x7f2f1c196c20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.368+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 >> 172.21.15.63:0/2968208126 conn(0x7f2f1c0fa840 msgr2=0x7f2f1c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:19.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.368+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 shutdown_connections 2024-05-09T06:45:19.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.368+0000 7f2f21bc4700 1 -- 172.21.15.63:0/2968208126 wait complete. 2024-05-09T06:45:19.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.477+0000 7fed6dc8c700 1 -- 172.21.15.63:0/2989096843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 msgr2=0x7fed680fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.477+0000 7fed6dc8c700 1 --2- 172.21.15.63:0/2989096843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 0x7fed680fffc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed58009910 tx=0x7fed58009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:19.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.477+0000 7fed6dc8c700 1 -- 172.21.15.63:0/2989096843 shutdown_connections 2024-05-09T06:45:19.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.477+0000 7fed6dc8c700 1 --2- 172.21.15.63:0/2989096843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed68100da0 0x7fed68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.477+0000 7fed6dc8c700 1 --2- 172.21.15.63:0/2989096843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 0x7fed680fffc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.477+0000 7fed6dc8c700 1 -- 172.21.15.63:0/2989096843 >> 172.21.15.63:0/2989096843 conn(0x7fed680fb120 msgr2=0x7fed680fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:19.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.477+0000 7fed6dc8c700 1 -- 172.21.15.63:0/2989096843 shutdown_connections 2024-05-09T06:45:19.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.477+0000 7fed6dc8c700 1 -- 172.21.15.63:0/2989096843 wait complete. 2024-05-09T06:45:19.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.478+0000 7fed6dc8c700 1 Processor -- start 2024-05-09T06:45:19.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.478+0000 7fed6dc8c700 1 -- start start 2024-05-09T06:45:19.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed6dc8c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 0x7fed68103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:19.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed6dc8c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed68100da0 0x7fed68102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:19.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed6dc8c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed681026c0 con 0x7fed680ffba0 2024-05-09T06:45:19.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed6dc8c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed68102800 con 0x7fed68100da0 2024-05-09T06:45:19.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed677fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 0x7fed68103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:19.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed677fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 0x7fed68103a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59400/0 (socket says 172.21.15.63:59400) 2024-05-09T06:45:19.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed677fe700 1 -- 172.21.15.63:0/1178318148 learned_addr learned my addr 172.21.15.63:0/1178318148 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:19.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed66ffd700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed68100da0 0x7fed68102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:19.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed677fe700 1 -- 172.21.15.63:0/1178318148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed68100da0 msgr2=0x7fed68102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed677fe700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed68100da0 0x7fed68102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed677fe700 1 -- 172.21.15.63:0/1178318148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed580095f0 con 0x7fed680ffba0 2024-05-09T06:45:19.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed677fe700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 0x7fed68103a90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed5800bd70 tx=0x7fed5800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:19.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.479+0000 7fed64ff9700 1 -- 172.21.15.63:0/1178318148 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed5801d070 con 0x7fed680ffba0 2024-05-09T06:45:19.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.480+0000 7fed64ff9700 1 -- 172.21.15.63:0/1178318148 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed58005dc0 con 0x7fed680ffba0 2024-05-09T06:45:19.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.480+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed68102a80 con 0x7fed680ffba0 2024-05-09T06:45:19.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.480+0000 7fed64ff9700 1 -- 172.21.15.63:0/1178318148 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed58021410 con 0x7fed680ffba0 2024-05-09T06:45:19.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.480+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed68102f70 con 0x7fed680ffba0 2024-05-09T06:45:19.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.481+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6804ebb0 con 0x7fed680ffba0 2024-05-09T06:45:19.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.482+0000 7fed64ff9700 1 -- 172.21.15.63:0/1178318148 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fed5800f7f0 con 0x7fed680ffba0 2024-05-09T06:45:19.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.482+0000 7fed64ff9700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fed5006e210 0x7fed500706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:19.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.482+0000 7fed64ff9700 1 -- 172.21.15.63:0/1178318148 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fed5808f320 con 0x7fed680ffba0 2024-05-09T06:45:19.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.483+0000 7fed66ffd700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fed5006e210 0x7fed500706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:19.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.483+0000 7fed66ffd700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fed5006e210 0x7fed500706d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed5c005950 tx=0x7fed5c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:19.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.487+0000 7fed64ff9700 1 -- 172.21.15.63:0/1178318148 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fed5805c0e0 con 0x7fed680ffba0 2024-05-09T06:45:19.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.797+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fed6819e8f0 con 0x7fed680ffba0 2024-05-09T06:45:19.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.798+0000 7fed64ff9700 1 -- 172.21.15.63:0/1178318148 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fed5805bc70 con 0x7fed680ffba0 2024-05-09T06:45:19.777 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:45:19.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 6 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2, 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 18.2.2-1297-g1758e8ea (1758e8ea56486d7ed4a57cf883ac5d1ad36580ad) reef (stable)": 2, 2024-05-09T06:45:19.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 12 2024-05-09T06:45:19.779 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:45:19.779 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:45:19.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fed5006e210 msgr2=0x7fed500706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fed5006e210 0x7fed500706d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed5c005950 tx=0x7fed5c00b410 comp rx=0 tx=0).stop 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 msgr2=0x7fed68103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 0x7fed68103a90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed5800bd70 tx=0x7fed5800bda0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 shutdown_connections 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fed5006e210 0x7fed500706d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed680ffba0 0x7fed68103a90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 --2- 172.21.15.63:0/1178318148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed68100da0 0x7fed68102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 >> 172.21.15.63:0/1178318148 conn(0x7fed680fb120 msgr2=0x7fed68068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 shutdown_connections 2024-05-09T06:45:19.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.800+0000 7fed6dc8c700 1 -- 172.21.15.63:0/1178318148 wait complete. 2024-05-09T06:45:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.911+0000 7f5d13947700 1 -- 172.21.15.63:0/1923196622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 msgr2=0x7f5d0c103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.911+0000 7f5d13947700 1 --2- 172.21.15.63:0/1923196622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 0x7f5d0c103aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009910 tx=0x7f5d08009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.911+0000 7f5d13947700 1 -- 172.21.15.63:0/1923196622 shutdown_connections 2024-05-09T06:45:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.911+0000 7f5d13947700 1 --2- 172.21.15.63:0/1923196622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d0c103fe0 0x7f5d0c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.911+0000 7f5d13947700 1 --2- 172.21.15.63:0/1923196622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 0x7f5d0c103aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.911+0000 7f5d13947700 1 -- 172.21.15.63:0/1923196622 >> 172.21.15.63:0/1923196622 conn(0x7f5d0c0fb0c0 msgr2=0x7f5d0c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.911+0000 7f5d13947700 1 -- 172.21.15.63:0/1923196622 shutdown_connections 2024-05-09T06:45:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.911+0000 7f5d13947700 1 -- 172.21.15.63:0/1923196622 wait complete. 2024-05-09T06:45:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.912+0000 7f5d13947700 1 Processor -- start 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.912+0000 7f5d13947700 1 -- start start 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.912+0000 7f5d13947700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 0x7f5d0c100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.912+0000 7f5d13947700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d0c103fe0 0x7f5d0c0ff660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.912+0000 7f5d13947700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0ffba0 con 0x7f5d0c1016b0 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.912+0000 7f5d13947700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0ffce0 con 0x7f5d0c103fe0 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.912+0000 7f5d116e3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 0x7f5d0c100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.912+0000 7f5d116e3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 0x7f5d0c100fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59408/0 (socket says 172.21.15.63:59408) 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.912+0000 7f5d116e3700 1 -- 172.21.15.63:0/3601496716 learned_addr learned my addr 172.21.15.63:0/3601496716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d10ee2700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d0c103fe0 0x7f5d0c0ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d116e3700 1 -- 172.21.15.63:0/3601496716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d0c103fe0 msgr2=0x7f5d0c0ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d116e3700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d0c103fe0 0x7f5d0c0ff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d116e3700 1 -- 172.21.15.63:0/3601496716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d080095f0 con 0x7f5d0c1016b0 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d10ee2700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d0c103fe0 0x7f5d0c0ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:45:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d116e3700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 0x7f5d0c100fc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d08006010 tx=0x7f5d0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d027fc700 1 -- 172.21.15.63:0/3601496716 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d0801d070 con 0x7f5d0c1016b0 2024-05-09T06:45:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d027fc700 1 -- 172.21.15.63:0/3601496716 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d0800f460 con 0x7f5d0c1016b0 2024-05-09T06:45:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d027fc700 1 -- 172.21.15.63:0/3601496716 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d08021640 con 0x7f5d0c1016b0 2024-05-09T06:45:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c0fff60 con 0x7f5d0c1016b0 2024-05-09T06:45:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.913+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c100480 con 0x7f5d0c1016b0 2024-05-09T06:45:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.914+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c04ebb0 con 0x7f5d0c1016b0 2024-05-09T06:45:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.915+0000 7f5d027fc700 1 -- 172.21.15.63:0/3601496716 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5d0800fad0 con 0x7f5d0c1016b0 2024-05-09T06:45:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.915+0000 7f5d027fc700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5cf806e210 0x7f5cf80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.915+0000 7f5d027fc700 1 -- 172.21.15.63:0/3601496716 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f5d0808f5d0 con 0x7f5d0c1016b0 2024-05-09T06:45:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.916+0000 7f5d10ee2700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5cf806e210 0x7f5cf80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.916+0000 7f5d10ee2700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5cf806e210 0x7f5cf80706d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc005fd0 tx=0x7f5cfc005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:19.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:19.920+0000 7f5d027fc700 1 -- 172.21.15.63:0/3601496716 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5d0805c390 con 0x7f5d0c1016b0 2024-05-09T06:45:20.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.181+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5d0c109780 con 0x7f5d0c1016b0 2024-05-09T06:45:20.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.181+0000 7f5d027fc700 1 -- 172.21.15.63:0/3601496716 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f5d0805bf20 con 0x7f5d0c1016b0 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:e21 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 16 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:44:47.035052+0000 2024-05-09T06:45:20.161 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 0 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=24277} 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:45:20.162 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 24277 members: 24277 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{0:24277} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.193:6824/296432349,v1:172.21.15.193:6825/296432349] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] compat {c=[1],r=[1],i=[7ff]}] 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:45:20.163 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/2182150991,v1:172.21.15.63:6829/2182150991] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:45:20.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5cf806e210 msgr2=0x7f5cf80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5cf806e210 0x7f5cf80706d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc005fd0 tx=0x7f5cfc005dc0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 msgr2=0x7f5d0c100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 0x7f5d0c100fc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d08006010 tx=0x7f5d0800be20 comp rx=0 tx=0).stop 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 shutdown_connections 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5cf806e210 0x7f5cf80706d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d0c1016b0 0x7f5d0c100fc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 --2- 172.21.15.63:0/3601496716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d0c103fe0 0x7f5d0c0ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 >> 172.21.15.63:0/3601496716 conn(0x7f5d0c0fb0c0 msgr2=0x7f5d0c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 shutdown_connections 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.183+0000 7f5d13947700 1 -- 172.21.15.63:0/3601496716 wait complete. 2024-05-09T06:45:20.165 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 21 2024-05-09T06:45:20.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.297+0000 7fdb43206700 1 -- 172.21.15.63:0/1954639061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c104060 msgr2=0x7fdb3c106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.297+0000 7fdb43206700 1 --2- 172.21.15.63:0/1954639061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c104060 0x7fdb3c106450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009910 tx=0x7fdb28009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:20.277 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:20 smithi063 ceph-mon[92765]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:20.277 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:20 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:20.277 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:20 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:20.277 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:20 smithi063 ceph-mon[92765]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:20.277 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:20 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1178318148' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.297+0000 7fdb43206700 1 -- 172.21.15.63:0/1954639061 shutdown_connections 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.297+0000 7fdb43206700 1 --2- 172.21.15.63:0/1954639061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c104060 0x7fdb3c106450 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.297+0000 7fdb43206700 1 --2- 172.21.15.63:0/1954639061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb3c101730 0x7fdb3c103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.297+0000 7fdb43206700 1 -- 172.21.15.63:0/1954639061 >> 172.21.15.63:0/1954639061 conn(0x7fdb3c0fb120 msgr2=0x7fdb3c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.298+0000 7fdb43206700 1 -- 172.21.15.63:0/1954639061 shutdown_connections 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.298+0000 7fdb43206700 1 -- 172.21.15.63:0/1954639061 wait complete. 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.299+0000 7fdb43206700 1 Processor -- start 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.299+0000 7fdb43206700 1 -- start start 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.299+0000 7fdb43206700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c101730 0x7fdb3c101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.299+0000 7fdb43206700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb3c104060 0x7fdb3c0ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.299+0000 7fdb43206700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0ffc20 con 0x7fdb3c101730 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.299+0000 7fdb43206700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0ffd60 con 0x7fdb3c104060 2024-05-09T06:45:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.299+0000 7fdb40fa2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c101730 0x7fdb3c101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.299+0000 7fdb40fa2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c101730 0x7fdb3c101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59422/0 (socket says 172.21.15.63:59422) 2024-05-09T06:45:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.299+0000 7fdb40fa2700 1 -- 172.21.15.63:0/370795164 learned_addr learned my addr 172.21.15.63:0/370795164 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.300+0000 7fdb3bfff700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb3c104060 0x7fdb3c0ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.300+0000 7fdb40fa2700 1 -- 172.21.15.63:0/370795164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb3c104060 msgr2=0x7fdb3c0ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.300+0000 7fdb40fa2700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb3c104060 0x7fdb3c0ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.300+0000 7fdb40fa2700 1 -- 172.21.15.63:0/370795164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280095f0 con 0x7fdb3c101730 2024-05-09T06:45:20.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.300+0000 7fdb40fa2700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c101730 0x7fdb3c101040 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000d8d0 tx=0x7fdb3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:20.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.300+0000 7fdb39ffb700 1 -- 172.21.15.63:0/370795164 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb30009940 con 0x7fdb3c101730 2024-05-09T06:45:20.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.300+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c100040 con 0x7fdb3c101730 2024-05-09T06:45:20.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.300+0000 7fdb39ffb700 1 -- 172.21.15.63:0/370795164 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb30010460 con 0x7fdb3c101730 2024-05-09T06:45:20.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.300+0000 7fdb39ffb700 1 -- 172.21.15.63:0/370795164 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3000f5f0 con 0x7fdb3c101730 2024-05-09T06:45:20.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.301+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c19e670 con 0x7fdb3c101730 2024-05-09T06:45:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.302+0000 7fdb39ffb700 1 -- 172.21.15.63:0/370795164 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdb300105d0 con 0x7fdb3c101730 2024-05-09T06:45:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.302+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c04ebb0 con 0x7fdb3c101730 2024-05-09T06:45:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.303+0000 7fdb39ffb700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb2c06e200 0x7fdb2c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.303+0000 7fdb39ffb700 1 -- 172.21.15.63:0/370795164 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fdb3008e240 con 0x7fdb3c101730 2024-05-09T06:45:20.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.303+0000 7fdb3bfff700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb2c06e200 0x7fdb2c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:20.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.304+0000 7fdb3bfff700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb2c06e200 0x7fdb2c0706c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb280051c0 tx=0x7fdb28005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:20.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.308+0000 7fdb39ffb700 1 -- 172.21.15.63:0/370795164 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdb3005af80 con 0x7fdb3c101730 2024-05-09T06:45:20.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:20 smithi193 ceph-mon[81665]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:20.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:20 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:20.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:20 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:20.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:20 smithi193 ceph-mon[81665]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:20.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:20 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1178318148' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:20.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.545+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb3c105ef0 con 0x7fdb2c06e200 2024-05-09T06:45:20.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.546+0000 7fdb39ffb700 1 -- 172.21.15.63:0/370795164 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdb3c105ef0 con 0x7fdb2c06e200 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:45:20.526 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "14/23 daemons upgraded", 2024-05-09T06:45:20.527 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mds daemons", 2024-05-09T06:45:20.527 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:45:20.527 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:45:20.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb2c06e200 msgr2=0x7fdb2c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb2c06e200 0x7fdb2c0706c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb280051c0 tx=0x7fdb28005a90 comp rx=0 tx=0).stop 2024-05-09T06:45:20.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c101730 msgr2=0x7fdb3c101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c101730 0x7fdb3c101040 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000d8d0 tx=0x7fdb3000dc90 comp rx=0 tx=0).stop 2024-05-09T06:45:20.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 shutdown_connections 2024-05-09T06:45:20.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb2c06e200 0x7fdb2c0706c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c101730 0x7fdb3c101040 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 --2- 172.21.15.63:0/370795164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb3c104060 0x7fdb3c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 >> 172.21.15.63:0/370795164 conn(0x7fdb3c0fb120 msgr2=0x7fdb3c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:20.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 shutdown_connections 2024-05-09T06:45:20.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.549+0000 7fdb43206700 1 -- 172.21.15.63:0/370795164 wait complete. 2024-05-09T06:45:20.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.661+0000 7f1606e2f700 1 -- 172.21.15.63:0/1195468650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16000ffb30 msgr2=0x7f16000fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.661+0000 7f1606e2f700 1 --2- 172.21.15.63:0/1195468650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16000ffb30 0x7f16000fff50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009910 tx=0x7f15f0009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:20.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.661+0000 7f1606e2f700 1 -- 172.21.15.63:0/1195468650 shutdown_connections 2024-05-09T06:45:20.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.661+0000 7f1606e2f700 1 --2- 172.21.15.63:0/1195468650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1600100d30 0x7f1600101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.661+0000 7f1606e2f700 1 --2- 172.21.15.63:0/1195468650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16000ffb30 0x7f16000fff50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.661+0000 7f1606e2f700 1 -- 172.21.15.63:0/1195468650 >> 172.21.15.63:0/1195468650 conn(0x7f16000fb0d0 msgr2=0x7f16000fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:20.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.661+0000 7f1606e2f700 1 -- 172.21.15.63:0/1195468650 shutdown_connections 2024-05-09T06:45:20.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.661+0000 7f1606e2f700 1 -- 172.21.15.63:0/1195468650 wait complete. 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f1606e2f700 1 Processor -- start 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f1606e2f700 1 -- start start 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f1606e2f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1600100d30 0x7f1600196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f1606e2f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1600196db0 0x7f160019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f1606e2f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16001972c0 con 0x7f1600196db0 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f1606e2f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1600197430 con 0x7f1600100d30 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f15fffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1600196db0 0x7f160019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f15fffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1600196db0 0x7f160019bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59440/0 (socket says 172.21.15.63:59440) 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f15fffff700 1 -- 172.21.15.63:0/1362454303 learned_addr learned my addr 172.21.15.63:0/1362454303 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.662+0000 7f1604bcb700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1600100d30 0x7f1600196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:20.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.663+0000 7f15fffff700 1 -- 172.21.15.63:0/1362454303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1600100d30 msgr2=0x7f1600196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.663+0000 7f15fffff700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1600100d30 0x7f1600196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.663+0000 7f15fffff700 1 -- 172.21.15.63:0/1362454303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f00095f0 con 0x7f1600196db0 2024-05-09T06:45:20.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.663+0000 7f15fffff700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1600196db0 0x7f160019bdd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c8a0 tx=0x7f15f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:20.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.663+0000 7f15fdffb700 1 -- 172.21.15.63:0/1362454303 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f400cea0 con 0x7f1600196db0 2024-05-09T06:45:20.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.663+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160019c370 con 0x7f1600196db0 2024-05-09T06:45:20.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.663+0000 7f15fdffb700 1 -- 172.21.15.63:0/1362454303 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15f4004830 con 0x7f1600196db0 2024-05-09T06:45:20.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.663+0000 7f15fdffb700 1 -- 172.21.15.63:0/1362454303 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f40055b0 con 0x7f1600196db0 2024-05-09T06:45:20.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.663+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160019c940 con 0x7f1600196db0 2024-05-09T06:45:20.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.664+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160004ebb0 con 0x7f1600196db0 2024-05-09T06:45:20.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.665+0000 7f15fdffb700 1 -- 172.21.15.63:0/1362454303 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f15f4007ad0 con 0x7f1600196db0 2024-05-09T06:45:20.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.665+0000 7f15fdffb700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f15e806e210 0x7f15e80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:20.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.666+0000 7f15fdffb700 1 -- 172.21.15.63:0/1362454303 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f15f408de00 con 0x7f1600196db0 2024-05-09T06:45:20.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.666+0000 7f1604bcb700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f15e806e210 0x7f15e80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:20.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.666+0000 7f1604bcb700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f15e806e210 0x7f15e80706d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f0005160 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:20.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.670+0000 7f15fdffb700 1 -- 172.21.15.63:0/1362454303 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f15f405abc0 con 0x7f1600196db0 2024-05-09T06:45:20.950 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:45:20.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.970+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1600108690 con 0x7f1600196db0 2024-05-09T06:45:20.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.970+0000 7f15fdffb700 1 -- 172.21.15.63:0/1362454303 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f15f401d020 con 0x7f1600196db0 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.972+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f15e806e210 msgr2=0x7f15e80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.972+0000 7f1606e2f700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f15e806e210 0x7f15e80706d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f0005160 comp rx=0 tx=0).stop 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.972+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1600196db0 msgr2=0x7f160019bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.972+0000 7f1606e2f700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1600196db0 0x7f160019bdd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c8a0 tx=0x7f15f400cc60 comp rx=0 tx=0).stop 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.972+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 shutdown_connections 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.973+0000 7f1606e2f700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f15e806e210 0x7f15e80706d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.973+0000 7f1606e2f700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1600100d30 0x7f1600196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.973+0000 7f1606e2f700 1 --2- 172.21.15.63:0/1362454303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1600196db0 0x7f160019bdd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.973+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 >> 172.21.15.63:0/1362454303 conn(0x7f16000fb0d0 msgr2=0x7f16000fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.973+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 shutdown_connections 2024-05-09T06:45:20.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:20.973+0000 7f1606e2f700 1 -- 172.21.15.63:0/1362454303 wait complete. 2024-05-09T06:45:21.209 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:21 smithi063 ceph-mon[92765]: pgmap v201: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:21.209 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:21 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3601496716' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:45:21.209 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:21 smithi063 ceph-mon[92765]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:21.209 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:21 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1362454303' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:45:21.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:21 smithi193 ceph-mon[81665]: pgmap v201: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:21.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:21 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3601496716' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:45:21.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:21 smithi193 ceph-mon[81665]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:21.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:21 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1362454303' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:45:24.083 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: pgmap v202: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:24.083 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:24.083 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:24.083 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:24.083 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:45:24.083 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:24.084 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:24.084 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:24.084 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:24.084 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:24.084 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:24.084 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi193.ggtmyb"]}]: dispatch 2024-05-09T06:45:24.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: pgmap v202: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:24.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:24.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:24.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:24.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:45:24.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:24.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:24.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:24.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:24.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:24.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:24.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi193.ggtmyb"]}]: dispatch 2024-05-09T06:45:25.038 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:24 smithi193 ceph-mon[81665]: Upgrade: It appears safe to stop mds.cephfs.smithi193.ggtmyb 2024-05-09T06:45:25.038 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:24 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:25.038 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:24 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ggtmyb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:45:25.038 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:24 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:25.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:24 smithi063 ceph-mon[92765]: Upgrade: It appears safe to stop mds.cephfs.smithi193.ggtmyb 2024-05-09T06:45:25.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:24 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:25.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:24 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ggtmyb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:45:25.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:24 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:26.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:25 smithi063 ceph-mon[92765]: pgmap v203: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:26.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:25 smithi063 ceph-mon[92765]: Upgrade: Updating mds.cephfs.smithi193.ggtmyb 2024-05-09T06:45:26.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:25 smithi063 ceph-mon[92765]: Deploying daemon mds.cephfs.smithi193.ggtmyb on smithi193 2024-05-09T06:45:26.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:25 smithi193 ceph-mon[81665]: pgmap v203: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:26.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:25 smithi193 ceph-mon[81665]: Upgrade: Updating mds.cephfs.smithi193.ggtmyb 2024-05-09T06:45:26.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:25 smithi193 ceph-mon[81665]: Deploying daemon mds.cephfs.smithi193.ggtmyb on smithi193 2024-05-09T06:45:27.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:26 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063[92761]: 2024-05-09T06:45:26.765+0000 7f21b78c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:45:28.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: pgmap v204: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:28.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T06:45:28.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:45:28.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: osdmap e95: 6 total, 6 up, 6 in 2024-05-09T06:45:28.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: Standby daemon mds.cephfs.smithi193.sygxnf assigned to filesystem cephfs as rank 0 2024-05-09T06:45:28.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T06:45:28.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-09T06:45:28.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: fsmap cephfs:1/1 {0=cephfs.smithi193.sygxnf=up:replay} 2 up:standby 2024-05-09T06:45:28.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:28.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:45:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: pgmap v204: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-05-09T06:45:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T06:45:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:45:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: osdmap e95: 6 total, 6 up, 6 in 2024-05-09T06:45:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: Standby daemon mds.cephfs.smithi193.sygxnf assigned to filesystem cephfs as rank 0 2024-05-09T06:45:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T06:45:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-09T06:45:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: fsmap cephfs:1/1 {0=cephfs.smithi193.sygxnf=up:replay} 2 up:standby 2024-05-09T06:45:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:28.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:45:29.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:28 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:45:29.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:28 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:45:30.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:29 smithi063 ceph-mon[92765]: pgmap v206: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 0 op/s 2024-05-09T06:45:30.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:29 smithi193 ceph-mon[81665]: pgmap v206: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 0 op/s 2024-05-09T06:45:32.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:31 smithi063 ceph-mon[92765]: pgmap v207: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s rd, 1 op/s 2024-05-09T06:45:32.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:31 smithi193 ceph-mon[81665]: pgmap v207: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s rd, 1 op/s 2024-05-09T06:45:34.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:33 smithi063 ceph-mon[92765]: pgmap v208: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-09T06:45:34.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:33 smithi193 ceph-mon[81665]: pgmap v208: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-09T06:45:36.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:35 smithi063 ceph-mon[92765]: pgmap v209: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-09T06:45:36.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:35 smithi193 ceph-mon[81665]: pgmap v209: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-09T06:45:37.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:36 smithi063 ceph-mon[92765]: reconnect by client.24327 192.168.0.1:0/1228019762 after 0.000999985 2024-05-09T06:45:37.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:36 smithi063 ceph-mon[92765]: reconnect by client.14540 192.168.0.1:0/2082070818 after 0.000999985 2024-05-09T06:45:37.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:36 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:reconnect 2024-05-09T06:45:37.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:36 smithi063 ceph-mon[92765]: fsmap cephfs:1/1 {0=cephfs.smithi193.sygxnf=up:reconnect} 2 up:standby 2024-05-09T06:45:37.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:36 smithi063 ceph-mon[92765]: pgmap v210: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-05-09T06:45:37.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:36 smithi193 ceph-mon[81665]: reconnect by client.24327 192.168.0.1:0/1228019762 after 0.000999985 2024-05-09T06:45:37.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:36 smithi193 ceph-mon[81665]: reconnect by client.14540 192.168.0.1:0/2082070818 after 0.000999985 2024-05-09T06:45:37.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:36 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:reconnect 2024-05-09T06:45:37.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:36 smithi193 ceph-mon[81665]: fsmap cephfs:1/1 {0=cephfs.smithi193.sygxnf=up:reconnect} 2 up:standby 2024-05-09T06:45:37.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:36 smithi193 ceph-mon[81665]: pgmap v210: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-05-09T06:45:37.851 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:37 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:rejoin 2024-05-09T06:45:37.851 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:37 smithi193 ceph-mon[81665]: fsmap cephfs:1/1 {0=cephfs.smithi193.sygxnf=up:rejoin} 2 up:standby 2024-05-09T06:45:37.851 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:37 smithi193 ceph-mon[81665]: daemon mds.cephfs.smithi193.sygxnf is now active in filesystem cephfs as rank 0 2024-05-09T06:45:38.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:37 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:rejoin 2024-05-09T06:45:38.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:37 smithi063 ceph-mon[92765]: fsmap cephfs:1/1 {0=cephfs.smithi193.sygxnf=up:rejoin} 2 up:standby 2024-05-09T06:45:38.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:37 smithi063 ceph-mon[92765]: daemon mds.cephfs.smithi193.sygxnf is now active in filesystem cephfs as rank 0 2024-05-09T06:45:39.040 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:38 smithi193 ceph-mon[81665]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T06:45:39.041 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:38 smithi193 ceph-mon[81665]: Cluster is now healthy 2024-05-09T06:45:39.041 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:38 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:active 2024-05-09T06:45:39.041 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:38 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi193.sygxnf=up:active} 2 up:standby 2024-05-09T06:45:39.041 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:38 smithi193 ceph-mon[81665]: pgmap v211: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-09T06:45:39.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:38 smithi063 ceph-mon[92765]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T06:45:39.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:38 smithi063 ceph-mon[92765]: Cluster is now healthy 2024-05-09T06:45:39.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:38 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.193:6826/354024957,v1:172.21.15.193:6827/354024957] up:active 2024-05-09T06:45:39.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:38 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.sygxnf=up:active} 2 up:standby 2024-05-09T06:45:39.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:38 smithi063 ceph-mon[92765]: pgmap v211: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-09T06:45:41.154 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:41 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.193:6824/2007543899,v1:172.21.15.193:6825/2007543899] up:boot 2024-05-09T06:45:41.154 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:41 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi193.sygxnf=up:active} 3 up:standby 2024-05-09T06:45:41.154 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:41 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:45:41.154 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:41 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:41.155 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:41 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:41.155 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:41 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:41.155 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:41 smithi193 ceph-mon[81665]: pgmap v212: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T06:45:41.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:41 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.193:6824/2007543899,v1:172.21.15.193:6825/2007543899] up:boot 2024-05-09T06:45:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:41 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi193.sygxnf=up:active} 3 up:standby 2024-05-09T06:45:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:41 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:45:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:41 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:41 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:41 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:41.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:41 smithi063 ceph-mon[92765]: pgmap v212: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T06:45:43.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:43.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:43 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:45:43.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:43 smithi193 ceph-mon[81665]: pgmap v213: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-09T06:45:43.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:43.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:43 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:45:43.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:43 smithi063 ceph-mon[92765]: pgmap v213: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-09T06:45:44.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:44 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:44.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:44 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:44.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:44 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:44.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:44 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:45.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:45 smithi193 ceph-mon[81665]: pgmap v214: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-05-09T06:45:45.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:45.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:45.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:45 smithi063 ceph-mon[92765]: pgmap v214: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-05-09T06:45:45.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:45.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:47.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:47 smithi193 ceph-mon[81665]: pgmap v215: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-05-09T06:45:47.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:47 smithi063 ceph-mon[92765]: pgmap v215: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-05-09T06:45:49.351 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:49.352 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:49.352 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:49.352 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:45:49.352 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:49.352 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:49.352 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:49.352 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:49.352 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:49.352 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:49.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi193.sygxnf"]}]: dispatch 2024-05-09T06:45:49.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: Upgrade: It appears safe to stop mds.cephfs.smithi193.sygxnf 2024-05-09T06:45:49.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:48 smithi063 ceph-mon[92765]: pgmap v216: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:49.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:49.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi193.sygxnf"]}]: dispatch 2024-05-09T06:45:49.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: Upgrade: It appears safe to stop mds.cephfs.smithi193.sygxnf 2024-05-09T06:45:49.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:48 smithi193 ceph-mon[81665]: pgmap v216: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-05-09T06:45:50.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:50 smithi193 ceph-mon[81665]: Upgrade: Updating mds.cephfs.smithi193.sygxnf 2024-05-09T06:45:50.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:50 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:50.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:50 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sygxnf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:45:50.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:50 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:50.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:50 smithi193 ceph-mon[81665]: Deploying daemon mds.cephfs.smithi193.sygxnf on smithi193 2024-05-09T06:45:51.072 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:50 smithi063 ceph-mon[92765]: Upgrade: Updating mds.cephfs.smithi193.sygxnf 2024-05-09T06:45:51.072 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:50 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:51.072 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:50 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sygxnf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T06:45:51.072 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:50 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:45:51.072 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:50 smithi063 ceph-mon[92765]: Deploying daemon mds.cephfs.smithi193.sygxnf on smithi193 2024-05-09T06:45:51.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.091+0000 7f0fc1270700 1 -- 172.21.15.63:0/2160416911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 msgr2=0x7f0fbc1063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.091+0000 7f0fc1270700 1 --2- 172.21.15.63:0/2160416911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 0x7f0fbc1063d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009910 tx=0x7f0fb0009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:51.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.091+0000 7f0fc1270700 1 -- 172.21.15.63:0/2160416911 shutdown_connections 2024-05-09T06:45:51.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.091+0000 7f0fc1270700 1 --2- 172.21.15.63:0/2160416911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 0x7f0fbc1063d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.091+0000 7f0fc1270700 1 --2- 172.21.15.63:0/2160416911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fbc1016b0 0x7f0fbc103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.091+0000 7f0fc1270700 1 -- 172.21.15.63:0/2160416911 >> 172.21.15.63:0/2160416911 conn(0x7f0fbc0fb0c0 msgr2=0x7f0fbc0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.091+0000 7f0fc1270700 1 -- 172.21.15.63:0/2160416911 shutdown_connections 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.091+0000 7f0fc1270700 1 -- 172.21.15.63:0/2160416911 wait complete. 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fc1270700 1 Processor -- start 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fc1270700 1 -- start start 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fc1270700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fbc1016b0 0x7f0fbc1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fc1270700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 0x7f0fbc196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fc1270700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc197140 con 0x7f0fbc103fe0 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fc1270700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc197280 con 0x7f0fbc1016b0 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fba7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 0x7f0fbc196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fba7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 0x7f0fbc196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53234/0 (socket says 172.21.15.63:53234) 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fba7fc700 1 -- 172.21.15.63:0/1766340033 learned_addr learned my addr 172.21.15.63:0/1766340033 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.092+0000 7f0fbaffd700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fbc1016b0 0x7f0fbc1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:51.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.093+0000 7f0fba7fc700 1 -- 172.21.15.63:0/1766340033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fbc1016b0 msgr2=0x7f0fbc1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.093+0000 7f0fba7fc700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fbc1016b0 0x7f0fbc1965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.093+0000 7f0fba7fc700 1 -- 172.21.15.63:0/1766340033 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00095f0 con 0x7f0fbc103fe0 2024-05-09T06:45:51.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.093+0000 7f0fba7fc700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 0x7f0fbc196b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00092e0 tx=0x7f0fb000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:51.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.093+0000 7f0fa3fff700 1 -- 172.21.15.63:0/1766340033 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fb001d070 con 0x7f0fbc103fe0 2024-05-09T06:45:51.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.093+0000 7f0fa3fff700 1 -- 172.21.15.63:0/1766340033 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fb000f460 con 0x7f0fbc103fe0 2024-05-09T06:45:51.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.093+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc19bc80 con 0x7f0fbc103fe0 2024-05-09T06:45:51.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.093+0000 7f0fa3fff700 1 -- 172.21.15.63:0/1766340033 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fb0021620 con 0x7f0fbc103fe0 2024-05-09T06:45:51.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.093+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc19c1f0 con 0x7f0fbc103fe0 2024-05-09T06:45:51.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.094+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fbc04ebb0 con 0x7f0fbc103fe0 2024-05-09T06:45:51.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.095+0000 7f0fa3fff700 1 -- 172.21.15.63:0/1766340033 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0fb000f5d0 con 0x7f0fbc103fe0 2024-05-09T06:45:51.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.095+0000 7f0fa3fff700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0fa406e200 0x7f0fa40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:51.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.095+0000 7f0fa3fff700 1 -- 172.21.15.63:0/1766340033 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0fb008f4f0 con 0x7f0fbc103fe0 2024-05-09T06:45:51.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.095+0000 7f0fbaffd700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0fa406e200 0x7f0fa40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:51.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.096+0000 7f0fbaffd700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0fa406e200 0x7f0fa40706c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005950 tx=0x7f0fac00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:51.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.100+0000 7f0fa3fff700 1 -- 172.21.15.63:0/1766340033 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0fb005c1e0 con 0x7f0fbc103fe0 2024-05-09T06:45:51.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.326+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fbc105ec0 con 0x7f0fa406e200 2024-05-09T06:45:51.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.327+0000 7f0fa3fff700 1 -- 172.21.15.63:0/1766340033 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0fbc105ec0 con 0x7f0fa406e200 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.329+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0fa406e200 msgr2=0x7f0fa40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.329+0000 7f0fc1270700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0fa406e200 0x7f0fa40706c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005950 tx=0x7f0fac00b410 comp rx=0 tx=0).stop 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.329+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 msgr2=0x7f0fbc196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.329+0000 7f0fc1270700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 0x7f0fbc196b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00092e0 tx=0x7f0fb000bd70 comp rx=0 tx=0).stop 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.329+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 shutdown_connections 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.329+0000 7f0fc1270700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0fa406e200 0x7f0fa40706c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.329+0000 7f0fc1270700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fbc1016b0 0x7f0fbc1965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.329+0000 7f0fc1270700 1 --2- 172.21.15.63:0/1766340033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fbc103fe0 0x7f0fbc196b20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.329+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 >> 172.21.15.63:0/1766340033 conn(0x7f0fbc0fb0c0 msgr2=0x7f0fbc0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.330+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 shutdown_connections 2024-05-09T06:45:51.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.330+0000 7f0fc1270700 1 -- 172.21.15.63:0/1766340033 wait complete. 2024-05-09T06:45:51.320 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:45:51.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.439+0000 7fdb8047b700 1 -- 172.21.15.63:0/3416314847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 msgr2=0x7fdb780fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.439+0000 7fdb8047b700 1 --2- 172.21.15.63:0/3416314847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 0x7fdb780fffc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb74009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.439+0000 7fdb8047b700 1 -- 172.21.15.63:0/3416314847 shutdown_connections 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.439+0000 7fdb8047b700 1 --2- 172.21.15.63:0/3416314847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb78100da0 0x7fdb78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 --2- 172.21.15.63:0/3416314847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 0x7fdb780fffc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 -- 172.21.15.63:0/3416314847 >> 172.21.15.63:0/3416314847 conn(0x7fdb780fb120 msgr2=0x7fdb780fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 -- 172.21.15.63:0/3416314847 shutdown_connections 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 -- 172.21.15.63:0/3416314847 wait complete. 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 Processor -- start 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 -- start start 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 0x7fdb78103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb78100da0 0x7fdb78102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:51.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb781026c0 con 0x7fdb780ffba0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb8047b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb78102800 con 0x7fdb78100da0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.440+0000 7fdb7e217700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 0x7fdb78103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb7e217700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 0x7fdb78103a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53258/0 (socket says 172.21.15.63:53258) 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb7e217700 1 -- 172.21.15.63:0/2522990510 learned_addr learned my addr 172.21.15.63:0/2522990510 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb7da16700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb78100da0 0x7fdb78102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb7e217700 1 -- 172.21.15.63:0/2522990510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb78100da0 msgr2=0x7fdb78102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb7e217700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb78100da0 0x7fdb78102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb7e217700 1 -- 172.21.15.63:0/2522990510 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb740095f0 con 0x7fdb780ffba0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb7e217700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 0x7fdb78103a90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb740059d0 tx=0x7fdb7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb6f7fe700 1 -- 172.21.15.63:0/2522990510 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7401d070 con 0x7fdb780ffba0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb6f7fe700 1 -- 172.21.15.63:0/2522990510 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb7400f460 con 0x7fdb780ffba0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb78102a80 con 0x7fdb780ffba0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb6f7fe700 1 -- 172.21.15.63:0/2522990510 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb74021640 con 0x7fdb780ffba0 2024-05-09T06:45:51.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.441+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb78102f70 con 0x7fdb780ffba0 2024-05-09T06:45:51.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.442+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7804ebb0 con 0x7fdb780ffba0 2024-05-09T06:45:51.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.443+0000 7fdb6f7fe700 1 -- 172.21.15.63:0/2522990510 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdb7400f5d0 con 0x7fdb780ffba0 2024-05-09T06:45:51.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.443+0000 7fdb6f7fe700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb6406e200 0x7fdb640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:51.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.443+0000 7fdb6f7fe700 1 -- 172.21.15.63:0/2522990510 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fdb7408f420 con 0x7fdb780ffba0 2024-05-09T06:45:51.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.443+0000 7fdb7da16700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb6406e200 0x7fdb640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:51.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.444+0000 7fdb7da16700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb6406e200 0x7fdb640706c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdb68007c60 tx=0x7fdb680073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:51.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.448+0000 7fdb6f7fe700 1 -- 172.21.15.63:0/2522990510 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdb7405c190 con 0x7fdb780ffba0 2024-05-09T06:45:51.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.672+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb78068d50 con 0x7fdb6406e200 2024-05-09T06:45:51.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.674+0000 7fdb6f7fe700 1 -- 172.21.15.63:0/2522990510 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdb78068d50 con 0x7fdb6406e200 2024-05-09T06:45:51.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb6406e200 msgr2=0x7fdb640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb6406e200 0x7fdb640706c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdb68007c60 tx=0x7fdb680073d0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 msgr2=0x7fdb78103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 0x7fdb78103a90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb740059d0 tx=0x7fdb7400bd70 comp rx=0 tx=0).stop 2024-05-09T06:45:51.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 shutdown_connections 2024-05-09T06:45:51.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdb6406e200 0x7fdb640706c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb780ffba0 0x7fdb78103a90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 --2- 172.21.15.63:0/2522990510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb78100da0 0x7fdb78102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 >> 172.21.15.63:0/2522990510 conn(0x7fdb780fb120 msgr2=0x7fdb78068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:51.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 shutdown_connections 2024-05-09T06:45:51.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.676+0000 7fdb8047b700 1 -- 172.21.15.63:0/2522990510 wait complete. 2024-05-09T06:45:51.732 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:51 smithi193 ceph-mon[81665]: pgmap v217: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s rd, 4.5 KiB/s wr, 5 op/s 2024-05-09T06:45:51.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:51 smithi063 ceph-mon[92765]: pgmap v217: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s rd, 4.5 KiB/s wr, 5 op/s 2024-05-09T06:45:51.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.802+0000 7f3d20d8c700 1 -- 172.21.15.63:0/3895388129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c104060 msgr2=0x7f3d1c106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.802+0000 7f3d20d8c700 1 --2- 172.21.15.63:0/3895388129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c104060 0x7f3d1c106450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009910 tx=0x7f3d10009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:51.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.803+0000 7f3d20d8c700 1 -- 172.21.15.63:0/3895388129 shutdown_connections 2024-05-09T06:45:51.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.803+0000 7f3d20d8c700 1 --2- 172.21.15.63:0/3895388129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c104060 0x7f3d1c106450 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.803+0000 7f3d20d8c700 1 --2- 172.21.15.63:0/3895388129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d1c101730 0x7f3d1c103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.803+0000 7f3d20d8c700 1 -- 172.21.15.63:0/3895388129 >> 172.21.15.63:0/3895388129 conn(0x7f3d1c0fb120 msgr2=0x7f3d1c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:51.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.803+0000 7f3d20d8c700 1 -- 172.21.15.63:0/3895388129 shutdown_connections 2024-05-09T06:45:51.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.803+0000 7f3d20d8c700 1 -- 172.21.15.63:0/3895388129 wait complete. 2024-05-09T06:45:51.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.803+0000 7f3d20d8c700 1 Processor -- start 2024-05-09T06:45:51.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d20d8c700 1 -- start start 2024-05-09T06:45:51.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d20d8c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c101730 0x7f3d1c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:51.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d20d8c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d1c104060 0x7f3d1c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:51.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d20d8c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c199320 con 0x7f3d1c101730 2024-05-09T06:45:51.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d20d8c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c199460 con 0x7f3d1c104060 2024-05-09T06:45:51.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d1a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c101730 0x7f3d1c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d19d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d1c104060 0x7f3d1c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d1a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c101730 0x7f3d1c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53278/0 (socket says 172.21.15.63:53278) 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d1a59c700 1 -- 172.21.15.63:0/2047996807 learned_addr learned my addr 172.21.15.63:0/2047996807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d1a59c700 1 -- 172.21.15.63:0/2047996807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d1c104060 msgr2=0x7f3d1c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d1a59c700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d1c104060 0x7f3d1c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d1a59c700 1 -- 172.21.15.63:0/2047996807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d100095f0 con 0x7f3d1c101730 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.804+0000 7f3d19d9b700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d1c104060 0x7f3d1c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.805+0000 7f3d1a59c700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c101730 0x7f3d1c198850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c930 tx=0x7f3d0c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.805+0000 7f3d0b7fe700 1 -- 172.21.15.63:0/2047996807 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0c007ad0 con 0x7f3d1c101730 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.805+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c19ded0 con 0x7f3d1c101730 2024-05-09T06:45:51.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.805+0000 7f3d0b7fe700 1 -- 172.21.15.63:0/2047996807 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d0c00ce80 con 0x7f3d1c101730 2024-05-09T06:45:51.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.805+0000 7f3d0b7fe700 1 -- 172.21.15.63:0/2047996807 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0c018630 con 0x7f3d1c101730 2024-05-09T06:45:51.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.805+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c19e390 con 0x7f3d1c101730 2024-05-09T06:45:51.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.806+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c0fcd10 con 0x7f3d1c101730 2024-05-09T06:45:51.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.807+0000 7f3d0b7fe700 1 -- 172.21.15.63:0/2047996807 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3d0c007c30 con 0x7f3d1c101730 2024-05-09T06:45:51.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.807+0000 7f3d0b7fe700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3d0406e200 0x7f3d040706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:51.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.807+0000 7f3d0b7fe700 1 -- 172.21.15.63:0/2047996807 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f3d0c08e290 con 0x7f3d1c101730 2024-05-09T06:45:51.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.808+0000 7f3d19d9b700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3d0406e200 0x7f3d040706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:51.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.808+0000 7f3d19d9b700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3d0406e200 0x7f3d040706c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009fd0 tx=0x7f3d10005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:51.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:51.812+0000 7f3d0b7fe700 1 -- 172.21.15.63:0/2047996807 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3d0c05af20 con 0x7f3d1c101730 2024-05-09T06:45:52.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.032+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d1c02cef0 con 0x7f3d0406e200 2024-05-09T06:45:52.013 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:51 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063[92761]: 2024-05-09T06:45:51.752+0000 7f21b78c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:45:52.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.045+0000 7f3d0b7fe700 1 -- 172.21.15.63:0/2047996807 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f3d1c02cef0 con 0x7f3d0406e200 2024-05-09T06:45:52.025 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:45:52.025 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (14m) 34s ago 16m 25.5M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:45:52.025 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (16m) 34s ago 16m 8820k - 18.2.2-1297-g1758e8ea 196422f1b641 a9ac69b38ae0 2024-05-09T06:45:52.025 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (15m) 8s ago 15m 8892k - 18.2.2-1297-g1758e8ea 196422f1b641 4ce21bc68e33 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (4m) 34s ago 16m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (4m) 8s ago 15m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (14m) 34s ago 16m 87.3M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (57s) 34s ago 10m 15.8M - 19.0.0-2515-g96805dcf d7acac7d5657 c7cf68a1a3e2 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (39s) 34s ago 10m 13.8M - 19.0.0-2515-g96805dcf d7acac7d5657 c24cf6de2c0f 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (12s) 8s ago 10m 12.5M - 19.0.0-2515-g96805dcf d7acac7d5657 b752027d6904 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (10m) 8s ago 10m 87.6M - 18.2.2-1297-g1758e8ea 196422f1b641 0b42635181eb 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (6m) 34s ago 17m 512M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (6m) 8s ago 15m 453M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (5m) 34s ago 17m 51.8M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (5m) 8s ago 15m 39.1M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (16m) 34s ago 16m 15.7M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:45:52.026 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (15m) 8s ago 15m 16.1M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:45:52.027 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (4m) 34s ago 14m 147M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:45:52.027 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (3m) 34s ago 13m 134M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:45:52.027 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (2m) 34s ago 13m 90.1M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:45:52.027 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (2m) 8s ago 13m 117M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:45:52.027 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (109s) 8s ago 12m 99.7M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 14c9b4920f5a 2024-05-09T06:45:52.027 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (75s) 8s ago 12m 99.8M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 d219b4bb599f 2024-05-09T06:45:52.027 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 34s ago 15m 63.6M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:45:52.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3d0406e200 msgr2=0x7f3d040706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3d0406e200 0x7f3d040706c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009fd0 tx=0x7f3d10005960 comp rx=0 tx=0).stop 2024-05-09T06:45:52.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c101730 msgr2=0x7f3d1c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c101730 0x7f3d1c198850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c930 tx=0x7f3d0c00cc40 comp rx=0 tx=0).stop 2024-05-09T06:45:52.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 shutdown_connections 2024-05-09T06:45:52.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3d0406e200 0x7f3d040706c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d1c101730 0x7f3d1c198850 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 --2- 172.21.15.63:0/2047996807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d1c104060 0x7f3d1c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 >> 172.21.15.63:0/2047996807 conn(0x7f3d1c0fb120 msgr2=0x7f3d1c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:52.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 shutdown_connections 2024-05-09T06:45:52.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.048+0000 7f3d20d8c700 1 -- 172.21.15.63:0/2047996807 wait complete. 2024-05-09T06:45:52.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.156+0000 7fcb1397d700 1 -- 172.21.15.63:0/1980640480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c074f20 msgr2=0x7fcb0c073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.156+0000 7fcb1397d700 1 --2- 172.21.15.63:0/1980640480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c074f20 0x7fcb0c073380 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009910 tx=0x7fcb08009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:52.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.156+0000 7fcb1397d700 1 -- 172.21.15.63:0/1980640480 shutdown_connections 2024-05-09T06:45:52.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.156+0000 7fcb1397d700 1 --2- 172.21.15.63:0/1980640480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c073950 0x7fcb0c073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.156+0000 7fcb1397d700 1 --2- 172.21.15.63:0/1980640480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c074f20 0x7fcb0c073380 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.156+0000 7fcb1397d700 1 -- 172.21.15.63:0/1980640480 >> 172.21.15.63:0/1980640480 conn(0x7fcb0c0fc210 msgr2=0x7fcb0c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.156+0000 7fcb1397d700 1 -- 172.21.15.63:0/1980640480 shutdown_connections 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.156+0000 7fcb1397d700 1 -- 172.21.15.63:0/1980640480 wait complete. 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.157+0000 7fcb1397d700 1 Processor -- start 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.157+0000 7fcb1397d700 1 -- start start 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.157+0000 7fcb1397d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c073950 0x7fcb0c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.157+0000 7fcb1397d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c074f20 0x7fcb0c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.157+0000 7fcb1397d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1993b0 con 0x7fcb0c073950 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.157+0000 7fcb1397d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1994f0 con 0x7fcb0c074f20 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.157+0000 7fcb11719700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c073950 0x7fcb0c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.157+0000 7fcb11719700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c073950 0x7fcb0c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53288/0 (socket says 172.21.15.63:53288) 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.157+0000 7fcb11719700 1 -- 172.21.15.63:0/1045549752 learned_addr learned my addr 172.21.15.63:0/1045549752 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb10f18700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c074f20 0x7fcb0c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:52.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb11719700 1 -- 172.21.15.63:0/1045549752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c074f20 msgr2=0x7fcb0c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb11719700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c074f20 0x7fcb0c198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb11719700 1 -- 172.21.15.63:0/1045549752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080095f0 con 0x7fcb0c073950 2024-05-09T06:45:52.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb11719700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c073950 0x7fcb0c198850 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb080059d0 tx=0x7fcb0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:52.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb027fc700 1 -- 172.21.15.63:0/1045549752 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb0801d070 con 0x7fcb0c073950 2024-05-09T06:45:52.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb027fc700 1 -- 172.21.15.63:0/1045549752 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb08005dc0 con 0x7fcb0c073950 2024-05-09T06:45:52.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c19def0 con 0x7fcb0c073950 2024-05-09T06:45:52.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb027fc700 1 -- 172.21.15.63:0/1045549752 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb08021410 con 0x7fcb0c073950 2024-05-09T06:45:52.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.158+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c19e3e0 con 0x7fcb0c073950 2024-05-09T06:45:52.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.159+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c1929e0 con 0x7fcb0c073950 2024-05-09T06:45:52.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.160+0000 7fcb027fc700 1 -- 172.21.15.63:0/1045549752 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcb0800f840 con 0x7fcb0c073950 2024-05-09T06:45:52.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.160+0000 7fcb027fc700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcaf806e200 0x7fcaf80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:52.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.161+0000 7fcb027fc700 1 -- 172.21.15.63:0/1045549752 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fcb0808f340 con 0x7fcb0c073950 2024-05-09T06:45:52.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.161+0000 7fcb10f18700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcaf806e200 0x7fcaf80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:52.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.161+0000 7fcb10f18700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcaf806e200 0x7fcaf80706c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcafc005950 tx=0x7fcafc0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:52.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.165+0000 7fcb027fc700 1 -- 172.21.15.63:0/1045549752 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcb0805c100 con 0x7fcb0c073950 2024-05-09T06:45:52.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.467+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb0c04ebb0 con 0x7fcb0c073950 2024-05-09T06:45:52.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.467+0000 7fcb027fc700 1 -- 172.21.15.63:0/1045549752 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fcb0805bc90 con 0x7fcb0c073950 2024-05-09T06:45:52.448 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:45:52.448 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:45:52.448 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:45:52.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:45:52.448 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:45:52.448 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:45:52.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:45:52.448 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:45:52.449 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 6 2024-05-09T06:45:52.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:45:52.449 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:45:52.449 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 3 2024-05-09T06:45:52.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:45:52.449 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:45:52.449 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 13 2024-05-09T06:45:52.449 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:45:52.449 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:45:52.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcaf806e200 msgr2=0x7fcaf80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcaf806e200 0x7fcaf80706c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcafc005950 tx=0x7fcafc0058e0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c073950 msgr2=0x7fcb0c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c073950 0x7fcb0c198850 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb080059d0 tx=0x7fcb0800be20 comp rx=0 tx=0).stop 2024-05-09T06:45:52.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 shutdown_connections 2024-05-09T06:45:52.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fcaf806e200 0x7fcaf80706c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb0c073950 0x7fcb0c198850 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 --2- 172.21.15.63:0/1045549752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c074f20 0x7fcb0c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 >> 172.21.15.63:0/1045549752 conn(0x7fcb0c0fc210 msgr2=0x7fcb0c106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:52.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 shutdown_connections 2024-05-09T06:45:52.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.470+0000 7fcb1397d700 1 -- 172.21.15.63:0/1045549752 wait complete. 2024-05-09T06:45:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.581+0000 7f3e97ec1700 1 -- 172.21.15.63:0/3728309096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 msgr2=0x7f3e90101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.581+0000 7f3e97ec1700 1 --2- 172.21.15.63:0/3728309096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 0x7f3e90101060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009910 tx=0x7f3e8c009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.581+0000 7f3e97ec1700 1 -- 172.21.15.63:0/3728309096 shutdown_connections 2024-05-09T06:45:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.581+0000 7f3e97ec1700 1 --2- 172.21.15.63:0/3728309096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e90101e40 0x7f3e901022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.581+0000 7f3e97ec1700 1 --2- 172.21.15.63:0/3728309096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 0x7f3e90101060 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.581+0000 7f3e97ec1700 1 -- 172.21.15.63:0/3728309096 >> 172.21.15.63:0/3728309096 conn(0x7f3e900fc1e0 msgr2=0x7f3e900fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:52.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.581+0000 7f3e97ec1700 1 -- 172.21.15.63:0/3728309096 shutdown_connections 2024-05-09T06:45:52.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.581+0000 7f3e97ec1700 1 -- 172.21.15.63:0/3728309096 wait complete. 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.582+0000 7f3e97ec1700 1 Processor -- start 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.582+0000 7f3e97ec1700 1 -- start start 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.582+0000 7f3e97ec1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 0x7f3e90196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.582+0000 7f3e97ec1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e90101e40 0x7f3e90196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.582+0000 7f3e97ec1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90197190 con 0x7f3e90100c40 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.582+0000 7f3e97ec1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901972d0 con 0x7f3e90101e40 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e95c5d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 0x7f3e90196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e95c5d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 0x7f3e90196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53298/0 (socket says 172.21.15.63:53298) 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e95c5d700 1 -- 172.21.15.63:0/373534991 learned_addr learned my addr 172.21.15.63:0/373534991 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e9545c700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e90101e40 0x7f3e90196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e95c5d700 1 -- 172.21.15.63:0/373534991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e90101e40 msgr2=0x7f3e90196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e95c5d700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e90101e40 0x7f3e90196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e95c5d700 1 -- 172.21.15.63:0/373534991 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0095f0 con 0x7f3e90100c40 2024-05-09T06:45:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e9545c700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e90101e40 0x7f3e90196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:45:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e95c5d700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 0x7f3e90196630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0059d0 tx=0x7f3e8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.583+0000 7f3e86ffd700 1 -- 172.21.15.63:0/373534991 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e8c01d070 con 0x7f3e90100c40 2024-05-09T06:45:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.584+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9019bcd0 con 0x7f3e90100c40 2024-05-09T06:45:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.584+0000 7f3e86ffd700 1 -- 172.21.15.63:0/373534991 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e8c005dc0 con 0x7f3e90100c40 2024-05-09T06:45:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.584+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9019c1c0 con 0x7f3e90100c40 2024-05-09T06:45:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.584+0000 7f3e86ffd700 1 -- 172.21.15.63:0/373534991 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e8c021410 con 0x7f3e90100c40 2024-05-09T06:45:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.585+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e901907c0 con 0x7f3e90100c40 2024-05-09T06:45:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.585+0000 7f3e86ffd700 1 -- 172.21.15.63:0/373534991 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3e8c02b440 con 0x7f3e90100c40 2024-05-09T06:45:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.586+0000 7f3e86ffd700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3e7c06e200 0x7f3e7c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.586+0000 7f3e86ffd700 1 -- 172.21.15.63:0/373534991 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f3e8c08f540 con 0x7f3e90100c40 2024-05-09T06:45:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.586+0000 7f3e9545c700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3e7c06e200 0x7f3e7c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.587+0000 7f3e9545c700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3e7c06e200 0x7f3e7c0706c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3e80005fd0 tx=0x7f3e80005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:52.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.591+0000 7f3e86ffd700 1 -- 172.21.15.63:0/373534991 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3e8c05c240 con 0x7f3e90100c40 2024-05-09T06:45:52.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:52.753 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:52.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T06:45:52.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:45:52.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: osdmap e96: 6 total, 6 up, 6 in 2024-05-09T06:45:52.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: Standby daemon mds.cephfs.smithi063.aoiibw assigned to filesystem cephfs as rank 0 2024-05-09T06:45:52.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T06:45:52.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-09T06:45:52.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: fsmap cephfs:1/1 {0=cephfs.smithi063.aoiibw=up:replay} 2 up:standby 2024-05-09T06:45:52.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:52 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1045549752' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:52.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.845+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3e9002cd40 con 0x7f3e90100c40 2024-05-09T06:45:52.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.845+0000 7f3e86ffd700 1 -- 172.21.15.63:0/373534991 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1749 (secure 0 0 0) 0x7f3e8c05bdd0 con 0x7f3e90100c40 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout:e29 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 29 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-09T06:45:52.827 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:45:51.758882+0000 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 96 2024-05-09T06:45:52.828 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34406} 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:45:52.829 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 0 members: 2024-05-09T06:45:52.830 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{0:34406} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:45:52.830 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:45:52.830 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:45:52.830 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:45:52.830 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:45:52.830 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/2182150991,v1:172.21.15.63:6829/2182150991] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:45:52.830 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/2007543899,v1:172.21.15.193:6825/2007543899] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:45:52.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.847+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3e7c06e200 msgr2=0x7f3e7c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.847+0000 7f3e97ec1700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3e7c06e200 0x7f3e7c0706c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3e80005fd0 tx=0x7f3e80005ee0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.847+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 msgr2=0x7f3e90196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.847+0000 7f3e97ec1700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 0x7f3e90196630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0059d0 tx=0x7f3e8c00be20 comp rx=0 tx=0).stop 2024-05-09T06:45:52.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.847+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 shutdown_connections 2024-05-09T06:45:52.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.847+0000 7f3e97ec1700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3e7c06e200 0x7f3e7c0706c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.847+0000 7f3e97ec1700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e90100c40 0x7f3e90196630 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.847+0000 7f3e97ec1700 1 --2- 172.21.15.63:0/373534991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e90101e40 0x7f3e90196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.847+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 >> 172.21.15.63:0/373534991 conn(0x7f3e900fc1e0 msgr2=0x7f3e90105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:52.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.848+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 shutdown_connections 2024-05-09T06:45:52.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.848+0000 7f3e97ec1700 1 -- 172.21.15.63:0/373534991 wait complete. 2024-05-09T06:45:52.832 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 29 2024-05-09T06:45:52.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.959+0000 7f2707977700 1 -- 172.21.15.63:0/3096436148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27000ffba0 msgr2=0x7f27000fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.959+0000 7f2707977700 1 --2- 172.21.15.63:0/3096436148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27000ffba0 0x7f27000fffc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009910 tx=0x7f26fc009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:52.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.959+0000 7f2707977700 1 -- 172.21.15.63:0/3096436148 shutdown_connections 2024-05-09T06:45:52.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.959+0000 7f2707977700 1 --2- 172.21.15.63:0/3096436148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2700100da0 0x7f2700101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.959+0000 7f2707977700 1 --2- 172.21.15.63:0/3096436148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27000ffba0 0x7f27000fffc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.959+0000 7f2707977700 1 -- 172.21.15.63:0/3096436148 >> 172.21.15.63:0/3096436148 conn(0x7f27000fb120 msgr2=0x7f27000fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:52.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.959+0000 7f2707977700 1 -- 172.21.15.63:0/3096436148 shutdown_connections 2024-05-09T06:45:52.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.959+0000 7f2707977700 1 -- 172.21.15.63:0/3096436148 wait complete. 2024-05-09T06:45:52.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.960+0000 7f2707977700 1 Processor -- start 2024-05-09T06:45:52.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.960+0000 7f2707977700 1 -- start start 2024-05-09T06:45:52.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.960+0000 7f2707977700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27000ffba0 0x7f2700195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:52.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.960+0000 7f2707977700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2700100da0 0x7f2700195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.960+0000 7f2707977700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27001961d0 con 0x7f2700100da0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.960+0000 7f2707977700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2700196310 con 0x7f27000ffba0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2704f12700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2700100da0 0x7f2700195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2704f12700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2700100da0 0x7f2700195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53316/0 (socket says 172.21.15.63:53316) 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2704f12700 1 -- 172.21.15.63:0/2569540054 learned_addr learned my addr 172.21.15.63:0/2569540054 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2705713700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27000ffba0 0x7f2700195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2704f12700 1 -- 172.21.15.63:0/2569540054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27000ffba0 msgr2=0x7f2700195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2704f12700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27000ffba0 0x7f2700195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2704f12700 1 -- 172.21.15.63:0/2569540054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0095f0 con 0x7f2700100da0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2705713700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27000ffba0 0x7f2700195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2704f12700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2700100da0 0x7f2700195bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26f000b4b0 tx=0x7f26f000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f26f67fc700 1 -- 172.21.15.63:0/2569540054 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26f0011800 con 0x7f2700100da0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f26f67fc700 1 -- 172.21.15.63:0/2569540054 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26f0011e20 con 0x7f2700100da0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.961+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270019ad70 con 0x7f2700100da0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.962+0000 7f26f67fc700 1 -- 172.21.15.63:0/2569540054 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26f000f2e0 con 0x7f2700100da0 2024-05-09T06:45:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.962+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270019b340 con 0x7f2700100da0 2024-05-09T06:45:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.963+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270018f7a0 con 0x7f2700100da0 2024-05-09T06:45:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.964+0000 7f26f67fc700 1 -- 172.21.15.63:0/2569540054 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f26f0010380 con 0x7f2700100da0 2024-05-09T06:45:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.964+0000 7f26f67fc700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f26ec06e200 0x7f26ec0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:52.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.964+0000 7f26f67fc700 1 -- 172.21.15.63:0/2569540054 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f26f008de10 con 0x7f2700100da0 2024-05-09T06:45:52.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.964+0000 7f2705713700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f26ec06e200 0x7f26ec0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:52.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.965+0000 7f2705713700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f26ec06e200 0x7f26ec0706c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0051c0 tx=0x7f26fc00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:52.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:52.969+0000 7f26f67fc700 1 -- 172.21.15.63:0/2569540054 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f26f005ab20 con 0x7f2700100da0 2024-05-09T06:45:53.024 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:53.024 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:53.024 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T06:45:53.024 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:45:53.025 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: osdmap e96: 6 total, 6 up, 6 in 2024-05-09T06:45:53.025 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: Standby daemon mds.cephfs.smithi063.aoiibw assigned to filesystem cephfs as rank 0 2024-05-09T06:45:53.025 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T06:45:53.025 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-09T06:45:53.025 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: fsmap cephfs:1/1 {0=cephfs.smithi063.aoiibw=up:replay} 2 up:standby 2024-05-09T06:45:53.025 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:52 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1045549752' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:45:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.200+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f270002cea0 con 0x7f26ec06e200 2024-05-09T06:45:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.201+0000 7f26f67fc700 1 -- 172.21.15.63:0/2569540054 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f270002cea0 con 0x7f26ec06e200 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "15/23 daemons upgraded", 2024-05-09T06:45:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mds daemons", 2024-05-09T06:45:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:45:53.183 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:45:53.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f26ec06e200 msgr2=0x7f26ec0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f26ec06e200 0x7f26ec0706c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0051c0 tx=0x7f26fc00b540 comp rx=0 tx=0).stop 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2700100da0 msgr2=0x7f2700195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2700100da0 0x7f2700195bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26f000b4b0 tx=0x7f26f000b870 comp rx=0 tx=0).stop 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 shutdown_connections 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f26ec06e200 0x7f26ec0706c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27000ffba0 0x7f2700195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 --2- 172.21.15.63:0/2569540054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2700100da0 0x7f2700195bb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 >> 172.21.15.63:0/2569540054 conn(0x7f27000fb120 msgr2=0x7f27001043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 shutdown_connections 2024-05-09T06:45:53.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.204+0000 7f2707977700 1 -- 172.21.15.63:0/2569540054 wait complete. 2024-05-09T06:45:53.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.314+0000 7f19680d8700 1 -- 172.21.15.63:0/766050551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 msgr2=0x7f1960103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:53.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.314+0000 7f19680d8700 1 --2- 172.21.15.63:0/766050551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 0x7f1960103aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f195c009910 tx=0x7f195c009c20 comp rx=0 tx=0).stop 2024-05-09T06:45:53.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.314+0000 7f19680d8700 1 -- 172.21.15.63:0/766050551 shutdown_connections 2024-05-09T06:45:53.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.315+0000 7f19680d8700 1 --2- 172.21.15.63:0/766050551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1960103fe0 0x7f19601063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:53.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.315+0000 7f19680d8700 1 --2- 172.21.15.63:0/766050551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 0x7f1960103aa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:53.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.315+0000 7f19680d8700 1 -- 172.21.15.63:0/766050551 >> 172.21.15.63:0/766050551 conn(0x7f19600fb0c0 msgr2=0x7f19600fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:53.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.315+0000 7f19680d8700 1 -- 172.21.15.63:0/766050551 shutdown_connections 2024-05-09T06:45:53.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.315+0000 7f19680d8700 1 -- 172.21.15.63:0/766050551 wait complete. 2024-05-09T06:45:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f19680d8700 1 Processor -- start 2024-05-09T06:45:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f19680d8700 1 -- start start 2024-05-09T06:45:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f19680d8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 0x7f19601965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f19680d8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1960103fe0 0x7f1960196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f19680d8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1960197140 con 0x7f19601016b0 2024-05-09T06:45:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f19680d8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1960197280 con 0x7f1960103fe0 2024-05-09T06:45:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f1965e74700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 0x7f19601965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f1965e74700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 0x7f19601965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53324/0 (socket says 172.21.15.63:53324) 2024-05-09T06:45:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f1965e74700 1 -- 172.21.15.63:0/3972811488 learned_addr learned my addr 172.21.15.63:0/3972811488 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:45:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.316+0000 7f1965673700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1960103fe0 0x7f1960196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.317+0000 7f1965e74700 1 -- 172.21.15.63:0/3972811488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1960103fe0 msgr2=0x7f1960196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.317+0000 7f1965e74700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1960103fe0 0x7f1960196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.317+0000 7f1965e74700 1 -- 172.21.15.63:0/3972811488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0095f0 con 0x7f19601016b0 2024-05-09T06:45:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.317+0000 7f1965e74700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 0x7f19601965e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f195c005b40 tx=0x7f195c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:53.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.317+0000 7f1956ffd700 1 -- 172.21.15.63:0/3972811488 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f195c01d070 con 0x7f19601016b0 2024-05-09T06:45:53.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.317+0000 7f1956ffd700 1 -- 172.21.15.63:0/3972811488 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f195c0050d0 con 0x7f19601016b0 2024-05-09T06:45:53.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.317+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196019bc80 con 0x7f19601016b0 2024-05-09T06:45:53.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.317+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196019c170 con 0x7f19601016b0 2024-05-09T06:45:53.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.317+0000 7f1956ffd700 1 -- 172.21.15.63:0/3972811488 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f195c022470 con 0x7f19601016b0 2024-05-09T06:45:53.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.319+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196004ebb0 con 0x7f19601016b0 2024-05-09T06:45:53.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.320+0000 7f1956ffd700 1 -- 172.21.15.63:0/3972811488 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f195c017400 con 0x7f19601016b0 2024-05-09T06:45:53.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.320+0000 7f1956ffd700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f194c06e210 0x7f194c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:45:53.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.320+0000 7f1956ffd700 1 -- 172.21.15.63:0/3972811488 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f195c08f7b0 con 0x7f19601016b0 2024-05-09T06:45:53.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.321+0000 7f1965673700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f194c06e210 0x7f194c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:45:53.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.321+0000 7f1965673700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f194c06e210 0x7f194c0706d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1950009c60 tx=0x7f1950009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:45:53.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.325+0000 7f1956ffd700 1 -- 172.21.15.63:0/3972811488 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f195c05c530 con 0x7f19601016b0 2024-05-09T06:45:53.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.651+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f196019c670 con 0x7f19601016b0 2024-05-09T06:45:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.651+0000 7f1956ffd700 1 -- 172.21.15.63:0/3972811488 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7f195c05c0c0 con 0x7f19601016b0 2024-05-09T06:45:53.632 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 filesystem is degraded 2024-05-09T06:45:53.632 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-09T06:45:53.632 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs is degraded 2024-05-09T06:45:53.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.653+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f194c06e210 msgr2=0x7f194c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:53.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.653+0000 7f19680d8700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f194c06e210 0x7f194c0706d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1950009c60 tx=0x7f1950009260 comp rx=0 tx=0).stop 2024-05-09T06:45:53.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.653+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 msgr2=0x7f19601965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:45:53.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.653+0000 7f19680d8700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 0x7f19601965e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f195c005b40 tx=0x7f195c00f690 comp rx=0 tx=0).stop 2024-05-09T06:45:53.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.653+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 shutdown_connections 2024-05-09T06:45:53.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.653+0000 7f19680d8700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f194c06e210 0x7f194c0706d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:53.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.653+0000 7f19680d8700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19601016b0 0x7f19601965e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:53.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.653+0000 7f19680d8700 1 --2- 172.21.15.63:0/3972811488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1960103fe0 0x7f1960196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:45:53.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.653+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 >> 172.21.15.63:0/3972811488 conn(0x7f19600fb0c0 msgr2=0x7f19600fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:45:53.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.654+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 shutdown_connections 2024-05-09T06:45:53.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:45:53.654+0000 7f19680d8700 1 -- 172.21.15.63:0/3972811488 wait complete. 2024-05-09T06:45:53.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:53 smithi193 ceph-mon[81665]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:53.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:53 smithi193 ceph-mon[81665]: pgmap v219: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 0 op/s 2024-05-09T06:45:53.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:53 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/373534991' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:45:53.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:53 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3972811488' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:45:54.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:53 smithi063 ceph-mon[92765]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:54.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:53 smithi063 ceph-mon[92765]: pgmap v219: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 0 op/s 2024-05-09T06:45:54.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:53 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/373534991' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:45:54.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:53 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3972811488' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:45:54.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:54 smithi063 ceph-mon[92765]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:55.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:54 smithi193 ceph-mon[81665]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:45:56.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:55 smithi063 ceph-mon[92765]: pgmap v220: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 0 op/s 2024-05-09T06:45:56.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:55 smithi193 ceph-mon[81665]: pgmap v220: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 0 op/s 2024-05-09T06:45:58.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:57 smithi193 ceph-mon[81665]: pgmap v221: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s rd, 2 op/s 2024-05-09T06:45:58.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:58.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:45:58.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:57 smithi063 ceph-mon[92765]: pgmap v221: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s rd, 2 op/s 2024-05-09T06:45:58.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:58.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:45:59.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:59 smithi063 ceph-mon[92765]: pgmap v222: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-09T06:45:59.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:59.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:45:59.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:45:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:00.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:59 smithi193 ceph-mon[81665]: pgmap v222: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-09T06:46:00.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:00.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:00.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:45:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:00.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:00 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.193:6826/790240017,v1:172.21.15.193:6827/790240017] up:boot 2024-05-09T06:46:00.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:00 smithi193 ceph-mon[81665]: fsmap cephfs:1/1 {0=cephfs.smithi063.aoiibw=up:replay} 3 up:standby 2024-05-09T06:46:00.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:00 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:46:01.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:00 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.193:6826/790240017,v1:172.21.15.193:6827/790240017] up:boot 2024-05-09T06:46:01.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:00 smithi063 ceph-mon[92765]: fsmap cephfs:1/1 {0=cephfs.smithi063.aoiibw=up:replay} 3 up:standby 2024-05-09T06:46:01.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:00 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:46:02.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:01 smithi063 ceph-mon[92765]: pgmap v223: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-09T06:46:02.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:01 smithi193 ceph-mon[81665]: pgmap v223: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-09T06:46:03.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:02 smithi063 ceph-mon[92765]: reconnect by client.14540 192.168.0.1:0/2082070818 after 0.000999991 2024-05-09T06:46:03.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:02 smithi063 ceph-mon[92765]: reconnect by client.24327 192.168.0.1:0/1228019762 after 0.00199998 2024-05-09T06:46:03.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:02 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] up:reconnect 2024-05-09T06:46:03.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:02 smithi063 ceph-mon[92765]: fsmap cephfs:1/1 {0=cephfs.smithi063.aoiibw=up:reconnect} 3 up:standby 2024-05-09T06:46:03.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:02 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:03.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:02 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:03.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:02 smithi193 ceph-mon[81665]: reconnect by client.14540 192.168.0.1:0/2082070818 after 0.000999991 2024-05-09T06:46:03.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:02 smithi193 ceph-mon[81665]: reconnect by client.24327 192.168.0.1:0/1228019762 after 0.00199998 2024-05-09T06:46:03.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:02 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] up:reconnect 2024-05-09T06:46:03.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:02 smithi193 ceph-mon[81665]: fsmap cephfs:1/1 {0=cephfs.smithi063.aoiibw=up:reconnect} 3 up:standby 2024-05-09T06:46:03.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:02 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:03.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:02 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:04.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:03 smithi063 ceph-mon[92765]: pgmap v224: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-05-09T06:46:04.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:03 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] up:rejoin 2024-05-09T06:46:04.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:03 smithi063 ceph-mon[92765]: fsmap cephfs:1/1 {0=cephfs.smithi063.aoiibw=up:rejoin} 3 up:standby 2024-05-09T06:46:04.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:03 smithi063 ceph-mon[92765]: daemon mds.cephfs.smithi063.aoiibw is now active in filesystem cephfs as rank 0 2024-05-09T06:46:04.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:04.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:04.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:03 smithi193 ceph-mon[81665]: pgmap v224: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-05-09T06:46:04.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:03 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] up:rejoin 2024-05-09T06:46:04.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:03 smithi193 ceph-mon[81665]: fsmap cephfs:1/1 {0=cephfs.smithi063.aoiibw=up:rejoin} 3 up:standby 2024-05-09T06:46:04.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:03 smithi193 ceph-mon[81665]: daemon mds.cephfs.smithi063.aoiibw is now active in filesystem cephfs as rank 0 2024-05-09T06:46:04.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:04.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:04.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:04 smithi063 ceph-mon[92765]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T06:46:04.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:04 smithi063 ceph-mon[92765]: Cluster is now healthy 2024-05-09T06:46:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:04 smithi063 ceph-mon[92765]: mds.? [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] up:active 2024-05-09T06:46:04.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:04 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 3 up:standby 2024-05-09T06:46:04.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:04 smithi193 ceph-mon[81665]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T06:46:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:04 smithi193 ceph-mon[81665]: Cluster is now healthy 2024-05-09T06:46:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:04 smithi193 ceph-mon[81665]: mds.? [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] up:active 2024-05-09T06:46:04.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:04 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 3 up:standby 2024-05-09T06:46:05.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:05 smithi193 ceph-mon[81665]: pgmap v225: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-09T06:46:06.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:05 smithi063 ceph-mon[92765]: pgmap v225: 65 pgs: 65 active+clean; 205 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: pgmap v226: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.aoiibw"}]': finished 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.fbzxbf"}]': finished 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.ggtmyb"}]': finished 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:46:08.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.sygxnf"}]': finished 2024-05-09T06:46:08.107 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:07 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-09T06:46:08.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: pgmap v226: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T06:46:08.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:08.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:46:08.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:08.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.aoiibw"}]: dispatch 2024-05-09T06:46:08.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.aoiibw"}]': finished 2024-05-09T06:46:08.124 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.fbzxbf"}]: dispatch 2024-05-09T06:46:08.124 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.fbzxbf"}]': finished 2024-05-09T06:46:08.124 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.ggtmyb"}]: dispatch 2024-05-09T06:46:08.124 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.ggtmyb"}]': finished 2024-05-09T06:46:08.124 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.sygxnf"}]: dispatch 2024-05-09T06:46:08.124 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.sygxnf"}]': finished 2024-05-09T06:46:08.124 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:07 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-09T06:46:08.947 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all mds 2024-05-09T06:46:08.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-09T06:46:08.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-09T06:46:08.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:46:08.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:08.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:08.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:08 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:09.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all mds 2024-05-09T06:46:09.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-09T06:46:09.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-09T06:46:09.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: fsmap cephfs:1 {0=cephfs.smithi063.aoiibw=up:active} 1 up:standby-replay 2 up:standby 2024-05-09T06:46:09.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:09.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:09.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:09.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:09.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:09.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:08 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:09.951 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:09 smithi063 ceph-mon[92765]: pgmap v227: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-09T06:46:09.952 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:09 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all rgw 2024-05-09T06:46:09.952 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:09 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all rbd-mirror 2024-05-09T06:46:10.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:09 smithi193 ceph-mon[81665]: pgmap v227: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-09T06:46:10.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:09 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all rgw 2024-05-09T06:46:10.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:09 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all rbd-mirror 2024-05-09T06:46:11.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:11 smithi063 ceph-mon[92765]: Upgrade: Updating ceph-exporter.smithi063 (1/2) 2024-05-09T06:46:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:46:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:11 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:11 smithi063 ceph-mon[92765]: Deploying daemon ceph-exporter.smithi063 on smithi063 2024-05-09T06:46:11.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:11 smithi063 ceph-mon[92765]: pgmap v228: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-05-09T06:46:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:11 smithi193 ceph-mon[81665]: Upgrade: Updating ceph-exporter.smithi063 (1/2) 2024-05-09T06:46:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:11.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi063", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:46:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:11 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:11 smithi193 ceph-mon[81665]: Deploying daemon ceph-exporter.smithi063 on smithi063 2024-05-09T06:46:11.372 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:11 smithi193 ceph-mon[81665]: pgmap v228: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-05-09T06:46:13.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:13.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:13 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:46:13.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:13 smithi063 ceph-mon[92765]: pgmap v229: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-09T06:46:13.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:13.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:13 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:46:13.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:13 smithi193 ceph-mon[81665]: pgmap v229: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-09T06:46:15.534 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:15 smithi193 ceph-mon[81665]: pgmap v230: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-09T06:46:15.535 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:15 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:15.535 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:15 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:15.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:15 smithi063 ceph-mon[92765]: pgmap v230: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-09T06:46:15.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:15 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:15.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:15 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:17.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:17 smithi193 ceph-mon[81665]: pgmap v231: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-09T06:46:17.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:17 smithi193 ceph-mon[81665]: Upgrade: Updating ceph-exporter.smithi193 (2/2) 2024-05-09T06:46:17.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:17 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:17.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:17 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:46:17.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:17 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:17.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:17 smithi193 ceph-mon[81665]: Deploying daemon ceph-exporter.smithi193 on smithi193 2024-05-09T06:46:17.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:17 smithi063 ceph-mon[92765]: pgmap v231: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-05-09T06:46:17.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:17 smithi063 ceph-mon[92765]: Upgrade: Updating ceph-exporter.smithi193 (2/2) 2024-05-09T06:46:17.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:17 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:17.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:17 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi193", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-09T06:46:17.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:17 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:17.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:17 smithi063 ceph-mon[92765]: Deploying daemon ceph-exporter.smithi193 on smithi193 2024-05-09T06:46:19.554 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:19 smithi193 ceph-mon[81665]: pgmap v232: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-05-09T06:46:19.673 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:19 smithi063 ceph-mon[92765]: pgmap v232: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-05-09T06:46:21.308 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:21 smithi193 ceph-mon[81665]: pgmap v233: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4 op/s 2024-05-09T06:46:21.308 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:21.308 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:21.308 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:21.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:21 smithi063 ceph-mon[92765]: pgmap v233: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4 op/s 2024-05-09T06:46:21.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:21.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:21.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:23.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.769+0000 7fdd26821700 1 -- 172.21.15.63:0/131790849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 msgr2=0x7fdd200fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:23.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.769+0000 7fdd26821700 1 --2- 172.21.15.63:0/131790849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 0x7fdd200fffc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009910 tx=0x7fdd10009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.769+0000 7fdd26821700 1 -- 172.21.15.63:0/131790849 shutdown_connections 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.769+0000 7fdd26821700 1 --2- 172.21.15.63:0/131790849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd20100da0 0x7fdd20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.769+0000 7fdd26821700 1 --2- 172.21.15.63:0/131790849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 0x7fdd200fffc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.769+0000 7fdd26821700 1 -- 172.21.15.63:0/131790849 >> 172.21.15.63:0/131790849 conn(0x7fdd200fb120 msgr2=0x7fdd200fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.769+0000 7fdd26821700 1 -- 172.21.15.63:0/131790849 shutdown_connections 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.769+0000 7fdd26821700 1 -- 172.21.15.63:0/131790849 wait complete. 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd26821700 1 Processor -- start 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd26821700 1 -- start start 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd26821700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 0x7fdd20196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd26821700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd20100da0 0x7fdd20196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd26821700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd201972b0 con 0x7fdd200ffba0 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd26821700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd201973f0 con 0x7fdd20100da0 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd1ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 0x7fdd20196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd1ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 0x7fdd20196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59716/0 (socket says 172.21.15.63:59716) 2024-05-09T06:46:23.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd1ffff700 1 -- 172.21.15.63:0/3552337408 learned_addr learned my addr 172.21.15.63:0/3552337408 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.770+0000 7fdd1f7fe700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd20100da0 0x7fdd20196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.771+0000 7fdd1ffff700 1 -- 172.21.15.63:0/3552337408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd20100da0 msgr2=0x7fdd20196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.771+0000 7fdd1ffff700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd20100da0 0x7fdd20196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.771+0000 7fdd1ffff700 1 -- 172.21.15.63:0/3552337408 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd100095f0 con 0x7fdd200ffba0 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.771+0000 7fdd1ffff700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 0x7fdd20196750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005f00 tx=0x7fdd1000b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.771+0000 7fdd1d7fa700 1 -- 172.21.15.63:0/3552337408 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd1001d070 con 0x7fdd200ffba0 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.771+0000 7fdd1d7fa700 1 -- 172.21.15.63:0/3552337408 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd100052a0 con 0x7fdd200ffba0 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.771+0000 7fdd1d7fa700 1 -- 172.21.15.63:0/3552337408 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd10017400 con 0x7fdd200ffba0 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.771+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2019bda0 con 0x7fdd200ffba0 2024-05-09T06:46:23.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.771+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2019c230 con 0x7fdd200ffba0 2024-05-09T06:46:23.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.772+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2004ec20 con 0x7fdd200ffba0 2024-05-09T06:46:23.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.773+0000 7fdd1d7fa700 1 -- 172.21.15.63:0/3552337408 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdd10003680 con 0x7fdd200ffba0 2024-05-09T06:46:23.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.773+0000 7fdd1d7fa700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd0806e200 0x7fdd080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:23.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.773+0000 7fdd1d7fa700 1 -- 172.21.15.63:0/3552337408 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fdd1008f6e0 con 0x7fdd200ffba0 2024-05-09T06:46:23.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.773+0000 7fdd1f7fe700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd0806e200 0x7fdd080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:23.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.774+0000 7fdd1f7fe700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd0806e200 0x7fdd080706c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009c60 tx=0x7fdd14009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:23.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:23.778+0000 7fdd1d7fa700 1 -- 172.21.15.63:0/3552337408 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdd2004ec20 con 0x7fdd200ffba0 2024-05-09T06:46:23.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:23 smithi063 ceph-mon[92765]: pgmap v234: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-05-09T06:46:23.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:23.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:23 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:23.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:23 smithi193 ceph-mon[81665]: pgmap v234: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-05-09T06:46:23.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:23.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:23 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:23.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.007+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd2004ec20 con 0x7fdd0806e200 2024-05-09T06:46:23.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.009+0000 7fdd1d7fa700 1 -- 172.21.15.63:0/3552337408 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fdd2004ec20 con 0x7fdd0806e200 2024-05-09T06:46:23.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd0806e200 msgr2=0x7fdd080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:23.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd0806e200 0x7fdd080706c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009c60 tx=0x7fdd14009260 comp rx=0 tx=0).stop 2024-05-09T06:46:23.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 msgr2=0x7fdd20196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:23.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 0x7fdd20196750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005f00 tx=0x7fdd1000b5e0 comp rx=0 tx=0).stop 2024-05-09T06:46:23.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 shutdown_connections 2024-05-09T06:46:23.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd0806e200 0x7fdd080706c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:23.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd200ffba0 0x7fdd20196750 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:23.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 --2- 172.21.15.63:0/3552337408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd20100da0 0x7fdd20196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:23.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 >> 172.21.15.63:0/3552337408 conn(0x7fdd200fb120 msgr2=0x7fdd20068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:23.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 shutdown_connections 2024-05-09T06:46:23.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.012+0000 7fdd26821700 1 -- 172.21.15.63:0/3552337408 wait complete. 2024-05-09T06:46:24.002 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:46:24.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.122+0000 7f1a20c45700 1 -- 172.21.15.63:0/2368760923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 msgr2=0x7f1a1c1063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.122+0000 7f1a20c45700 1 --2- 172.21.15.63:0/2368760923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 0x7f1a1c1063f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009910 tx=0x7f1a10009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:24.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.122+0000 7f1a20c45700 1 -- 172.21.15.63:0/2368760923 shutdown_connections 2024-05-09T06:46:24.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.122+0000 7f1a20c45700 1 --2- 172.21.15.63:0/2368760923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 0x7f1a1c1063f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.122+0000 7f1a20c45700 1 --2- 172.21.15.63:0/2368760923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a1c1016d0 0x7f1a1c103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.122+0000 7f1a20c45700 1 -- 172.21.15.63:0/2368760923 >> 172.21.15.63:0/2368760923 conn(0x7f1a1c0fb120 msgr2=0x7f1a1c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:24.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.122+0000 7f1a20c45700 1 -- 172.21.15.63:0/2368760923 shutdown_connections 2024-05-09T06:46:24.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.122+0000 7f1a20c45700 1 -- 172.21.15.63:0/2368760923 wait complete. 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a20c45700 1 Processor -- start 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a20c45700 1 -- start start 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a20c45700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a1c1016d0 0x7f1a1c100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a20c45700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 0x7f1a1c0ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a20c45700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0ffbc0 con 0x7f1a1c104000 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a20c45700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0ffd00 con 0x7f1a1c1016d0 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a19d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 0x7f1a1c0ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a19d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 0x7f1a1c0ff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59744/0 (socket says 172.21.15.63:59744) 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a1a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a1c1016d0 0x7f1a1c100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a19d9b700 1 -- 172.21.15.63:0/294596197 learned_addr learned my addr 172.21.15.63:0/294596197 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a19d9b700 1 -- 172.21.15.63:0/294596197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a1c1016d0 msgr2=0x7f1a1c100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.123+0000 7f1a19d9b700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a1c1016d0 0x7f1a1c100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.124+0000 7f1a19d9b700 1 -- 172.21.15.63:0/294596197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a100095f0 con 0x7f1a1c104000 2024-05-09T06:46:24.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.124+0000 7f1a19d9b700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 0x7f1a1c0ff680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a100092e0 tx=0x7f1a1000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:24.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.124+0000 7f1a0b7fe700 1 -- 172.21.15.63:0/294596197 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1001d070 con 0x7f1a1c104000 2024-05-09T06:46:24.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.124+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c0fff80 con 0x7f1a1c104000 2024-05-09T06:46:24.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.124+0000 7f1a0b7fe700 1 -- 172.21.15.63:0/294596197 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a1000f460 con 0x7f1a1c104000 2024-05-09T06:46:24.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.124+0000 7f1a0b7fe700 1 -- 172.21.15.63:0/294596197 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a10021620 con 0x7f1a1c104000 2024-05-09T06:46:24.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.124+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1c100470 con 0x7f1a1c104000 2024-05-09T06:46:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.125+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1c04ebb0 con 0x7f1a1c104000 2024-05-09T06:46:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.125+0000 7f1a0b7fe700 1 -- 172.21.15.63:0/294596197 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1a1000f5d0 con 0x7f1a1c104000 2024-05-09T06:46:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.126+0000 7f1a0b7fe700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f1a0406e200 0x7f1a040706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.126+0000 7f1a0b7fe700 1 -- 172.21.15.63:0/294596197 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f1a1008f580 con 0x7f1a1c104000 2024-05-09T06:46:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.126+0000 7f1a1a59c700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f1a0406e200 0x7f1a040706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.126+0000 7f1a1a59c700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f1a0406e200 0x7f1a040706c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007be0 tx=0x7f1a0c0073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:24.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.131+0000 7f1a0b7fe700 1 -- 172.21.15.63:0/294596197 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1a1005c210 con 0x7f1a1c104000 2024-05-09T06:46:24.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.360+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a1c105fd0 con 0x7f1a0406e200 2024-05-09T06:46:24.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.362+0000 7f1a0b7fe700 1 -- 172.21.15.63:0/294596197 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f1a1c105fd0 con 0x7f1a0406e200 2024-05-09T06:46:24.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f1a0406e200 msgr2=0x7f1a040706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f1a0406e200 0x7f1a040706c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007be0 tx=0x7f1a0c0073d0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 msgr2=0x7f1a1c0ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 0x7f1a1c0ff680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a100092e0 tx=0x7f1a1000bd70 comp rx=0 tx=0).stop 2024-05-09T06:46:24.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 shutdown_connections 2024-05-09T06:46:24.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f1a0406e200 0x7f1a040706c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a1c1016d0 0x7f1a1c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 --2- 172.21.15.63:0/294596197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a1c104000 0x7f1a1c0ff680 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 >> 172.21.15.63:0/294596197 conn(0x7f1a1c0fb120 msgr2=0x7f1a1c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:24.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 shutdown_connections 2024-05-09T06:46:24.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.364+0000 7f1a20c45700 1 -- 172.21.15.63:0/294596197 wait complete. 2024-05-09T06:46:24.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.482+0000 7f417c7d3700 1 -- 172.21.15.63:0/2260614546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41740ffba0 msgr2=0x7f41740fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.482+0000 7f417c7d3700 1 --2- 172.21.15.63:0/2260614546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41740ffba0 0x7f41740fffc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4170009910 tx=0x7f4170009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:24.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.482+0000 7f417c7d3700 1 -- 172.21.15.63:0/2260614546 shutdown_connections 2024-05-09T06:46:24.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.482+0000 7f417c7d3700 1 --2- 172.21.15.63:0/2260614546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4174100da0 0x7f4174101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.482+0000 7f417c7d3700 1 --2- 172.21.15.63:0/2260614546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41740ffba0 0x7f41740fffc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.482+0000 7f417c7d3700 1 -- 172.21.15.63:0/2260614546 >> 172.21.15.63:0/2260614546 conn(0x7f41740fb120 msgr2=0x7f41740fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:24.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.482+0000 7f417c7d3700 1 -- 172.21.15.63:0/2260614546 shutdown_connections 2024-05-09T06:46:24.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.482+0000 7f417c7d3700 1 -- 172.21.15.63:0/2260614546 wait complete. 2024-05-09T06:46:24.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.483+0000 7f417c7d3700 1 Processor -- start 2024-05-09T06:46:24.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.483+0000 7f417c7d3700 1 -- start start 2024-05-09T06:46:24.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.483+0000 7f417c7d3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41740ffba0 0x7f4174103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:24.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.483+0000 7f417c7d3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4174100da0 0x7f4174102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:24.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.483+0000 7f417c7d3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41741026c0 con 0x7f4174100da0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.483+0000 7f417c7d3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4174102800 con 0x7f41740ffba0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f4179d6e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4174100da0 0x7f4174102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f417a56f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41740ffba0 0x7f4174103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f4179d6e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4174100da0 0x7f4174102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59754/0 (socket says 172.21.15.63:59754) 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f4179d6e700 1 -- 172.21.15.63:0/262621792 learned_addr learned my addr 172.21.15.63:0/262621792 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f4179d6e700 1 -- 172.21.15.63:0/262621792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41740ffba0 msgr2=0x7f4174103a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f4179d6e700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41740ffba0 0x7f4174103a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f4179d6e700 1 -- 172.21.15.63:0/262621792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700095f0 con 0x7f4174100da0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f417a56f700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41740ffba0 0x7f4174103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f4179d6e700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4174100da0 0x7f4174102180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f416400e920 tx=0x7f416400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f416b7fe700 1 -- 172.21.15.63:0/262621792 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f416400c9d0 con 0x7f4174100da0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4174102ae0 con 0x7f4174100da0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f416b7fe700 1 -- 172.21.15.63:0/262621792 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4164010430 con 0x7f4174100da0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f416b7fe700 1 -- 172.21.15.63:0/262621792 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4164018630 con 0x7f4174100da0 2024-05-09T06:46:24.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.484+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417419e7b0 con 0x7f4174100da0 2024-05-09T06:46:24.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.485+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417404ebb0 con 0x7f4174100da0 2024-05-09T06:46:24.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.486+0000 7f416b7fe700 1 -- 172.21.15.63:0/262621792 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f41640105a0 con 0x7f4174100da0 2024-05-09T06:46:24.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.486+0000 7f416b7fe700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f416006e210 0x7f41600706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:24.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.487+0000 7f416b7fe700 1 -- 172.21.15.63:0/262621792 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f4164014070 con 0x7f4174100da0 2024-05-09T06:46:24.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.487+0000 7f417a56f700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f416006e210 0x7f41600706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:24.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.487+0000 7f417a56f700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f416006e210 0x7f41600706d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f417000b800 tx=0x7f4170009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:24.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.491+0000 7f416b7fe700 1 -- 172.21.15.63:0/262621792 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4164053740 con 0x7f4174100da0 2024-05-09T06:46:24.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.715+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4174068d90 con 0x7f416006e210 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.728+0000 7f416b7fe700 1 -- 172.21.15.63:0/262621792 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f4174068d90 con 0x7f416006e210 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (15m) 66s ago 16m 25.5M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 starting - - - - 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (5s) 2s ago 16m 7277k - 19.0.0-2515-g96805dcf d7acac7d5657 bd88664dfa9d 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (5m) 66s ago 17m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (5m) 2s ago 15m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (15m) 66s ago 16m 87.3M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (90s) 66s ago 11m 15.8M - 19.0.0-2515-g96805dcf d7acac7d5657 c7cf68a1a3e2 2024-05-09T06:46:24.707 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (72s) 66s ago 11m 13.8M - 19.0.0-2515-g96805dcf d7acac7d5657 c24cf6de2c0f 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (45s) 2s ago 11m 13.7M - 19.0.0-2515-g96805dcf d7acac7d5657 b752027d6904 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (25s) 2s ago 11m 13.5M - 19.0.0-2515-g96805dcf d7acac7d5657 bf31d952d7fe 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (7m) 66s ago 18m 512M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (6m) 2s ago 15m 453M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (6m) 66s ago 18m 51.8M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (5m) 2s ago 15m 38.5M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (16m) 66s ago 16m 15.7M - 1.5.0 0da6a335fe13 f202ec485ed3 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (15m) 2s ago 15m 16.0M - 1.5.0 0da6a335fe13 400af60dafb8 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (4m) 66s ago 14m 147M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (4m) 66s ago 14m 134M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (3m) 66s ago 14m 90.1M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (2m) 2s ago 13m 119M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (2m) 2s ago 13m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 14c9b4920f5a 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (108s) 2s ago 12m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 d219b4bb599f 2024-05-09T06:46:24.708 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 66s ago 16m 63.6M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:46:24.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f416006e210 msgr2=0x7f41600706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f416006e210 0x7f41600706d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f417000b800 tx=0x7f4170009f70 comp rx=0 tx=0).stop 2024-05-09T06:46:24.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4174100da0 msgr2=0x7f4174102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4174100da0 0x7f4174102180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f416400e920 tx=0x7f416400ec30 comp rx=0 tx=0).stop 2024-05-09T06:46:24.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 shutdown_connections 2024-05-09T06:46:24.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f416006e210 0x7f41600706d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41740ffba0 0x7f4174103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 --2- 172.21.15.63:0/262621792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4174100da0 0x7f4174102180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 >> 172.21.15.63:0/262621792 conn(0x7f41740fb120 msgr2=0x7f4174068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:24.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 shutdown_connections 2024-05-09T06:46:24.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.730+0000 7f417c7d3700 1 -- 172.21.15.63:0/262621792 wait complete. 2024-05-09T06:46:24.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:24 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:24.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:24 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:24.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.847+0000 7f4b8a925700 1 -- 172.21.15.63:0/1392380572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 msgr2=0x7f4b84100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.847+0000 7f4b8a925700 1 --2- 172.21.15.63:0/1392380572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 0x7f4b84100210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009910 tx=0x7f4b78009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:24.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.847+0000 7f4b8a925700 1 -- 172.21.15.63:0/1392380572 shutdown_connections 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.847+0000 7f4b8a925700 1 --2- 172.21.15.63:0/1392380572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 0x7f4b84100210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.847+0000 7f4b8a925700 1 --2- 172.21.15.63:0/1392380572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b84101130 0x7f4b84101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.847+0000 7f4b8a925700 1 -- 172.21.15.63:0/1392380572 >> 172.21.15.63:0/1392380572 conn(0x7f4b840fa840 msgr2=0x7f4b840fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.847+0000 7f4b8a925700 1 -- 172.21.15.63:0/1392380572 shutdown_connections 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.847+0000 7f4b8a925700 1 -- 172.21.15.63:0/1392380572 wait complete. 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b8a925700 1 Processor -- start 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b8a925700 1 -- start start 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b8a925700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 0x7f4b841954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b8a925700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b84101130 0x7f4b84195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b8a925700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84195f90 con 0x7f4b840ffd90 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b8a925700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b841960d0 con 0x7f4b84101130 2024-05-09T06:46:24.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b83fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 0x7f4b841954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b83fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 0x7f4b841954c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35632/0 (socket says 172.21.15.63:35632) 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b83fff700 1 -- 172.21.15.63:0/457315273 learned_addr learned my addr 172.21.15.63:0/457315273 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.848+0000 7f4b837fe700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b84101130 0x7f4b84195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.849+0000 7f4b83fff700 1 -- 172.21.15.63:0/457315273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b84101130 msgr2=0x7f4b84195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.849+0000 7f4b83fff700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b84101130 0x7f4b84195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.849+0000 7f4b83fff700 1 -- 172.21.15.63:0/457315273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b780095f0 con 0x7f4b840ffd90 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.849+0000 7f4b83fff700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 0x7f4b841954c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400c910 tx=0x7f4b7400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.849+0000 7f4b817fa700 1 -- 172.21.15.63:0/457315273 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b740041f0 con 0x7f4b840ffd90 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.849+0000 7f4b817fa700 1 -- 172.21.15.63:0/457315273 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b74007dc0 con 0x7f4b840ffd90 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.849+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8419ab40 con 0x7f4b840ffd90 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.849+0000 7f4b817fa700 1 -- 172.21.15.63:0/457315273 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b74003e70 con 0x7f4b840ffd90 2024-05-09T06:46:24.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.849+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8419b060 con 0x7f4b840ffd90 2024-05-09T06:46:24.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.850+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8418f6c0 con 0x7f4b840ffd90 2024-05-09T06:46:24.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.851+0000 7f4b817fa700 1 -- 172.21.15.63:0/457315273 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4b7400f4a0 con 0x7f4b840ffd90 2024-05-09T06:46:24.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.851+0000 7f4b817fa700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b6c06e1b0 0x7f4b6c070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:24.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.851+0000 7f4b817fa700 1 -- 172.21.15.63:0/457315273 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f4b7408e070 con 0x7f4b840ffd90 2024-05-09T06:46:24.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.851+0000 7f4b837fe700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b6c06e1b0 0x7f4b6c070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:24.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.852+0000 7f4b837fe700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b6c06e1b0 0x7f4b6c070670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b780051c0 tx=0x7f4b7800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:24.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:24.856+0000 7f4b817fa700 1 -- 172.21.15.63:0/457315273 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4b7405ad00 con 0x7f4b840ffd90 2024-05-09T06:46:24.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:24 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:24.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:24 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.157+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b8402ce80 con 0x7f4b840ffd90 2024-05-09T06:46:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.157+0000 7f4b817fa700 1 -- 172.21.15.63:0/457315273 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4b7405a890 con 0x7f4b840ffd90 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 6 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:46:25.137 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 4 2024-05-09T06:46:25.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:46:25.138 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:46:25.138 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 14 2024-05-09T06:46:25.138 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:46:25.138 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:46:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b6c06e1b0 msgr2=0x7f4b6c070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b6c06e1b0 0x7f4b6c070670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b780051c0 tx=0x7f4b7800b540 comp rx=0 tx=0).stop 2024-05-09T06:46:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 msgr2=0x7f4b841954c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 0x7f4b841954c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400c910 tx=0x7f4b7400ccd0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 shutdown_connections 2024-05-09T06:46:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f4b6c06e1b0 0x7f4b6c070670 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b840ffd90 0x7f4b841954c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 --2- 172.21.15.63:0/457315273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b84101130 0x7f4b84195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 >> 172.21.15.63:0/457315273 conn(0x7f4b840fa840 msgr2=0x7f4b84104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 shutdown_connections 2024-05-09T06:46:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.160+0000 7f4b8a925700 1 -- 172.21.15.63:0/457315273 wait complete. 2024-05-09T06:46:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.274+0000 7fe5392a3700 1 -- 172.21.15.63:0/1333247694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe534102200 msgr2=0x7fe534102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.274+0000 7fe5392a3700 1 --2- 172.21.15.63:0/1333247694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe534102200 0x7fe534102620 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe524009910 tx=0x7fe524009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.274+0000 7fe5392a3700 1 -- 172.21.15.63:0/1333247694 shutdown_connections 2024-05-09T06:46:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.274+0000 7fe5392a3700 1 --2- 172.21.15.63:0/1333247694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5340fec80 0x7fe5340ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.274+0000 7fe5392a3700 1 --2- 172.21.15.63:0/1333247694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe534102200 0x7fe534102620 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.274+0000 7fe5392a3700 1 -- 172.21.15.63:0/1333247694 >> 172.21.15.63:0/1333247694 conn(0x7fe5340fa840 msgr2=0x7fe5340fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:25.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.274+0000 7fe5392a3700 1 -- 172.21.15.63:0/1333247694 shutdown_connections 2024-05-09T06:46:25.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.274+0000 7fe5392a3700 1 -- 172.21.15.63:0/1333247694 wait complete. 2024-05-09T06:46:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.275+0000 7fe5392a3700 1 Processor -- start 2024-05-09T06:46:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.275+0000 7fe5392a3700 1 -- start start 2024-05-09T06:46:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.275+0000 7fe5392a3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5340fec80 0x7fe5341018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.275+0000 7fe5392a3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe534102200 0x7fe5340fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.275+0000 7fe5392a3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe534100490 con 0x7fe5340fec80 2024-05-09T06:46:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.275+0000 7fe5392a3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5341005d0 con 0x7fe534102200 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe532ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5340fec80 0x7fe5341018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe532ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5340fec80 0x7fe5341018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35650/0 (socket says 172.21.15.63:35650) 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe532ffd700 1 -- 172.21.15.63:0/117443636 learned_addr learned my addr 172.21.15.63:0/117443636 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe5327fc700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe534102200 0x7fe5340fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe532ffd700 1 -- 172.21.15.63:0/117443636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe534102200 msgr2=0x7fe5340fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe532ffd700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe534102200 0x7fe5340fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe532ffd700 1 -- 172.21.15.63:0/117443636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5240095f0 con 0x7fe5340fec80 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe5327fc700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe534102200 0x7fe5340fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe532ffd700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5340fec80 0x7fe5341018b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe5240055f0 tx=0x7fe524005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe51bfff700 1 -- 172.21.15.63:0/117443636 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe52401d070 con 0x7fe5340fec80 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.276+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe534100850 con 0x7fe5340fec80 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.277+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe534100dc0 con 0x7fe5340fec80 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.277+0000 7fe51bfff700 1 -- 172.21.15.63:0/117443636 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe52400f460 con 0x7fe5340fec80 2024-05-09T06:46:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.277+0000 7fe51bfff700 1 -- 172.21.15.63:0/117443636 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe524021620 con 0x7fe5340fec80 2024-05-09T06:46:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.278+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe520005320 con 0x7fe5340fec80 2024-05-09T06:46:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.279+0000 7fe51bfff700 1 -- 172.21.15.63:0/117443636 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe52400f5d0 con 0x7fe5340fec80 2024-05-09T06:46:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.279+0000 7fe51bfff700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe51c06e200 0x7fe51c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:25.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.279+0000 7fe51bfff700 1 -- 172.21.15.63:0/117443636 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fe52408f390 con 0x7fe5340fec80 2024-05-09T06:46:25.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.279+0000 7fe5327fc700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe51c06e200 0x7fe51c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:25.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.280+0000 7fe5327fc700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe51c06e200 0x7fe51c0706c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe528005950 tx=0x7fe52800a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.284+0000 7fe51bfff700 1 -- 172.21.15.63:0/117443636 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe52405c0a0 con 0x7fe5340fec80 2024-05-09T06:46:25.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.536+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe520005de0 con 0x7fe5340fec80 2024-05-09T06:46:25.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.537+0000 7fe51bfff700 1 -- 172.21.15.63:0/117443636 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1963 (secure 0 0 0) 0x7fe52405bc30 con 0x7fe5340fec80 2024-05-09T06:46:25.516 INFO:teuthology.orchestra.run.smithi063.stdout:e34 2024-05-09T06:46:25.516 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:46:25.516 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:46:25.516 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 34 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:46:08.230738+0000 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:46:25.517 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 96 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34406} 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:46:25.518 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 34406 members: 34406 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{0:34406} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:34410} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/2182150991,v1:172.21.15.63:6829/2182150991] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/2007543899,v1:172.21.15.193:6825/2007543899] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:46:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/790240017,v1:172.21.15.193:6827/790240017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:46:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe51c06e200 msgr2=0x7fe51c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe51c06e200 0x7fe51c0706c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe528005950 tx=0x7fe52800a300 comp rx=0 tx=0).stop 2024-05-09T06:46:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5340fec80 msgr2=0x7fe5341018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5340fec80 0x7fe5341018b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe5240055f0 tx=0x7fe524005620 comp rx=0 tx=0).stop 2024-05-09T06:46:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 shutdown_connections 2024-05-09T06:46:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe51c06e200 0x7fe51c0706c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5340fec80 0x7fe5341018b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 --2- 172.21.15.63:0/117443636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe534102200 0x7fe5340fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 >> 172.21.15.63:0/117443636 conn(0x7fe5340fa840 msgr2=0x7fe5341055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:25.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 shutdown_connections 2024-05-09T06:46:25.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.539+0000 7fe5392a3700 1 -- 172.21.15.63:0/117443636 wait complete. 2024-05-09T06:46:25.521 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 34 2024-05-09T06:46:25.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.658+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/3499915895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 msgr2=0x7f0d840ff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.658+0000 7f0d8ac6e700 1 --2- 172.21.15.63:0/3499915895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 0x7f0d840ff310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009910 tx=0x7f0d78009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.658+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/3499915895 shutdown_connections 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.658+0000 7f0d8ac6e700 1 --2- 172.21.15.63:0/3499915895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d84101ed0 0x7f0d84102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.658+0000 7f0d8ac6e700 1 --2- 172.21.15.63:0/3499915895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 0x7f0d840ff310 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.658+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/3499915895 >> 172.21.15.63:0/3499915895 conn(0x7f0d840fa850 msgr2=0x7f0d840fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.658+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/3499915895 shutdown_connections 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.658+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/3499915895 wait complete. 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d8ac6e700 1 Processor -- start 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d8ac6e700 1 -- start start 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d8ac6e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 0x7f0d84101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d8ac6e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d84101ed0 0x7f0d840ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d8ac6e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d841003c0 con 0x7f0d840feef0 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d8ac6e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84100500 con 0x7f0d84101ed0 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d88a0a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 0x7f0d84101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:25.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d88a0a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 0x7f0d84101790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35666/0 (socket says 172.21.15.63:35666) 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d88a0a700 1 -- 172.21.15.63:0/64024922 learned_addr learned my addr 172.21.15.63:0/64024922 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.659+0000 7f0d83fff700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d84101ed0 0x7f0d840ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d88a0a700 1 -- 172.21.15.63:0/64024922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d84101ed0 msgr2=0x7f0d840ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d88a0a700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d84101ed0 0x7f0d840ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d88a0a700 1 -- 172.21.15.63:0/64024922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d780095f0 con 0x7f0d840feef0 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d83fff700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d84101ed0 0x7f0d840ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d88a0a700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 0x7f0d84101790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d78005f00 tx=0x7f0d7800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d81ffb700 1 -- 172.21.15.63:0/64024922 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d7801d070 con 0x7f0d840feef0 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d81ffb700 1 -- 172.21.15.63:0/64024922 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d7800fe30 con 0x7f0d840feef0 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d84100780 con 0x7f0d840feef0 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d81ffb700 1 -- 172.21.15.63:0/64024922 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d78017400 con 0x7f0d840feef0 2024-05-09T06:46:25.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.660+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d84100cf0 con 0x7f0d840feef0 2024-05-09T06:46:25.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.662+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d841907b0 con 0x7f0d840feef0 2024-05-09T06:46:25.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.662+0000 7f0d81ffb700 1 -- 172.21.15.63:0/64024922 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0d7800f970 con 0x7f0d840feef0 2024-05-09T06:46:25.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.662+0000 7f0d81ffb700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0d7406e200 0x7f0d740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:25.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.662+0000 7f0d81ffb700 1 -- 172.21.15.63:0/64024922 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f0d7808f630 con 0x7f0d840feef0 2024-05-09T06:46:25.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.662+0000 7f0d83fff700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0d7406e200 0x7f0d740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:25.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.663+0000 7f0d83fff700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0d7406e200 0x7f0d740706c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009be0 tx=0x7f0d70009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:25.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.667+0000 7f0d81ffb700 1 -- 172.21.15.63:0/64024922 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0d7805c2c0 con 0x7f0d840feef0 2024-05-09T06:46:25.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:25 smithi193 ceph-mon[81665]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:25.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:25 smithi193 ceph-mon[81665]: pgmap v235: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 2 op/s 2024-05-09T06:46:25.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:25 smithi193 ceph-mon[81665]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:25.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:25 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/457315273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:25.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:25 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/117443636' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:46:25.782 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:25 smithi063 ceph-mon[92765]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:25.782 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:25 smithi063 ceph-mon[92765]: pgmap v235: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 2 op/s 2024-05-09T06:46:25.782 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:25 smithi063 ceph-mon[92765]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:25.783 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:25 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/457315273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:25.783 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:25 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/117443636' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:46:25.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.912+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d84100cf0 con 0x7f0d7406e200 2024-05-09T06:46:25.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.913+0000 7f0d81ffb700 1 -- 172.21.15.63:0/64024922 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f0d84100cf0 con 0x7f0d7406e200 2024-05-09T06:46:25.892 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "mds" 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "17/23 daemons upgraded", 2024-05-09T06:46:25.893 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-05-09T06:46:25.894 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:46:25.894 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:46:25.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0d7406e200 msgr2=0x7f0d740706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0d7406e200 0x7f0d740706c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009be0 tx=0x7f0d70009260 comp rx=0 tx=0).stop 2024-05-09T06:46:25.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 msgr2=0x7f0d84101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:25.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 0x7f0d84101790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d78005f00 tx=0x7f0d7800f510 comp rx=0 tx=0).stop 2024-05-09T06:46:25.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 shutdown_connections 2024-05-09T06:46:25.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0d7406e200 0x7f0d740706c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d840feef0 0x7f0d84101790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 --2- 172.21.15.63:0/64024922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d84101ed0 0x7f0d840ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:25.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 >> 172.21.15.63:0/64024922 conn(0x7f0d840fa850 msgr2=0x7f0d841054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:25.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 shutdown_connections 2024-05-09T06:46:25.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:25.916+0000 7f0d8ac6e700 1 -- 172.21.15.63:0/64024922 wait complete. 2024-05-09T06:46:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.024+0000 7efd2cfcb700 1 -- 172.21.15.63:0/2947587053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd281016c0 msgr2=0x7efd28103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.024+0000 7efd2cfcb700 1 --2- 172.21.15.63:0/2947587053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd281016c0 0x7efd28103ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd18009910 tx=0x7efd18009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.025+0000 7efd2cfcb700 1 -- 172.21.15.63:0/2947587053 shutdown_connections 2024-05-09T06:46:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.025+0000 7efd2cfcb700 1 --2- 172.21.15.63:0/2947587053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd28103ff0 0x7efd281063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.025+0000 7efd2cfcb700 1 --2- 172.21.15.63:0/2947587053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd281016c0 0x7efd28103ab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.025+0000 7efd2cfcb700 1 -- 172.21.15.63:0/2947587053 >> 172.21.15.63:0/2947587053 conn(0x7efd280fb0d0 msgr2=0x7efd280fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:26.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.025+0000 7efd2cfcb700 1 -- 172.21.15.63:0/2947587053 shutdown_connections 2024-05-09T06:46:26.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.025+0000 7efd2cfcb700 1 -- 172.21.15.63:0/2947587053 wait complete. 2024-05-09T06:46:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.025+0000 7efd2cfcb700 1 Processor -- start 2024-05-09T06:46:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.025+0000 7efd2cfcb700 1 -- start start 2024-05-09T06:46:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd2cfcb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd281016c0 0x7efd28100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:26.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd2cfcb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd28103ff0 0x7efd280ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd2cfcb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280ffbb0 con 0x7efd28103ff0 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd2cfcb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280ffcf0 con 0x7efd281016c0 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd25d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd28103ff0 0x7efd280ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd25d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd28103ff0 0x7efd280ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35696/0 (socket says 172.21.15.63:35696) 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd25d9b700 1 -- 172.21.15.63:0/3319781336 learned_addr learned my addr 172.21.15.63:0/3319781336 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd2659c700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd281016c0 0x7efd28100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd25d9b700 1 -- 172.21.15.63:0/3319781336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd281016c0 msgr2=0x7efd28100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd25d9b700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd281016c0 0x7efd28100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd25d9b700 1 -- 172.21.15.63:0/3319781336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd180095f0 con 0x7efd28103ff0 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd25d9b700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd28103ff0 0x7efd280ff670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00d8d0 tx=0x7efd1c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd177fe700 1 -- 172.21.15.63:0/3319781336 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd1c0098a0 con 0x7efd28103ff0 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd177fe700 1 -- 172.21.15.63:0/3319781336 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd1c010460 con 0x7efd28103ff0 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.026+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd280fffd0 con 0x7efd28103ff0 2024-05-09T06:46:26.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.027+0000 7efd177fe700 1 -- 172.21.15.63:0/3319781336 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd1c00f5f0 con 0x7efd28103ff0 2024-05-09T06:46:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.027+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2819e6b0 con 0x7efd28103ff0 2024-05-09T06:46:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.027+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2804ebb0 con 0x7efd28103ff0 2024-05-09T06:46:26.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.028+0000 7efd177fe700 1 -- 172.21.15.63:0/3319781336 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efd1c009a00 con 0x7efd28103ff0 2024-05-09T06:46:26.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.029+0000 7efd177fe700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efd1006e210 0x7efd100706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:26.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.029+0000 7efd177fe700 1 -- 172.21.15.63:0/3319781336 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7efd1c08e1c0 con 0x7efd28103ff0 2024-05-09T06:46:26.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.029+0000 7efd2659c700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efd1006e210 0x7efd100706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:26.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.029+0000 7efd2659c700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efd1006e210 0x7efd100706d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd18006010 tx=0x7efd1800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:26.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.033+0000 7efd177fe700 1 -- 172.21.15.63:0/3319781336 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efd1c05ae50 con 0x7efd28103ff0 2024-05-09T06:46:26.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.348+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efd2819e8e0 con 0x7efd28103ff0 2024-05-09T06:46:26.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.348+0000 7efd177fe700 1 -- 172.21.15.63:0/3319781336 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7efd1c05a9e0 con 0x7efd28103ff0 2024-05-09T06:46:26.328 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:46:26.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efd1006e210 msgr2=0x7efd100706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:26.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efd1006e210 0x7efd100706d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd18006010 tx=0x7efd1800b540 comp rx=0 tx=0).stop 2024-05-09T06:46:26.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd28103ff0 msgr2=0x7efd280ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd28103ff0 0x7efd280ff670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00d8d0 tx=0x7efd1c00dbe0 comp rx=0 tx=0).stop 2024-05-09T06:46:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 shutdown_connections 2024-05-09T06:46:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7efd1006e210 0x7efd100706d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd281016c0 0x7efd28100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 --2- 172.21.15.63:0/3319781336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd28103ff0 0x7efd280ff670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 >> 172.21.15.63:0/3319781336 conn(0x7efd280fb0d0 msgr2=0x7efd280fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 shutdown_connections 2024-05-09T06:46:26.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:26.350+0000 7efd2cfcb700 1 -- 172.21.15.63:0/3319781336 wait complete. 2024-05-09T06:46:26.717 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:26 smithi193 ceph-mon[81665]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:26.717 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:26 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3319781336' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:46:27.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:26 smithi063 ceph-mon[92765]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:27.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:26 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3319781336' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:46:27.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:27 smithi063 ceph-mon[92765]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:27.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:27 smithi063 ceph-mon[92765]: pgmap v236: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 2 op/s 2024-05-09T06:46:27.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:46:27.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:27.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:27.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:27.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:28.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:27 smithi193 ceph-mon[81665]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:27 smithi193 ceph-mon[81665]: pgmap v236: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 2 op/s 2024-05-09T06:46:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:46:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:28.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:29.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:29 smithi063 ceph-mon[92765]: pgmap v237: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 2 op/s 2024-05-09T06:46:29.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:29.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:29 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:29.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:29 smithi193 ceph-mon[81665]: pgmap v237: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 2 op/s 2024-05-09T06:46:29.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:29.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:29 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:31.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:31 smithi193 ceph-mon[81665]: pgmap v238: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-05-09T06:46:31.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:31 smithi063 ceph-mon[92765]: pgmap v238: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-05-09T06:46:33.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: pgmap v239: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-05-09T06:46:33.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:33.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:46:33.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:33.376 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all ceph-exporter 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi063"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi063"}]': finished 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi193"}]: dispatch 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi193"}]': finished 2024-05-09T06:46:33.377 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all iscsi 2024-05-09T06:46:33.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all nfs 2024-05-09T06:46:33.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: Upgrade: Setting container_image for all nvmeof 2024-05-09T06:46:33.378 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:33 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: pgmap v239: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-05-09T06:46:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:46:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:33.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all ceph-exporter 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi063"}]: dispatch 2024-05-09T06:46:33.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi063"}]': finished 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi193"}]: dispatch 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi193"}]': finished 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all iscsi 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all nfs 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: Upgrade: Setting container_image for all nvmeof 2024-05-09T06:46:33.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:33 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:35.416 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:35 smithi063 ceph-mon[92765]: pgmap v240: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:35.416 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:35 smithi063 ceph-mon[92765]: Upgrade: Updating node-exporter.smithi063 (1/2) 2024-05-09T06:46:35.416 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:35 smithi063 ceph-mon[92765]: Deploying daemon node-exporter.smithi063 on smithi063 2024-05-09T06:46:35.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:35 smithi193 ceph-mon[81665]: pgmap v240: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:35.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:35 smithi193 ceph-mon[81665]: Upgrade: Updating node-exporter.smithi063 (1/2) 2024-05-09T06:46:35.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:35 smithi193 ceph-mon[81665]: Deploying daemon node-exporter.smithi063 on smithi063 2024-05-09T06:46:37.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:37 smithi063 ceph-mon[92765]: pgmap v241: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:37.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:37.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:37 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:37.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:37 smithi193 ceph-mon[81665]: pgmap v241: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:37.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:37.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:37 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:39.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:39 smithi063 ceph-mon[92765]: pgmap v242: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:46:39.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:39 smithi193 ceph-mon[81665]: pgmap v242: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:46:40.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:40 smithi063 ceph-mon[92765]: Upgrade: Updating node-exporter.smithi193 (2/2) 2024-05-09T06:46:40.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:40 smithi063 ceph-mon[92765]: Deploying daemon node-exporter.smithi193 on smithi193 2024-05-09T06:46:40.605 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:40 smithi193 ceph-mon[81665]: Upgrade: Updating node-exporter.smithi193 (2/2) 2024-05-09T06:46:40.605 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:40 smithi193 ceph-mon[81665]: Deploying daemon node-exporter.smithi193 on smithi193 2024-05-09T06:46:41.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:41 smithi063 ceph-mon[92765]: pgmap v243: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:41.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:41 smithi193 ceph-mon[81665]: pgmap v243: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:42.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:42.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:42.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:42.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:46:42.688 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:42.688 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:42.688 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:42.688 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:46:43.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:43 smithi063 ceph-mon[92765]: pgmap v244: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:43.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:43 smithi193 ceph-mon[81665]: pgmap v244: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:45.271 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:45.271 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:45 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:45.271 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:45 smithi063 ceph-mon[92765]: pgmap v245: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:45 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:45 smithi193 ceph-mon[81665]: pgmap v245: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:46.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:46 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:46.371 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:46 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:46.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:46 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:46.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:46 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:47.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:47 smithi063 ceph-mon[92765]: pgmap v246: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:47.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:47.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:47.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:47 smithi193 ceph-mon[81665]: pgmap v246: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:47.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:47.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:49.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:49.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:49.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:48 smithi063 ceph-mon[92765]: pgmap v247: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:46:49.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:49.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:49.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:48 smithi193 ceph-mon[81665]: pgmap v247: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:46:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:49 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:50.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:50.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:49 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:50.988 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:50 smithi063 ceph-mon[92765]: pgmap v248: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:51.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:50 smithi193 ceph-mon[81665]: pgmap v248: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:52.433 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:46:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:46:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:52.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:53.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:53 smithi063 ceph-mon[92765]: pgmap v249: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:53.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:53 smithi193 ceph-mon[81665]: pgmap v249: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:54.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:54 smithi063 ceph-mon[92765]: Upgrade: Updating prometheus.smithi063 2024-05-09T06:46:54.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:54 smithi063 ceph-mon[92765]: Deploying daemon prometheus.smithi063 on smithi063 2024-05-09T06:46:54.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:54 smithi193 ceph-mon[81665]: Upgrade: Updating prometheus.smithi063 2024-05-09T06:46:54.705 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:54 smithi193 ceph-mon[81665]: Deploying daemon prometheus.smithi063 on smithi063 2024-05-09T06:46:55.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:55 smithi193 ceph-mon[81665]: pgmap v250: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:55.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:55 smithi063 ceph-mon[92765]: pgmap v250: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:46:56.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.475+0000 7f776a0df700 1 -- 172.21.15.63:0/1559818383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 msgr2=0x7f77640ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:56.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.475+0000 7f776a0df700 1 --2- 172.21.15.63:0/1559818383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 0x7f77640ff330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7754009910 tx=0x7f7754009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:56.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.475+0000 7f776a0df700 1 -- 172.21.15.63:0/1559818383 shutdown_connections 2024-05-09T06:46:56.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.475+0000 7f776a0df700 1 --2- 172.21.15.63:0/1559818383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7764101ef0 0x7f7764102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.475+0000 7f776a0df700 1 --2- 172.21.15.63:0/1559818383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 0x7f77640ff330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.475+0000 7f776a0df700 1 -- 172.21.15.63:0/1559818383 >> 172.21.15.63:0/1559818383 conn(0x7f77640fa870 msgr2=0x7f77640fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.475+0000 7f776a0df700 1 -- 172.21.15.63:0/1559818383 shutdown_connections 2024-05-09T06:46:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.475+0000 7f776a0df700 1 -- 172.21.15.63:0/1559818383 wait complete. 2024-05-09T06:46:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.476+0000 7f776a0df700 1 Processor -- start 2024-05-09T06:46:56.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.476+0000 7f776a0df700 1 -- start start 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f776a0df700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 0x7f7764196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f776a0df700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7764101ef0 0x7f7764196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f776a0df700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77641971d0 con 0x7f77640fef10 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f776a0df700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7764197310 con 0x7f7764101ef0 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f77637fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 0x7f7764196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f77637fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 0x7f7764196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33828/0 (socket says 172.21.15.63:33828) 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f77637fe700 1 -- 172.21.15.63:0/3672395522 learned_addr learned my addr 172.21.15.63:0/3672395522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f7762ffd700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7764101ef0 0x7f7764196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f77637fe700 1 -- 172.21.15.63:0/3672395522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7764101ef0 msgr2=0x7f7764196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f77637fe700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7764101ef0 0x7f7764196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f77637fe700 1 -- 172.21.15.63:0/3672395522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77540095f0 con 0x7f77640fef10 2024-05-09T06:46:56.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f77637fe700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 0x7f7764196670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f77540059d0 tx=0x7f775400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:56.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f7760ff9700 1 -- 172.21.15.63:0/3672395522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775401d070 con 0x7f77640fef10 2024-05-09T06:46:56.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f7760ff9700 1 -- 172.21.15.63:0/3672395522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7754005dc0 con 0x7f77640fef10 2024-05-09T06:46:56.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.477+0000 7f7760ff9700 1 -- 172.21.15.63:0/3672395522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7754021410 con 0x7f77640fef10 2024-05-09T06:46:56.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.478+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776419bcc0 con 0x7f77640fef10 2024-05-09T06:46:56.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.478+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776419c1b0 con 0x7f77640fef10 2024-05-09T06:46:56.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.479+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77641907d0 con 0x7f77640fef10 2024-05-09T06:46:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.479+0000 7f7760ff9700 1 -- 172.21.15.63:0/3672395522 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f775400f840 con 0x7f77640fef10 2024-05-09T06:46:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.480+0000 7f7760ff9700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f774c06e200 0x7f774c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.480+0000 7f7760ff9700 1 -- 172.21.15.63:0/3672395522 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f775408f340 con 0x7f77640fef10 2024-05-09T06:46:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.480+0000 7f7762ffd700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f774c06e200 0x7f774c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.480+0000 7f7762ffd700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f774c06e200 0x7f774c0706c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7758005950 tx=0x7f77580058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:56.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.485+0000 7f7760ff9700 1 -- 172.21.15.63:0/3672395522 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f775405c100 con 0x7f77640fef10 2024-05-09T06:46:56.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.712+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f776402ce00 con 0x7f774c06e200 2024-05-09T06:46:56.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.713+0000 7f7760ff9700 1 -- 172.21.15.63:0/3672395522 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f776402ce00 con 0x7f774c06e200 2024-05-09T06:46:56.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f774c06e200 msgr2=0x7f774c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f774c06e200 0x7f774c0706c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7758005950 tx=0x7f77580058e0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 msgr2=0x7f7764196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 0x7f7764196670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f77540059d0 tx=0x7f775400be20 comp rx=0 tx=0).stop 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 shutdown_connections 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f774c06e200 0x7f774c0706c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77640fef10 0x7f7764196670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 --2- 172.21.15.63:0/3672395522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7764101ef0 0x7f7764196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 >> 172.21.15.63:0/3672395522 conn(0x7f77640fa870 msgr2=0x7f77641054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 shutdown_connections 2024-05-09T06:46:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.715+0000 7f776a0df700 1 -- 172.21.15.63:0/3672395522 wait complete. 2024-05-09T06:46:56.705 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:46:56.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.841+0000 7fa715869700 1 -- 172.21.15.63:0/1299013917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 msgr2=0x7fa710101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.841+0000 7fa715869700 1 --2- 172.21.15.63:0/1299013917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 0x7fa710101060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa700009910 tx=0x7fa700009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.841+0000 7fa715869700 1 -- 172.21.15.63:0/1299013917 shutdown_connections 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.841+0000 7fa715869700 1 --2- 172.21.15.63:0/1299013917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa710101e40 0x7fa7101022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.841+0000 7fa715869700 1 --2- 172.21.15.63:0/1299013917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 0x7fa710101060 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.841+0000 7fa715869700 1 -- 172.21.15.63:0/1299013917 >> 172.21.15.63:0/1299013917 conn(0x7fa7100fc1e0 msgr2=0x7fa7100fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.841+0000 7fa715869700 1 -- 172.21.15.63:0/1299013917 shutdown_connections 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.841+0000 7fa715869700 1 -- 172.21.15.63:0/1299013917 wait complete. 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa715869700 1 Processor -- start 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa715869700 1 -- start start 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa715869700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 0x7fa710074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa715869700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa710101e40 0x7fa7100732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa715869700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710075180 con 0x7fa710100c40 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa715869700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710073820 con 0x7fa710101e40 2024-05-09T06:46:56.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa70effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 0x7fa710074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:56.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa70effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 0x7fa710074bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33846/0 (socket says 172.21.15.63:33846) 2024-05-09T06:46:56.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa70effd700 1 -- 172.21.15.63:0/1835060022 learned_addr learned my addr 172.21.15.63:0/1835060022 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:56.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa707fff700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa710101e40 0x7fa7100732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:56.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa70effd700 1 -- 172.21.15.63:0/1835060022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa710101e40 msgr2=0x7fa7100732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:56.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa70effd700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa710101e40 0x7fa7100732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:56.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.842+0000 7fa70effd700 1 -- 172.21.15.63:0/1835060022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7000095f0 con 0x7fa710100c40 2024-05-09T06:46:56.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.843+0000 7fa70effd700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 0x7fa710074bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa70000bd70 tx=0x7fa70000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:56.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.843+0000 7fa714867700 1 -- 172.21.15.63:0/1835060022 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa70001d070 con 0x7fa710100c40 2024-05-09T06:46:56.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.843+0000 7fa714867700 1 -- 172.21.15.63:0/1835060022 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa70000f460 con 0x7fa710100c40 2024-05-09T06:46:56.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.843+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa710073aa0 con 0x7fa710100c40 2024-05-09T06:46:56.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.843+0000 7fa714867700 1 -- 172.21.15.63:0/1835060022 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa700021640 con 0x7fa710100c40 2024-05-09T06:46:56.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.843+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa710074010 con 0x7fa710100c40 2024-05-09T06:46:56.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.844+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7101907c0 con 0x7fa710100c40 2024-05-09T06:46:56.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.844+0000 7fa714867700 1 -- 172.21.15.63:0/1835060022 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa70000fad0 con 0x7fa710100c40 2024-05-09T06:46:56.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.845+0000 7fa714867700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa6fc06e210 0x7fa6fc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:56.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.845+0000 7fa714867700 1 -- 172.21.15.63:0/1835060022 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fa70008f6f0 con 0x7fa710100c40 2024-05-09T06:46:56.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.845+0000 7fa707fff700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa6fc06e210 0x7fa6fc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:56.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.846+0000 7fa707fff700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa6fc06e210 0x7fa6fc0706d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800ac50 tx=0x7fa6f800a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:56.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:56.850+0000 7fa714867700 1 -- 172.21.15.63:0/1835060022 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa70005c380 con 0x7fa710100c40 2024-05-09T06:46:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.094+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa710061330 con 0x7fa6fc06e210 2024-05-09T06:46:57.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.096+0000 7fa714867700 1 -- 172.21.15.63:0/1835060022 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7fa710061330 con 0x7fa6fc06e210 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.098+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa6fc06e210 msgr2=0x7fa6fc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.098+0000 7fa715869700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa6fc06e210 0x7fa6fc0706d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800ac50 tx=0x7fa6f800a380 comp rx=0 tx=0).stop 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.098+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 msgr2=0x7fa710074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.098+0000 7fa715869700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 0x7fa710074bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa70000bd70 tx=0x7fa70000bda0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.098+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 shutdown_connections 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.098+0000 7fa715869700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa6fc06e210 0x7fa6fc0706d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.099+0000 7fa715869700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa710100c40 0x7fa710074bf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.099+0000 7fa715869700 1 --2- 172.21.15.63:0/1835060022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa710101e40 0x7fa7100732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.099+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 >> 172.21.15.63:0/1835060022 conn(0x7fa7100fc1e0 msgr2=0x7fa710105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.099+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 shutdown_connections 2024-05-09T06:46:57.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.099+0000 7fa715869700 1 -- 172.21.15.63:0/1835060022 wait complete. 2024-05-09T06:46:57.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.223+0000 7f86f1046700 1 -- 172.21.15.63:0/339432863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 msgr2=0x7f86ec1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.223+0000 7f86f1046700 1 --2- 172.21.15.63:0/339432863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 0x7f86ec1022a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009910 tx=0x7f86d4009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.223+0000 7f86f1046700 1 -- 172.21.15.63:0/339432863 shutdown_connections 2024-05-09T06:46:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.223+0000 7f86f1046700 1 --2- 172.21.15.63:0/339432863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 0x7f86ec1022a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.223+0000 7f86f1046700 1 --2- 172.21.15.63:0/339432863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86ec100c40 0x7f86ec101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.223+0000 7f86f1046700 1 -- 172.21.15.63:0/339432863 >> 172.21.15.63:0/339432863 conn(0x7f86ec0fc1e0 msgr2=0x7f86ec0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.223+0000 7f86f1046700 1 -- 172.21.15.63:0/339432863 shutdown_connections 2024-05-09T06:46:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.223+0000 7f86f1046700 1 -- 172.21.15.63:0/339432863 wait complete. 2024-05-09T06:46:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.224+0000 7f86f1046700 1 Processor -- start 2024-05-09T06:46:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.224+0000 7f86f1046700 1 -- start start 2024-05-09T06:46:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86f1046700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86ec100c40 0x7f86ec196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86f1046700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 0x7f86ec196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86f1046700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec1971b0 con 0x7f86ec101e40 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86f1046700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec1972f0 con 0x7f86ec100c40 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86e259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 0x7f86ec196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86e259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 0x7f86ec196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33866/0 (socket says 172.21.15.63:33866) 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86e259c700 1 -- 172.21.15.63:0/22011010 learned_addr learned my addr 172.21.15.63:0/22011010 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86ead9d700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86ec100c40 0x7f86ec196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86e259c700 1 -- 172.21.15.63:0/22011010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86ec100c40 msgr2=0x7f86ec196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86e259c700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86ec100c40 0x7f86ec196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86e259c700 1 -- 172.21.15.63:0/22011010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d40095f0 con 0x7f86ec101e40 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86ead9d700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86ec100c40 0x7f86ec196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86e259c700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 0x7f86ec196b90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86d40092e0 tx=0x7f86d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86e8d99700 1 -- 172.21.15.63:0/22011010 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86d401d070 con 0x7f86ec101e40 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.225+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ec19bca0 con 0x7f86ec101e40 2024-05-09T06:46:57.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.226+0000 7f86e8d99700 1 -- 172.21.15.63:0/22011010 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86d400f460 con 0x7f86ec101e40 2024-05-09T06:46:57.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.226+0000 7f86e8d99700 1 -- 172.21.15.63:0/22011010 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86d4021640 con 0x7f86ec101e40 2024-05-09T06:46:57.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.226+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec19c210 con 0x7f86ec101e40 2024-05-09T06:46:57.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.227+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ec1907c0 con 0x7f86ec101e40 2024-05-09T06:46:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.227+0000 7f86e8d99700 1 -- 172.21.15.63:0/22011010 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f86d400f5d0 con 0x7f86ec101e40 2024-05-09T06:46:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.228+0000 7f86e8d99700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f86d806e200 0x7f86d80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.228+0000 7f86e8d99700 1 -- 172.21.15.63:0/22011010 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f86d408f6e0 con 0x7f86ec101e40 2024-05-09T06:46:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.228+0000 7f86ead9d700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f86d806e200 0x7f86d80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.228+0000 7f86ead9d700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f86d806e200 0x7f86d80706c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005fd0 tx=0x7f86dc005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:57.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.234+0000 7f86e8d99700 1 -- 172.21.15.63:0/22011010 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f86d405c370 con 0x7f86ec101e40 2024-05-09T06:46:57.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.453+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f86ec02ce00 con 0x7f86d806e200 2024-05-09T06:46:57.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.464+0000 7f86e8d99700 1 -- 172.21.15.63:0/22011010 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f86ec02ce00 con 0x7f86d806e200 2024-05-09T06:46:57.444 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:46:57.444 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (15m) 10s ago 17m 25.5M - 0.25.0 c8568f914cd2 b2b98f6bab65 2024-05-09T06:46:57.444 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (43s) 10s ago 17m 8056k - 19.0.0-2515-g96805dcf d7acac7d5657 1993c9f6fdcd 2024-05-09T06:46:57.444 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (37s) 13s ago 16m 7570k - 19.0.0-2515-g96805dcf d7acac7d5657 bd88664dfa9d 2024-05-09T06:46:57.444 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (5m) 10s ago 17m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:46:57.444 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (5m) 13s ago 16m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (15m) 10s ago 17m 86.7M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (2m) 10s ago 11m 87.6M - 19.0.0-2515-g96805dcf d7acac7d5657 c7cf68a1a3e2 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (105s) 10s ago 11m 84.5M - 19.0.0-2515-g96805dcf d7acac7d5657 c24cf6de2c0f 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (77s) 13s ago 11m 13.8M - 19.0.0-2515-g96805dcf d7acac7d5657 b752027d6904 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (58s) 13s ago 11m 13.7M - 19.0.0-2515-g96805dcf d7acac7d5657 bf31d952d7fe 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (7m) 10s ago 18m 515M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (7m) 13s ago 16m 453M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (6m) 10s ago 19m 55.5M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (6m) 13s ago 16m 40.9M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (20s) 10s ago 17m 3397k - 1.5.0 0da6a335fe13 1f7eb6ee1981 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (16s) 13s ago 16m 5473k - 1.5.0 0da6a335fe13 21d4f1f51ed5 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (5m) 10s ago 15m 151M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (4m) 10s ago 15m 124M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:46:57.445 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 10s ago 14m 99.1M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:46:57.446 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (3m) 13s ago 14m 118M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:46:57.446 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (2m) 13s ago 13m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 14c9b4920f5a 2024-05-09T06:46:57.446 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (2m) 13s ago 13m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 d219b4bb599f 2024-05-09T06:46:57.446 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (7m) 10s ago 16m 69.6M - 2.43.0 a07b618ecd1d 44ea3508523a 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f86d806e200 msgr2=0x7f86d80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f86d806e200 0x7f86d80706c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005fd0 tx=0x7f86dc005dc0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 msgr2=0x7f86ec196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 0x7f86ec196b90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86d40092e0 tx=0x7f86d400be20 comp rx=0 tx=0).stop 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 shutdown_connections 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f86d806e200 0x7f86d80706c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86ec100c40 0x7f86ec196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 --2- 172.21.15.63:0/22011010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86ec101e40 0x7f86ec196b90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 >> 172.21.15.63:0/22011010 conn(0x7f86ec0fc1e0 msgr2=0x7f86ec1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:57.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 shutdown_connections 2024-05-09T06:46:57.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.467+0000 7f86f1046700 1 -- 172.21.15.63:0/22011010 wait complete. 2024-05-09T06:46:57.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.575+0000 7f978b27b700 1 -- 172.21.15.63:0/132812377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9784100da0 msgr2=0x7f9784101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.575+0000 7f978b27b700 1 --2- 172.21.15.63:0/132812377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9784100da0 0x7f9784101200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9774009910 tx=0x7f9774009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:57.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.575+0000 7f978b27b700 1 -- 172.21.15.63:0/132812377 shutdown_connections 2024-05-09T06:46:57.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.575+0000 7f978b27b700 1 --2- 172.21.15.63:0/132812377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9784100da0 0x7f9784101200 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.575+0000 7f978b27b700 1 --2- 172.21.15.63:0/132812377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97840ffba0 0x7f97840fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.575+0000 7f978b27b700 1 -- 172.21.15.63:0/132812377 >> 172.21.15.63:0/132812377 conn(0x7f97840fb120 msgr2=0x7f97840fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:57.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.575+0000 7f978b27b700 1 -- 172.21.15.63:0/132812377 shutdown_connections 2024-05-09T06:46:57.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.575+0000 7f978b27b700 1 -- 172.21.15.63:0/132812377 wait complete. 2024-05-09T06:46:57.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f978b27b700 1 Processor -- start 2024-05-09T06:46:57.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f978b27b700 1 -- start start 2024-05-09T06:46:57.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f978b27b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97840ffba0 0x7f9784103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:57.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f978b27b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9784100da0 0x7f9784102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:57.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f978b27b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9784104070 con 0x7f97840ffba0 2024-05-09T06:46:57.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f978b27b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97841026c0 con 0x7f9784100da0 2024-05-09T06:46:57.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f9789017700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97840ffba0 0x7f9784103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:57.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f9789017700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97840ffba0 0x7f9784103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33894/0 (socket says 172.21.15.63:33894) 2024-05-09T06:46:57.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f9789017700 1 -- 172.21.15.63:0/245082274 learned_addr learned my addr 172.21.15.63:0/245082274 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:57.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.576+0000 7f9788816700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9784100da0 0x7f9784102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:57.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f9789017700 1 -- 172.21.15.63:0/245082274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9784100da0 msgr2=0x7f9784102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f9789017700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9784100da0 0x7f9784102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f9789017700 1 -- 172.21.15.63:0/245082274 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97740095f0 con 0x7f97840ffba0 2024-05-09T06:46:57.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f9788816700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9784100da0 0x7f9784102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:46:57.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f9789017700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97840ffba0 0x7f9784103ae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f978000c910 tx=0x7f978000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:57.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f977a7fc700 1 -- 172.21.15.63:0/245082274 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97800041f0 con 0x7f97840ffba0 2024-05-09T06:46:57.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f977a7fc700 1 -- 172.21.15.63:0/245082274 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9780007dc0 con 0x7f97840ffba0 2024-05-09T06:46:57.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97841029a0 con 0x7f97840ffba0 2024-05-09T06:46:57.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f977a7fc700 1 -- 172.21.15.63:0/245082274 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9780003e70 con 0x7f97840ffba0 2024-05-09T06:46:57.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.577+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9784102ef0 con 0x7f97840ffba0 2024-05-09T06:46:57.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.578+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978404ebb0 con 0x7f97840ffba0 2024-05-09T06:46:57.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.579+0000 7f977a7fc700 1 -- 172.21.15.63:0/245082274 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9780004350 con 0x7f97840ffba0 2024-05-09T06:46:57.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.579+0000 7f977a7fc700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f977006e1a0 0x7f9770070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:57.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.579+0000 7f977a7fc700 1 -- 172.21.15.63:0/245082274 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f978008e0c0 con 0x7f97840ffba0 2024-05-09T06:46:57.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.579+0000 7f9788816700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f977006e1a0 0x7f9770070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:57.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.579+0000 7f9788816700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f977006e1a0 0x7f9770070660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9774005b40 tx=0x7f9774005ab0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:57.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.584+0000 7f977a7fc700 1 -- 172.21.15.63:0/245082274 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f978005ad50 con 0x7f97840ffba0 2024-05-09T06:46:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:57 smithi063 ceph-mon[92765]: pgmap v251: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:46:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.890+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f978419e900 con 0x7f97840ffba0 2024-05-09T06:46:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.891+0000 7f977a7fc700 1 -- 172.21.15.63:0/245082274 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f978005a8e0 con 0x7f97840ffba0 2024-05-09T06:46:57.870 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:46:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:46:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:46:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:46:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:46:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:46:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:46:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:46:57.871 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 6 2024-05-09T06:46:57.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:46:57.871 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:46:57.871 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 4 2024-05-09T06:46:57.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:46:57.871 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:46:57.871 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 14 2024-05-09T06:46:57.871 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:46:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:46:57.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:57 smithi193 ceph-mon[81665]: pgmap v251: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:46:57.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:46:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f977006e1a0 msgr2=0x7f9770070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f977006e1a0 0x7f9770070660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9774005b40 tx=0x7f9774005ab0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97840ffba0 msgr2=0x7f9784103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97840ffba0 0x7f9784103ae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f978000c910 tx=0x7f978000ccd0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 shutdown_connections 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f977006e1a0 0x7f9770070660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97840ffba0 0x7f9784103ae0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 --2- 172.21.15.63:0/245082274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9784100da0 0x7f9784102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 >> 172.21.15.63:0/245082274 conn(0x7f97840fb120 msgr2=0x7f9784068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 shutdown_connections 2024-05-09T06:46:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:57.893+0000 7f978b27b700 1 -- 172.21.15.63:0/245082274 wait complete. 2024-05-09T06:46:57.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.006+0000 7faf286f3700 1 -- 172.21.15.63:0/1453055704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 msgr2=0x7faf20103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.006+0000 7faf286f3700 1 --2- 172.21.15.63:0/1453055704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 0x7faf20103b20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009910 tx=0x7faf1c009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:57.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.006+0000 7faf286f3700 1 -- 172.21.15.63:0/1453055704 shutdown_connections 2024-05-09T06:46:57.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.006+0000 7faf286f3700 1 --2- 172.21.15.63:0/1453055704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf20104060 0x7faf20106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.006+0000 7faf286f3700 1 --2- 172.21.15.63:0/1453055704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 0x7faf20103b20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.006+0000 7faf286f3700 1 -- 172.21.15.63:0/1453055704 >> 172.21.15.63:0/1453055704 conn(0x7faf200fb120 msgr2=0x7faf200fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:57.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.006+0000 7faf286f3700 1 -- 172.21.15.63:0/1453055704 shutdown_connections 2024-05-09T06:46:57.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.006+0000 7faf286f3700 1 -- 172.21.15.63:0/1453055704 wait complete. 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.006+0000 7faf286f3700 1 Processor -- start 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf286f3700 1 -- start start 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf286f3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 0x7faf20101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf286f3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf20104060 0x7faf200ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf286f3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf200ffc20 con 0x7faf20101730 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf286f3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf200ffd60 con 0x7faf20104060 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf2648f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 0x7faf20101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf2648f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 0x7faf20101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33908/0 (socket says 172.21.15.63:33908) 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf2648f700 1 -- 172.21.15.63:0/3795319504 learned_addr learned my addr 172.21.15.63:0/3795319504 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:57.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf25c8e700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf20104060 0x7faf200ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:57.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf2648f700 1 -- 172.21.15.63:0/3795319504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf20104060 msgr2=0x7faf200ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:57.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf2648f700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf20104060 0x7faf200ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:57.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf2648f700 1 -- 172.21.15.63:0/3795319504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c0095f0 con 0x7faf20101730 2024-05-09T06:46:57.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.007+0000 7faf2648f700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 0x7faf20101040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0092e0 tx=0x7faf1c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:57.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.008+0000 7faf177fe700 1 -- 172.21.15.63:0/3795319504 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf1c01d070 con 0x7faf20101730 2024-05-09T06:46:57.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.008+0000 7faf177fe700 1 -- 172.21.15.63:0/3795319504 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf1c00f460 con 0x7faf20101730 2024-05-09T06:46:57.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.008+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf200fffe0 con 0x7faf20101730 2024-05-09T06:46:57.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.008+0000 7faf177fe700 1 -- 172.21.15.63:0/3795319504 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf1c021620 con 0x7faf20101730 2024-05-09T06:46:57.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.008+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf201004d0 con 0x7faf20101730 2024-05-09T06:46:57.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.009+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2004ebb0 con 0x7faf20101730 2024-05-09T06:46:57.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.009+0000 7faf177fe700 1 -- 172.21.15.63:0/3795319504 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faf1c00f5d0 con 0x7faf20101730 2024-05-09T06:46:57.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.010+0000 7faf177fe700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faf0c06e200 0x7faf0c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:57.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.010+0000 7faf177fe700 1 -- 172.21.15.63:0/3795319504 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7faf1c08f580 con 0x7faf20101730 2024-05-09T06:46:57.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.010+0000 7faf25c8e700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faf0c06e200 0x7faf0c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:57.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.010+0000 7faf25c8e700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faf0c06e200 0x7faf0c0706c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faf10005950 tx=0x7faf1000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:57.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.014+0000 7faf177fe700 1 -- 172.21.15.63:0/3795319504 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faf1c05c210 con 0x7faf20101730 2024-05-09T06:46:58.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.301+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faf2019e870 con 0x7faf20101730 2024-05-09T06:46:58.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.301+0000 7faf177fe700 1 -- 172.21.15.63:0/3795319504 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1963 (secure 0 0 0) 0x7faf1c05bda0 con 0x7faf20101730 2024-05-09T06:46:58.282 INFO:teuthology.orchestra.run.smithi063.stdout:e34 2024-05-09T06:46:58.282 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:46:58.282 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:46:58.283 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:46:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:46:58.283 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:46:58.283 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:46:58.283 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 34 2024-05-09T06:46:58.283 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:46:58.283 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:46:58.283 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:46:08.230738+0000 2024-05-09T06:46:58.283 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 96 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:46:58.284 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34406} 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 34406 members: 34406 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{0:34406} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:34410} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/2182150991,v1:172.21.15.63:6829/2182150991] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:46:58.285 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:46:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:46:58.286 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/2007543899,v1:172.21.15.193:6825/2007543899] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:46:58.286 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/790240017,v1:172.21.15.193:6827/790240017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:46:58.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faf0c06e200 msgr2=0x7faf0c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faf0c06e200 0x7faf0c0706c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faf10005950 tx=0x7faf1000b410 comp rx=0 tx=0).stop 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 msgr2=0x7faf20101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 0x7faf20101040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0092e0 tx=0x7faf1c00bd70 comp rx=0 tx=0).stop 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 shutdown_connections 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faf0c06e200 0x7faf0c0706c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf20101730 0x7faf20101040 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 --2- 172.21.15.63:0/3795319504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf20104060 0x7faf200ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 >> 172.21.15.63:0/3795319504 conn(0x7faf200fb120 msgr2=0x7faf200fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 shutdown_connections 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.304+0000 7faf286f3700 1 -- 172.21.15.63:0/3795319504 wait complete. 2024-05-09T06:46:58.287 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 34 2024-05-09T06:46:58.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 -- 172.21.15.63:0/794111913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 msgr2=0x7fa228102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:58.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 --2- 172.21.15.63:0/794111913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 0x7fa228102280 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa218009910 tx=0x7fa218009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:58.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 -- 172.21.15.63:0/794111913 shutdown_connections 2024-05-09T06:46:58.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 --2- 172.21.15.63:0/794111913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 0x7fa228102280 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 --2- 172.21.15.63:0/794111913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa228100c20 0x7fa228101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 -- 172.21.15.63:0/794111913 >> 172.21.15.63:0/794111913 conn(0x7fa2280fc200 msgr2=0x7fa2280fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:58.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 -- 172.21.15.63:0/794111913 shutdown_connections 2024-05-09T06:46:58.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 -- 172.21.15.63:0/794111913 wait complete. 2024-05-09T06:46:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 Processor -- start 2024-05-09T06:46:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.424+0000 7fa22edeb700 1 -- start start 2024-05-09T06:46:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa22edeb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa228100c20 0x7fa228196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa22edeb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 0x7fa228196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa22edeb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2281971c0 con 0x7fa228101e20 2024-05-09T06:46:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa22edeb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228197300 con 0x7fa228100c20 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa21ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 0x7fa228196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa21ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 0x7fa228196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33928/0 (socket says 172.21.15.63:33928) 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa21ffff700 1 -- 172.21.15.63:0/698417461 learned_addr learned my addr 172.21.15.63:0/698417461 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa22cb87700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa228100c20 0x7fa228196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa21ffff700 1 -- 172.21.15.63:0/698417461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa228100c20 msgr2=0x7fa228196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa21ffff700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa228100c20 0x7fa228196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa21ffff700 1 -- 172.21.15.63:0/698417461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2180095f0 con 0x7fa228101e20 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.425+0000 7fa21ffff700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 0x7fa228196ba0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa218005b40 tx=0x7fa21800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.426+0000 7fa21dffb700 1 -- 172.21.15.63:0/698417461 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa21801d070 con 0x7fa228101e20 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.426+0000 7fa21dffb700 1 -- 172.21.15.63:0/698417461 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2180050d0 con 0x7fa228101e20 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.426+0000 7fa21dffb700 1 -- 172.21.15.63:0/698417461 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2180226a0 con 0x7fa228101e20 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.426+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22819bcb0 con 0x7fa228101e20 2024-05-09T06:46:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.426+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22819c220 con 0x7fa228101e20 2024-05-09T06:46:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.427+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2281907c0 con 0x7fa228101e20 2024-05-09T06:46:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.428+0000 7fa21dffb700 1 -- 172.21.15.63:0/698417461 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa218003680 con 0x7fa228101e20 2024-05-09T06:46:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.428+0000 7fa21dffb700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa21006e210 0x7fa2100706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.428+0000 7fa21dffb700 1 -- 172.21.15.63:0/698417461 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fa21808f7f0 con 0x7fa228101e20 2024-05-09T06:46:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.428+0000 7fa22cb87700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa21006e210 0x7fa2100706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:58.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.429+0000 7fa22cb87700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa21006e210 0x7fa2100706d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa228101c80 tx=0x7fa224009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:58.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.433+0000 7fa21dffb700 1 -- 172.21.15.63:0/698417461 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa21805c500 con 0x7fa228101e20 2024-05-09T06:46:58.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:58 smithi063 ceph-mon[92765]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:58.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:58 smithi063 ceph-mon[92765]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:58.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:58 smithi063 ceph-mon[92765]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:58.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:58 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/245082274' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:58.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:58 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3795319504' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:46:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.666+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa22802cea0 con 0x7fa21006e210 2024-05-09T06:46:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.667+0000 7fa21dffb700 1 -- 172.21.15.63:0/698417461 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7fa22802cea0 con 0x7fa21006e210 2024-05-09T06:46:58.646 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:46:58.646 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:46:58.646 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:46:58.646 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:46:58.646 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:46:58.646 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout: "mds", 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph-exporter" 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "18/23 daemons upgraded", 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading prometheus daemons", 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:46:58.647 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:46:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa21006e210 msgr2=0x7fa2100706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa21006e210 0x7fa2100706d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa228101c80 tx=0x7fa224009260 comp rx=0 tx=0).stop 2024-05-09T06:46:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 msgr2=0x7fa228196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 0x7fa228196ba0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa218005b40 tx=0x7fa21800f740 comp rx=0 tx=0).stop 2024-05-09T06:46:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 shutdown_connections 2024-05-09T06:46:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa21006e210 0x7fa2100706d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa228100c20 0x7fa228196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 --2- 172.21.15.63:0/698417461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa228101e20 0x7fa228196ba0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 >> 172.21.15.63:0/698417461 conn(0x7fa2280fc200 msgr2=0x7fa228105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 shutdown_connections 2024-05-09T06:46:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.669+0000 7fa22edeb700 1 -- 172.21.15.63:0/698417461 wait complete. 2024-05-09T06:46:58.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.780+0000 7f62191e7700 1 -- 172.21.15.63:0/1135017177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 msgr2=0x7f6214102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:58.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.780+0000 7f62191e7700 1 --2- 172.21.15.63:0/1135017177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 0x7f6214102280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009910 tx=0x7f61fc009c20 comp rx=0 tx=0).stop 2024-05-09T06:46:58.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.780+0000 7f62191e7700 1 -- 172.21.15.63:0/1135017177 shutdown_connections 2024-05-09T06:46:58.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.780+0000 7f62191e7700 1 --2- 172.21.15.63:0/1135017177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 0x7f6214102280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.780+0000 7f62191e7700 1 --2- 172.21.15.63:0/1135017177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6214100c20 0x7f6214101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.780+0000 7f62191e7700 1 -- 172.21.15.63:0/1135017177 >> 172.21.15.63:0/1135017177 conn(0x7f62140fc200 msgr2=0x7f62140fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:58.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.780+0000 7f62191e7700 1 -- 172.21.15.63:0/1135017177 shutdown_connections 2024-05-09T06:46:58.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.780+0000 7f62191e7700 1 -- 172.21.15.63:0/1135017177 wait complete. 2024-05-09T06:46:58.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f62191e7700 1 Processor -- start 2024-05-09T06:46:58.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f62191e7700 1 -- start start 2024-05-09T06:46:58.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f62191e7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6214100c20 0x7f6214196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f62191e7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 0x7f6214196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f62191e7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6214197170 con 0x7f6214101e20 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f62191e7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62141972b0 con 0x7f6214100c20 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f620a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 0x7f6214196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f620a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 0x7f6214196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33952/0 (socket says 172.21.15.63:33952) 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f620a59c700 1 -- 172.21.15.63:0/1026447200 learned_addr learned my addr 172.21.15.63:0/1026447200 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f6212d9d700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6214100c20 0x7f6214196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f620a59c700 1 -- 172.21.15.63:0/1026447200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6214100c20 msgr2=0x7f6214196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f620a59c700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6214100c20 0x7f6214196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:58.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.781+0000 7f620a59c700 1 -- 172.21.15.63:0/1026447200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0095f0 con 0x7f6214101e20 2024-05-09T06:46:58.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.782+0000 7f620a59c700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 0x7f6214196b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0092e0 tx=0x7f61fc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:58.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.782+0000 7f6210d99700 1 -- 172.21.15.63:0/1026447200 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc01d070 con 0x7f6214101e20 2024-05-09T06:46:58.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.782+0000 7f6210d99700 1 -- 172.21.15.63:0/1026447200 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61fc00f460 con 0x7f6214101e20 2024-05-09T06:46:58.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.782+0000 7f6210d99700 1 -- 172.21.15.63:0/1026447200 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc021620 con 0x7f6214101e20 2024-05-09T06:46:58.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.782+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621419bbf0 con 0x7f6214101e20 2024-05-09T06:46:58.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.782+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621419c110 con 0x7f6214101e20 2024-05-09T06:46:58.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.783+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62141907a0 con 0x7f6214101e20 2024-05-09T06:46:58.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.784+0000 7f6210d99700 1 -- 172.21.15.63:0/1026447200 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f61fc00f5d0 con 0x7f6214101e20 2024-05-09T06:46:58.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.784+0000 7f6210d99700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f620006e200 0x7f62000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:46:58.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.784+0000 7f6210d99700 1 -- 172.21.15.63:0/1026447200 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f61fc08f5b0 con 0x7f6214101e20 2024-05-09T06:46:58.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.784+0000 7f6212d9d700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f620006e200 0x7f62000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:46:58.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.785+0000 7f6212d9d700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f620006e200 0x7f62000706c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6214101c80 tx=0x7f620400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:46:58.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:58.789+0000 7f6210d99700 1 -- 172.21.15.63:0/1026447200 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f61fc05c1e0 con 0x7f6214101e20 2024-05-09T06:46:58.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:58 smithi193 ceph-mon[81665]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:58.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:58 smithi193 ceph-mon[81665]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:58.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:58 smithi193 ceph-mon[81665]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:58.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:58 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/245082274' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:46:58.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:58 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3795319504' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:46:59.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.107+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f621404ebb0 con 0x7f6214101e20 2024-05-09T06:46:59.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.107+0000 7f6210d99700 1 -- 172.21.15.63:0/1026447200 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f61fc05bd70 con 0x7f6214101e20 2024-05-09T06:46:59.086 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:46:59.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f620006e200 msgr2=0x7f62000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:59.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f620006e200 0x7f62000706c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6214101c80 tx=0x7f620400b410 comp rx=0 tx=0).stop 2024-05-09T06:46:59.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 msgr2=0x7f6214196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:46:59.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 0x7f6214196b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0092e0 tx=0x7f61fc00bd70 comp rx=0 tx=0).stop 2024-05-09T06:46:59.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 shutdown_connections 2024-05-09T06:46:59.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f620006e200 0x7f62000706c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:59.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6214100c20 0x7f6214196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:59.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 --2- 172.21.15.63:0/1026447200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6214101e20 0x7f6214196b50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:46:59.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 >> 172.21.15.63:0/1026447200 conn(0x7f62140fc200 msgr2=0x7f6214105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:46:59.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 shutdown_connections 2024-05-09T06:46:59.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:46:59.109+0000 7f62191e7700 1 -- 172.21.15.63:0/1026447200 wait complete. 2024-05-09T06:46:59.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:59 smithi063 ceph-mon[92765]: pgmap v252: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:46:59.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:59 smithi063 ceph-mon[92765]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:59.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:59.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:59.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:59 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:59.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:46:59 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1026447200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:46:59.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:59 smithi193 ceph-mon[81665]: pgmap v252: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:46:59.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:59 smithi193 ceph-mon[81665]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:46:59.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:59.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:46:59.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:59 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:46:59.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:46:59 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1026447200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:47:01.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:01 smithi063 ceph-mon[92765]: pgmap v253: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:01.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:01 smithi193 ceph-mon[81665]: pgmap v253: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:03.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:03 smithi063 ceph-mon[92765]: pgmap v254: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:03.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:03.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:03 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:03.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:03 smithi193 ceph-mon[81665]: pgmap v254: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:03.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:03.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:03 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:05.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:05 smithi063 ceph-mon[92765]: pgmap v255: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:05.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:05.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:05 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:05.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:05 smithi193 ceph-mon[81665]: pgmap v255: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:05.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:05.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:05 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:07.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:07 smithi063 ceph-mon[92765]: pgmap v256: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:07.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:07 smithi193 ceph-mon[81665]: pgmap v256: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:09.256 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: pgmap v257: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:09.256 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.257 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.258 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:09 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: pgmap v257: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:09.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:09 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:11.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:11 smithi063 ceph-mon[92765]: Upgrade: Updating alertmanager.smithi063 2024-05-09T06:47:11.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:11 smithi063 ceph-mon[92765]: Deploying daemon alertmanager.smithi063 on smithi063 2024-05-09T06:47:11.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:11 smithi063 ceph-mon[92765]: pgmap v258: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:11.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:11 smithi193 ceph-mon[81665]: Upgrade: Updating alertmanager.smithi063 2024-05-09T06:47:11.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:11 smithi193 ceph-mon[81665]: Deploying daemon alertmanager.smithi063 on smithi063 2024-05-09T06:47:11.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:11 smithi193 ceph-mon[81665]: pgmap v258: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:12.328 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:47:12.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:47:13.430 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:13 smithi063 ceph-mon[92765]: pgmap v259: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:13.522 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:13 smithi193 ceph-mon[81665]: pgmap v259: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:15.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:15 smithi063 ceph-mon[92765]: pgmap v260: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:15.614 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:15 smithi193 ceph-mon[81665]: pgmap v260: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:17.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:16 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:17.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:16 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:17.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:16 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:17.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:16 smithi063 ceph-mon[92765]: pgmap v261: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:17.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:16 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:17.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:16 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:17.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:16 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:17.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:16 smithi193 ceph-mon[81665]: pgmap v261: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:19.355 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:19 smithi063 ceph-mon[92765]: pgmap v262: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:19.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:19 smithi193 ceph-mon[81665]: pgmap v262: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:22.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:21 smithi063 ceph-mon[92765]: pgmap v263: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:22.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:22.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:21 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:22.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:21 smithi193 ceph-mon[81665]: pgmap v263: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:22.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:22.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:21 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:23.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:23.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:22 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:23.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:22 smithi063 ceph-mon[92765]: pgmap v264: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:23.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:22 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:23.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:22 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:23.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:22 smithi193 ceph-mon[81665]: pgmap v264: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:25.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:25 smithi063 ceph-mon[92765]: pgmap v265: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:25.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:25 smithi193 ceph-mon[81665]: pgmap v265: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:26.333 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:26.333 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:26.333 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:26.333 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:26.333 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:26.333 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-09T06:47:26.333 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.334 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.335 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.335 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:26 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-09T06:47:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-09T06:47:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:26.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:26.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:26 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:27.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:27 smithi063 ceph-mon[92765]: pgmap v266: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:27.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:47:27.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:27 smithi193 ceph-mon[81665]: pgmap v266: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:27.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:47:28.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:28 smithi063 ceph-mon[92765]: Upgrade: Updating grafana.smithi063 2024-05-09T06:47:28.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:28 smithi063 ceph-mon[92765]: Deploying daemon grafana.smithi063 on smithi063 2024-05-09T06:47:28.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:28 smithi193 ceph-mon[81665]: Upgrade: Updating grafana.smithi063 2024-05-09T06:47:28.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:28 smithi193 ceph-mon[81665]: Deploying daemon grafana.smithi063 on smithi063 2024-05-09T06:47:29.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.234+0000 7f632fb0f700 1 -- 172.21.15.63:0/3216601709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 msgr2=0x7f6328101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.234+0000 7f632fb0f700 1 --2- 172.21.15.63:0/3216601709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 0x7f6328101190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f631c009910 tx=0x7f631c009c20 comp rx=0 tx=0).stop 2024-05-09T06:47:29.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.234+0000 7f632fb0f700 1 -- 172.21.15.63:0/3216601709 shutdown_connections 2024-05-09T06:47:29.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.234+0000 7f632fb0f700 1 --2- 172.21.15.63:0/3216601709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 0x7f6328101190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.234+0000 7f632fb0f700 1 --2- 172.21.15.63:0/3216601709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63280ffb30 0x7f63280fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.234+0000 7f632fb0f700 1 -- 172.21.15.63:0/3216601709 >> 172.21.15.63:0/3216601709 conn(0x7f63280fb0d0 msgr2=0x7f63280fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.234+0000 7f632fb0f700 1 -- 172.21.15.63:0/3216601709 shutdown_connections 2024-05-09T06:47:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.235+0000 7f632fb0f700 1 -- 172.21.15.63:0/3216601709 wait complete. 2024-05-09T06:47:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.235+0000 7f632fb0f700 1 Processor -- start 2024-05-09T06:47:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.235+0000 7f632fb0f700 1 -- start start 2024-05-09T06:47:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.235+0000 7f632fb0f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63280ffb30 0x7f6328196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.235+0000 7f632fb0f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 0x7f6328196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.235+0000 7f632fb0f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63281971c0 con 0x7f6328100d30 2024-05-09T06:47:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.235+0000 7f632fb0f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6328197300 con 0x7f63280ffb30 2024-05-09T06:47:29.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.235+0000 7f632d0aa700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 0x7f6328196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:29.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632d0aa700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 0x7f6328196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33350/0 (socket says 172.21.15.63:33350) 2024-05-09T06:47:29.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632d0aa700 1 -- 172.21.15.63:0/2778108928 learned_addr learned my addr 172.21.15.63:0/2778108928 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632d8ab700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63280ffb30 0x7f6328196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632d0aa700 1 -- 172.21.15.63:0/2778108928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63280ffb30 msgr2=0x7f6328196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632d0aa700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63280ffb30 0x7f6328196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632d0aa700 1 -- 172.21.15.63:0/2778108928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0095f0 con 0x7f6328100d30 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632d8ab700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63280ffb30 0x7f6328196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632d0aa700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 0x7f6328196ba0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f631c006010 tx=0x7f631c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f6326ffd700 1 -- 172.21.15.63:0/2778108928 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631c01d070 con 0x7f6328100d30 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632819bcb0 con 0x7f6328100d30 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f6326ffd700 1 -- 172.21.15.63:0/2778108928 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f631c005dc0 con 0x7f6328100d30 2024-05-09T06:47:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f6326ffd700 1 -- 172.21.15.63:0/2778108928 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631c021640 con 0x7f6328100d30 2024-05-09T06:47:29.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.236+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632819c220 con 0x7f6328100d30 2024-05-09T06:47:29.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.238+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632804ebb0 con 0x7f6328100d30 2024-05-09T06:47:29.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.238+0000 7f6326ffd700 1 -- 172.21.15.63:0/2778108928 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f631c00f880 con 0x7f6328100d30 2024-05-09T06:47:29.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.239+0000 7f6326ffd700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f631006e210 0x7f63100706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:29.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.239+0000 7f6326ffd700 1 -- 172.21.15.63:0/2778108928 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f631c08f5d0 con 0x7f6328100d30 2024-05-09T06:47:29.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.239+0000 7f632d8ab700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f631006e210 0x7f63100706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:29.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.239+0000 7f632d8ab700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f631006e210 0x7f63100706d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6328100b90 tx=0x7f6318009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:29.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.244+0000 7f6326ffd700 1 -- 172.21.15.63:0/2778108928 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f631c05c260 con 0x7f6328100d30 2024-05-09T06:47:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.485+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6328069800 con 0x7f631006e210 2024-05-09T06:47:29.465 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:29 smithi063 ceph-mon[92765]: pgmap v267: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:29.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.486+0000 7f6326ffd700 1 -- 172.21.15.63:0/2778108928 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7f6328069800 con 0x7f631006e210 2024-05-09T06:47:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f631006e210 msgr2=0x7f63100706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f631006e210 0x7f63100706d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6328100b90 tx=0x7f6318009040 comp rx=0 tx=0).stop 2024-05-09T06:47:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 msgr2=0x7f6328196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 0x7f6328196ba0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f631c006010 tx=0x7f631c00be20 comp rx=0 tx=0).stop 2024-05-09T06:47:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 shutdown_connections 2024-05-09T06:47:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f631006e210 0x7f63100706d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63280ffb30 0x7f6328196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 --2- 172.21.15.63:0/2778108928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6328100d30 0x7f6328196ba0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 >> 172.21.15.63:0/2778108928 conn(0x7f63280fb0d0 msgr2=0x7f63280690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.488+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 shutdown_connections 2024-05-09T06:47:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.489+0000 7f632fb0f700 1 -- 172.21.15.63:0/2778108928 wait complete. 2024-05-09T06:47:29.479 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:47:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.601+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/3823824227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f0100d30 msgr2=0x7fc5f0101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.601+0000 7fc5f7fc9700 1 --2- 172.21.15.63:0/3823824227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f0100d30 0x7fc5f0101190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009910 tx=0x7fc5e0009c20 comp rx=0 tx=0).stop 2024-05-09T06:47:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.601+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/3823824227 shutdown_connections 2024-05-09T06:47:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.601+0000 7fc5f7fc9700 1 --2- 172.21.15.63:0/3823824227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f0100d30 0x7fc5f0101190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.601+0000 7fc5f7fc9700 1 --2- 172.21.15.63:0/3823824227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f00ffb30 0x7fc5f00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.601+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/3823824227 >> 172.21.15.63:0/3823824227 conn(0x7fc5f00fb0d0 msgr2=0x7fc5f00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.602+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/3823824227 shutdown_connections 2024-05-09T06:47:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.602+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/3823824227 wait complete. 2024-05-09T06:47:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.602+0000 7fc5f7fc9700 1 Processor -- start 2024-05-09T06:47:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.602+0000 7fc5f7fc9700 1 -- start start 2024-05-09T06:47:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f7fc9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f00ffb30 0x7fc5f0196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f7fc9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f0100d30 0x7fc5f0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f7fc9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0197160 con 0x7fc5f00ffb30 2024-05-09T06:47:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f7fc9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01972a0 con 0x7fc5f0100d30 2024-05-09T06:47:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f5d65700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f00ffb30 0x7fc5f0196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f5d65700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f00ffb30 0x7fc5f0196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33374/0 (socket says 172.21.15.63:33374) 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f5d65700 1 -- 172.21.15.63:0/1177461486 learned_addr learned my addr 172.21.15.63:0/1177461486 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f5564700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f0100d30 0x7fc5f0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f5d65700 1 -- 172.21.15.63:0/1177461486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f0100d30 msgr2=0x7fc5f0196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f5d65700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f0100d30 0x7fc5f0196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f5d65700 1 -- 172.21.15.63:0/1177461486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e00095f0 con 0x7fc5f00ffb30 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5f5d65700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f00ffb30 0x7fc5f0196600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00d900 tx=0x7fc5ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.603+0000 7fc5e6ffd700 1 -- 172.21.15.63:0/1177461486 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5ec0041f0 con 0x7fc5f00ffb30 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.604+0000 7fc5e6ffd700 1 -- 172.21.15.63:0/1177461486 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5ec004350 con 0x7fc5f00ffb30 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.604+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f019bd00 con 0x7fc5f00ffb30 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.604+0000 7fc5e6ffd700 1 -- 172.21.15.63:0/1177461486 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5ec003cf0 con 0x7fc5f00ffb30 2024-05-09T06:47:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.604+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f019c270 con 0x7fc5f00ffb30 2024-05-09T06:47:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.605+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f004ec20 con 0x7fc5f00ffb30 2024-05-09T06:47:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.605+0000 7fc5e6ffd700 1 -- 172.21.15.63:0/1177461486 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc5ec009730 con 0x7fc5f00ffb30 2024-05-09T06:47:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.606+0000 7fc5e6ffd700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5dc06e200 0x7fc5dc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.606+0000 7fc5e6ffd700 1 -- 172.21.15.63:0/1177461486 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fc5ec08e190 con 0x7fc5f00ffb30 2024-05-09T06:47:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.606+0000 7fc5f5564700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5dc06e200 0x7fc5dc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.606+0000 7fc5f5564700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5dc06e200 0x7fc5dc0706c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0005090 tx=0x7fc5e0005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.610+0000 7fc5e6ffd700 1 -- 172.21.15.63:0/1177461486 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc5f004ec20 con 0x7fc5f00ffb30 2024-05-09T06:47:29.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:29 smithi193 ceph-mon[81665]: pgmap v267: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:29.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.864+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5f004ebb0 con 0x7fc5dc06e200 2024-05-09T06:47:29.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.865+0000 7fc5e6ffd700 1 -- 172.21.15.63:0/1177461486 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7fc5f004ebb0 con 0x7fc5dc06e200 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5dc06e200 msgr2=0x7fc5dc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5dc06e200 0x7fc5dc0706c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0005090 tx=0x7fc5e0005960 comp rx=0 tx=0).stop 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f00ffb30 msgr2=0x7fc5f0196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f00ffb30 0x7fc5f0196600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00d900 tx=0x7fc5ec00dc10 comp rx=0 tx=0).stop 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 shutdown_connections 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5dc06e200 0x7fc5dc0706c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5f00ffb30 0x7fc5f0196600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 --2- 172.21.15.63:0/1177461486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f0100d30 0x7fc5f0196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 >> 172.21.15.63:0/1177461486 conn(0x7fc5f00fb0d0 msgr2=0x7fc5f00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 shutdown_connections 2024-05-09T06:47:29.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.868+0000 7fc5f7fc9700 1 -- 172.21.15.63:0/1177461486 wait complete. 2024-05-09T06:47:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.984+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/2221741700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 msgr2=0x7f7c980fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.984+0000 7f7c9e2b0700 1 --2- 172.21.15.63:0/2221741700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 0x7f7c980fff40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009910 tx=0x7f7c88009c20 comp rx=0 tx=0).stop 2024-05-09T06:47:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.984+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/2221741700 shutdown_connections 2024-05-09T06:47:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.984+0000 7f7c9e2b0700 1 --2- 172.21.15.63:0/2221741700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98100d20 0x7f7c98101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.984+0000 7f7c9e2b0700 1 --2- 172.21.15.63:0/2221741700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 0x7f7c980fff40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.984+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/2221741700 >> 172.21.15.63:0/2221741700 conn(0x7f7c980fb0c0 msgr2=0x7f7c980fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:29.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.984+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/2221741700 shutdown_connections 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.984+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/2221741700 wait complete. 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c9e2b0700 1 Processor -- start 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c9e2b0700 1 -- start start 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c9e2b0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 0x7f7c98196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c9e2b0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98100d20 0x7f7c98196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c9e2b0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98197190 con 0x7f7c980ffb20 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c9e2b0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c981972d0 con 0x7f7c98100d20 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c97fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 0x7f7c98196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c97fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 0x7f7c98196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33398/0 (socket says 172.21.15.63:33398) 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c97fff700 1 -- 172.21.15.63:0/3712056527 learned_addr learned my addr 172.21.15.63:0/3712056527 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.985+0000 7f7c8ffff700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98100d20 0x7f7c98196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.986+0000 7f7c97fff700 1 -- 172.21.15.63:0/3712056527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98100d20 msgr2=0x7f7c98196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.986+0000 7f7c97fff700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98100d20 0x7f7c98196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:29.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.986+0000 7f7c97fff700 1 -- 172.21.15.63:0/3712056527 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c880095f0 con 0x7f7c980ffb20 2024-05-09T06:47:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.986+0000 7f7c97fff700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 0x7f7c98196630 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7c880059d0 tx=0x7f7c8800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.986+0000 7f7c95ffb700 1 -- 172.21.15.63:0/3712056527 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c8801d070 con 0x7f7c980ffb20 2024-05-09T06:47:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.986+0000 7f7c95ffb700 1 -- 172.21.15.63:0/3712056527 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c88005dc0 con 0x7f7c980ffb20 2024-05-09T06:47:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.986+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9819bcd0 con 0x7f7c980ffb20 2024-05-09T06:47:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.986+0000 7f7c95ffb700 1 -- 172.21.15.63:0/3712056527 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c88021410 con 0x7f7c980ffb20 2024-05-09T06:47:29.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.986+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9819c1c0 con 0x7f7c980ffb20 2024-05-09T06:47:29.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.987+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9804ebb0 con 0x7f7c980ffb20 2024-05-09T06:47:29.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.988+0000 7f7c95ffb700 1 -- 172.21.15.63:0/3712056527 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7c8800f840 con 0x7f7c980ffb20 2024-05-09T06:47:29.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.988+0000 7f7c95ffb700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c8406e200 0x7f7c840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:29.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.988+0000 7f7c95ffb700 1 -- 172.21.15.63:0/3712056527 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f7c8808f340 con 0x7f7c980ffb20 2024-05-09T06:47:29.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.988+0000 7f7c8ffff700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c8406e200 0x7f7c840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:29.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.989+0000 7f7c8ffff700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c8406e200 0x7f7c840706c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005950 tx=0x7f7c800058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:29.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:29.993+0000 7f7c95ffb700 1 -- 172.21.15.63:0/3712056527 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7c8805c100 con 0x7f7c980ffb20 2024-05-09T06:47:30.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.276+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c98069660 con 0x7f7c8406e200 2024-05-09T06:47:30.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.289+0000 7f7c95ffb700 1 -- 172.21.15.63:0/3712056527 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7c98069660 con 0x7f7c8406e200 2024-05-09T06:47:30.269 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:47:30.269 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (14s) 9s ago 17m 15.2M - 0.25.0 c8568f914cd2 b9acb0012a9e 2024-05-09T06:47:30.269 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (76s) 9s ago 18m 8108k - 19.0.0-2515-g96805dcf d7acac7d5657 1993c9f6fdcd 2024-05-09T06:47:30.269 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (70s) 46s ago 17m 7570k - 19.0.0-2515-g96805dcf d7acac7d5657 bd88664dfa9d 2024-05-09T06:47:30.269 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (6m) 9s ago 18m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (6m) 46s ago 17m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (16m) 9s ago 17m 87.4M - 9.4.7 954c08fa6188 aa9e6c32129b 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (2m) 9s ago 12m 87.6M - 19.0.0-2515-g96805dcf d7acac7d5657 c7cf68a1a3e2 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (2m) 9s ago 12m 84.5M - 19.0.0-2515-g96805dcf d7acac7d5657 c24cf6de2c0f 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (110s) 46s ago 12m 13.8M - 19.0.0-2515-g96805dcf d7acac7d5657 b752027d6904 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (91s) 46s ago 12m 13.7M - 19.0.0-2515-g96805dcf d7acac7d5657 bf31d952d7fe 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (8m) 9s ago 19m 518M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (7m) 46s ago 16m 453M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (7m) 9s ago 19m 56.7M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (7m) 46s ago 16m 40.9M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (53s) 9s ago 18m 7704k - 1.5.0 0da6a335fe13 1f7eb6ee1981 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (49s) 46s ago 16m 5473k - 1.5.0 0da6a335fe13 21d4f1f51ed5 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (5m) 9s ago 16m 151M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:47:30.270 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 9s ago 15m 124M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:47:30.271 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 9s ago 15m 99.2M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:47:30.271 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (4m) 46s ago 14m 118M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:47:30.271 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (3m) 46s ago 14m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 14c9b4920f5a 2024-05-09T06:47:30.271 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (2m) 46s ago 13m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 d219b4bb599f 2024-05-09T06:47:30.271 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (31s) 9s ago 17m 47.1M - 2.43.0 a07b618ecd1d 6ee3f1928d73 2024-05-09T06:47:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.291+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c8406e200 msgr2=0x7f7c840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.291+0000 7f7c9e2b0700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c8406e200 0x7f7c840706c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005950 tx=0x7f7c800058e0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.291+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 msgr2=0x7f7c98196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:30.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.291+0000 7f7c9e2b0700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 0x7f7c98196630 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7c880059d0 tx=0x7f7c8800be20 comp rx=0 tx=0).stop 2024-05-09T06:47:30.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.292+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 shutdown_connections 2024-05-09T06:47:30.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.292+0000 7f7c9e2b0700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c8406e200 0x7f7c840706c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.292+0000 7f7c9e2b0700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c980ffb20 0x7f7c98196630 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.292+0000 7f7c9e2b0700 1 --2- 172.21.15.63:0/3712056527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98100d20 0x7f7c98196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.292+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 >> 172.21.15.63:0/3712056527 conn(0x7f7c980fb0c0 msgr2=0x7f7c98068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:30.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.292+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 shutdown_connections 2024-05-09T06:47:30.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.292+0000 7f7c9e2b0700 1 -- 172.21.15.63:0/3712056527 wait complete. 2024-05-09T06:47:30.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.408+0000 7fbe3a925700 1 -- 172.21.15.63:0/119923414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 msgr2=0x7fbe34102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:30.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.408+0000 7fbe3a925700 1 --2- 172.21.15.63:0/119923414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 0x7fbe34102320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009910 tx=0x7fbe28009c20 comp rx=0 tx=0).stop 2024-05-09T06:47:30.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.408+0000 7fbe3a925700 1 -- 172.21.15.63:0/119923414 shutdown_connections 2024-05-09T06:47:30.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.408+0000 7fbe3a925700 1 --2- 172.21.15.63:0/119923414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 0x7fbe34102320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.408+0000 7fbe3a925700 1 --2- 172.21.15.63:0/119923414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe340feee0 0x7fbe340ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.408+0000 7fbe3a925700 1 -- 172.21.15.63:0/119923414 >> 172.21.15.63:0/119923414 conn(0x7fbe340fa840 msgr2=0x7fbe340fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:30.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.408+0000 7fbe3a925700 1 -- 172.21.15.63:0/119923414 shutdown_connections 2024-05-09T06:47:30.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.408+0000 7fbe3a925700 1 -- 172.21.15.63:0/119923414 wait complete. 2024-05-09T06:47:30.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.409+0000 7fbe3a925700 1 Processor -- start 2024-05-09T06:47:30.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.409+0000 7fbe3a925700 1 -- start start 2024-05-09T06:47:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe3a925700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe340feee0 0x7fbe34071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe3a925700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 0x7fbe340724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe3a925700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe34072af0 con 0x7fbe34101ec0 2024-05-09T06:47:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe3a925700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe341a4360 con 0x7fbe340feee0 2024-05-09T06:47:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 0x7fbe340724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe33fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe340feee0 0x7fbe34071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 0x7fbe340724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33428/0 (socket says 172.21.15.63:33428) 2024-05-09T06:47:30.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe337fe700 1 -- 172.21.15.63:0/731360842 learned_addr learned my addr 172.21.15.63:0/731360842 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:47:30.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe337fe700 1 -- 172.21.15.63:0/731360842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe340feee0 msgr2=0x7fbe34071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:30.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe337fe700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe340feee0 0x7fbe34071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.410+0000 7fbe337fe700 1 -- 172.21.15.63:0/731360842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe280095f0 con 0x7fbe34101ec0 2024-05-09T06:47:30.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.411+0000 7fbe337fe700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 0x7fbe340724d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe280092e0 tx=0x7fbe2800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:30.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.411+0000 7fbe317fa700 1 -- 172.21.15.63:0/731360842 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe2801d070 con 0x7fbe34101ec0 2024-05-09T06:47:30.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.411+0000 7fbe317fa700 1 -- 172.21.15.63:0/731360842 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe2800f460 con 0x7fbe34101ec0 2024-05-09T06:47:30.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.411+0000 7fbe317fa700 1 -- 172.21.15.63:0/731360842 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe28021620 con 0x7fbe34101ec0 2024-05-09T06:47:30.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.411+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe341a4500 con 0x7fbe34101ec0 2024-05-09T06:47:30.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.411+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe341a49f0 con 0x7fbe34101ec0 2024-05-09T06:47:30.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.412+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3406c0b0 con 0x7fbe34101ec0 2024-05-09T06:47:30.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.413+0000 7fbe317fa700 1 -- 172.21.15.63:0/731360842 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbe2800f5d0 con 0x7fbe34101ec0 2024-05-09T06:47:30.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.414+0000 7fbe317fa700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe1c06e200 0x7fbe1c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:30.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.414+0000 7fbe317fa700 1 -- 172.21.15.63:0/731360842 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fbe2808f570 con 0x7fbe34101ec0 2024-05-09T06:47:30.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.414+0000 7fbe33fff700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe1c06e200 0x7fbe1c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:30.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.415+0000 7fbe33fff700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe1c06e200 0x7fbe1c0706c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe341a02d0 tx=0x7fbe2400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:30.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.420+0000 7fbe317fa700 1 -- 172.21.15.63:0/731360842 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbe280586d0 con 0x7fbe34101ec0 2024-05-09T06:47:30.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:30 smithi063 ceph-mon[92765]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:47:30.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:30 smithi193 ceph-mon[81665]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:47:30.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.777+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbe3404ebb0 con 0x7fbe34101ec0 2024-05-09T06:47:30.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.778+0000 7fbe317fa700 1 -- 172.21.15.63:0/731360842 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbe2805bd90 con 0x7fbe34101ec0 2024-05-09T06:47:30.758 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:47:30.758 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:47:30.758 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 6 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 4 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 14 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:47:30.759 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:47:30.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe1c06e200 msgr2=0x7fbe1c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:30.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe1c06e200 0x7fbe1c0706c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe341a02d0 tx=0x7fbe2400b410 comp rx=0 tx=0).stop 2024-05-09T06:47:30.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 msgr2=0x7fbe340724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:30.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 0x7fbe340724d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe280092e0 tx=0x7fbe2800bd70 comp rx=0 tx=0).stop 2024-05-09T06:47:30.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 shutdown_connections 2024-05-09T06:47:30.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fbe1c06e200 0x7fbe1c0706c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe340feee0 0x7fbe34071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 --2- 172.21.15.63:0/731360842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe34101ec0 0x7fbe340724d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 >> 172.21.15.63:0/731360842 conn(0x7fbe340fa840 msgr2=0x7fbe341054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:30.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 shutdown_connections 2024-05-09T06:47:30.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.780+0000 7fbe3a925700 1 -- 172.21.15.63:0/731360842 wait complete. 2024-05-09T06:47:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.908+0000 7fea7f8d0700 1 -- 172.21.15.63:0/742589056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea780ffb30 msgr2=0x7fea780fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.908+0000 7fea7f8d0700 1 --2- 172.21.15.63:0/742589056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea780ffb30 0x7fea780fff50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fea74009910 tx=0x7fea74009c20 comp rx=0 tx=0).stop 2024-05-09T06:47:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.908+0000 7fea7f8d0700 1 -- 172.21.15.63:0/742589056 shutdown_connections 2024-05-09T06:47:30.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.908+0000 7fea7f8d0700 1 --2- 172.21.15.63:0/742589056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea78100d30 0x7fea78101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.908+0000 7fea7f8d0700 1 --2- 172.21.15.63:0/742589056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea780ffb30 0x7fea780fff50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.908+0000 7fea7f8d0700 1 -- 172.21.15.63:0/742589056 >> 172.21.15.63:0/742589056 conn(0x7fea780fb0d0 msgr2=0x7fea780fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.908+0000 7fea7f8d0700 1 -- 172.21.15.63:0/742589056 shutdown_connections 2024-05-09T06:47:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.908+0000 7fea7f8d0700 1 -- 172.21.15.63:0/742589056 wait complete. 2024-05-09T06:47:30.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7f8d0700 1 Processor -- start 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7f8d0700 1 -- start start 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7f8d0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea780ffb30 0x7fea78196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7f8d0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea78100d30 0x7fea78196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7f8d0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea78197360 con 0x7fea78100d30 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7f8d0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea781974a0 con 0x7fea780ffb30 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7ce6b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea78100d30 0x7fea78196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7ce6b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea78100d30 0x7fea78196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33446/0 (socket says 172.21.15.63:33446) 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7ce6b700 1 -- 172.21.15.63:0/775955318 learned_addr learned my addr 172.21.15.63:0/775955318 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7d66c700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea780ffb30 0x7fea78196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7ce6b700 1 -- 172.21.15.63:0/775955318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea780ffb30 msgr2=0x7fea78196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7ce6b700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea780ffb30 0x7fea78196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7ce6b700 1 -- 172.21.15.63:0/775955318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea740095f0 con 0x7fea78100d30 2024-05-09T06:47:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.909+0000 7fea7d66c700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea780ffb30 0x7fea78196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:47:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.910+0000 7fea7ce6b700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea78100d30 0x7fea78196d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fea6800b4b0 tx=0x7fea6800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.910+0000 7fea6e7fc700 1 -- 172.21.15.63:0/775955318 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea68011800 con 0x7fea78100d30 2024-05-09T06:47:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.910+0000 7fea6e7fc700 1 -- 172.21.15.63:0/775955318 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea68011e20 con 0x7fea78100d30 2024-05-09T06:47:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.910+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7819beb0 con 0x7fea78100d30 2024-05-09T06:47:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.910+0000 7fea6e7fc700 1 -- 172.21.15.63:0/775955318 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea6800f2e0 con 0x7fea78100d30 2024-05-09T06:47:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.910+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7819c450 con 0x7fea78100d30 2024-05-09T06:47:30.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.911+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7804ebb0 con 0x7fea78100d30 2024-05-09T06:47:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.911+0000 7fea6e7fc700 1 -- 172.21.15.63:0/775955318 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fea68010380 con 0x7fea78100d30 2024-05-09T06:47:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.912+0000 7fea6e7fc700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fea6406e200 0x7fea640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.912+0000 7fea7d66c700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fea6406e200 0x7fea640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.912+0000 7fea6e7fc700 1 -- 172.21.15.63:0/775955318 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fea6808df10 con 0x7fea78100d30 2024-05-09T06:47:30.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.913+0000 7fea7d66c700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fea6406e200 0x7fea640706c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fea740051c0 tx=0x7fea74005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:30.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:30.917+0000 7fea6e7fc700 1 -- 172.21.15.63:0/775955318 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fea6805aba0 con 0x7fea78100d30 2024-05-09T06:47:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.202+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fea78108710 con 0x7fea78100d30 2024-05-09T06:47:31.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.202+0000 7fea6e7fc700 1 -- 172.21.15.63:0/775955318 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1963 (secure 0 0 0) 0x7fea6805a730 con 0x7fea78100d30 2024-05-09T06:47:31.183 INFO:teuthology.orchestra.run.smithi063.stdout:e34 2024-05-09T06:47:31.183 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:47:31.183 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:47:31.183 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:47:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:47:31.183 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 34 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:46:08.230738+0000 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:47:31.184 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 96 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34406} 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:47:31.185 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 34406 members: 34406 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{0:34406} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:34410} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/2182150991,v1:172.21.15.63:6829/2182150991] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/2007543899,v1:172.21.15.193:6825/2007543899] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:47:31.186 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/790240017,v1:172.21.15.193:6827/790240017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:47:31.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fea6406e200 msgr2=0x7fea640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:31.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fea6406e200 0x7fea640706c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fea740051c0 tx=0x7fea74005960 comp rx=0 tx=0).stop 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea78100d30 msgr2=0x7fea78196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea78100d30 0x7fea78196d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fea6800b4b0 tx=0x7fea6800b870 comp rx=0 tx=0).stop 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 shutdown_connections 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fea6406e200 0x7fea640706c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea780ffb30 0x7fea78196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 --2- 172.21.15.63:0/775955318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea78100d30 0x7fea78196d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 >> 172.21.15.63:0/775955318 conn(0x7fea780fb0d0 msgr2=0x7fea780fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 shutdown_connections 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.205+0000 7fea7f8d0700 1 -- 172.21.15.63:0/775955318 wait complete. 2024-05-09T06:47:31.188 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 34 2024-05-09T06:47:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.329+0000 7f2f0602d700 1 -- 172.21.15.63:0/2049805885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 msgr2=0x7f2f00101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.329+0000 7f2f0602d700 1 --2- 172.21.15.63:0/2049805885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 0x7f2f00101200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009910 tx=0x7f2ef4009c20 comp rx=0 tx=0).stop 2024-05-09T06:47:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.329+0000 7f2f0602d700 1 -- 172.21.15.63:0/2049805885 shutdown_connections 2024-05-09T06:47:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.329+0000 7f2f0602d700 1 --2- 172.21.15.63:0/2049805885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 0x7f2f00101200 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.329+0000 7f2f0602d700 1 --2- 172.21.15.63:0/2049805885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f000ffba0 0x7f2f000fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.329+0000 7f2f0602d700 1 -- 172.21.15.63:0/2049805885 >> 172.21.15.63:0/2049805885 conn(0x7f2f000fb120 msgr2=0x7f2f000fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.329+0000 7f2f0602d700 1 -- 172.21.15.63:0/2049805885 shutdown_connections 2024-05-09T06:47:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.329+0000 7f2f0602d700 1 -- 172.21.15.63:0/2049805885 wait complete. 2024-05-09T06:47:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.330+0000 7f2f0602d700 1 Processor -- start 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.330+0000 7f2f0602d700 1 -- start start 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.330+0000 7f2f0602d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f000ffba0 0x7f2f001966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.330+0000 7f2f0602d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 0x7f2f00196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.330+0000 7f2f0602d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00197230 con 0x7f2f00100da0 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.330+0000 7f2f0602d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00197370 con 0x7f2f000ffba0 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.330+0000 7f2efeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 0x7f2f00196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.330+0000 7f2efeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 0x7f2f00196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33452/0 (socket says 172.21.15.63:33452) 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.330+0000 7f2efeffd700 1 -- 172.21.15.63:0/1324239361 learned_addr learned my addr 172.21.15.63:0/1324239361 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2eff7fe700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f000ffba0 0x7f2f001966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2efeffd700 1 -- 172.21.15.63:0/1324239361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f000ffba0 msgr2=0x7f2f001966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2efeffd700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f000ffba0 0x7f2f001966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2efeffd700 1 -- 172.21.15.63:0/1324239361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef40095f0 con 0x7f2f00100da0 2024-05-09T06:47:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2eff7fe700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f000ffba0 0x7f2f001966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:47:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2efeffd700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 0x7f2f00196c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400b800 tx=0x7f2ef400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2efcff9700 1 -- 172.21.15.63:0/1324239361 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef401d070 con 0x7f2f00100da0 2024-05-09T06:47:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2efcff9700 1 -- 172.21.15.63:0/1324239361 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ef40052a0 con 0x7f2f00100da0 2024-05-09T06:47:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0019bd20 con 0x7f2f00100da0 2024-05-09T06:47:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2efcff9700 1 -- 172.21.15.63:0/1324239361 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef4017400 con 0x7f2f00100da0 2024-05-09T06:47:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.331+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0019c210 con 0x7f2f00100da0 2024-05-09T06:47:31.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.333+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0004ebb0 con 0x7f2f00100da0 2024-05-09T06:47:31.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.333+0000 7f2efcff9700 1 -- 172.21.15.63:0/1324239361 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2ef40037f0 con 0x7f2f00100da0 2024-05-09T06:47:31.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.334+0000 7f2efcff9700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2ee806e200 0x7f2ee80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:31.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.334+0000 7f2efcff9700 1 -- 172.21.15.63:0/1324239361 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f2ef408f830 con 0x7f2f00100da0 2024-05-09T06:47:31.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.334+0000 7f2eff7fe700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2ee806e200 0x7f2ee80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:31.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.335+0000 7f2eff7fe700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2ee806e200 0x7f2ee80706c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2f00100c00 tx=0x7f2ef0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.343+0000 7f2efcff9700 1 -- 172.21.15.63:0/1324239361 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2ef405c4c0 con 0x7f2f00100da0 2024-05-09T06:47:31.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.603+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f00068ce0 con 0x7f2ee806e200 2024-05-09T06:47:31.584 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:31 smithi063 ceph-mon[92765]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:47:31.584 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:31 smithi063 ceph-mon[92765]: pgmap v268: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:31.584 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:31 smithi063 ceph-mon[92765]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:47:31.584 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:31 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/731360842' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:31.584 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:31 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/775955318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:47:31.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.605+0000 7f2efcff9700 1 -- 172.21.15.63:0/1324239361 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7f2f00068ce0 con 0x7f2ee806e200 2024-05-09T06:47:31.585 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:47:31.585 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac", 2024-05-09T06:47:31.585 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-05-09T06:47:31.585 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-09T06:47:31.585 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: "mon", 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: "mds", 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph-exporter" 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "18/23 daemons upgraded", 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading grafana daemons", 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:47:31.586 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2ee806e200 msgr2=0x7f2ee80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2ee806e200 0x7f2ee80706c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2f00100c00 tx=0x7f2ef0009260 comp rx=0 tx=0).stop 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 msgr2=0x7f2f00196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 0x7f2f00196c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400b800 tx=0x7f2ef400f510 comp rx=0 tx=0).stop 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 shutdown_connections 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f2ee806e200 0x7f2ee80706c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f000ffba0 0x7f2f001966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 --2- 172.21.15.63:0/1324239361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f00100da0 0x7f2f00196c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 >> 172.21.15.63:0/1324239361 conn(0x7f2f000fb120 msgr2=0x7f2f000685d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 shutdown_connections 2024-05-09T06:47:31.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.608+0000 7f2f0602d700 1 -- 172.21.15.63:0/1324239361 wait complete. 2024-05-09T06:47:31.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:31 smithi193 ceph-mon[81665]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:47:31.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:31 smithi193 ceph-mon[81665]: pgmap v268: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:31.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:31 smithi193 ceph-mon[81665]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:47:31.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:31 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/731360842' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:31.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:31 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/775955318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:47:31.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.754+0000 7f709b77a700 1 -- 172.21.15.63:0/3228263744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70941011d0 msgr2=0x7f70941015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:31.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.754+0000 7f709b77a700 1 --2- 172.21.15.63:0/3228263744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70941011d0 0x7f70941015f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7090009910 tx=0x7f7090009c20 comp rx=0 tx=0).stop 2024-05-09T06:47:31.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.754+0000 7f709b77a700 1 -- 172.21.15.63:0/3228263744 shutdown_connections 2024-05-09T06:47:31.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.754+0000 7f709b77a700 1 --2- 172.21.15.63:0/3228263744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70940ffda0 0x7f7094100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.754+0000 7f709b77a700 1 --2- 172.21.15.63:0/3228263744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70941011d0 0x7f70941015f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.754+0000 7f709b77a700 1 -- 172.21.15.63:0/3228263744 >> 172.21.15.63:0/3228263744 conn(0x7f70940fa870 msgr2=0x7f70940fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:31.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.754+0000 7f709b77a700 1 -- 172.21.15.63:0/3228263744 shutdown_connections 2024-05-09T06:47:31.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.754+0000 7f709b77a700 1 -- 172.21.15.63:0/3228263744 wait complete. 2024-05-09T06:47:31.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.755+0000 7f709b77a700 1 Processor -- start 2024-05-09T06:47:31.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.755+0000 7f709b77a700 1 -- start start 2024-05-09T06:47:31.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.755+0000 7f709b77a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70940ffda0 0x7f7094198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:31.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.755+0000 7f709b77a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70941011d0 0x7f7094198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:31.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.755+0000 7f709b77a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70941993f0 con 0x7f70940ffda0 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.755+0000 7f709b77a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7094199530 con 0x7f70941011d0 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.755+0000 7f7099516700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70940ffda0 0x7f7094198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.755+0000 7f7099516700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70940ffda0 0x7f7094198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33476/0 (socket says 172.21.15.63:33476) 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.756+0000 7f7099516700 1 -- 172.21.15.63:0/854005571 learned_addr learned my addr 172.21.15.63:0/854005571 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.756+0000 7f7098d15700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70941011d0 0x7f7094198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.756+0000 7f7099516700 1 -- 172.21.15.63:0/854005571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70941011d0 msgr2=0x7f7094198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.756+0000 7f7099516700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70941011d0 0x7f7094198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.756+0000 7f7099516700 1 -- 172.21.15.63:0/854005571 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900095f0 con 0x7f70940ffda0 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.756+0000 7f7098d15700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70941011d0 0x7f7094198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.756+0000 7f7099516700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70940ffda0 0x7f7094198890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7090006010 tx=0x7f709000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:31.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.756+0000 7f708a7fc700 1 -- 172.21.15.63:0/854005571 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709001d070 con 0x7f70940ffda0 2024-05-09T06:47:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.757+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709419df30 con 0x7f70940ffda0 2024-05-09T06:47:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.757+0000 7f708a7fc700 1 -- 172.21.15.63:0/854005571 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f709000f460 con 0x7f70940ffda0 2024-05-09T06:47:31.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.757+0000 7f708a7fc700 1 -- 172.21.15.63:0/854005571 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7090021620 con 0x7f70940ffda0 2024-05-09T06:47:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.757+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709419e420 con 0x7f70940ffda0 2024-05-09T06:47:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.758+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7094107220 con 0x7f70940ffda0 2024-05-09T06:47:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.760+0000 7f708a7fc700 1 -- 172.21.15.63:0/854005571 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f709002b440 con 0x7f70940ffda0 2024-05-09T06:47:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.760+0000 7f708a7fc700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f708006e200 0x7f70800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:47:31.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.761+0000 7f708a7fc700 1 -- 172.21.15.63:0/854005571 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f709008f570 con 0x7f70940ffda0 2024-05-09T06:47:31.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.761+0000 7f7098d15700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f708006e200 0x7f70800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:47:31.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.761+0000 7f7098d15700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f708006e200 0x7f70800706c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7084005950 tx=0x7f708400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:47:31.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:31.766+0000 7f708a7fc700 1 -- 172.21.15.63:0/854005571 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f709005d200 con 0x7f70940ffda0 2024-05-09T06:47:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.124+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f709404ec00 con 0x7f70940ffda0 2024-05-09T06:47:32.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.125+0000 7f708a7fc700 1 -- 172.21.15.63:0/854005571 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f709005cd90 con 0x7f70940ffda0 2024-05-09T06:47:32.105 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:47:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f708006e200 msgr2=0x7f70800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f708006e200 0x7f70800706c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7084005950 tx=0x7f708400b410 comp rx=0 tx=0).stop 2024-05-09T06:47:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70940ffda0 msgr2=0x7f7094198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:47:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70940ffda0 0x7f7094198890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7090006010 tx=0x7f709000bd70 comp rx=0 tx=0).stop 2024-05-09T06:47:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 shutdown_connections 2024-05-09T06:47:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f708006e200 0x7f70800706c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:32.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70940ffda0 0x7f7094198890 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 --2- 172.21.15.63:0/854005571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70941011d0 0x7f7094198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:47:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 >> 172.21.15.63:0/854005571 conn(0x7f70940fa870 msgr2=0x7f7094104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:47:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 shutdown_connections 2024-05-09T06:47:32.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:47:32.127+0000 7f709b77a700 1 -- 172.21.15.63:0/854005571 wait complete. 2024-05-09T06:47:32.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:32 smithi063 ceph-mon[92765]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:47:32.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:32 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/854005571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:47:32.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:32 smithi193 ceph-mon[81665]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:47:32.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:32 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/854005571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:47:33.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:33 smithi063 ceph-mon[92765]: pgmap v269: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:33.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:33 smithi193 ceph-mon[81665]: pgmap v269: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:35.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:35 smithi063 ceph-mon[92765]: pgmap v270: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:35.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:35 smithi193 ceph-mon[81665]: pgmap v270: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:37.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:37 smithi063 ceph-mon[92765]: pgmap v271: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:37.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:37 smithi193 ceph-mon[81665]: pgmap v271: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:39.564 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:39 smithi063 ceph-mon[92765]: pgmap v272: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:39.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:39 smithi193 ceph-mon[81665]: pgmap v272: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:41.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:41 smithi063 ceph-mon[92765]: pgmap v273: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:41.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:41 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:41.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:41 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:41.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:41 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:41.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:41 smithi193 ceph-mon[81665]: pgmap v273: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:41.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:41 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:41.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:41 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:41.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:41 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:42.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:47:42.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:47:43.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:43 smithi063 ceph-mon[92765]: pgmap v274: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:43.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:43 smithi193 ceph-mon[81665]: pgmap v274: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:45.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:45 smithi063 ceph-mon[92765]: pgmap v275: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:45.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:45 smithi193 ceph-mon[81665]: pgmap v275: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:47.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:47.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:47 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:47.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:47 smithi063 ceph-mon[92765]: pgmap v276: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:47.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:47.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:47 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:47.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:47 smithi193 ceph-mon[81665]: pgmap v276: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:48.542 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:48.542 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:48 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:48.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:48.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:48 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:49.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:49 smithi063 ceph-mon[92765]: pgmap v277: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:49.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:49 smithi193 ceph-mon[81665]: pgmap v277: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:51.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: pgmap v278: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:51.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:51.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:51.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:51.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:51.605 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-09T06:47:51.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-09T06:47:51.608 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-09T06:47:51.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:51 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:51.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: pgmap v278: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-09T06:47:51.622 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-09T06:47:51.623 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-09T06:47:51.624 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-09T06:47:51.625 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:51 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:52.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-09T06:47:52.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: Upgrade: Finalizing container_image settings 2024-05-09T06:47:52.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: Upgrade: Complete! 2024-05-09T06:47:52.226 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:52.226 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:52.226 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:52.226 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:52.226 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:52.226 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:52.226 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:52.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-09T06:47:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: Upgrade: Finalizing container_image settings 2024-05-09T06:47:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: Upgrade: Complete! 2024-05-09T06:47:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:47:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:47:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:47:52.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:53.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:53 smithi063 ceph-mon[92765]: pgmap v279: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:53.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:53 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:53.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:53 smithi193 ceph-mon[81665]: pgmap v279: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:53.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:53 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:47:55.801 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:55 smithi063 ceph-mon[92765]: pgmap v280: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:55.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:55 smithi193 ceph-mon[81665]: pgmap v280: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:47:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:57 smithi063 ceph-mon[92765]: pgmap v281: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:47:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:57 smithi193 ceph-mon[81665]: pgmap v281: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:47:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:47:59.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:47:59 smithi063 ceph-mon[92765]: pgmap v282: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:47:59.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:47:59 smithi193 ceph-mon[81665]: pgmap v282: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:01.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:01 smithi063 ceph-mon[92765]: pgmap v283: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:01.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:01 smithi193 ceph-mon[81665]: pgmap v283: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:02.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.253+0000 7f64bef0a700 1 -- 172.21.15.63:0/2468431415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b80ffb50 msgr2=0x7f64b80fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:02.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.253+0000 7f64bef0a700 1 --2- 172.21.15.63:0/2468431415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b80ffb50 0x7f64b80fff70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009910 tx=0x7f64ac009c20 comp rx=0 tx=0).stop 2024-05-09T06:48:02.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.253+0000 7f64bef0a700 1 -- 172.21.15.63:0/2468431415 shutdown_connections 2024-05-09T06:48:02.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.253+0000 7f64bef0a700 1 --2- 172.21.15.63:0/2468431415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64b8100d50 0x7f64b81011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:02.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.253+0000 7f64bef0a700 1 --2- 172.21.15.63:0/2468431415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b80ffb50 0x7f64b80fff70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:02.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.253+0000 7f64bef0a700 1 -- 172.21.15.63:0/2468431415 >> 172.21.15.63:0/2468431415 conn(0x7f64b80fb0f0 msgr2=0x7f64b80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:02.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.253+0000 7f64bef0a700 1 -- 172.21.15.63:0/2468431415 shutdown_connections 2024-05-09T06:48:02.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.253+0000 7f64bef0a700 1 -- 172.21.15.63:0/2468431415 wait complete. 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64bef0a700 1 Processor -- start 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64bef0a700 1 -- start start 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64bef0a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64b80ffb50 0x7f64b8103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64bef0a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b8100d50 0x7f64b8102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64bef0a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8102670 con 0x7f64b8100d50 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64bef0a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b81027b0 con 0x7f64b80ffb50 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64b7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b8100d50 0x7f64b8102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64b7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b8100d50 0x7f64b8102130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54024/0 (socket says 172.21.15.63:54024) 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64b7fff700 1 -- 172.21.15.63:0/292500439 learned_addr learned my addr 172.21.15.63:0/292500439 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.254+0000 7f64bcca6700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64b80ffb50 0x7f64b8103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.255+0000 7f64b7fff700 1 -- 172.21.15.63:0/292500439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64b80ffb50 msgr2=0x7f64b8103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.255+0000 7f64b7fff700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64b80ffb50 0x7f64b8103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:02.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.255+0000 7f64b7fff700 1 -- 172.21.15.63:0/292500439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac0095f0 con 0x7f64b8100d50 2024-05-09T06:48:02.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.255+0000 7f64b7fff700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b8100d50 0x7f64b8102130 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d8d0 tx=0x7f64a400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:02.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.255+0000 7f64b5ffb700 1 -- 172.21.15.63:0/292500439 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64a40098a0 con 0x7f64b8100d50 2024-05-09T06:48:02.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.255+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b8102a90 con 0x7f64b8100d50 2024-05-09T06:48:02.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.255+0000 7f64b5ffb700 1 -- 172.21.15.63:0/292500439 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64a4010460 con 0x7f64b8100d50 2024-05-09T06:48:02.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.255+0000 7f64b5ffb700 1 -- 172.21.15.63:0/292500439 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64a400f5f0 con 0x7f64b8100d50 2024-05-09T06:48:02.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.255+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b819e6d0 con 0x7f64b8100d50 2024-05-09T06:48:02.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.256+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b804ebb0 con 0x7f64b8100d50 2024-05-09T06:48:02.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.257+0000 7f64b5ffb700 1 -- 172.21.15.63:0/292500439 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f64a4009a00 con 0x7f64b8100d50 2024-05-09T06:48:02.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.257+0000 7f64b5ffb700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f64a806e200 0x7f64a80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:02.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.257+0000 7f64b5ffb700 1 -- 172.21.15.63:0/292500439 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f64a408e1d0 con 0x7f64b8100d50 2024-05-09T06:48:02.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.257+0000 7f64bcca6700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f64a806e200 0x7f64a80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:02.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.258+0000 7f64bcca6700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f64a806e200 0x7f64a80706c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64ac000c00 tx=0x7f64ac005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:02.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.262+0000 7f64b5ffb700 1 -- 172.21.15.63:0/292500439 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f64a405ae60 con 0x7f64b8100d50 2024-05-09T06:48:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.493+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64b8069920 con 0x7f64a806e200 2024-05-09T06:48:02.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.494+0000 7f64b5ffb700 1 -- 172.21.15.63:0/292500439 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f64b8069920 con 0x7f64a806e200 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.496+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f64a806e200 msgr2=0x7f64a80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.496+0000 7f64bef0a700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f64a806e200 0x7f64a80706c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64ac000c00 tx=0x7f64ac005960 comp rx=0 tx=0).stop 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.496+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b8100d50 msgr2=0x7f64b8102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.496+0000 7f64bef0a700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b8100d50 0x7f64b8102130 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d8d0 tx=0x7f64a400dbe0 comp rx=0 tx=0).stop 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.496+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 shutdown_connections 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.496+0000 7f64bef0a700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f64a806e200 0x7f64a80706c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.496+0000 7f64bef0a700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64b80ffb50 0x7f64b8103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.496+0000 7f64bef0a700 1 --2- 172.21.15.63:0/292500439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b8100d50 0x7f64b8102130 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.496+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 >> 172.21.15.63:0/292500439 conn(0x7f64b80fb0f0 msgr2=0x7f64b8069210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.497+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 shutdown_connections 2024-05-09T06:48:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:02.497+0000 7f64bef0a700 1 -- 172.21.15.63:0/292500439 wait complete. 2024-05-09T06:48:03.242 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph orch ps' 2024-05-09T06:48:03.510 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:48:03.745 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:03 smithi063 ceph-mon[92765]: pgmap v284: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:03.745 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:03 smithi063 ceph-mon[92765]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:48:03.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:03 smithi193 ceph-mon[81665]: pgmap v284: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:03.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:03 smithi193 ceph-mon[81665]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:48:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.128+0000 7f6def8cf700 1 -- 172.21.15.63:0/3861831081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de8100e30 msgr2=0x7f6de8101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.128+0000 7f6def8cf700 1 --2- 172.21.15.63:0/3861831081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de8100e30 0x7f6de8101210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800b0a0 tx=0x7f6dd800b3b0 comp rx=0 tx=0).stop 2024-05-09T06:48:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.128+0000 7f6def8cf700 1 -- 172.21.15.63:0/3861831081 shutdown_connections 2024-05-09T06:48:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.128+0000 7f6def8cf700 1 --2- 172.21.15.63:0/3861831081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6de81017e0 0x7f6de8105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.128+0000 7f6def8cf700 1 --2- 172.21.15.63:0/3861831081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de8100e30 0x7f6de8101210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.128+0000 7f6def8cf700 1 -- 172.21.15.63:0/3861831081 >> 172.21.15.63:0/3861831081 conn(0x7f6de80fc6e0 msgr2=0x7f6de80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.128+0000 7f6def8cf700 1 -- 172.21.15.63:0/3861831081 shutdown_connections 2024-05-09T06:48:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.128+0000 7f6def8cf700 1 -- 172.21.15.63:0/3861831081 wait complete. 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.129+0000 7f6def8cf700 1 Processor -- start 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.129+0000 7f6def8cf700 1 -- start start 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.129+0000 7f6def8cf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6de8100e30 0x7f6de80731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.129+0000 7f6def8cf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de81017e0 0x7f6de8073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.129+0000 7f6def8cf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8073c70 con 0x7f6de81017e0 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.129+0000 7f6def8cf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8073db0 con 0x7f6de8100e30 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.129+0000 7f6dece6a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de81017e0 0x7f6de8073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.129+0000 7f6dece6a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de81017e0 0x7f6de8073730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50842/0 (socket says 172.21.15.63:50842) 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.129+0000 7f6dece6a700 1 -- 172.21.15.63:0/283711173 learned_addr learned my addr 172.21.15.63:0/283711173 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6ded66b700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6de8100e30 0x7f6de80731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6dece6a700 1 -- 172.21.15.63:0/283711173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6de8100e30 msgr2=0x7f6de80731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6dece6a700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6de8100e30 0x7f6de80731f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6dece6a700 1 -- 172.21.15.63:0/283711173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd8009d00 con 0x7f6de81017e0 2024-05-09T06:48:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6dece6a700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de81017e0 0x7f6de8073730 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00d900 tx=0x7f6ddc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6de67fc700 1 -- 172.21.15.63:0/283711173 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ddc009900 con 0x7f6de81017e0 2024-05-09T06:48:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de81a2d60 con 0x7f6de81017e0 2024-05-09T06:48:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6de67fc700 1 -- 172.21.15.63:0/283711173 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ddc010460 con 0x7f6de81017e0 2024-05-09T06:48:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6de67fc700 1 -- 172.21.15.63:0/283711173 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ddc00f5f0 con 0x7f6de81017e0 2024-05-09T06:48:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.130+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de81a30c0 con 0x7f6de81017e0 2024-05-09T06:48:05.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.131+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de81091d0 con 0x7f6de81017e0 2024-05-09T06:48:05.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.132+0000 7f6de67fc700 1 -- 172.21.15.63:0/283711173 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6ddc0105d0 con 0x7f6de81017e0 2024-05-09T06:48:05.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.132+0000 7f6de67fc700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6dd006e200 0x7f6dd00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:05.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.132+0000 7f6de67fc700 1 -- 172.21.15.63:0/283711173 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f6ddc08e190 con 0x7f6de81017e0 2024-05-09T06:48:05.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.132+0000 7f6ded66b700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6dd006e200 0x7f6dd00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:05.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.133+0000 7f6ded66b700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6dd006e200 0x7f6dd00706c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80062a0 tx=0x7f6dd80061f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:05.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.137+0000 7f6de67fc700 1 -- 172.21.15.63:0/283711173 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6ddc0572f0 con 0x7f6de81017e0 2024-05-09T06:48:05.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.352+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6de80746f0 con 0x7f6dd006e200 2024-05-09T06:48:05.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.365+0000 7f6de67fc700 1 -- 172.21.15.63:0/283711173 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6de80746f0 con 0x7f6dd006e200 2024-05-09T06:48:05.344 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:48:05.344 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (50s) 19s ago 18m 15.2M - 0.25.0 c8568f914cd2 b9acb0012a9e 2024-05-09T06:48:05.344 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (111s) 19s ago 18m 8140k - 19.0.0-2515-g96805dcf d7acac7d5657 1993c9f6fdcd 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (105s) 81s ago 17m 7570k - 19.0.0-2515-g96805dcf d7acac7d5657 bd88664dfa9d 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (6m) 19s ago 18m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (6m) 81s ago 17m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (24s) 19s ago 18m 54.7M - 9.4.12 2bacad6d85d8 a8eca89f71c1 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (3m) 19s ago 12m 87.7M - 19.0.0-2515-g96805dcf d7acac7d5657 c7cf68a1a3e2 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (2m) 19s ago 12m 84.5M - 19.0.0-2515-g96805dcf d7acac7d5657 c24cf6de2c0f 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (2m) 81s ago 12m 13.8M - 19.0.0-2515-g96805dcf d7acac7d5657 b752027d6904 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (2m) 81s ago 12m 13.7M - 19.0.0-2515-g96805dcf d7acac7d5657 bf31d952d7fe 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (9m) 19s ago 19m 524M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (8m) 81s ago 17m 453M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (8m) 19s ago 20m 57.3M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (7m) 81s ago 17m 40.9M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (88s) 19s ago 18m 8284k - 1.5.0 0da6a335fe13 1f7eb6ee1981 2024-05-09T06:48:05.345 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (84s) 81s ago 17m 5473k - 1.5.0 0da6a335fe13 21d4f1f51ed5 2024-05-09T06:48:05.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (6m) 19s ago 16m 151M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:48:05.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 19s ago 16m 124M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:48:05.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (5m) 19s ago 15m 99.3M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:48:05.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (4m) 81s ago 15m 118M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:48:05.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (4m) 81s ago 14m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 14c9b4920f5a 2024-05-09T06:48:05.346 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (3m) 81s ago 14m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 d219b4bb599f 2024-05-09T06:48:05.346 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (66s) 19s ago 18m 47.2M - 2.43.0 a07b618ecd1d 6ee3f1928d73 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6dd006e200 msgr2=0x7f6dd00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6dd006e200 0x7f6dd00706c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80062a0 tx=0x7f6dd80061f0 comp rx=0 tx=0).stop 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de81017e0 msgr2=0x7f6de8073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de81017e0 0x7f6de8073730 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00d900 tx=0x7f6ddc00dcc0 comp rx=0 tx=0).stop 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 shutdown_connections 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6dd006e200 0x7f6dd00706c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6de8100e30 0x7f6de80731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 --2- 172.21.15.63:0/283711173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6de81017e0 0x7f6de8073730 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 >> 172.21.15.63:0/283711173 conn(0x7f6de80fc6e0 msgr2=0x7f6de80fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 shutdown_connections 2024-05-09T06:48:05.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:05.367+0000 7f6def8cf700 1 -- 172.21.15.63:0/283711173 wait complete. 2024-05-09T06:48:05.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:05 smithi063 ceph-mon[92765]: pgmap v285: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:05.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:05 smithi193 ceph-mon[81665]: pgmap v285: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:05.954 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph orch upgrade status' 2024-05-09T06:48:06.177 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:48:06.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:06 smithi063 ceph-mon[92765]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:48:06.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:06 smithi193 ceph-mon[81665]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:48:07.531 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:07 smithi063 ceph-mon[92765]: pgmap v286: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:07.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:07 smithi193 ceph-mon[81665]: pgmap v286: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:09.766 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:09 smithi063 ceph-mon[92765]: pgmap v287: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:09.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:09 smithi193 ceph-mon[81665]: pgmap v287: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:10.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.550+0000 7f2689863700 1 -- 172.21.15.63:0/2630583005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 msgr2=0x7f2684105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.550+0000 7f2689863700 1 --2- 172.21.15.63:0/2630583005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 0x7f2684105800 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f266c009910 tx=0x7f266c009c20 comp rx=0 tx=0).stop 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.550+0000 7f2689863700 1 -- 172.21.15.63:0/2630583005 shutdown_connections 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.550+0000 7f2689863700 1 --2- 172.21.15.63:0/2630583005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 0x7f2684105800 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.550+0000 7f2689863700 1 --2- 172.21.15.63:0/2630583005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684100e00 0x7f26841011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.550+0000 7f2689863700 1 -- 172.21.15.63:0/2630583005 >> 172.21.15.63:0/2630583005 conn(0x7f26840fc6b0 msgr2=0x7f26840fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.550+0000 7f2689863700 1 -- 172.21.15.63:0/2630583005 shutdown_connections 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.550+0000 7f2689863700 1 -- 172.21.15.63:0/2630583005 wait complete. 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.551+0000 7f2689863700 1 Processor -- start 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.551+0000 7f2689863700 1 -- start start 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.551+0000 7f2689863700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684100e00 0x7f268419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:10.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f2689863700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 0x7f268419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f2689863700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268419d460 con 0x7f26841017b0 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f2689863700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2684196a40 con 0x7f2684100e00 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f267a7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 0x7f268419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f267a7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 0x7f268419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50858/0 (socket says 172.21.15.63:50858) 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f267a7fc700 1 -- 172.21.15.63:0/1068911031 learned_addr learned my addr 172.21.15.63:0/1068911031 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f2682ffd700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684100e00 0x7f268419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f267a7fc700 1 -- 172.21.15.63:0/1068911031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684100e00 msgr2=0x7f268419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f267a7fc700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684100e00 0x7f268419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f267a7fc700 1 -- 172.21.15.63:0/1068911031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0095f0 con 0x7f26841017b0 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f267a7fc700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 0x7f268419ce40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f266c0092e0 tx=0x7f266c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f2688861700 1 -- 172.21.15.63:0/1068911031 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f266c01d070 con 0x7f26841017b0 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f2688861700 1 -- 172.21.15.63:0/1068911031 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f266c00f460 con 0x7f26841017b0 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2684196cc0 con 0x7f26841017b0 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f2688861700 1 -- 172.21.15.63:0/1068911031 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f266c017610 con 0x7f26841017b0 2024-05-09T06:48:10.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.552+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26841971b0 con 0x7f26841017b0 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.553+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2684109150 con 0x7f26841017b0 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.554+0000 7f2688861700 1 -- 172.21.15.63:0/1068911031 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f266c00f5d0 con 0x7f26841017b0 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.554+0000 7f2688861700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f267006e210 0x7f26700706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.554+0000 7f2688861700 1 -- 172.21.15.63:0/1068911031 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f266c08f810 con 0x7f26841017b0 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.555+0000 7f2682ffd700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f267006e210 0x7f26700706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.555+0000 7f2682ffd700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f267006e210 0x7f26700706d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2674006fd0 tx=0x7f2674008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.559+0000 7f2688861700 1 -- 172.21.15.63:0/1068911031 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f266c05c4a0 con 0x7f26841017b0 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.786+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2684197f00 con 0x7f267006e210 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.788+0000 7f2688861700 1 -- 172.21.15.63:0/1068911031 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f2684197f00 con 0x7f267006e210 2024-05-09T06:48:10.173 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": null, 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": false, 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "", 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": null, 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f267006e210 msgr2=0x7f26700706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f267006e210 0x7f26700706d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2674006fd0 tx=0x7f2674008040 comp rx=0 tx=0).stop 2024-05-09T06:48:10.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 msgr2=0x7f268419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:10.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 0x7f268419ce40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f266c0092e0 tx=0x7f266c00be20 comp rx=0 tx=0).stop 2024-05-09T06:48:10.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 shutdown_connections 2024-05-09T06:48:10.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f267006e210 0x7f26700706d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:10.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684100e00 0x7f268419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:10.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 --2- 172.21.15.63:0/1068911031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26841017b0 0x7f268419ce40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:10.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 >> 172.21.15.63:0/1068911031 conn(0x7f26840fc6b0 msgr2=0x7f26840fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:10.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 shutdown_connections 2024-05-09T06:48:10.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:09.790+0000 7f2689863700 1 -- 172.21.15.63:0/1068911031 wait complete. 2024-05-09T06:48:10.828 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph health detail' 2024-05-09T06:48:11.056 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:48:11.773 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:11 smithi063 ceph-mon[92765]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:48:11.773 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:11 smithi063 ceph-mon[92765]: pgmap v288: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:11.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:11 smithi193 ceph-mon[81665]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:48:11.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:11 smithi193 ceph-mon[81665]: pgmap v288: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:12.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.671+0000 7f98740cd700 1 -- 172.21.15.63:0/717001592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 msgr2=0x7f986c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:12.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.671+0000 7f98740cd700 1 --2- 172.21.15.63:0/717001592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 0x7f986c10abe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f985c009940 tx=0x7f985c009c50 comp rx=0 tx=0).stop 2024-05-09T06:48:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.671+0000 7f98740cd700 1 -- 172.21.15.63:0/717001592 shutdown_connections 2024-05-09T06:48:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.671+0000 7f98740cd700 1 --2- 172.21.15.63:0/717001592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 0x7f986c10abe0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.671+0000 7f98740cd700 1 --2- 172.21.15.63:0/717001592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f986c101dd0 0x7f986c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.671+0000 7f98740cd700 1 -- 172.21.15.63:0/717001592 >> 172.21.15.63:0/717001592 conn(0x7f986c076750 msgr2=0x7f986c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.671+0000 7f98740cd700 1 -- 172.21.15.63:0/717001592 shutdown_connections 2024-05-09T06:48:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.671+0000 7f98740cd700 1 -- 172.21.15.63:0/717001592 wait complete. 2024-05-09T06:48:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.672+0000 7f98740cd700 1 Processor -- start 2024-05-09T06:48:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.672+0000 7f98740cd700 1 -- start start 2024-05-09T06:48:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.672+0000 7f98740cd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f986c101dd0 0x7f986c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.672+0000 7f98740cd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 0x7f986c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.672+0000 7f98740cd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c19d450 con 0x7f986c1026f0 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.672+0000 7f98740cd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c196a30 con 0x7f986c101dd0 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.672+0000 7f9871668700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 0x7f986c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.672+0000 7f9871668700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 0x7f986c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50876/0 (socket says 172.21.15.63:50876) 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.672+0000 7f9871668700 1 -- 172.21.15.63:0/2918510277 learned_addr learned my addr 172.21.15.63:0/2918510277 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f9871e69700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f986c101dd0 0x7f986c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f9871668700 1 -- 172.21.15.63:0/2918510277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f986c101dd0 msgr2=0x7f986c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f9871668700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f986c101dd0 0x7f986c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f9871668700 1 -- 172.21.15.63:0/2918510277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0095f0 con 0x7f986c1026f0 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f9871668700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 0x7f986c19ce30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f985c0055f0 tx=0x7f985c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f9862ffd700 1 -- 172.21.15.63:0/2918510277 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f985c01d070 con 0x7f986c1026f0 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c196cb0 con 0x7f986c1026f0 2024-05-09T06:48:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f9862ffd700 1 -- 172.21.15.63:0/2918510277 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f985c00bed0 con 0x7f986c1026f0 2024-05-09T06:48:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f9862ffd700 1 -- 172.21.15.63:0/2918510277 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f985c021410 con 0x7f986c1026f0 2024-05-09T06:48:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.673+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c1971a0 con 0x7f986c1026f0 2024-05-09T06:48:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.674+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c105560 con 0x7f986c1026f0 2024-05-09T06:48:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.675+0000 7f9862ffd700 1 -- 172.21.15.63:0/2918510277 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f985c00f7f0 con 0x7f986c1026f0 2024-05-09T06:48:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.675+0000 7f9862ffd700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f985806e200 0x7f98580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.675+0000 7f9862ffd700 1 -- 172.21.15.63:0/2918510277 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f985c08f540 con 0x7f986c1026f0 2024-05-09T06:48:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.675+0000 7f9871e69700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f985806e200 0x7f98580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.676+0000 7f9871e69700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f985806e200 0x7f98580706c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9868009f20 tx=0x7f98680094f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.680+0000 7f9862ffd700 1 -- 172.21.15.63:0/2918510277 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f985c05c1d0 con 0x7f986c1026f0 2024-05-09T06:48:12.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:48:12.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:48:12.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.976+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f986c197ef0 con 0x7f986c1026f0 2024-05-09T06:48:12.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.976+0000 7f9862ffd700 1 -- 172.21.15.63:0/2918510277 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f985c05bd60 con 0x7f986c1026f0 2024-05-09T06:48:12.955 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-05-09T06:48:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f985806e200 msgr2=0x7f98580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f985806e200 0x7f98580706c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9868009f20 tx=0x7f98680094f0 comp rx=0 tx=0).stop 2024-05-09T06:48:12.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 msgr2=0x7f986c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:12.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 0x7f986c19ce30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f985c0055f0 tx=0x7f985c0056d0 comp rx=0 tx=0).stop 2024-05-09T06:48:12.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 shutdown_connections 2024-05-09T06:48:12.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f985806e200 0x7f98580706c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:12.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f986c101dd0 0x7f986c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:12.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 --2- 172.21.15.63:0/2918510277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f986c1026f0 0x7f986c19ce30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:12.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 >> 172.21.15.63:0/2918510277 conn(0x7f986c076750 msgr2=0x7f986c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:12.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 shutdown_connections 2024-05-09T06:48:12.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:12.978+0000 7f98740cd700 1 -- 172.21.15.63:0/2918510277 wait complete. 2024-05-09T06:48:13.529 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph versions' 2024-05-09T06:48:13.748 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:48:13.771 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:13 smithi193 ceph-mon[81665]: pgmap v289: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:13.771 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:13 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2918510277' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:48:13.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:13 smithi063 ceph-mon[92765]: pgmap v289: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:13.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:13 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2918510277' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-09T06:48:15.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.338+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1901529310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 msgr2=0x7f7c64105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:15.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.338+0000 7f7c6c16e700 1 --2- 172.21.15.63:0/1901529310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 0x7f7c64105850 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009910 tx=0x7f7c54009c20 comp rx=0 tx=0).stop 2024-05-09T06:48:15.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.338+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1901529310 shutdown_connections 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.338+0000 7f7c6c16e700 1 --2- 172.21.15.63:0/1901529310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 0x7f7c64105850 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.338+0000 7f7c6c16e700 1 --2- 172.21.15.63:0/1901529310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c64100e50 0x7f7c64101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.338+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1901529310 >> 172.21.15.63:0/1901529310 conn(0x7f7c640fc720 msgr2=0x7f7c640feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.339+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1901529310 shutdown_connections 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.339+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1901529310 wait complete. 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.339+0000 7f7c6c16e700 1 Processor -- start 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.339+0000 7f7c6c16e700 1 -- start start 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.339+0000 7f7c6c16e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c64100e50 0x7f7c6419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.339+0000 7f7c6c16e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 0x7f7c6419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.339+0000 7f7c6c16e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6419d390 con 0x7f7c64101800 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.339+0000 7f7c6c16e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c64196970 con 0x7f7c64100e50 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c69709700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 0x7f7c6419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:15.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c69709700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 0x7f7c6419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39800/0 (socket says 172.21.15.63:39800) 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c69709700 1 -- 172.21.15.63:0/1458212812 learned_addr learned my addr 172.21.15.63:0/1458212812 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c69f0a700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c64100e50 0x7f7c6419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c69709700 1 -- 172.21.15.63:0/1458212812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c64100e50 msgr2=0x7f7c6419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c69709700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c64100e50 0x7f7c6419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c69709700 1 -- 172.21.15.63:0/1458212812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c540095f0 con 0x7f7c64101800 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c69709700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 0x7f7c6419cd70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7c54000c00 tx=0x7f7c5400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c5affd700 1 -- 172.21.15.63:0/1458212812 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c5401d070 con 0x7f7c64101800 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c5affd700 1 -- 172.21.15.63:0/1458212812 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c54005700 con 0x7f7c64101800 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c5affd700 1 -- 172.21.15.63:0/1458212812 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c5400fda0 con 0x7f7c64101800 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c64196bf0 con 0x7f7c64101800 2024-05-09T06:48:15.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.340+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c641970e0 con 0x7f7c64101800 2024-05-09T06:48:15.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.342+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c64109250 con 0x7f7c64101800 2024-05-09T06:48:15.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.342+0000 7f7c5affd700 1 -- 172.21.15.63:0/1458212812 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7c5400f5d0 con 0x7f7c64101800 2024-05-09T06:48:15.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.343+0000 7f7c5affd700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c5006e210 0x7f7c500706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:48:15.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.343+0000 7f7c5affd700 1 -- 172.21.15.63:0/1458212812 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f7c5408f5d0 con 0x7f7c64101800 2024-05-09T06:48:15.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.343+0000 7f7c69f0a700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c5006e210 0x7f7c500706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:48:15.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.343+0000 7f7c69f0a700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c5006e210 0x7f7c500706d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c60007be0 tx=0x7f7c600073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:48:15.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.347+0000 7f7c5affd700 1 -- 172.21.15.63:0/1458212812 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7c5405c2e0 con 0x7f7c64101800 2024-05-09T06:48:15.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.644+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7c64102330 con 0x7f7c64101800 2024-05-09T06:48:15.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.644+0000 7f7c5affd700 1 -- 172.21.15.63:0/1458212812 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7c5405be70 con 0x7f7c64101800 2024-05-09T06:48:15.623 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:48:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 6 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 4 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 14 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:48:15.624 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:48:15.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.646+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c5006e210 msgr2=0x7f7c500706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:15.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.646+0000 7f7c6c16e700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c5006e210 0x7f7c500706d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c60007be0 tx=0x7f7c600073d0 comp rx=0 tx=0).stop 2024-05-09T06:48:15.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.646+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 msgr2=0x7f7c6419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:48:15.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.646+0000 7f7c6c16e700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 0x7f7c6419cd70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7c54000c00 tx=0x7f7c5400bd70 comp rx=0 tx=0).stop 2024-05-09T06:48:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.646+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 shutdown_connections 2024-05-09T06:48:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.646+0000 7f7c6c16e700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f7c5006e210 0x7f7c500706d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.646+0000 7f7c6c16e700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c64100e50 0x7f7c6419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.646+0000 7f7c6c16e700 1 --2- 172.21.15.63:0/1458212812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c64101800 0x7f7c6419cd70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:48:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.646+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 >> 172.21.15.63:0/1458212812 conn(0x7f7c640fc720 msgr2=0x7f7c640fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:48:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.647+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 shutdown_connections 2024-05-09T06:48:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:48:15.647+0000 7f7c6c16e700 1 -- 172.21.15.63:0/1458212812 wait complete. 2024-05-09T06:48:15.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:15 smithi063 ceph-mon[92765]: pgmap v290: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:15.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:15 smithi193 ceph-mon[81665]: pgmap v290: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:16.258 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-05-09T06:48:16.475 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:48:16.771 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:16 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1458212812' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:48:16.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:16 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1458212812' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:48:17.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:17 smithi063 ceph-mon[92765]: pgmap v291: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:17.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:17 smithi193 ceph-mon[81665]: pgmap v291: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:18.138 INFO:teuthology.orchestra.run.smithi063.stdout:wait for servicemap items w/ changing names to refresh 2024-05-09T06:48:18.658 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'sleep 60' 2024-05-09T06:48:18.877 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:48:19.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:19 smithi063 ceph-mon[92765]: pgmap v292: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:19.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:19 smithi193 ceph-mon[81665]: pgmap v292: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:21.858 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:21 smithi063 ceph-mon[92765]: pgmap v293: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:21.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:21 smithi193 ceph-mon[81665]: pgmap v293: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:23.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:23 smithi063 ceph-mon[92765]: pgmap v294: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:23.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:23 smithi193 ceph-mon[81665]: pgmap v294: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:25.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:25 smithi063 ceph-mon[92765]: pgmap v295: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:25.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:25 smithi193 ceph-mon[81665]: pgmap v295: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:27.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:27 smithi063 ceph-mon[92765]: pgmap v296: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:27.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:48:27.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:27 smithi193 ceph-mon[81665]: pgmap v296: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:27.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:48:29.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:29 smithi063 ceph-mon[92765]: pgmap v297: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:29.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:29 smithi193 ceph-mon[81665]: pgmap v297: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:31.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:31 smithi193 ceph-mon[81665]: pgmap v298: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:31.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:31 smithi063 ceph-mon[92765]: pgmap v298: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:33.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:33 smithi063 ceph-mon[92765]: pgmap v299: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:33.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:33 smithi193 ceph-mon[81665]: pgmap v299: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:35.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:35 smithi063 ceph-mon[92765]: pgmap v300: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:35.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:35 smithi193 ceph-mon[81665]: pgmap v300: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:37.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:37 smithi063 ceph-mon[92765]: pgmap v301: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:37.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:37 smithi193 ceph-mon[81665]: pgmap v301: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:39.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:39 smithi063 ceph-mon[92765]: pgmap v302: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:39.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:39 smithi193 ceph-mon[81665]: pgmap v302: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:41.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:41 smithi063 ceph-mon[92765]: pgmap v303: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:41.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:41 smithi193 ceph-mon[81665]: pgmap v303: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:42.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:48:42.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:48:43.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:43 smithi063 ceph-mon[92765]: pgmap v304: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:43.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:43 smithi193 ceph-mon[81665]: pgmap v304: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:45.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:45 smithi063 ceph-mon[92765]: pgmap v305: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:45.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:45 smithi193 ceph-mon[81665]: pgmap v305: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:47.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:47 smithi063 ceph-mon[92765]: pgmap v306: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:47.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:47 smithi193 ceph-mon[81665]: pgmap v306: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:49.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:49 smithi063 ceph-mon[92765]: pgmap v307: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:49.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:49 smithi193 ceph-mon[81665]: pgmap v307: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:51.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:51 smithi063 ceph-mon[92765]: pgmap v308: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:51.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:51 smithi193 ceph-mon[81665]: pgmap v308: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:52.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:52 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:48:52.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:52 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:48:53.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:53 smithi063 ceph-mon[92765]: pgmap v309: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:53.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:53 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:48:53.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:53 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:48:53.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:53 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:48:53.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:53 smithi193 ceph-mon[81665]: pgmap v309: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:53.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:53 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:48:53.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:53 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:48:53.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:53 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:48:55.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:55 smithi063 ceph-mon[92765]: pgmap v310: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:55.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:55 smithi193 ceph-mon[81665]: pgmap v310: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:48:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:57 smithi063 ceph-mon[92765]: pgmap v311: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:48:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:57 smithi193 ceph-mon[81665]: pgmap v311: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:48:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:48:59.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:48:59 smithi063 ceph-mon[92765]: pgmap v312: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:48:59.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:48:59 smithi193 ceph-mon[81665]: pgmap v312: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:01.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:01 smithi063 ceph-mon[92765]: pgmap v313: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:01.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:01 smithi193 ceph-mon[81665]: pgmap v313: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:03.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:03 smithi063 ceph-mon[92765]: pgmap v314: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:03.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:03 smithi193 ceph-mon[81665]: pgmap v314: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:05.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:05 smithi063 ceph-mon[92765]: pgmap v315: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:05.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:05 smithi193 ceph-mon[81665]: pgmap v315: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:07.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:07 smithi193 ceph-mon[81665]: pgmap v316: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:08.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:07 smithi063 ceph-mon[92765]: pgmap v316: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:09.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:09 smithi193 ceph-mon[81665]: pgmap v317: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:10.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:09 smithi063 ceph-mon[92765]: pgmap v317: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:11.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:11 smithi193 ceph-mon[81665]: pgmap v318: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:12.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:11 smithi063 ceph-mon[92765]: pgmap v318: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:12.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:49:13.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:49:13.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:13 smithi193 ceph-mon[81665]: pgmap v319: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:14.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:13 smithi063 ceph-mon[92765]: pgmap v319: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:16.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:15 smithi063 ceph-mon[92765]: pgmap v320: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:16.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:15 smithi193 ceph-mon[81665]: pgmap v320: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:18.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:17 smithi063 ceph-mon[92765]: pgmap v321: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:18.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:17 smithi193 ceph-mon[81665]: pgmap v321: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:19.906 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:19 smithi063 ceph-mon[92765]: pgmap v322: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:20.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:19 smithi193 ceph-mon[81665]: pgmap v322: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:21.210 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph orch ps' 2024-05-09T06:49:21.440 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:49:21.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:21 smithi063 ceph-mon[92765]: pgmap v323: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:22.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:21 smithi193 ceph-mon[81665]: pgmap v323: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:23.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.021+0000 7f29687ad700 1 -- 172.21.15.63:0/1898608602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 msgr2=0x7f2960105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:23.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.021+0000 7f29687ad700 1 --2- 172.21.15.63:0/1898608602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 0x7f2960105e90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f295c009910 tx=0x7f295c009c20 comp rx=0 tx=0).stop 2024-05-09T06:49:23.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.021+0000 7f29687ad700 1 -- 172.21.15.63:0/1898608602 shutdown_connections 2024-05-09T06:49:23.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.021+0000 7f29687ad700 1 --2- 172.21.15.63:0/1898608602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29600ffab0 0x7f29600fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:23.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.021+0000 7f29687ad700 1 --2- 172.21.15.63:0/1898608602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 0x7f2960105e90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:23.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.021+0000 7f29687ad700 1 -- 172.21.15.63:0/1898608602 >> 172.21.15.63:0/1898608602 conn(0x7f2960076750 msgr2=0x7f29600fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.021+0000 7f29687ad700 1 -- 172.21.15.63:0/1898608602 shutdown_connections 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.021+0000 7f29687ad700 1 -- 172.21.15.63:0/1898608602 wait complete. 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f29687ad700 1 Processor -- start 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f29687ad700 1 -- start start 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f29687ad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29600ffab0 0x7f296019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f29687ad700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 0x7f296019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f29687ad700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296019d2b0 con 0x7f2960105ab0 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f29687ad700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2960196920 con 0x7f29600ffab0 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f2965d48700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 0x7f296019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f2965d48700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 0x7f296019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52864/0 (socket says 172.21.15.63:52864) 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f2965d48700 1 -- 172.21.15.63:0/316183371 learned_addr learned my addr 172.21.15.63:0/316183371 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f2966549700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29600ffab0 0x7f296019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f2965d48700 1 -- 172.21.15.63:0/316183371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29600ffab0 msgr2=0x7f296019c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f2965d48700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29600ffab0 0x7f296019c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.022+0000 7f2965d48700 1 -- 172.21.15.63:0/316183371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c0095f0 con 0x7f2960105ab0 2024-05-09T06:49:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.023+0000 7f2965d48700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 0x7f296019cd20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f295400c930 tx=0x7f295400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.023+0000 7f29537fe700 1 -- 172.21.15.63:0/316183371 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2954007ad0 con 0x7f2960105ab0 2024-05-09T06:49:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.023+0000 7f29537fe700 1 -- 172.21.15.63:0/316183371 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2954007c30 con 0x7f2960105ab0 2024-05-09T06:49:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.023+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2960196c00 con 0x7f2960105ab0 2024-05-09T06:49:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.023+0000 7f29537fe700 1 -- 172.21.15.63:0/316183371 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2954018610 con 0x7f2960105ab0 2024-05-09T06:49:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.023+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2960197150 con 0x7f2960105ab0 2024-05-09T06:49:23.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.024+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29601011f0 con 0x7f2960105ab0 2024-05-09T06:49:23.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.025+0000 7f29537fe700 1 -- 172.21.15.63:0/316183371 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f295400f450 con 0x7f2960105ab0 2024-05-09T06:49:23.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.026+0000 7f29537fe700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f294c06e210 0x7f294c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:23.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.026+0000 7f29537fe700 1 -- 172.21.15.63:0/316183371 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f295408e4c0 con 0x7f2960105ab0 2024-05-09T06:49:23.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.026+0000 7f2966549700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f294c06e210 0x7f294c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:23.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.026+0000 7f2966549700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f294c06e210 0x7f294c0706d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f295c00b5c0 tx=0x7f295c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:23.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.030+0000 7f29537fe700 1 -- 172.21.15.63:0/316183371 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f295405b130 con 0x7f2960105ab0 2024-05-09T06:49:23.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.248+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 --> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f296010cd20 con 0x7f294c06e210 2024-05-09T06:49:23.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.259+0000 7f29537fe700 1 -- 172.21.15.63:0/316183371 <== mgr.34102 v2:172.21.15.63:6800/2497028716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f296010cd20 con 0x7f294c06e210 2024-05-09T06:49:23.238 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T06:49:23.238 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (2m) 97s ago 19m 15.2M - 0.25.0 c8568f914cd2 b9acb0012a9e 2024-05-09T06:49:23.238 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi063 smithi063 running (3m) 97s ago 20m 8140k - 19.0.0-2515-g96805dcf d7acac7d5657 1993c9f6fdcd 2024-05-09T06:49:23.238 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-exporter.smithi193 smithi193 running (3m) 2m ago 18m 7570k - 19.0.0-2515-g96805dcf d7acac7d5657 bd88664dfa9d 2024-05-09T06:49:23.238 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (8m) 97s ago 20m 6660k - 19.0.0-2515-g96805dcf d7acac7d5657 7ed781f4474d 2024-05-09T06:49:23.238 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi193 smithi193 running (8m) 2m ago 18m 6665k - 19.0.0-2515-g96805dcf d7acac7d5657 2b494cf8aaeb 2024-05-09T06:49:23.238 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (102s) 97s ago 19m 54.7M - 9.4.12 2bacad6d85d8 a8eca89f71c1 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.aoiibw smithi063 running (4m) 97s ago 14m 87.7M - 19.0.0-2515-g96805dcf d7acac7d5657 c7cf68a1a3e2 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.fbzxbf smithi063 running (4m) 97s ago 14m 84.5M - 19.0.0-2515-g96805dcf d7acac7d5657 c24cf6de2c0f 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.ggtmyb smithi193 running (3m) 2m ago 14m 13.8M - 19.0.0-2515-g96805dcf d7acac7d5657 b752027d6904 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi193.sygxnf smithi193 running (3m) 2m ago 14m 13.7M - 19.0.0-2515-g96805dcf d7acac7d5657 bf31d952d7fe 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.egfguy smithi063 *:8443,9283,8765 running (10m) 97s ago 21m 524M - 19.0.0-2515-g96805dcf d7acac7d5657 8302ed683b99 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi193.mmmqzh smithi193 *:8443,9283,8765 running (9m) 2m ago 18m 453M - 19.0.0-2515-g96805dcf d7acac7d5657 415a67121fb1 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (9m) 97s ago 21m 57.3M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 8b7491de89ee 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi193 smithi193 running (8m) 2m ago 18m 40.9M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 dafb35e76458 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (2m) 97s ago 19m 8284k - 1.5.0 0da6a335fe13 1f7eb6ee1981 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi193 smithi193 *:9100 running (2m) 2m ago 18m 5473k - 1.5.0 0da6a335fe13 21d4f1f51ed5 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (7m) 97s ago 17m 151M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 6e387840e5b7 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (7m) 97s ago 17m 124M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 f1a348429588 2024-05-09T06:49:23.239 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (6m) 97s ago 17m 99.3M 1216M 19.0.0-2515-g96805dcf d7acac7d5657 313cdd3a9867 2024-05-09T06:49:23.240 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi193 running (5m) 2m ago 16m 118M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 96afdd638dd5 2024-05-09T06:49:23.240 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi193 running (5m) 2m ago 16m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 14c9b4920f5a 2024-05-09T06:49:23.240 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi193 running (4m) 2m ago 15m 101M 2240M 19.0.0-2515-g96805dcf d7acac7d5657 d219b4bb599f 2024-05-09T06:49:23.240 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (2m) 97s ago 19m 47.2M - 2.43.0 a07b618ecd1d 6ee3f1928d73 2024-05-09T06:49:23.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f294c06e210 msgr2=0x7f294c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:23.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f294c06e210 0x7f294c0706d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f295c00b5c0 tx=0x7f295c01a040 comp rx=0 tx=0).stop 2024-05-09T06:49:23.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 msgr2=0x7f296019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:23.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 0x7f296019cd20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f295400c930 tx=0x7f295400ccf0 comp rx=0 tx=0).stop 2024-05-09T06:49:23.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 shutdown_connections 2024-05-09T06:49:23.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f294c06e210 0x7f294c0706d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:23.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29600ffab0 0x7f296019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:23.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 --2- 172.21.15.63:0/316183371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2960105ab0 0x7f296019cd20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:23.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 >> 172.21.15.63:0/316183371 conn(0x7f2960076750 msgr2=0x7f2960104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:23.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 shutdown_connections 2024-05-09T06:49:23.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:23.261+0000 7f29687ad700 1 -- 172.21.15.63:0/316183371 wait complete. 2024-05-09T06:49:23.805 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:23 smithi063 ceph-mon[92765]: pgmap v324: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:23.829 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph versions' 2024-05-09T06:49:24.051 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:49:24.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:23 smithi193 ceph-mon[81665]: pgmap v324: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:24.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:24 smithi063 ceph-mon[92765]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:49:25.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:24 smithi193 ceph-mon[81665]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:49:25.780 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:25 smithi063 ceph-mon[92765]: pgmap v325: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.801+0000 7f5ca39e0700 1 -- 172.21.15.63:0/3813764491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c100e30 msgr2=0x7f5c9c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:25.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.801+0000 7f5ca39e0700 1 --2- 172.21.15.63:0/3813764491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c100e30 0x7f5c9c101210 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009910 tx=0x7f5c98009c20 comp rx=0 tx=0).stop 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.801+0000 7f5ca39e0700 1 -- 172.21.15.63:0/3813764491 shutdown_connections 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.801+0000 7f5ca39e0700 1 --2- 172.21.15.63:0/3813764491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c1017e0 0x7f5c9c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.801+0000 7f5ca39e0700 1 --2- 172.21.15.63:0/3813764491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c100e30 0x7f5c9c101210 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.801+0000 7f5ca39e0700 1 -- 172.21.15.63:0/3813764491 >> 172.21.15.63:0/3813764491 conn(0x7f5c9c0fc6e0 msgr2=0x7f5c9c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.801+0000 7f5ca39e0700 1 -- 172.21.15.63:0/3813764491 shutdown_connections 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.801+0000 7f5ca39e0700 1 -- 172.21.15.63:0/3813764491 wait complete. 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca39e0700 1 Processor -- start 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca39e0700 1 -- start start 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca39e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c100e30 0x7f5c9c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca39e0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c1017e0 0x7f5c9c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca39e0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c199a90 con 0x7f5c9c1017e0 2024-05-09T06:49:25.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca39e0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c19d760 con 0x7f5c9c100e30 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca0f7b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c1017e0 0x7f5c9c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca0f7b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c1017e0 0x7f5c9c1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35338/0 (socket says 172.21.15.63:35338) 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca0f7b700 1 -- 172.21.15.63:0/2046958482 learned_addr learned my addr 172.21.15.63:0/2046958482 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.802+0000 7f5ca177c700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c100e30 0x7f5c9c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.803+0000 7f5ca0f7b700 1 -- 172.21.15.63:0/2046958482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c100e30 msgr2=0x7f5c9c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.803+0000 7f5ca0f7b700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c100e30 0x7f5c9c198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.803+0000 7f5ca0f7b700 1 -- 172.21.15.63:0/2046958482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c980095f0 con 0x7f5c9c1017e0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.803+0000 7f5ca0f7b700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c1017e0 0x7f5c9c1993b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00b480 tx=0x7f5c8c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.803+0000 7f5c927fc700 1 -- 172.21.15.63:0/2046958482 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c8c011800 con 0x7f5c9c1017e0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.803+0000 7f5c927fc700 1 -- 172.21.15.63:0/2046958482 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c8c011e20 con 0x7f5c9c1017e0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.803+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c19da40 con 0x7f5c9c1017e0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.803+0000 7f5c927fc700 1 -- 172.21.15.63:0/2046958482 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c8c010ea0 con 0x7f5c9c1017e0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.803+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c19df90 con 0x7f5c9c1017e0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.804+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c102310 con 0x7f5c9c1017e0 2024-05-09T06:49:25.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.805+0000 7f5c927fc700 1 -- 172.21.15.63:0/2046958482 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5c8c010380 con 0x7f5c9c1017e0 2024-05-09T06:49:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.805+0000 7f5c927fc700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5c8806e200 0x7f5c880706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.805+0000 7f5c927fc700 1 -- 172.21.15.63:0/2046958482 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f5c8c08de10 con 0x7f5c9c1017e0 2024-05-09T06:49:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.805+0000 7f5ca177c700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5c8806e200 0x7f5c880706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.806+0000 7f5ca177c700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5c8806e200 0x7f5c880706c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c98000c00 tx=0x7f5c9801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:25.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:25.810+0000 7f5c927fc700 1 -- 172.21.15.63:0/2046958482 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5c8c056f70 con 0x7f5c9c1017e0 2024-05-09T06:49:26.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.110+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5c9c04ebb0 con 0x7f5c9c1017e0 2024-05-09T06:49:26.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.111+0000 7f5c927fc700 1 -- 172.21.15.63:0/2046958482 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5c8c05a630 con 0x7f5c9c1017e0 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 2 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 6 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-05-09T06:49:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 4 2024-05-09T06:49:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T06:49:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-05-09T06:49:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)": 14 2024-05-09T06:49:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T06:49:26.091 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T06:49:26.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5c8806e200 msgr2=0x7f5c880706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:26.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5c8806e200 0x7f5c880706c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c98000c00 tx=0x7f5c9801a040 comp rx=0 tx=0).stop 2024-05-09T06:49:26.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c1017e0 msgr2=0x7f5c9c1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:26.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c1017e0 0x7f5c9c1993b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00b480 tx=0x7f5c8c00b790 comp rx=0 tx=0).stop 2024-05-09T06:49:26.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 shutdown_connections 2024-05-09T06:49:26.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5c8806e200 0x7f5c880706c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:26.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c9c100e30 0x7f5c9c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:26.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 --2- 172.21.15.63:0/2046958482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c9c1017e0 0x7f5c9c1993b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:26.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 >> 172.21.15.63:0/2046958482 conn(0x7f5c9c0fc6e0 msgr2=0x7f5c9c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:26.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 shutdown_connections 2024-05-09T06:49:26.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:26.113+0000 7f5ca39e0700 1 -- 172.21.15.63:0/2046958482 wait complete. 2024-05-09T06:49:26.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:25 smithi193 ceph-mon[81665]: pgmap v325: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:26.672 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:26 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2046958482' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:49:26.693 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-05-09T06:49:26.921 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:49:27.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:26 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2046958482' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:49:27.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:27 smithi063 ceph-mon[92765]: pgmap v326: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:27.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:49:28.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:27 smithi193 ceph-mon[81665]: pgmap v326: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:28.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:49:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.548+0000 7f5379f0c700 1 -- 172.21.15.63:0/214224075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 msgr2=0x7f537410aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.548+0000 7f5379f0c700 1 --2- 172.21.15.63:0/214224075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 0x7f537410aba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5368009910 tx=0x7f5368009c20 comp rx=0 tx=0).stop 2024-05-09T06:49:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.548+0000 7f5379f0c700 1 -- 172.21.15.63:0/214224075 shutdown_connections 2024-05-09T06:49:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.548+0000 7f5379f0c700 1 --2- 172.21.15.63:0/214224075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 0x7f537410aba0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.548+0000 7f5379f0c700 1 --2- 172.21.15.63:0/214224075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5374101d00 0x7f53741020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.548+0000 7f5379f0c700 1 -- 172.21.15.63:0/214224075 >> 172.21.15.63:0/214224075 conn(0x7f53740fb5e0 msgr2=0x7f53740fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.548+0000 7f5379f0c700 1 -- 172.21.15.63:0/214224075 shutdown_connections 2024-05-09T06:49:28.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.548+0000 7f5379f0c700 1 -- 172.21.15.63:0/214224075 wait complete. 2024-05-09T06:49:28.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.548+0000 7f5379f0c700 1 Processor -- start 2024-05-09T06:49:28.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5379f0c700 1 -- start start 2024-05-09T06:49:28.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5379f0c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5374101d00 0x7f537419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:28.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5379f0c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 0x7f537419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:28.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5379f0c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537419f5a0 con 0x7f53741026b0 2024-05-09T06:49:28.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5379f0c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5374198b80 con 0x7f5374101d00 2024-05-09T06:49:28.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5372ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 0x7f537419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:28.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5372ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 0x7f537419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35362/0 (socket says 172.21.15.63:35362) 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5372ffd700 1 -- 172.21.15.63:0/1505595032 learned_addr learned my addr 172.21.15.63:0/1505595032 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f53737fe700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5374101d00 0x7f537419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5372ffd700 1 -- 172.21.15.63:0/1505595032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5374101d00 msgr2=0x7f537419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5372ffd700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5374101d00 0x7f537419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.549+0000 7f5372ffd700 1 -- 172.21.15.63:0/1505595032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53680095f0 con 0x7f53741026b0 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.550+0000 7f5372ffd700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 0x7f537419ef80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f536800bd70 tx=0x7f536800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.550+0000 7f5370ff9700 1 -- 172.21.15.63:0/1505595032 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f536801d070 con 0x7f53741026b0 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.550+0000 7f5370ff9700 1 -- 172.21.15.63:0/1505595032 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53680056e0 con 0x7f53741026b0 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.550+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5374198e00 con 0x7f53741026b0 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.550+0000 7f5370ff9700 1 -- 172.21.15.63:0/1505595032 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f536800fe50 con 0x7f53741026b0 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.550+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53741992f0 con 0x7f53741026b0 2024-05-09T06:49:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.551+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537410a700 con 0x7f53741026b0 2024-05-09T06:49:28.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.551+0000 7f5370ff9700 1 -- 172.21.15.63:0/1505595032 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f536800f680 con 0x7f53741026b0 2024-05-09T06:49:28.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.552+0000 7f5370ff9700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f535c06e200 0x7f535c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:28.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.552+0000 7f5370ff9700 1 -- 172.21.15.63:0/1505595032 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f536808f6f0 con 0x7f53741026b0 2024-05-09T06:49:28.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.552+0000 7f53737fe700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f535c06e200 0x7f535c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:28.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.552+0000 7f53737fe700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f535c06e200 0x7f535c0706c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5364007be0 tx=0x7f53640073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:28.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.557+0000 7f5370ff9700 1 -- 172.21.15.63:0/1505595032 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f536805c2e0 con 0x7f53741026b0 2024-05-09T06:49:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.853+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5374069a00 con 0x7f53741026b0 2024-05-09T06:49:28.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.854+0000 7f5370ff9700 1 -- 172.21.15.63:0/1505595032 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f536805be70 con 0x7f53741026b0 2024-05-09T06:49:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f535c06e200 msgr2=0x7f535c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f535c06e200 0x7f535c0706c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5364007be0 tx=0x7f53640073d0 comp rx=0 tx=0).stop 2024-05-09T06:49:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 msgr2=0x7f537419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 0x7f537419ef80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f536800bd70 tx=0x7f536800bda0 comp rx=0 tx=0).stop 2024-05-09T06:49:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 shutdown_connections 2024-05-09T06:49:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f535c06e200 0x7f535c0706c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5374101d00 0x7f537419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 --2- 172.21.15.63:0/1505595032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53741026b0 0x7f537419ef80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:28.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 >> 172.21.15.63:0/1505595032 conn(0x7f53740fb5e0 msgr2=0x7f53741011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 shutdown_connections 2024-05-09T06:49:28.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:28.856+0000 7f5379f0c700 1 -- 172.21.15.63:0/1505595032 wait complete. 2024-05-09T06:49:28.845 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-05-09T06:49:29.423 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -e sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-05-09T06:49:29.641 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:49:29.765 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:29 smithi063 ceph-mon[92765]: pgmap v327: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:29.765 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:29 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1505595032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:49:30.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:29 smithi193 ceph-mon[81665]: pgmap v327: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:30.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:29 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1505595032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:49:31.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.338+0000 7f637ead4700 1 -- 172.21.15.63:0/2497777163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 msgr2=0x7f63780fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.338+0000 7f637ead4700 1 --2- 172.21.15.63:0/2497777163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 0x7f63780fff60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6360009910 tx=0x7f6360009c20 comp rx=0 tx=0).stop 2024-05-09T06:49:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.338+0000 7f637ead4700 1 -- 172.21.15.63:0/2497777163 shutdown_connections 2024-05-09T06:49:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.338+0000 7f637ead4700 1 --2- 172.21.15.63:0/2497777163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 0x7f63780fff60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.338+0000 7f637ead4700 1 --2- 172.21.15.63:0/2497777163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6378105b70 0x7f6378105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.338+0000 7f637ead4700 1 -- 172.21.15.63:0/2497777163 >> 172.21.15.63:0/2497777163 conn(0x7f63780fb610 msgr2=0x7f63780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.338+0000 7f637ead4700 1 -- 172.21.15.63:0/2497777163 shutdown_connections 2024-05-09T06:49:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.338+0000 7f637ead4700 1 -- 172.21.15.63:0/2497777163 wait complete. 2024-05-09T06:49:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.339+0000 7f637ead4700 1 Processor -- start 2024-05-09T06:49:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.339+0000 7f637ead4700 1 -- start start 2024-05-09T06:49:31.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.339+0000 7f637ead4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 0x7f63780fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:31.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.339+0000 7f637ead4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6378105b70 0x7f63780fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:31.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.339+0000 7f637ead4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780fc820 con 0x7f63780ffae0 2024-05-09T06:49:31.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.339+0000 7f637ead4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780fc960 con 0x7f6378105b70 2024-05-09T06:49:31.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.339+0000 7f637c870700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 0x7f63780fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:31.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.339+0000 7f637c870700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 0x7f63780fbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35384/0 (socket says 172.21.15.63:35384) 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.339+0000 7f637c870700 1 -- 172.21.15.63:0/3191974803 learned_addr learned my addr 172.21.15.63:0/3191974803 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f6377fff700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6378105b70 0x7f63780fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f637c870700 1 -- 172.21.15.63:0/3191974803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6378105b70 msgr2=0x7f63780fc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f637c870700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6378105b70 0x7f63780fc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f637c870700 1 -- 172.21.15.63:0/3191974803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600095f0 con 0x7f63780ffae0 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f6377fff700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6378105b70 0x7f63780fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f637c870700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 0x7f63780fbda0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f63680060b0 tx=0x7f636800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f6375ffb700 1 -- 172.21.15.63:0/3191974803 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f636800dda0 con 0x7f63780ffae0 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f6375ffb700 1 -- 172.21.15.63:0/3191974803 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f636800f040 con 0x7f63780ffae0 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63781a2d00 con 0x7f63780ffae0 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f6375ffb700 1 -- 172.21.15.63:0/3191974803 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6368014790 con 0x7f63780ffae0 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.340+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63781a3060 con 0x7f63780ffae0 2024-05-09T06:49:31.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.342+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6378109110 con 0x7f63780ffae0 2024-05-09T06:49:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.342+0000 7f6375ffb700 1 -- 172.21.15.63:0/3191974803 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6368014980 con 0x7f63780ffae0 2024-05-09T06:49:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.342+0000 7f6375ffb700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f636406e200 0x7f63640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.343+0000 7f6375ffb700 1 -- 172.21.15.63:0/3191974803 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f636808dd60 con 0x7f63780ffae0 2024-05-09T06:49:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.343+0000 7f6377fff700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f636406e200 0x7f63640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:31.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.343+0000 7f6377fff700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f636406e200 0x7f63640706c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f636000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:31.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.347+0000 7f6375ffb700 1 -- 172.21.15.63:0/3191974803 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6368056f70 con 0x7f63780ffae0 2024-05-09T06:49:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.648+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f637810cce0 con 0x7f63780ffae0 2024-05-09T06:49:31.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.649+0000 7f6375ffb700 1 -- 172.21.15.63:0/3191974803 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f636805a630 con 0x7f63780ffae0 2024-05-09T06:49:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f636406e200 msgr2=0x7f63640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:31.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f636406e200 0x7f63640706c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f636000b540 comp rx=0 tx=0).stop 2024-05-09T06:49:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 msgr2=0x7f63780fbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 0x7f63780fbda0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f63680060b0 tx=0x7f636800d3b0 comp rx=0 tx=0).stop 2024-05-09T06:49:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 shutdown_connections 2024-05-09T06:49:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f636406e200 0x7f63640706c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63780ffae0 0x7f63780fbda0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 --2- 172.21.15.63:0/3191974803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6378105b70 0x7f63780fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 >> 172.21.15.63:0/3191974803 conn(0x7f63780fb610 msgr2=0x7f6378069660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 shutdown_connections 2024-05-09T06:49:31.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:31.651+0000 7f637ead4700 1 -- 172.21.15.63:0/3191974803 wait complete. 2024-05-09T06:49:31.640 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)" 2024-05-09T06:49:31.893 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:31 smithi063 ceph-mon[92765]: pgmap v328: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:31.893 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:31 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3191974803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:49:31.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:31 smithi193 ceph-mon[81665]: pgmap v328: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:31.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:31 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3191974803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-09T06:49:32.247 DEBUG:teuthology.parallel:result is None 2024-05-09T06:49:32.248 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T06:49:32.260 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:49:32.260 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T06:49:32.478 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/mon.smithi063/config 2024-05-09T06:49:33.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:33 smithi063 ceph-mon[92765]: pgmap v329: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:33.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.996+0000 7f2873ff6700 1 -- 172.21.15.63:0/1539658655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 msgr2=0x7f286c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:33.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.996+0000 7f2873ff6700 1 --2- 172.21.15.63:0/1539658655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 0x7f286c101210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2868009910 tx=0x7f2868009c20 comp rx=0 tx=0).stop 2024-05-09T06:49:33.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.996+0000 7f2873ff6700 1 -- 172.21.15.63:0/1539658655 shutdown_connections 2024-05-09T06:49:33.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.996+0000 7f2873ff6700 1 --2- 172.21.15.63:0/1539658655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f286c1017e0 0x7f286c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:33.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.996+0000 7f2873ff6700 1 --2- 172.21.15.63:0/1539658655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 0x7f286c101210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:33.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.996+0000 7f2873ff6700 1 -- 172.21.15.63:0/1539658655 >> 172.21.15.63:0/1539658655 conn(0x7f286c0fc6e0 msgr2=0x7f286c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:33.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.996+0000 7f2873ff6700 1 -- 172.21.15.63:0/1539658655 shutdown_connections 2024-05-09T06:49:33.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.996+0000 7f2873ff6700 1 -- 172.21.15.63:0/1539658655 wait complete. 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.997+0000 7f2873ff6700 1 Processor -- start 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.997+0000 7f2873ff6700 1 -- start start 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.997+0000 7f2873ff6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 0x7f286c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2873ff6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f286c1017e0 0x7f286c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2873ff6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c19d330 con 0x7f286c100e30 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2873ff6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c1969a0 con 0x7f286c1017e0 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2871d92700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 0x7f286c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2871d92700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 0x7f286c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35400/0 (socket says 172.21.15.63:35400) 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2871d92700 1 -- 172.21.15.63:0/2799502423 learned_addr learned my addr 172.21.15.63:0/2799502423 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2871591700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f286c1017e0 0x7f286c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2871d92700 1 -- 172.21.15.63:0/2799502423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f286c1017e0 msgr2=0x7f286c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:33.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2871d92700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f286c1017e0 0x7f286c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:33.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2871d92700 1 -- 172.21.15.63:0/2799502423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680095f0 con 0x7f286c100e30 2024-05-09T06:49:33.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2871d92700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 0x7f286c19c860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28680092e0 tx=0x7f286800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:33.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.998+0000 7f2862ffd700 1 -- 172.21.15.63:0/2799502423 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286801d070 con 0x7f286c100e30 2024-05-09T06:49:33.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.999+0000 7f2862ffd700 1 -- 172.21.15.63:0/2799502423 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f286800f460 con 0x7f286c100e30 2024-05-09T06:49:33.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.999+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c196c20 con 0x7f286c100e30 2024-05-09T06:49:33.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.999+0000 7f2862ffd700 1 -- 172.21.15.63:0/2799502423 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2868017630 con 0x7f286c100e30 2024-05-09T06:49:33.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:33.999+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c197110 con 0x7f286c100e30 2024-05-09T06:49:33.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.000+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286c1091d0 con 0x7f286c100e30 2024-05-09T06:49:33.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.000+0000 7f2862ffd700 1 -- 172.21.15.63:0/2799502423 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f28680178a0 con 0x7f286c100e30 2024-05-09T06:49:33.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.001+0000 7f2862ffd700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f285806e210 0x7f28580706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:33.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.001+0000 7f2862ffd700 1 -- 172.21.15.63:0/2799502423 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f286808f6f0 con 0x7f286c100e30 2024-05-09T06:49:33.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.001+0000 7f2871591700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f285806e210 0x7f28580706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:33.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.001+0000 7f2871591700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f285806e210 0x7f28580706d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f286c197b60 tx=0x7f285c00a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:33.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.006+0000 7f2862ffd700 1 -- 172.21.15.63:0/2799502423 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f286805c470 con 0x7f286c100e30 2024-05-09T06:49:34.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:33 smithi193 ceph-mon[81665]: pgmap v329: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:34.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.260+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f286c197e60 con 0x7f286c100e30 2024-05-09T06:49:34.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.260+0000 7f2862ffd700 1 -- 172.21.15.63:0/2799502423 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1963 (secure 0 0 0) 0x7f286805c000 con 0x7f286c100e30 2024-05-09T06:49:34.239 INFO:teuthology.orchestra.run.smithi063.stdout:e34 2024-05-09T06:49:34.239 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T06:49:34.239 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-09T06:49:34.239 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-09T06:49:34.239 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 34 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T06:35:03.455858+0000 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-09T06:46:08.230738+0000 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-09T06:49:34.240 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 96 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:in 0 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34406} 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-05-09T06:49:34.241 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 34406 members: 34406 2024-05-09T06:49:34.242 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.aoiibw{0:34406} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.63:6826/566601990,v1:172.21.15.63:6827/566601990] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:49:34.242 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.fbzxbf{0:34410} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/2182150991,v1:172.21.15.63:6829/2182150991] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:49:34.242 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:34.242 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:34.242 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-09T06:49:34.242 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:34.242 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.ggtmyb{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/2007543899,v1:172.21.15.193:6825/2007543899] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:49:34.242 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi193.sygxnf{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/790240017,v1:172.21.15.193:6827/790240017] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f285806e210 msgr2=0x7f28580706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f285806e210 0x7f28580706d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f286c197b60 tx=0x7f285c00a380 comp rx=0 tx=0).stop 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 msgr2=0x7f286c19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 0x7f286c19c860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28680092e0 tx=0x7f286800bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 shutdown_connections 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f285806e210 0x7f28580706d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f286c100e30 0x7f286c19c860 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 --2- 172.21.15.63:0/2799502423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f286c1017e0 0x7f286c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 >> 172.21.15.63:0/2799502423 conn(0x7f286c0fc6e0 msgr2=0x7f286c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 shutdown_connections 2024-05-09T06:49:34.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:34.262+0000 7f2873ff6700 1 -- 172.21.15.63:0/2799502423 wait complete. 2024-05-09T06:49:34.244 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 34 2024-05-09T06:49:34.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:34 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2799502423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:49:34.885 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-05-09T06:49:34.896 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:49:35.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.024+0000 7f102f9dd640 1 -- 172.21.15.63:0/249466330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 msgr2=0x7f1028101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.024+0000 7f102f9dd640 1 --2- 172.21.15.63:0/249466330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 0x7f1028101640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1024009880 tx=0x7f102402bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:35.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.024+0000 7f102f9dd640 1 -- 172.21.15.63:0/249466330 shutdown_connections 2024-05-09T06:49:35.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.024+0000 7f102f9dd640 1 --2- 172.21.15.63:0/249466330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028101b80 0x7f102810a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.024+0000 7f102f9dd640 1 --2- 172.21.15.63:0/249466330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 0x7f1028101640 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.024+0000 7f102f9dd640 1 -- 172.21.15.63:0/249466330 >> 172.21.15.63:0/249466330 conn(0x7f1028075c40 msgr2=0x7f10280fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:35.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.024+0000 7f102f9dd640 1 -- 172.21.15.63:0/249466330 shutdown_connections 2024-05-09T06:49:35.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.024+0000 7f102f9dd640 1 -- 172.21.15.63:0/249466330 wait complete. 2024-05-09T06:49:35.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102f9dd640 1 Processor -- start 2024-05-09T06:49:35.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102f9dd640 1 -- start start 2024-05-09T06:49:35.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102f9dd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 0x7f1028196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102f9dd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028101b80 0x7f1028196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102f9dd640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028196cd0 con 0x7f1028101260 2024-05-09T06:49:35.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102f9dd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102819a9e0 con 0x7f1028101b80 2024-05-09T06:49:35.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102d779640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 0x7f1028196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102d779640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 0x7f1028196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40024/0 (socket says 172.21.15.63:40024) 2024-05-09T06:49:35.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102d779640 1 -- 172.21.15.63:0/263776603 learned_addr learned my addr 172.21.15.63:0/263776603 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:35.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102cf78640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028101b80 0x7f1028196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102d779640 1 -- 172.21.15.63:0/263776603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028101b80 msgr2=0x7f1028196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102d779640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028101b80 0x7f1028196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.025+0000 7f102d779640 1 -- 172.21.15.63:0/263776603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1024009560 con 0x7f1028101260 2024-05-09T06:49:35.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.026+0000 7f102d779640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 0x7f1028196100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f102402c2b0 tx=0x7f102402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:35.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.026+0000 7f101e7fc640 1 -- 172.21.15.63:0/263776603 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1024004520 con 0x7f1028101260 2024-05-09T06:49:35.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.026+0000 7f101e7fc640 1 -- 172.21.15.63:0/263776603 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1024003ab0 con 0x7f1028101260 2024-05-09T06:49:35.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.026+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102819ac60 con 0x7f1028101260 2024-05-09T06:49:35.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.026+0000 7f101e7fc640 1 -- 172.21.15.63:0/263776603 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1024005500 con 0x7f1028101260 2024-05-09T06:49:35.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.026+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102819b1c0 con 0x7f1028101260 2024-05-09T06:49:35.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.027+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1028067c80 con 0x7f1028101260 2024-05-09T06:49:35.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.027+0000 7f101e7fc640 1 -- 172.21.15.63:0/263776603 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1024003c20 con 0x7f1028101260 2024-05-09T06:49:35.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.028+0000 7f101e7fc640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f101406e170 0x7f1014070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.028+0000 7f101e7fc640 1 -- 172.21.15.63:0/263776603 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f10240b0ce0 con 0x7f1028101260 2024-05-09T06:49:35.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.028+0000 7f102cf78640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f101406e170 0x7f1014070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.028+0000 7f102cf78640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f101406e170 0x7f1014070630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1018005fd0 tx=0x7f1018009510 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:35.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.033+0000 7f101e7fc640 1 -- 172.21.15.63:0/263776603 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1024079ec0 con 0x7f1028101260 2024-05-09T06:49:35.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:34 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2799502423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T06:49:35.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.219+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10281048a0 con 0x7f1028101260 2024-05-09T06:49:35.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.220+0000 7f101e7fc640 1 -- 172.21.15.63:0/263776603 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17332 (secure 0 0 0) 0x7f102407d580 con 0x7f1028101260 2024-05-09T06:49:35.200 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:35.200 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":96,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","created":"2024-05-09T06:27:53.398729+0000","modified":"2024-05-09T06:45:51.753800+0000","last_up_change":"2024-05-09T06:44:43.058485+0000","last_in_change":"2024-05-09T06:33:25.853191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:32:32.042216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-09T06:35:02.184376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-09T06:35:02.871327+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eea045d9-973d-4960-a34e-689f9403d55f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":89,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6803","nonce":1522382249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6805","nonce":1522382249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6809","nonce":1522382249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6807","nonce":1522382249}]},"public_addr":"172.21.15.63:6803/1522382249","cluster_addr":"172.21.15.63:6805/1522382249","heartbeat_back_addr":"172.21.15.63:6809/1522382249","heartbeat_front_addr":"172.21.15.63:6807/1522382249","state":["exists","up"]},{"osd":1,"uuid":"da187f97-762d-40d3-94d0-fef449ce1dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":66,"up_thru":89,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6811","nonce":2256761098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6813","nonce":2256761098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6817","nonce":2256761098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6815","nonce":2256761098}]},"public_addr":"172.21.15.63:6811/2256761098","cluster_addr":"172.21.15.63:6813/2256761098","heartbeat_back_addr":"172.21.15.63:6817/2256761098","heartbeat_front_addr":"172.21.15.63:6815/2256761098","state":["exists","up"]},{"osd":2,"uuid":"6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":70,"up_from":74,"up_thru":89,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":616478},{"type":"v1","addr":"172.21.15.63:6819","nonce":616478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":616478},{"type":"v1","addr":"172.21.15.63:6821","nonce":616478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":616478},{"type":"v1","addr":"172.21.15.63:6825","nonce":616478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":616478},{"type":"v1","addr":"172.21.15.63:6823","nonce":616478}]},"public_addr":"172.21.15.63:6819/616478","cluster_addr":"172.21.15.63:6821/616478","heartbeat_back_addr":"172.21.15.63:6825/616478","heartbeat_front_addr":"172.21.15.63:6823/616478","state":["exists","up"]},{"osd":3,"uuid":"ff1a9d2a-4481-4e61-aaa5-e5046f3c0055","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":75,"up_from":79,"up_thru":89,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6801","nonce":2657331774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6803","nonce":2657331774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6807","nonce":2657331774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6805","nonce":2657331774}]},"public_addr":"172.21.15.193:6801/2657331774","cluster_addr":"172.21.15.193:6803/2657331774","heartbeat_back_addr":"172.21.15.193:6807/2657331774","heartbeat_front_addr":"172.21.15.193:6805/2657331774","state":["exists","up"]},{"osd":4,"uuid":"23f0fabc-4f38-438c-a691-307209bb0f31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":80,"up_from":84,"up_thru":89,"down_at":81,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6809","nonce":909880686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6811","nonce":909880686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6815","nonce":909880686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6813","nonce":909880686}]},"public_addr":"172.21.15.193:6809/909880686","cluster_addr":"172.21.15.193:6811/909880686","heartbeat_back_addr":"172.21.15.193:6815/909880686","heartbeat_front_addr":"172.21.15.193:6813/909880686","state":["exists","up"]},{"osd":5,"uuid":"eb6226de-63c7-4200-8978-4ee9491c1427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":85,"up_from":89,"up_thru":89,"down_at":86,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6817","nonce":2078959605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6819","nonce":2078959605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6823","nonce":2078959605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6821","nonce":2078959605}]},"public_addr":"172.21.15.193:6817/2078959605","cluster_addr":"172.21.15.193:6819/2078959605","heartbeat_back_addr":"172.21.15.193:6823/2078959605","heartbeat_front_addr":"172.21.15.193:6821/2078959605","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:32.202943+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:59.919211+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:28.132162+0000","dead_epoch":70},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:52.038182+0000","dead_epoch":75},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:17.334991+0000","dead_epoch":80},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:43.406303+0000","dead_epoch":85}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":0,"to":2}]},{"pgid":"2.9","mappings":[{"from":0,"to":2}]},{"pgid":"2.18","mappings":[{"from":3,"to":1}]},{"pgid":"2.19","mappings":[{"from":0,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"2.1d","mappings":[{"from":3,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":0}]},{"pgid":"3.4","mappings":[{"from":3,"to":0}]},{"pgid":"3.7","mappings":[{"from":4,"to":5}]},{"pgid":"3.b","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6827/354024957":"2024-05-10T06:45:51.753775+0000","172.21.15.193:6826/354024957":"2024-05-10T06:45:51.753775+0000","172.21.15.193:6824/296432349":"2024-05-10T06:45:26.766385+0000","172.21.15.63:0/2992018924":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/1538574058":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/2580859343":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/4061231873":"2024-05-10T06:29:03.468046+0000","172.21.15.193:0/3682498688":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6801/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/2209960091":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/576235286":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/750127211":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/1465547870":"2024-05-10T06:30:11.518272+0000","172.21.15.193:6825/296432349":"2024-05-10T06:45:26.766385+0000","172.21.15.63:0/1507850411":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/4049710905":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/3349953626":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6826/1318147125":"2024-05-10T06:35:21.124789+0000","172.21.15.63:0/1971919126":"2024-05-10T06:28:29.800161+0000","172.21.15.63:6800/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.193:0/3815434278":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6800/1655504434":"2024-05-10T06:40:12.081321+0000","172.21.15.193:0/3760737819":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6801/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6800/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/452783656":"2024-05-10T06:37:59.346754+0000","172.21.15.63:6827/1318147125":"2024-05-10T06:35:21.124789+0000","172.21.15.63:6801/4130975635":"2024-05-10T06:37:59.346754+0000","172.21.15.193:6828/1432929717":"2024-05-10T06:39:11.862568+0000","172.21.15.193:0/389588125":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6801/1655504434":"2024-05-10T06:40:12.081321+0000","172.21.15.63:6801/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/3728039632":"2024-05-10T06:28:29.800161+0000","172.21.15.193:6829/1432929717":"2024-05-10T06:39:11.862568+0000","172.21.15.63:0/4215281381":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/494791920":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.193:0/398074734":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6800/4130975635":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/2254623220":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/2340959335":"2024-05-10T06:40:12.081321+0000","172.21.15.193:0/3446861232":"2024-05-10T06:39:11.862568+0000","172.21.15.63:0/2926131890":"2024-05-10T06:40:12.081321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f101406e170 msgr2=0x7f1014070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f101406e170 0x7f1014070630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1018005fd0 tx=0x7f1018009510 comp rx=0 tx=0).stop 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 msgr2=0x7f1028196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 0x7f1028196100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f102402c2b0 tx=0x7f102402c850 comp rx=0 tx=0).stop 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 shutdown_connections 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f101406e170 0x7f1014070630 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1028101b80 0x7f1028196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 --2- 172.21.15.63:0/263776603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1028101260 0x7f1028196100 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 >> 172.21.15.63:0/263776603 conn(0x7f1028075c40 msgr2=0x7f1028102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:35.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 shutdown_connections 2024-05-09T06:49:35.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.223+0000 7f102f9dd640 1 -- 172.21.15.63:0/263776603 wait complete. 2024-05-09T06:49:35.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:32:32.042216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-09T06:35:02.184376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-09T06:35:02.871327+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:49:35.214 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:49:35.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.343+0000 7fd040537640 1 -- 172.21.15.63:0/310917012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0380ff220 msgr2=0x7fd0380ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.343+0000 7fd040537640 1 --2- 172.21.15.63:0/310917012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0380ff220 0x7fd0380ff600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd034009880 tx=0x7fd03402bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.343+0000 7fd040537640 1 -- 172.21.15.63:0/310917012 shutdown_connections 2024-05-09T06:49:35.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.343+0000 7fd040537640 1 --2- 172.21.15.63:0/310917012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0380ffbd0 0x7fd038103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.343+0000 7fd040537640 1 --2- 172.21.15.63:0/310917012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0380ff220 0x7fd0380ff600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.343+0000 7fd040537640 1 -- 172.21.15.63:0/310917012 >> 172.21.15.63:0/310917012 conn(0x7fd0380faac0 msgr2=0x7fd0380fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:35.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.343+0000 7fd040537640 1 -- 172.21.15.63:0/310917012 shutdown_connections 2024-05-09T06:49:35.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.343+0000 7fd040537640 1 -- 172.21.15.63:0/310917012 wait complete. 2024-05-09T06:49:35.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd040537640 1 Processor -- start 2024-05-09T06:49:35.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd040537640 1 -- start start 2024-05-09T06:49:35.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd040537640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0380ff220 0x7fd038195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd040537640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd038195590 0x7fd038199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd040537640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038195b00 con 0x7fd038195590 2024-05-09T06:49:35.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd040537640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038195c70 con 0x7fd0380ff220 2024-05-09T06:49:35.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd03dad2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd038195590 0x7fd038199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd03dad2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd038195590 0x7fd038199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40040/0 (socket says 172.21.15.63:40040) 2024-05-09T06:49:35.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd03dad2640 1 -- 172.21.15.63:0/3482600394 learned_addr learned my addr 172.21.15.63:0/3482600394 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:35.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd03e2d3640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0380ff220 0x7fd038195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd03dad2640 1 -- 172.21.15.63:0/3482600394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0380ff220 msgr2=0x7fd038195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd03dad2640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0380ff220 0x7fd038195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd03dad2640 1 -- 172.21.15.63:0/3482600394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd034009560 con 0x7fd038195590 2024-05-09T06:49:35.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.344+0000 7fd03dad2640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd038195590 0x7fd038199930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c8f0 tx=0x7fd02800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:35.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.345+0000 7fd02f7fe640 1 -- 172.21.15.63:0/3482600394 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd028004410 con 0x7fd038195590 2024-05-09T06:49:35.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.345+0000 7fd02f7fe640 1 -- 172.21.15.63:0/3482600394 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd028007c60 con 0x7fd038195590 2024-05-09T06:49:35.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.345+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd038199ed0 con 0x7fd038195590 2024-05-09T06:49:35.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.345+0000 7fd02f7fe640 1 -- 172.21.15.63:0/3482600394 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd02800f450 con 0x7fd038195590 2024-05-09T06:49:35.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.345+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03819a3e0 con 0x7fd038195590 2024-05-09T06:49:35.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.346+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0381006f0 con 0x7fd038195590 2024-05-09T06:49:35.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.346+0000 7fd02f7fe640 1 -- 172.21.15.63:0/3482600394 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd0280077a0 con 0x7fd038195590 2024-05-09T06:49:35.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.347+0000 7fd02f7fe640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd02406e180 0x7fd024070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.347+0000 7fd02f7fe640 1 -- 172.21.15.63:0/3482600394 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fd02808e240 con 0x7fd038195590 2024-05-09T06:49:35.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.347+0000 7fd03e2d3640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd02406e180 0x7fd024070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.347+0000 7fd03e2d3640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd02406e180 0x7fd024070640 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd034006010 tx=0x7fd034000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:35.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.351+0000 7fd02f7fe640 1 -- 172.21.15.63:0/3482600394 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd028057420 con 0x7fd038195590 2024-05-09T06:49:35.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.536+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd038107520 con 0x7fd038195590 2024-05-09T06:49:35.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.536+0000 7fd02f7fe640 1 -- 172.21.15.63:0/3482600394 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd02805aae0 con 0x7fd038195590 2024-05-09T06:49:35.515 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-09T06:49:35.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd02406e180 msgr2=0x7fd024070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd02406e180 0x7fd024070640 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd034006010 tx=0x7fd034000bc0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd038195590 msgr2=0x7fd038199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd038195590 0x7fd038199930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c8f0 tx=0x7fd02800cdb0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 shutdown_connections 2024-05-09T06:49:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd02406e180 0x7fd024070640 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd038195590 0x7fd038199930 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 --2- 172.21.15.63:0/3482600394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0380ff220 0x7fd038195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 >> 172.21.15.63:0/3482600394 conn(0x7fd0380faac0 msgr2=0x7fd0380fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 shutdown_connections 2024-05-09T06:49:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.538+0000 7fd040537640 1 -- 172.21.15.63:0/3482600394 wait complete. 2024-05-09T06:49:35.529 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-09T06:49:35.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 -- 172.21.15.63:0/3573191303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ffc50 msgr2=0x7f51b0103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 --2- 172.21.15.63:0/3573191303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ffc50 0x7f51b0103be0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009880 tx=0x7f51a402bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 -- 172.21.15.63:0/3573191303 shutdown_connections 2024-05-09T06:49:35.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 --2- 172.21.15.63:0/3573191303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ffc50 0x7f51b0103be0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 --2- 172.21.15.63:0/3573191303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b00ff2a0 0x7f51b00ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 -- 172.21.15.63:0/3573191303 >> 172.21.15.63:0/3573191303 conn(0x7f51b0074aa0 msgr2=0x7f51b0074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:35.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 -- 172.21.15.63:0/3573191303 shutdown_connections 2024-05-09T06:49:35.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 -- 172.21.15.63:0/3573191303 wait complete. 2024-05-09T06:49:35.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 Processor -- start 2024-05-09T06:49:35.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.658+0000 7f51b67e7640 1 -- start start 2024-05-09T06:49:35.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51b67e7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ff2a0 0x7f51b0196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51b67e7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b01974f0 0x7f51b0197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51b67e7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0068370 con 0x7f51b00ff2a0 2024-05-09T06:49:35.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51b67e7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b01a0020 con 0x7f51b01974f0 2024-05-09T06:49:35.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51affff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ff2a0 0x7f51b0196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51affff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ff2a0 0x7f51b0196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40072/0 (socket says 172.21.15.63:40072) 2024-05-09T06:49:35.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51affff640 1 -- 172.21.15.63:0/3078902637 learned_addr learned my addr 172.21.15.63:0/3078902637 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:35.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51af7fe640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b01974f0 0x7f51b0197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51affff640 1 -- 172.21.15.63:0/3078902637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b01974f0 msgr2=0x7f51b0197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51affff640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b01974f0 0x7f51b0197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51affff640 1 -- 172.21.15.63:0/3078902637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4009560 con 0x7f51b00ff2a0 2024-05-09T06:49:35.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51af7fe640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b01974f0 0x7f51b0197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:49:35.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.659+0000 7f51affff640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ff2a0 0x7f51b0196fb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f51a000d900 tx=0x7f51a000ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:35.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.660+0000 7f51ad7fa640 1 -- 172.21.15.63:0/3078902637 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a0009c00 con 0x7f51b00ff2a0 2024-05-09T06:49:35.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.660+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b01a0280 con 0x7f51b00ff2a0 2024-05-09T06:49:35.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.660+0000 7f51ad7fa640 1 -- 172.21.15.63:0/3078902637 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51a000b660 con 0x7f51b00ff2a0 2024-05-09T06:49:35.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.660+0000 7f51ad7fa640 1 -- 172.21.15.63:0/3078902637 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a000f680 con 0x7f51b00ff2a0 2024-05-09T06:49:35.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.660+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b01a07e0 con 0x7f51b00ff2a0 2024-05-09T06:49:35.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.660+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b01006c0 con 0x7f51b00ff2a0 2024-05-09T06:49:35.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.661+0000 7f51ad7fa640 1 -- 172.21.15.63:0/3078902637 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f51a0010ab0 con 0x7f51b00ff2a0 2024-05-09T06:49:35.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.661+0000 7f51ad7fa640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f519806e170 0x7f5198070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.662+0000 7f51ad7fa640 1 -- 172.21.15.63:0/3078902637 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f51a008e130 con 0x7f51b00ff2a0 2024-05-09T06:49:35.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.662+0000 7f51af7fe640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f519806e170 0x7f5198070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.662+0000 7f51af7fe640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f519806e170 0x7f5198070630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006010 tx=0x7f51a4005a20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:35.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.666+0000 7f51ad7fa640 1 -- 172.21.15.63:0/3078902637 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f51a0057290 con 0x7f51b00ff2a0 2024-05-09T06:49:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.848+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f51b004e160 con 0x7f51b00ff2a0 2024-05-09T06:49:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.849+0000 7f51ad7fa640 1 -- 172.21.15.63:0/3078902637 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f51a005a950 con 0x7f51b00ff2a0 2024-05-09T06:49:35.828 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-09T06:49:35.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f519806e170 msgr2=0x7f5198070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f519806e170 0x7f5198070630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006010 tx=0x7f51a4005a20 comp rx=0 tx=0).stop 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ff2a0 msgr2=0x7f51b0196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ff2a0 0x7f51b0196fb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f51a000d900 tx=0x7f51a000ddc0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 shutdown_connections 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f519806e170 0x7f5198070630 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b01974f0 0x7f51b0197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 --2- 172.21.15.63:0/3078902637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51b00ff2a0 0x7f51b0196fb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 >> 172.21.15.63:0/3078902637 conn(0x7f51b0074aa0 msgr2=0x7f51b0102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 shutdown_connections 2024-05-09T06:49:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.851+0000 7f51b67e7640 1 -- 172.21.15.63:0/3078902637 wait complete. 2024-05-09T06:49:35.839 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:35 smithi063 ceph-mon[92765]: pgmap v330: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:35.840 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:35 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/263776603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:49:35.840 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:35 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3482600394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:49:35.841 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-09T06:49:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.969+0000 7f09809dd640 1 -- 172.21.15.63:0/1169408061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 msgr2=0x7f097c10a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.969+0000 7f09809dd640 1 --2- 172.21.15.63:0/1169408061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 0x7f097c10a070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f096c009880 tx=0x7f096c02bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.969+0000 7f09809dd640 1 -- 172.21.15.63:0/1169408061 shutdown_connections 2024-05-09T06:49:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.969+0000 7f09809dd640 1 --2- 172.21.15.63:0/1169408061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 0x7f097c10a070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.969+0000 7f09809dd640 1 --2- 172.21.15.63:0/1169408061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f097c101270 0x7f097c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.969+0000 7f09809dd640 1 -- 172.21.15.63:0/1169408061 >> 172.21.15.63:0/1169408061 conn(0x7f097c0faa90 msgr2=0x7f097c0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.969+0000 7f09809dd640 1 -- 172.21.15.63:0/1169408061 shutdown_connections 2024-05-09T06:49:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.969+0000 7f09809dd640 1 -- 172.21.15.63:0/1169408061 wait complete. 2024-05-09T06:49:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f09809dd640 1 Processor -- start 2024-05-09T06:49:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f09809dd640 1 -- start start 2024-05-09T06:49:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f09809dd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f097c101270 0x7f097c0feec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f09809dd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 0x7f097c0ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f09809dd640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0ff940 con 0x7f097c101b90 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f09809dd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0ffab0 con 0x7f097c101270 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f0979d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 0x7f097c0ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f0979d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 0x7f097c0ff400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40092/0 (socket says 172.21.15.63:40092) 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f0979d9b640 1 -- 172.21.15.63:0/2428588321 learned_addr learned my addr 172.21.15.63:0/2428588321 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f097a59c640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f097c101270 0x7f097c0feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f0979d9b640 1 -- 172.21.15.63:0/2428588321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f097c101270 msgr2=0x7f097c0feec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f0979d9b640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f097c101270 0x7f097c0feec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.970+0000 7f0979d9b640 1 -- 172.21.15.63:0/2428588321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c009560 con 0x7f097c101b90 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.971+0000 7f0979d9b640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 0x7f097c0ff400 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f096c02c2b0 tx=0x7f096c004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.971+0000 7f09737fe640 1 -- 172.21.15.63:0/2428588321 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f096c004420 con 0x7f097c101b90 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.971+0000 7f09737fe640 1 -- 172.21.15.63:0/2428588321 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f096c003ab0 con 0x7f097c101b90 2024-05-09T06:49:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.971+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097c0ffd30 con 0x7f097c101b90 2024-05-09T06:49:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.971+0000 7f09737fe640 1 -- 172.21.15.63:0/2428588321 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f096c005460 con 0x7f097c101b90 2024-05-09T06:49:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.971+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097c1a25c0 con 0x7f097c101b90 2024-05-09T06:49:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.972+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097c068e60 con 0x7f097c101b90 2024-05-09T06:49:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.972+0000 7f09737fe640 1 -- 172.21.15.63:0/2428588321 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f096c003c20 con 0x7f097c101b90 2024-05-09T06:49:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.973+0000 7f09737fe640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f096806e170 0x7f0968070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.973+0000 7f09737fe640 1 -- 172.21.15.63:0/2428588321 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f096c0b09b0 con 0x7f097c101b90 2024-05-09T06:49:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.973+0000 7f097a59c640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f096806e170 0x7f0968070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.973+0000 7f097a59c640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f096806e170 0x7f0968070630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0964009bd0 tx=0x7f0964005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:35.978+0000 7f09737fe640 1 -- 172.21.15.63:0/2428588321 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f096c079b90 con 0x7f097c101b90 2024-05-09T06:49:36.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:35 smithi193 ceph-mon[81665]: pgmap v330: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:36.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:35 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/263776603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:49:36.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:35 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3482600394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:49:36.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.165+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f097c1003a0 con 0x7f097c101b90 2024-05-09T06:49:36.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.165+0000 7f09737fe640 1 -- 172.21.15.63:0/2428588321 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f096c07d250 con 0x7f097c101b90 2024-05-09T06:49:36.144 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-09T06:49:36.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f096806e170 msgr2=0x7f0968070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f096806e170 0x7f0968070630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0964009bd0 tx=0x7f0964005f90 comp rx=0 tx=0).stop 2024-05-09T06:49:36.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 msgr2=0x7f097c0ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 0x7f097c0ff400 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f096c02c2b0 tx=0x7f096c004960 comp rx=0 tx=0).stop 2024-05-09T06:49:36.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 shutdown_connections 2024-05-09T06:49:36.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f096806e170 0x7f0968070630 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c101b90 0x7f097c0ff400 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 --2- 172.21.15.63:0/2428588321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f097c101270 0x7f097c0feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 >> 172.21.15.63:0/2428588321 conn(0x7f097c0faa90 msgr2=0x7f097c102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:36.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 shutdown_connections 2024-05-09T06:49:36.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.167+0000 7f09809dd640 1 -- 172.21.15.63:0/2428588321 wait complete. 2024-05-09T06:49:36.157 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.285+0000 7f2771450640 1 -- 172.21.15.63:0/2237147505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100cc0 msgr2=0x7f276c104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.285+0000 7f2771450640 1 --2- 172.21.15.63:0/2237147505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100cc0 0x7f276c104d00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f275c009880 tx=0x7f275c02bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.285+0000 7f2771450640 1 -- 172.21.15.63:0/2237147505 shutdown_connections 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.285+0000 7f2771450640 1 --2- 172.21.15.63:0/2237147505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100cc0 0x7f276c104d00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.285+0000 7f2771450640 1 --2- 172.21.15.63:0/2237147505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f276c100310 0x7f276c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.285+0000 7f2771450640 1 -- 172.21.15.63:0/2237147505 >> 172.21.15.63:0/2237147505 conn(0x7f276c0fbbd0 msgr2=0x7f276c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.285+0000 7f2771450640 1 -- 172.21.15.63:0/2237147505 shutdown_connections 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.285+0000 7f2771450640 1 -- 172.21.15.63:0/2237147505 wait complete. 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f2771450640 1 Processor -- start 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f2771450640 1 -- start start 2024-05-09T06:49:36.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f2771450640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100310 0x7f276c196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f2771450640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f276c196690 0x7f276c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f2771450640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c196c90 con 0x7f276c100310 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f2771450640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c196e00 con 0x7f276c196690 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f276affd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100310 0x7f276c196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f276affd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100310 0x7f276c196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40116/0 (socket says 172.21.15.63:40116) 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f276affd640 1 -- 172.21.15.63:0/2102944578 learned_addr learned my addr 172.21.15.63:0/2102944578 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f276a7fc640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f276c196690 0x7f276c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f276affd640 1 -- 172.21.15.63:0/2102944578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f276c196690 msgr2=0x7f276c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f276affd640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f276c196690 0x7f276c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f276affd640 1 -- 172.21.15.63:0/2102944578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c009560 con 0x7f276c100310 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f276a7fc640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f276c196690 0x7f276c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.286+0000 7f276affd640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100310 0x7f276c196150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f275400e9a0 tx=0x7f275400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.287+0000 7f2763fff640 1 -- 172.21.15.63:0/2102944578 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f275400cc90 con 0x7f276c100310 2024-05-09T06:49:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.287+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c19b030 con 0x7f276c100310 2024-05-09T06:49:36.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.287+0000 7f2763fff640 1 -- 172.21.15.63:0/2102944578 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f275400cdf0 con 0x7f276c100310 2024-05-09T06:49:36.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.287+0000 7f2763fff640 1 -- 172.21.15.63:0/2102944578 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27540187f0 con 0x7f276c100310 2024-05-09T06:49:36.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.287+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c19b590 con 0x7f276c100310 2024-05-09T06:49:36.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.288+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276c19b870 con 0x7f276c100310 2024-05-09T06:49:36.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.288+0000 7f2763fff640 1 -- 172.21.15.63:0/2102944578 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2754010c10 con 0x7f276c100310 2024-05-09T06:49:36.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.289+0000 7f2763fff640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f275806e170 0x7f2758070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:36.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.289+0000 7f2763fff640 1 -- 172.21.15.63:0/2102944578 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f2754014070 con 0x7f276c100310 2024-05-09T06:49:36.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.289+0000 7f276a7fc640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f275806e170 0x7f2758070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:36.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.289+0000 7f276a7fc640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f275806e170 0x7f2758070630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f275c036040 tx=0x7f275c000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:36.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.293+0000 7f2763fff640 1 -- 172.21.15.63:0/2102944578 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2754053740 con 0x7f276c100310 2024-05-09T06:49:36.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.515+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f276c197450 con 0x7f276c100310 2024-05-09T06:49:36.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.515+0000 7f2763fff640 1 -- 172.21.15.63:0/2102944578 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5215 (secure 0 0 0) 0x7f2754057960 con 0x7f276c100310 2024-05-09T06:49:36.495 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:36.495 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi193.ggtmyb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/2007543899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2007543899},{"type":"v1","addr":"172.21.15.193:6825","nonce":2007543899}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44347,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/790240017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":790240017},{"type":"v1","addr":"172.21.15.193:6827","nonce":790240017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:46:08.230738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":0,"incarnation":29,"state":"up:active","state_seq":20,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-05-09T06:49:36.495 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 34 2024-05-09T06:49:36.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f275806e170 msgr2=0x7f2758070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f275806e170 0x7f2758070630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f275c036040 tx=0x7f275c000bc0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100310 msgr2=0x7f276c196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100310 0x7f276c196150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f275400e9a0 tx=0x7f275400ee60 comp rx=0 tx=0).stop 2024-05-09T06:49:36.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 shutdown_connections 2024-05-09T06:49:36.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f275806e170 0x7f2758070630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f276c196690 0x7f276c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 --2- 172.21.15.63:0/2102944578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f276c100310 0x7f276c196150 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 >> 172.21.15.63:0/2102944578 conn(0x7f276c0fbbd0 msgr2=0x7f276c103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:36.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 shutdown_connections 2024-05-09T06:49:36.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.517+0000 7f2771450640 1 -- 172.21.15.63:0/2102944578 wait complete. 2024-05-09T06:49:36.507 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-05-09T06:49:36.508 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-05-09T06:49:36.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.634+0000 7f40cd511640 1 -- 172.21.15.63:0/1074961572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 msgr2=0x7f40c8101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.634+0000 7f40cd511640 1 --2- 172.21.15.63:0/1074961572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 0x7f40c8101650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009880 tx=0x7f40b802bd90 comp rx=0 tx=0).stop 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.634+0000 7f40cd511640 1 -- 172.21.15.63:0/1074961572 shutdown_connections 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.634+0000 7f40cd511640 1 --2- 172.21.15.63:0/1074961572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c8101b90 0x7f40c810a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.634+0000 7f40cd511640 1 --2- 172.21.15.63:0/1074961572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 0x7f40c8101650 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.634+0000 7f40cd511640 1 -- 172.21.15.63:0/1074961572 >> 172.21.15.63:0/1074961572 conn(0x7f40c80faa90 msgr2=0x7f40c80fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.634+0000 7f40cd511640 1 -- 172.21.15.63:0/1074961572 shutdown_connections 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.634+0000 7f40cd511640 1 -- 172.21.15.63:0/1074961572 wait complete. 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.634+0000 7f40cd511640 1 Processor -- start 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40cd511640 1 -- start start 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40cd511640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 0x7f40c8196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40cd511640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c8101b90 0x7f40c8196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40cd511640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8196c90 con 0x7f40c8101270 2024-05-09T06:49:36.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40cd511640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8196e00 con 0x7f40c8101b90 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 0x7f40c8196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 0x7f40c8196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40144/0 (socket says 172.21.15.63:40144) 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40c6ffd640 1 -- 172.21.15.63:0/1388736588 learned_addr learned my addr 172.21.15.63:0/1388736588 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40c67fc640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c8101b90 0x7f40c8196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40c6ffd640 1 -- 172.21.15.63:0/1388736588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c8101b90 msgr2=0x7f40c8196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40c6ffd640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c8101b90 0x7f40c8196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40c6ffd640 1 -- 172.21.15.63:0/1388736588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8009560 con 0x7f40c8101270 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40c67fc640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c8101b90 0x7f40c8196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.635+0000 7f40c6ffd640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 0x7f40c8196150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40b8006010 tx=0x7f40b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.636+0000 7f40bffff640 1 -- 172.21.15.63:0/1388736588 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b80037c0 con 0x7f40c8101270 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.636+0000 7f40bffff640 1 -- 172.21.15.63:0/1388736588 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40b8004aa0 con 0x7f40c8101270 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.636+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c819ab20 con 0x7f40c8101270 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.636+0000 7f40bffff640 1 -- 172.21.15.63:0/1388736588 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b8005660 con 0x7f40c8101270 2024-05-09T06:49:36.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.636+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c819b080 con 0x7f40c8101270 2024-05-09T06:49:36.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.637+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c8068e60 con 0x7f40c8101270 2024-05-09T06:49:36.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.637+0000 7f40bffff640 1 -- 172.21.15.63:0/1388736588 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f40b8004ed0 con 0x7f40c8101270 2024-05-09T06:49:36.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.638+0000 7f40bffff640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f40b406e170 0x7f40b4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:36.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.638+0000 7f40bffff640 1 -- 172.21.15.63:0/1388736588 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f40b80b0f10 con 0x7f40c8101270 2024-05-09T06:49:36.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.638+0000 7f40c67fc640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f40b406e170 0x7f40b4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:36.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.638+0000 7f40c67fc640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f40b406e170 0x7f40b4070630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005e10 tx=0x7f40b0005da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:36.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.642+0000 7f40bffff640 1 -- 172.21.15.63:0/1388736588 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f40b807a210 con 0x7f40c8101270 2024-05-09T06:49:36.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.865+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f40c804e160 con 0x7f40c8101270 2024-05-09T06:49:36.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.865+0000 7f40bffff640 1 -- 172.21.15.63:0/1388736588 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v34) v1 ==== 107+0+4088 (secure 0 0 0) 0x7f40b8043090 con 0x7f40c8101270 2024-05-09T06:49:36.844 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:36.844 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14504,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1277475441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1277475441},{"type":"v1","addr":"172.21.15.63:6827","nonce":1277475441}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:44:47.035052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi193.ggtmyb","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6825/296432349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":296432349},{"type":"v1","addr":"172.21.15.193:6825","nonce":296432349}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-05-09T06:49:36.845 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 16 2024-05-09T06:49:36.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f40b406e170 msgr2=0x7f40b4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f40b406e170 0x7f40b4070630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005e10 tx=0x7f40b0005da0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 msgr2=0x7f40c8196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 0x7f40c8196150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40b8006010 tx=0x7f40b8004270 comp rx=0 tx=0).stop 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 shutdown_connections 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f40b406e170 0x7f40b4070630 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c8101b90 0x7f40c8196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 --2- 172.21.15.63:0/1388736588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40c8101270 0x7f40c8196150 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 >> 172.21.15.63:0/1388736588 conn(0x7f40c80faa90 msgr2=0x7f40c8102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.867+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 shutdown_connections 2024-05-09T06:49:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.868+0000 7f40cd511640 1 -- 172.21.15.63:0/1388736588 wait complete. 2024-05-09T06:49:36.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:36 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3078902637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-09T06:49:36.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:36 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2428588321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-09T06:49:36.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:36 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2102944578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T06:49:36.858 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 16 2024-05-09T06:49:36.858 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.986+0000 7fc5b9058640 1 -- 172.21.15.63:0/650491180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 msgr2=0x7fc5b40ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.986+0000 7fc5b9058640 1 --2- 172.21.15.63:0/650491180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 0x7fc5b40ff550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009880 tx=0x7fc5a402bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.986+0000 7fc5b9058640 1 -- 172.21.15.63:0/650491180 shutdown_connections 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.986+0000 7fc5b9058640 1 --2- 172.21.15.63:0/650491180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b40ffb20 0x7fc5b4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.986+0000 7fc5b9058640 1 --2- 172.21.15.63:0/650491180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 0x7fc5b40ff550 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.986+0000 7fc5b9058640 1 -- 172.21.15.63:0/650491180 >> 172.21.15.63:0/650491180 conn(0x7fc5b4075bd0 msgr2=0x7fc5b40fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.986+0000 7fc5b9058640 1 -- 172.21.15.63:0/650491180 shutdown_connections 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.986+0000 7fc5b9058640 1 -- 172.21.15.63:0/650491180 wait complete. 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b9058640 1 Processor -- start 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b9058640 1 -- start start 2024-05-09T06:49:36.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b9058640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 0x7fc5b40faf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b9058640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b40fb4d0 0x7fc5b40fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b9058640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40fcef0 con 0x7fc5b40ff170 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b9058640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b41a2290 con 0x7fc5b40fb4d0 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b2d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 0x7fc5b40faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b2d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 0x7fc5b40faf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40160/0 (socket says 172.21.15.63:40160) 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b2d9d640 1 -- 172.21.15.63:0/295744383 learned_addr learned my addr 172.21.15.63:0/295744383 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5abfff640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b40fb4d0 0x7fc5b40fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b2d9d640 1 -- 172.21.15.63:0/295744383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b40fb4d0 msgr2=0x7fc5b40fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b2d9d640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b40fb4d0 0x7fc5b40fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.987+0000 7fc5b2d9d640 1 -- 172.21.15.63:0/295744383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4009560 con 0x7fc5b40ff170 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.988+0000 7fc5b2d9d640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 0x7fc5b40faf90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc5a402c220 tx=0x7fc5a4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.988+0000 7fc5b0d99640 1 -- 172.21.15.63:0/295744383 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a40035f0 con 0x7fc5b40ff170 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.988+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b41a2430 con 0x7fc5b40ff170 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.988+0000 7fc5b0d99640 1 -- 172.21.15.63:0/295744383 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5a4003c10 con 0x7fc5b40ff170 2024-05-09T06:49:36.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.988+0000 7fc5b0d99640 1 -- 172.21.15.63:0/295744383 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a4034440 con 0x7fc5b40ff170 2024-05-09T06:49:36.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.988+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b41a2960 con 0x7fc5b40ff170 2024-05-09T06:49:36.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.989+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b4100640 con 0x7fc5b40ff170 2024-05-09T06:49:36.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.990+0000 7fc5b0d99640 1 -- 172.21.15.63:0/295744383 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc5a4004f60 con 0x7fc5b40ff170 2024-05-09T06:49:36.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.990+0000 7fc5b0d99640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5a006e170 0x7fc5a0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:36.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.990+0000 7fc5b0d99640 1 -- 172.21.15.63:0/295744383 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fc5a40b0730 con 0x7fc5b40ff170 2024-05-09T06:49:36.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.990+0000 7fc5abfff640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5a006e170 0x7fc5a0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:36.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.991+0000 7fc5abfff640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5a006e170 0x7fc5a0070630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c005e20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:36.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:36.997+0000 7fc5b0d99640 1 -- 172.21.15.63:0/295744383 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc5a4079890 con 0x7fc5b40ff170 2024-05-09T06:49:36.984 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:36 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3078902637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-09T06:49:36.984 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:36 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2428588321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-09T06:49:36.984 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:36 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2102944578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T06:49:37.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.226+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fc5b40ffb80 con 0x7fc5b40ff170 2024-05-09T06:49:37.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.226+0000 7fc5b0d99640 1 -- 172.21.15.63:0/295744383 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v34) v1 ==== 107+0+4864 (secure 0 0 0) 0x7fc5a407cf50 con 0x7fc5b40ff170 2024-05-09T06:49:37.205 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:37.205 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14504,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1277475441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1277475441},{"type":"v1","addr":"172.21.15.63:6827","nonce":1277475441}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34390,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/311216115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":311216115},{"type":"v1","addr":"172.21.15.63:6829","nonce":311216115}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:44:47.035052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi193.ggtmyb","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6825/296432349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":296432349},{"type":"v1","addr":"172.21.15.193:6825","nonce":296432349}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-05-09T06:49:37.205 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 17 2024-05-09T06:49:37.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5a006e170 msgr2=0x7fc5a0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5a006e170 0x7fc5a0070630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c005e20 comp rx=0 tx=0).stop 2024-05-09T06:49:37.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 msgr2=0x7fc5b40faf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 0x7fc5b40faf90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc5a402c220 tx=0x7fc5a4004040 comp rx=0 tx=0).stop 2024-05-09T06:49:37.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 shutdown_connections 2024-05-09T06:49:37.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc5a006e170 0x7fc5a0070630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b40fb4d0 0x7fc5b40fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 --2- 172.21.15.63:0/295744383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5b40ff170 0x7fc5b40faf90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 >> 172.21.15.63:0/295744383 conn(0x7fc5b4075bd0 msgr2=0x7fc5b4067430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 shutdown_connections 2024-05-09T06:49:37.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.228+0000 7fc5b9058640 1 -- 172.21.15.63:0/295744383 wait complete. 2024-05-09T06:49:37.218 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 17 2024-05-09T06:49:37.219 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-05-09T06:49:37.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.345+0000 7f25f217c640 1 -- 172.21.15.63:0/3112710147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100cb0 msgr2=0x7f25ec104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.345+0000 7f25f217c640 1 --2- 172.21.15.63:0/3112710147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100cb0 0x7f25ec104cf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009880 tx=0x7f25e002bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.345+0000 7f25f217c640 1 -- 172.21.15.63:0/3112710147 shutdown_connections 2024-05-09T06:49:37.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.345+0000 7f25f217c640 1 --2- 172.21.15.63:0/3112710147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100cb0 0x7f25ec104cf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.345+0000 7f25f217c640 1 --2- 172.21.15.63:0/3112710147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ec100300 0x7f25ec1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.345+0000 7f25f217c640 1 -- 172.21.15.63:0/3112710147 >> 172.21.15.63:0/3112710147 conn(0x7f25ec0fbb60 msgr2=0x7f25ec0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:37.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.345+0000 7f25f217c640 1 -- 172.21.15.63:0/3112710147 shutdown_connections 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.345+0000 7f25f217c640 1 -- 172.21.15.63:0/3112710147 wait complete. 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.346+0000 7f25f217c640 1 Processor -- start 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.346+0000 7f25f217c640 1 -- start start 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.346+0000 7f25f217c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100300 0x7f25ec198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.346+0000 7f25f217c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ec1989b0 0x7f25ec19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.346+0000 7f25f217c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec198fb0 con 0x7f25ec100300 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.346+0000 7f25f217c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec199120 con 0x7f25ec1989b0 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.346+0000 7f25eb7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100300 0x7f25ec198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.346+0000 7f25eb7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100300 0x7f25ec198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40172/0 (socket says 172.21.15.63:40172) 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.346+0000 7f25eb7fe640 1 -- 172.21.15.63:0/26786850 learned_addr learned my addr 172.21.15.63:0/26786850 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25eaffd640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ec1989b0 0x7f25ec19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25eb7fe640 1 -- 172.21.15.63:0/26786850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ec1989b0 msgr2=0x7f25ec19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25eb7fe640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ec1989b0 0x7f25ec19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25eb7fe640 1 -- 172.21.15.63:0/26786850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0009560 con 0x7f25ec100300 2024-05-09T06:49:37.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25eaffd640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ec1989b0 0x7f25ec19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:37.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25eb7fe640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100300 0x7f25ec198470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00b450 tx=0x7f25dc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:37.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25e8ff9640 1 -- 172.21.15.63:0/26786850 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25dc011860 con 0x7f25ec100300 2024-05-09T06:49:37.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25e8ff9640 1 -- 172.21.15.63:0/26786850 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25dc011e80 con 0x7f25ec100300 2024-05-09T06:49:37.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec19d350 con 0x7f25ec100300 2024-05-09T06:49:37.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25e8ff9640 1 -- 172.21.15.63:0/26786850 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25dc00f570 con 0x7f25ec100300 2024-05-09T06:49:37.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.347+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec19d8b0 con 0x7f25ec100300 2024-05-09T06:49:37.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.348+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ec1017d0 con 0x7f25ec100300 2024-05-09T06:49:37.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.349+0000 7f25e8ff9640 1 -- 172.21.15.63:0/26786850 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f25dc0103e0 con 0x7f25ec100300 2024-05-09T06:49:37.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.349+0000 7f25e8ff9640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f25d406e170 0x7f25d4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:37.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.349+0000 7f25e8ff9640 1 -- 172.21.15.63:0/26786850 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f25dc08e220 con 0x7f25ec100300 2024-05-09T06:49:37.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.349+0000 7f25eaffd640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f25d406e170 0x7f25d4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:37.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.350+0000 7f25eaffd640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f25d406e170 0x7f25d4070630 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25e0036040 tx=0x7f25e0000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:37.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.354+0000 7f25e8ff9640 1 -- 172.21.15.63:0/26786850 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f25dc057380 con 0x7f25ec100300 2024-05-09T06:49:37.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.574+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f25ec19db30 con 0x7f25ec100300 2024-05-09T06:49:37.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.574+0000 7f25e8ff9640 1 -- 172.21.15.63:0/26786850 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v34) v1 ==== 107+0+4085 (secure 0 0 0) 0x7f25dc05aa40 con 0x7f25ec100300 2024-05-09T06:49:37.553 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:37.553 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34390,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/311216115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":311216115},{"type":"v1","addr":"172.21.15.63:6829","nonce":311216115}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:44:47.035052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi193.ggtmyb","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6825/296432349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":296432349},{"type":"v1","addr":"172.21.15.193:6825","nonce":296432349}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-05-09T06:49:37.554 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 18 2024-05-09T06:49:37.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.576+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f25d406e170 msgr2=0x7f25d4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.576+0000 7f25f217c640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f25d406e170 0x7f25d4070630 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25e0036040 tx=0x7f25e0000bc0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.576+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100300 msgr2=0x7f25ec198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.576+0000 7f25f217c640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100300 0x7f25ec198470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00b450 tx=0x7f25dc00b910 comp rx=0 tx=0).stop 2024-05-09T06:49:37.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.576+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 shutdown_connections 2024-05-09T06:49:37.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.576+0000 7f25f217c640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f25d406e170 0x7f25d4070630 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.576+0000 7f25f217c640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ec1989b0 0x7f25ec19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.576+0000 7f25f217c640 1 --2- 172.21.15.63:0/26786850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25ec100300 0x7f25ec198470 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.577+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 >> 172.21.15.63:0/26786850 conn(0x7f25ec0fbb60 msgr2=0x7f25ec073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:37.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.577+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 shutdown_connections 2024-05-09T06:49:37.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.577+0000 7f25f217c640 1 -- 172.21.15.63:0/26786850 wait complete. 2024-05-09T06:49:37.567 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-05-09T06:49:37.567 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-05-09T06:49:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.693+0000 7f0f32f80640 1 -- 172.21.15.63:0/2755567564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c0ff210 msgr2=0x7f0f2c0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.693+0000 7f0f32f80640 1 --2- 172.21.15.63:0/2755567564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c0ff210 0x7f0f2c0ff5f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009880 tx=0x7f0f2002bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.693+0000 7f0f32f80640 1 -- 172.21.15.63:0/2755567564 shutdown_connections 2024-05-09T06:49:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.693+0000 7f0f32f80640 1 --2- 172.21.15.63:0/2755567564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f2c0ffbc0 0x7f0f2c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.693+0000 7f0f32f80640 1 --2- 172.21.15.63:0/2755567564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c0ff210 0x7f0f2c0ff5f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.693+0000 7f0f32f80640 1 -- 172.21.15.63:0/2755567564 >> 172.21.15.63:0/2755567564 conn(0x7f0f2c0faa90 msgr2=0x7f0f2c0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.693+0000 7f0f32f80640 1 -- 172.21.15.63:0/2755567564 shutdown_connections 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.693+0000 7f0f32f80640 1 -- 172.21.15.63:0/2755567564 wait complete. 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f32f80640 1 Processor -- start 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f32f80640 1 -- start start 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f32f80640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f2c0ff210 0x7f0f2c100ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f32f80640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c1011e0 0x7f0f2c101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f32f80640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c102c00 con 0x7f0f2c1011e0 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f32f80640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c1a22e0 con 0x7f0f2c0ff210 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f2bfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c1011e0 0x7f0f2c101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f2bfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c1011e0 0x7f0f2c101640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40202/0 (socket says 172.21.15.63:40202) 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f2bfff640 1 -- 172.21.15.63:0/2678316771 learned_addr learned my addr 172.21.15.63:0/2678316771 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f30d1c640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f2c0ff210 0x7f0f2c100ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.694+0000 7f0f2bfff640 1 -- 172.21.15.63:0/2678316771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f2c0ff210 msgr2=0x7f0f2c100ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.695+0000 7f0f2bfff640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f2c0ff210 0x7f0f2c100ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.695+0000 7f0f2bfff640 1 -- 172.21.15.63:0/2678316771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20009560 con 0x7f0f2c1011e0 2024-05-09T06:49:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.695+0000 7f0f30d1c640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f2c0ff210 0x7f0f2c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.695+0000 7f0f2bfff640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c1011e0 0x7f0f2c101640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005fd0 tx=0x7f0f1800dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.695+0000 7f0f29ffb640 1 -- 172.21.15.63:0/2678316771 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f18009c00 con 0x7f0f2c1011e0 2024-05-09T06:49:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.695+0000 7f0f29ffb640 1 -- 172.21.15.63:0/2678316771 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f1800de90 con 0x7f0f2c1011e0 2024-05-09T06:49:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.695+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c1a24e0 con 0x7f0f2c1011e0 2024-05-09T06:49:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.695+0000 7f0f29ffb640 1 -- 172.21.15.63:0/2678316771 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f18003ea0 con 0x7f0f2c1011e0 2024-05-09T06:49:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.695+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c1a2a10 con 0x7f0f2c1011e0 2024-05-09T06:49:37.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.696+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c1006e0 con 0x7f0f2c1011e0 2024-05-09T06:49:37.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.696+0000 7f0f29ffb640 1 -- 172.21.15.63:0/2678316771 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0f18004000 con 0x7f0f2c1011e0 2024-05-09T06:49:37.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.697+0000 7f0f29ffb640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0f1c06e170 0x7f0f1c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:37.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.697+0000 7f0f29ffb640 1 -- 172.21.15.63:0/2678316771 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f0f1808e130 con 0x7f0f2c1011e0 2024-05-09T06:49:37.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.697+0000 7f0f30d1c640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0f1c06e170 0x7f0f1c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:37.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.697+0000 7f0f30d1c640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0f1c06e170 0x7f0f1c070630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0f20036040 tx=0x7f0f20000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:37.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.702+0000 7f0f29ffb640 1 -- 172.21.15.63:0/2678316771 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0f18057310 con 0x7f0f2c1011e0 2024-05-09T06:49:37.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:37 smithi063 ceph-mon[92765]: pgmap v331: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:37.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:37 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1388736588' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-05-09T06:49:37.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:37 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/295744383' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-05-09T06:49:37.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:37 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/26786850' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-05-09T06:49:37.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.923+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f0f2c0ffbc0 con 0x7f0f2c1011e0 2024-05-09T06:49:37.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.924+0000 7f0f29ffb640 1 -- 172.21.15.63:0/2678316771 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v34) v1 ==== 107+0+4929 (secure 0 0 0) 0x7f0f1805a9d0 con 0x7f0f2c1011e0 2024-05-09T06:49:37.902 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:37.903 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34390,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/311216115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":311216115},{"type":"v1","addr":"172.21.15.63:6829","nonce":311216115}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:44:47.035052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi193.ggtmyb","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6825/296432349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":296432349},{"type":"v1","addr":"172.21.15.193:6825","nonce":296432349}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-05-09T06:49:37.903 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 19 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0f1c06e170 msgr2=0x7f0f1c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0f1c06e170 0x7f0f1c070630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0f20036040 tx=0x7f0f20000bc0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c1011e0 msgr2=0x7f0f2c101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c1011e0 0x7f0f2c101640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005fd0 tx=0x7f0f1800dd60 comp rx=0 tx=0).stop 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 shutdown_connections 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f0f1c06e170 0x7f0f1c070630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c1011e0 0x7f0f2c101640 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 --2- 172.21.15.63:0/2678316771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f2c0ff210 0x7f0f2c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 >> 172.21.15.63:0/2678316771 conn(0x7f0f2c0faa90 msgr2=0x7f0f2c0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:37.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 shutdown_connections 2024-05-09T06:49:37.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:37.926+0000 7f0f32f80640 1 -- 172.21.15.63:0/2678316771 wait complete. 2024-05-09T06:49:37.916 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-05-09T06:49:37.916 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-05-09T06:49:38.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.043+0000 7fb93d64a640 1 -- 172.21.15.63:0/835396940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067fc0 msgr2=0x7fb938104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.043+0000 7fb93d64a640 1 --2- 172.21.15.63:0/835396940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067fc0 0x7fb938104d00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb920009880 tx=0x7fb92002bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.043+0000 7fb93d64a640 1 -- 172.21.15.63:0/835396940 shutdown_connections 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.043+0000 7fb93d64a640 1 --2- 172.21.15.63:0/835396940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067fc0 0x7fb938104d00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.043+0000 7fb93d64a640 1 --2- 172.21.15.63:0/835396940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb938067610 0x7fb9380679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.043+0000 7fb93d64a640 1 -- 172.21.15.63:0/835396940 >> 172.21.15.63:0/835396940 conn(0x7fb938074aa0 msgr2=0x7fb938074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.043+0000 7fb93d64a640 1 -- 172.21.15.63:0/835396940 shutdown_connections 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.043+0000 7fb93d64a640 1 -- 172.21.15.63:0/835396940 wait complete. 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb93d64a640 1 Processor -- start 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb93d64a640 1 -- start start 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb93d64a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067610 0x7fb9380ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb93d64a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9380ff640 0x7fb9380ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb93d64a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb938100fd0 con 0x7fb938067610 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb93d64a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381a2390 con 0x7fb9380ff640 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067610 0x7fb9380ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067610 0x7fb9380ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40226/0 (socket says 172.21.15.63:40226) 2024-05-09T06:49:38.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb936ffd640 1 -- 172.21.15.63:0/842719666 learned_addr learned my addr 172.21.15.63:0/842719666 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb92ffff640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9380ff640 0x7fb9380ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb936ffd640 1 -- 172.21.15.63:0/842719666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9380ff640 msgr2=0x7fb9380ffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb936ffd640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9380ff640 0x7fb9380ffaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb936ffd640 1 -- 172.21.15.63:0/842719666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb920009560 con 0x7fb938067610 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.044+0000 7fb936ffd640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067610 0x7fb9380ff100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c8a0 tx=0x7fb92800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.045+0000 7fb934ff9640 1 -- 172.21.15.63:0/842719666 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb928007b80 con 0x7fb938067610 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.045+0000 7fb934ff9640 1 -- 172.21.15.63:0/842719666 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb928007ce0 con 0x7fb938067610 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.045+0000 7fb934ff9640 1 -- 172.21.15.63:0/842719666 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb928018810 con 0x7fb938067610 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.045+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9381a2530 con 0x7fb938067610 2024-05-09T06:49:38.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.045+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9381a2a60 con 0x7fb938067610 2024-05-09T06:49:38.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.046+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9381017e0 con 0x7fb938067610 2024-05-09T06:49:38.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.046+0000 7fb934ff9640 1 -- 172.21.15.63:0/842719666 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb928018970 con 0x7fb938067610 2024-05-09T06:49:38.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.047+0000 7fb934ff9640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb92406e1c0 0x7fb924070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:38.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.047+0000 7fb934ff9640 1 -- 172.21.15.63:0/842719666 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fb92808e510 con 0x7fb938067610 2024-05-09T06:49:38.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.047+0000 7fb92ffff640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb92406e1c0 0x7fb924070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:38.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.047+0000 7fb92ffff640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb92406e1c0 0x7fb924070680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb920036040 tx=0x7fb920000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:38.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.051+0000 7fb934ff9640 1 -- 172.21.15.63:0/842719666 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb928057650 con 0x7fb938067610 2024-05-09T06:49:38.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:37 smithi193 ceph-mon[81665]: pgmap v331: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:38.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:37 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1388736588' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-05-09T06:49:38.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:37 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/295744383' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-05-09T06:49:38.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:37 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/26786850' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-05-09T06:49:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.273+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fb93804e160 con 0x7fb938067610 2024-05-09T06:49:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.274+0000 7fb934ff9640 1 -- 172.21.15.63:0/842719666 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v34) v1 ==== 107+0+4153 (secure 0 0 0) 0x7fb92805ad10 con 0x7fb938067610 2024-05-09T06:49:38.253 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:38.253 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:44:47.035052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi193.ggtmyb","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6825/296432349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":296432349},{"type":"v1","addr":"172.21.15.193:6825","nonce":296432349}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-05-09T06:49:38.254 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 20 2024-05-09T06:49:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb92406e1c0 msgr2=0x7fb924070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb92406e1c0 0x7fb924070680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb920036040 tx=0x7fb920000bc0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067610 msgr2=0x7fb9380ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067610 0x7fb9380ff100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c8a0 tx=0x7fb92800cd60 comp rx=0 tx=0).stop 2024-05-09T06:49:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 shutdown_connections 2024-05-09T06:49:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb92406e1c0 0x7fb924070680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9380ff640 0x7fb9380ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 --2- 172.21.15.63:0/842719666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb938067610 0x7fb9380ff100 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 >> 172.21.15.63:0/842719666 conn(0x7fb938074aa0 msgr2=0x7fb938103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:38.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 shutdown_connections 2024-05-09T06:49:38.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.276+0000 7fb93d64a640 1 -- 172.21.15.63:0/842719666 wait complete. 2024-05-09T06:49:38.266 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-05-09T06:49:38.266 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-05-09T06:49:38.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.393+0000 7fa402adf640 1 -- 172.21.15.63:0/84404778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc0fe760 msgr2=0x7fa3fc0febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.393+0000 7fa402adf640 1 --2- 172.21.15.63:0/84404778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc0fe760 0x7fa3fc0febc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009c20 tx=0x7fa3e402c1b0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.393+0000 7fa402adf640 1 -- 172.21.15.63:0/84404778 shutdown_connections 2024-05-09T06:49:38.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.393+0000 7fa402adf640 1 --2- 172.21.15.63:0/84404778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc0fe760 0x7fa3fc0febc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.393+0000 7fa402adf640 1 --2- 172.21.15.63:0/84404778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3fc1047c0 0x7fa3fc104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.393+0000 7fa402adf640 1 -- 172.21.15.63:0/84404778 >> 172.21.15.63:0/84404778 conn(0x7fa3fc0fa680 msgr2=0x7fa3fc0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:38.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.393+0000 7fa402adf640 1 -- 172.21.15.63:0/84404778 shutdown_connections 2024-05-09T06:49:38.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.393+0000 7fa402adf640 1 -- 172.21.15.63:0/84404778 wait complete. 2024-05-09T06:49:38.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa402adf640 1 Processor -- start 2024-05-09T06:49:38.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa402adf640 1 -- start start 2024-05-09T06:49:38.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa402adf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3fc0fe760 0x7fa3fc195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:38.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa402adf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc1047c0 0x7fa3fc195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:38.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa402adf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc195be0 con 0x7fa3fc1047c0 2024-05-09T06:49:38.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa402adf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc1998a0 con 0x7fa3fc0fe760 2024-05-09T06:49:38.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc1047c0 0x7fa3fc195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:38.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc1047c0 0x7fa3fc195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40242/0 (socket says 172.21.15.63:40242) 2024-05-09T06:49:38.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa3fbfff640 1 -- 172.21.15.63:0/2070774695 learned_addr learned my addr 172.21.15.63:0/2070774695 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:38.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa40087b640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3fc0fe760 0x7fa3fc195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:38.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa3fbfff640 1 -- 172.21.15.63:0/2070774695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3fc0fe760 msgr2=0x7fa3fc195010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa3fbfff640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3fc0fe760 0x7fa3fc195010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa3fbfff640 1 -- 172.21.15.63:0/2070774695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e4009900 con 0x7fa3fc1047c0 2024-05-09T06:49:38.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.394+0000 7fa3fbfff640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc1047c0 0x7fa3fc195550 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa3e402c6f0 tx=0x7fa3e4032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:38.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.395+0000 7fa3f9ffb640 1 -- 172.21.15.63:0/2070774695 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3e4034820 con 0x7fa3fc1047c0 2024-05-09T06:49:38.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.395+0000 7fa3f9ffb640 1 -- 172.21.15.63:0/2070774695 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3e4003e30 con 0x7fa3fc1047c0 2024-05-09T06:49:38.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.395+0000 7fa3f9ffb640 1 -- 172.21.15.63:0/2070774695 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3e403f440 con 0x7fa3fc1047c0 2024-05-09T06:49:38.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.395+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc199b20 con 0x7fa3fc1047c0 2024-05-09T06:49:38.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.395+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc19a020 con 0x7fa3fc1047c0 2024-05-09T06:49:38.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.396+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc0ffe80 con 0x7fa3fc1047c0 2024-05-09T06:49:38.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.396+0000 7fa3f9ffb640 1 -- 172.21.15.63:0/2070774695 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa3e403b040 con 0x7fa3fc1047c0 2024-05-09T06:49:38.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.397+0000 7fa3f9ffb640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa3e806e170 0x7fa3e8070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:38.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.397+0000 7fa3f9ffb640 1 -- 172.21.15.63:0/2070774695 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fa3e40b0f70 con 0x7fa3fc1047c0 2024-05-09T06:49:38.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.397+0000 7fa40087b640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa3e806e170 0x7fa3e8070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:38.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.397+0000 7fa40087b640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa3e806e170 0x7fa3e8070630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0058c0 tx=0x7fa3ec00a2d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:38.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.402+0000 7fa3f9ffb640 1 -- 172.21.15.63:0/2070774695 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa3e40e5740 con 0x7fa3fc1047c0 2024-05-09T06:49:38.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.624+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fa3fc04e160 con 0x7fa3fc1047c0 2024-05-09T06:49:38.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.624+0000 7fa3f9ffb640 1 -- 172.21.15.63:0/2070774695 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v34) v1 ==== 107+0+5000 (secure 0 0 0) 0x7fa3e407a150 con 0x7fa3fc1047c0 2024-05-09T06:49:38.604 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:38.604 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:44:47.035052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi193.ggtmyb","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6825/296432349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":296432349},{"type":"v1","addr":"172.21.15.193:6825","nonce":296432349}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-05-09T06:49:38.604 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 21 2024-05-09T06:49:38.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.626+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa3e806e170 msgr2=0x7fa3e8070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.626+0000 7fa402adf640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa3e806e170 0x7fa3e8070630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0058c0 tx=0x7fa3ec00a2d0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.626+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc1047c0 msgr2=0x7fa3fc195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.626+0000 7fa402adf640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc1047c0 0x7fa3fc195550 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa3e402c6f0 tx=0x7fa3e4032570 comp rx=0 tx=0).stop 2024-05-09T06:49:38.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.626+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 shutdown_connections 2024-05-09T06:49:38.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.626+0000 7fa402adf640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa3e806e170 0x7fa3e8070630 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.626+0000 7fa402adf640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3fc1047c0 0x7fa3fc195550 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.626+0000 7fa402adf640 1 --2- 172.21.15.63:0/2070774695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3fc0fe760 0x7fa3fc195010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.626+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 >> 172.21.15.63:0/2070774695 conn(0x7fa3fc0fa680 msgr2=0x7fa3fc0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:38.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.627+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 shutdown_connections 2024-05-09T06:49:38.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.627+0000 7fa402adf640 1 -- 172.21.15.63:0/2070774695 wait complete. 2024-05-09T06:49:38.617 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-05-09T06:49:38.618 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-05-09T06:49:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.748+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1270705712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c8100ca0 msgr2=0x7fc3c8104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.748+0000 7fc3cdaa1640 1 --2- 172.21.15.63:0/1270705712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c8100ca0 0x7fc3c8104ce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009880 tx=0x7fc3bc02bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.748+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1270705712 shutdown_connections 2024-05-09T06:49:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.748+0000 7fc3cdaa1640 1 --2- 172.21.15.63:0/1270705712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c8100ca0 0x7fc3c8104ce0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.748+0000 7fc3cdaa1640 1 --2- 172.21.15.63:0/1270705712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c81002f0 0x7fc3c81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.748+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1270705712 >> 172.21.15.63:0/1270705712 conn(0x7fc3c80fbb90 msgr2=0x7fc3c80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.748+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1270705712 shutdown_connections 2024-05-09T06:49:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.748+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1270705712 wait complete. 2024-05-09T06:49:38.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.748+0000 7fc3cdaa1640 1 Processor -- start 2024-05-09T06:49:38.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3cdaa1640 1 -- start start 2024-05-09T06:49:38.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3cdaa1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c81002f0 0x7fc3c8196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:38.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3cdaa1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c81966b0 0x7fc3c819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:38.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3cdaa1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c8196cb0 con 0x7fc3c81002f0 2024-05-09T06:49:38.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3cdaa1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c8196e20 con 0x7fc3c81966b0 2024-05-09T06:49:38.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3c77fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c81002f0 0x7fc3c8196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3c77fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c81002f0 0x7fc3c8196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40254/0 (socket says 172.21.15.63:40254) 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3c77fe640 1 -- 172.21.15.63:0/1425957355 learned_addr learned my addr 172.21.15.63:0/1425957355 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3c6ffd640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c81966b0 0x7fc3c819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3c77fe640 1 -- 172.21.15.63:0/1425957355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c81966b0 msgr2=0x7fc3c819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3c77fe640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c81966b0 0x7fc3c819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.749+0000 7fc3c77fe640 1 -- 172.21.15.63:0/1425957355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc009560 con 0x7fc3c81002f0 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.750+0000 7fc3c77fe640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c81002f0 0x7fc3c8196170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800eb40 tx=0x7fc3b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.750+0000 7fc3c4ff9640 1 -- 172.21.15.63:0/1425957355 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3b800cee0 con 0x7fc3c81002f0 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.750+0000 7fc3c4ff9640 1 -- 172.21.15.63:0/1425957355 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3b8005020 con 0x7fc3c81002f0 2024-05-09T06:49:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.750+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c819b050 con 0x7fc3c81002f0 2024-05-09T06:49:38.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.750+0000 7fc3c4ff9640 1 -- 172.21.15.63:0/1425957355 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3b8010640 con 0x7fc3c81002f0 2024-05-09T06:49:38.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.750+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c819b5b0 con 0x7fc3c81002f0 2024-05-09T06:49:38.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.751+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c81017c0 con 0x7fc3c81002f0 2024-05-09T06:49:38.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.751+0000 7fc3c4ff9640 1 -- 172.21.15.63:0/1425957355 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc3b8005190 con 0x7fc3c81002f0 2024-05-09T06:49:38.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.752+0000 7fc3c4ff9640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc3b006e170 0x7fc3b0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:38.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.752+0000 7fc3c4ff9640 1 -- 172.21.15.63:0/1425957355 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fc3b8014070 con 0x7fc3c81002f0 2024-05-09T06:49:38.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.752+0000 7fc3c6ffd640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc3b006e170 0x7fc3b0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:38.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.752+0000 7fc3c6ffd640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc3b006e170 0x7fc3b0070630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc006010 tx=0x7fc3bc02c800 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:38.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.756+0000 7fc3c4ff9640 1 -- 172.21.15.63:0/1425957355 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc3b8057c50 con 0x7fc3c81002f0 2024-05-09T06:49:38.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.981+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fc3c8197470 con 0x7fc3c81002f0 2024-05-09T06:49:38.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.982+0000 7fc3c4ff9640 1 -- 172.21.15.63:0/1425957355 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v34) v1 ==== 107+0+4202 (secure 0 0 0) 0x7fc3b805b310 con 0x7fc3c81002f0 2024-05-09T06:49:38.961 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:38.961 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:45:26.766672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T06:49:38.961 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 22 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc3b006e170 msgr2=0x7fc3b0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc3b006e170 0x7fc3b0070630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc006010 tx=0x7fc3bc02c800 comp rx=0 tx=0).stop 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c81002f0 msgr2=0x7fc3c8196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c81002f0 0x7fc3c8196170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800eb40 tx=0x7fc3b800c2d0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 shutdown_connections 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc3b006e170 0x7fc3b0070630 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c81966b0 0x7fc3c819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 --2- 172.21.15.63:0/1425957355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c81002f0 0x7fc3c8196170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 >> 172.21.15.63:0/1425957355 conn(0x7fc3c80fbb90 msgr2=0x7fc3c80fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 shutdown_connections 2024-05-09T06:49:38.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:38.984+0000 7fc3cdaa1640 1 -- 172.21.15.63:0/1425957355 wait complete. 2024-05-09T06:49:38.973 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:38 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2678316771' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-09T06:49:38.973 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:38 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/842719666' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-09T06:49:38.973 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:38 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2070774695' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-09T06:49:38.974 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-05-09T06:49:38.974 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 -- 172.21.15.63:0/3819387051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ffc50 msgr2=0x7f3ab8103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 --2- 172.21.15.63:0/3819387051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ffc50 0x7f3ab8103be0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009c20 tx=0x7f3aac02c1b0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 -- 172.21.15.63:0/3819387051 shutdown_connections 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 --2- 172.21.15.63:0/3819387051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ffc50 0x7f3ab8103be0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 --2- 172.21.15.63:0/3819387051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab80ff2a0 0x7f3ab80ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 -- 172.21.15.63:0/3819387051 >> 172.21.15.63:0/3819387051 conn(0x7f3ab8074aa0 msgr2=0x7f3ab8074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 -- 172.21.15.63:0/3819387051 shutdown_connections 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 -- 172.21.15.63:0/3819387051 wait complete. 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 Processor -- start 2024-05-09T06:49:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.100+0000 7f3abfc0d640 1 -- start start 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abfc0d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ff2a0 0x7f3ab8196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abfc0d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab8196640 0x7f3ab819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abfc0d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8196bb0 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abfc0d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8196d20 con 0x7f3ab8196640 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abd9a9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ff2a0 0x7f3ab8196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abd9a9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ff2a0 0x7f3ab8196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40272/0 (socket says 172.21.15.63:40272) 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abd9a9640 1 -- 172.21.15.63:0/2081245699 learned_addr learned my addr 172.21.15.63:0/2081245699 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abd1a8640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab8196640 0x7f3ab819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abd9a9640 1 -- 172.21.15.63:0/2081245699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab8196640 msgr2=0x7f3ab819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abd9a9640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab8196640 0x7f3ab819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abd9a9640 1 -- 172.21.15.63:0/2081245699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac009900 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3abd9a9640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ff2a0 0x7f3ab8196100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400c8a0 tx=0x7f3ab400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.101+0000 7f3aaaffd640 1 -- 172.21.15.63:0/2081245699 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ab4007b80 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.102+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab819af80 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.102+0000 7f3aaaffd640 1 -- 172.21.15.63:0/2081245699 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ab4007ce0 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.102+0000 7f3aaaffd640 1 -- 172.21.15.63:0/2081245699 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ab4018810 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.102+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab819b490 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.103+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab81006c0 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.103+0000 7f3aaaffd640 1 -- 172.21.15.63:0/2081245699 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3ab4018a80 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.103+0000 7f3aaaffd640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3aa406e170 0x7f3aa4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.104+0000 7f3aaaffd640 1 -- 172.21.15.63:0/2081245699 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f3ab408e4d0 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.104+0000 7f3abd1a8640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3aa406e170 0x7f3aa4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.104+0000 7f3abd1a8640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3aa406e170 0x7f3aa4070630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3aac02c7c0 tx=0x7f3aac005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:39.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.108+0000 7f3aaaffd640 1 -- 172.21.15.63:0/2081245699 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3ab4057630 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:38 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2678316771' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-09T06:49:39.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:38 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/842719666' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-09T06:49:39.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:38 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2070774695' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-09T06:49:39.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.333+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f3ab8197400 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.334+0000 7f3aaaffd640 1 -- 172.21.15.63:0/2081245699 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v34) v1 ==== 107+0+4213 (secure 0 0 0) 0x7f3ab405acf0 con 0x7f3ab80ff2a0 2024-05-09T06:49:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:39.313 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:45:26.771829+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":0,"incarnation":23,"state":"up:replay","state_seq":2,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T06:49:39.313 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 23 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3aa406e170 msgr2=0x7f3aa4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3aa406e170 0x7f3aa4070630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3aac02c7c0 tx=0x7f3aac005f90 comp rx=0 tx=0).stop 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ff2a0 msgr2=0x7f3ab8196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ff2a0 0x7f3ab8196100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400c8a0 tx=0x7f3ab400cd60 comp rx=0 tx=0).stop 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 shutdown_connections 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f3aa406e170 0x7f3aa4070630 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab8196640 0x7f3ab819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 --2- 172.21.15.63:0/2081245699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ab80ff2a0 0x7f3ab8196100 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 >> 172.21.15.63:0/2081245699 conn(0x7f3ab8074aa0 msgr2=0x7f3ab8109380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:39.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 shutdown_connections 2024-05-09T06:49:39.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.336+0000 7f3abfc0d640 1 -- 172.21.15.63:0/2081245699 wait complete. 2024-05-09T06:49:39.326 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-05-09T06:49:39.326 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-05-09T06:49:39.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.452+0000 7fb331b0a640 1 -- 172.21.15.63:0/1273866176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 msgr2=0x7fb32c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.452+0000 7fb331b0a640 1 --2- 172.21.15.63:0/1273866176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 0x7fb32c1006e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009c20 tx=0x7fb31c02c1b0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.452+0000 7fb331b0a640 1 -- 172.21.15.63:0/1273866176 shutdown_connections 2024-05-09T06:49:39.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.452+0000 7fb331b0a640 1 --2- 172.21.15.63:0/1273866176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32c100cb0 0x7fb32c104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.452+0000 7fb331b0a640 1 --2- 172.21.15.63:0/1273866176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 0x7fb32c1006e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.452+0000 7fb331b0a640 1 -- 172.21.15.63:0/1273866176 >> 172.21.15.63:0/1273866176 conn(0x7fb32c0fbb60 msgr2=0x7fb32c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.452+0000 7fb331b0a640 1 -- 172.21.15.63:0/1273866176 shutdown_connections 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.452+0000 7fb331b0a640 1 -- 172.21.15.63:0/1273866176 wait complete. 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.453+0000 7fb331b0a640 1 Processor -- start 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.453+0000 7fb331b0a640 1 -- start start 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.453+0000 7fb331b0a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 0x7fb32c196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.453+0000 7fb331b0a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32c1966d0 0x7fb32c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.453+0000 7fb331b0a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c196cd0 con 0x7fb32c100300 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.453+0000 7fb331b0a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c196e40 con 0x7fb32c1966d0 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb32b7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 0x7fb32c196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb32b7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 0x7fb32c196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40292/0 (socket says 172.21.15.63:40292) 2024-05-09T06:49:39.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb32b7fe640 1 -- 172.21.15.63:0/3852116842 learned_addr learned my addr 172.21.15.63:0/3852116842 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb32affd640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32c1966d0 0x7fb32c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb32b7fe640 1 -- 172.21.15.63:0/3852116842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32c1966d0 msgr2=0x7fb32c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb32b7fe640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32c1966d0 0x7fb32c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb32b7fe640 1 -- 172.21.15.63:0/3852116842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c009900 con 0x7fb32c100300 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb32b7fe640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 0x7fb32c196190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb31c000f80 tx=0x7fb31c032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb328ff9640 1 -- 172.21.15.63:0/3852116842 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c034820 con 0x7fb32c100300 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb328ff9640 1 -- 172.21.15.63:0/3852116842 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb31c003e30 con 0x7fb32c100300 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c19afc0 con 0x7fb32c100300 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb328ff9640 1 -- 172.21.15.63:0/3852116842 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c03f440 con 0x7fb32c100300 2024-05-09T06:49:39.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.454+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c19b520 con 0x7fb32c100300 2024-05-09T06:49:39.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.455+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c1017d0 con 0x7fb32c100300 2024-05-09T06:49:39.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.456+0000 7fb328ff9640 1 -- 172.21.15.63:0/3852116842 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb31c03b040 con 0x7fb32c100300 2024-05-09T06:49:39.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.456+0000 7fb328ff9640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb31406e170 0x7fb314070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:39.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.456+0000 7fb32affd640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb31406e170 0x7fb314070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:39.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.457+0000 7fb328ff9640 1 -- 172.21.15.63:0/3852116842 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fb31c0b0fe0 con 0x7fb32c100300 2024-05-09T06:49:39.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.457+0000 7fb32affd640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb31406e170 0x7fb314070630 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb320005fd0 tx=0x7fb320005e20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:39.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.461+0000 7fb328ff9640 1 -- 172.21.15.63:0/3852116842 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb31c07a1c0 con 0x7fb32c100300 2024-05-09T06:49:39.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.685+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fb32c19b790 con 0x7fb32c100300 2024-05-09T06:49:39.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.686+0000 7fb328ff9640 1 -- 172.21.15.63:0/3852116842 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v34) v1 ==== 107+0+4218 (secure 0 0 0) 0x7fb31c07d880 con 0x7fb32c100300 2024-05-09T06:49:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:39.665 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:45:35.408932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":0,"incarnation":23,"state":"up:reconnect","state_seq":155,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T06:49:39.665 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-05-09T06:49:39.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.688+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb31406e170 msgr2=0x7fb314070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.688+0000 7fb331b0a640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb31406e170 0x7fb314070630 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb320005fd0 tx=0x7fb320005e20 comp rx=0 tx=0).stop 2024-05-09T06:49:39.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.688+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 msgr2=0x7fb32c196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.688+0000 7fb331b0a640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 0x7fb32c196190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb31c000f80 tx=0x7fb31c032570 comp rx=0 tx=0).stop 2024-05-09T06:49:39.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.688+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 shutdown_connections 2024-05-09T06:49:39.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.688+0000 7fb331b0a640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fb31406e170 0x7fb314070630 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.688+0000 7fb331b0a640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32c1966d0 0x7fb32c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.688+0000 7fb331b0a640 1 --2- 172.21.15.63:0/3852116842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb32c100300 0x7fb32c196190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.688+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 >> 172.21.15.63:0/3852116842 conn(0x7fb32c0fbb60 msgr2=0x7fb32c103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:39.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.689+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 shutdown_connections 2024-05-09T06:49:39.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.689+0000 7fb331b0a640 1 -- 172.21.15.63:0/3852116842 wait complete. 2024-05-09T06:49:39.679 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-05-09T06:49:39.679 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-05-09T06:49:39.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.806+0000 7f816bc0d640 1 -- 172.21.15.63:0/2979708444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 msgr2=0x7f8164101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.806+0000 7f816bc0d640 1 --2- 172.21.15.63:0/2979708444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 0x7f8164101650 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8154009880 tx=0x7f815402bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:39.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.806+0000 7f816bc0d640 1 -- 172.21.15.63:0/2979708444 shutdown_connections 2024-05-09T06:49:39.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.806+0000 7f816bc0d640 1 --2- 172.21.15.63:0/2979708444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164101b90 0x7f816410a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.806+0000 7f816bc0d640 1 --2- 172.21.15.63:0/2979708444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 0x7f8164101650 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.806+0000 7f816bc0d640 1 -- 172.21.15.63:0/2979708444 >> 172.21.15.63:0/2979708444 conn(0x7f81640faa90 msgr2=0x7f81640fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:39.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.806+0000 7f816bc0d640 1 -- 172.21.15.63:0/2979708444 shutdown_connections 2024-05-09T06:49:39.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.806+0000 7f816bc0d640 1 -- 172.21.15.63:0/2979708444 wait complete. 2024-05-09T06:49:39.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f816bc0d640 1 Processor -- start 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f816bc0d640 1 -- start start 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f816bc0d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 0x7f8164196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f816bc0d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164101b90 0x7f8164196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f816bc0d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8164196c90 con 0x7f8164101270 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f816bc0d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8164196e00 con 0x7f8164101b90 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f81699a9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 0x7f8164196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f81699a9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 0x7f8164196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40316/0 (socket says 172.21.15.63:40316) 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f81699a9640 1 -- 172.21.15.63:0/2786159496 learned_addr learned my addr 172.21.15.63:0/2786159496 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f81691a8640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164101b90 0x7f8164196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f81699a9640 1 -- 172.21.15.63:0/2786159496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164101b90 msgr2=0x7f8164196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f81699a9640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164101b90 0x7f8164196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.807+0000 7f81699a9640 1 -- 172.21.15.63:0/2786159496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8154009560 con 0x7f8164101270 2024-05-09T06:49:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.808+0000 7f81699a9640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 0x7f8164196150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f815402c2b0 tx=0x7f815402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.808+0000 7f8162ffd640 1 -- 172.21.15.63:0/2786159496 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81540035f0 con 0x7f8164101270 2024-05-09T06:49:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.808+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816419ab20 con 0x7f8164101270 2024-05-09T06:49:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.808+0000 7f8162ffd640 1 -- 172.21.15.63:0/2786159496 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8154003c10 con 0x7f8164101270 2024-05-09T06:49:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.808+0000 7f8162ffd640 1 -- 172.21.15.63:0/2786159496 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8154005690 con 0x7f8164101270 2024-05-09T06:49:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.808+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816419b080 con 0x7f8164101270 2024-05-09T06:49:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.809+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8164068e60 con 0x7f8164101270 2024-05-09T06:49:39.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.809+0000 7f8162ffd640 1 -- 172.21.15.63:0/2786159496 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8154004ed0 con 0x7f8164101270 2024-05-09T06:49:39.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.810+0000 7f8162ffd640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f814c06e120 0x7f814c0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:39.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.810+0000 7f8162ffd640 1 -- 172.21.15.63:0/2786159496 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f81540b0ea0 con 0x7f8164101270 2024-05-09T06:49:39.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.810+0000 7f81691a8640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f814c06e120 0x7f814c0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:39.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.810+0000 7f81691a8640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f814c06e120 0x7f814c0705e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8158007350 tx=0x7f81580072e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:39.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:39.814+0000 7f8162ffd640 1 -- 172.21.15.63:0/2786159496 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f815407a000 con 0x7f8164101270 2024-05-09T06:49:40.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.039+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f816404e160 con 0x7f8164101270 2024-05-09T06:49:40.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.040+0000 7f8162ffd640 1 -- 172.21.15.63:0/2786159496 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v34) v1 ==== 107+0+4215 (secure 0 0 0) 0x7f815407d6c0 con 0x7f8164101270 2024-05-09T06:49:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:40.019 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:45:35.700972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":0,"incarnation":23,"state":"up:rejoin","state_seq":156,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T06:49:40.020 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 25 2024-05-09T06:49:40.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f814c06e120 msgr2=0x7f814c0705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f814c06e120 0x7f814c0705e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8158007350 tx=0x7f81580072e0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 msgr2=0x7f8164196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 0x7f8164196150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f815402c2b0 tx=0x7f815402c8e0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 shutdown_connections 2024-05-09T06:49:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f814c06e120 0x7f814c0705e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164101b90 0x7f8164196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 --2- 172.21.15.63:0/2786159496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8164101270 0x7f8164196150 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 >> 172.21.15.63:0/2786159496 conn(0x7f81640faa90 msgr2=0x7f8164102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 shutdown_connections 2024-05-09T06:49:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.042+0000 7f816bc0d640 1 -- 172.21.15.63:0/2786159496 wait complete. 2024-05-09T06:49:40.031 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:39 smithi063 ceph-mon[92765]: pgmap v332: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:40.031 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:39 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1425957355' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-09T06:49:40.031 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:39 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2081245699' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-09T06:49:40.031 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:39 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3852116842' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-09T06:49:40.032 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-05-09T06:49:40.033 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-05-09T06:49:40.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:39 smithi193 ceph-mon[81665]: pgmap v332: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:40.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:39 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1425957355' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-09T06:49:40.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:39 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2081245699' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-09T06:49:40.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:39 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3852116842' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-09T06:49:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.161+0000 7f01f2b4e640 1 -- 172.21.15.63:0/1037809327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec0ffb90 msgr2=0x7f01ec103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.161+0000 7f01f2b4e640 1 --2- 172.21.15.63:0/1037809327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec0ffb90 0x7f01ec103bd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009880 tx=0x7f01d802bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.161+0000 7f01f2b4e640 1 -- 172.21.15.63:0/1037809327 shutdown_connections 2024-05-09T06:49:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.161+0000 7f01f2b4e640 1 --2- 172.21.15.63:0/1037809327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec0ffb90 0x7f01ec103bd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.161+0000 7f01f2b4e640 1 --2- 172.21.15.63:0/1037809327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec0ff1e0 0x7f01ec0ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.161+0000 7f01f2b4e640 1 -- 172.21.15.63:0/1037809327 >> 172.21.15.63:0/1037809327 conn(0x7f01ec0faa60 msgr2=0x7f01ec0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.161+0000 7f01f2b4e640 1 -- 172.21.15.63:0/1037809327 shutdown_connections 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.161+0000 7f01f2b4e640 1 -- 172.21.15.63:0/1037809327 wait complete. 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01f2b4e640 1 Processor -- start 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01f2b4e640 1 -- start start 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01f2b4e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec0ff1e0 0x7f01ec195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01f2b4e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec1956c0 0x7f01ec199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01f2b4e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec195cc0 con 0x7f01ec1956c0 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01f2b4e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec195e30 con 0x7f01ec0ff1e0 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01ebfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec1956c0 0x7f01ec199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01ebfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec1956c0 0x7f01ec199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40334/0 (socket says 172.21.15.63:40334) 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01ebfff640 1 -- 172.21.15.63:0/224192352 learned_addr learned my addr 172.21.15.63:0/224192352 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01f08ea640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec0ff1e0 0x7f01ec195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01ebfff640 1 -- 172.21.15.63:0/224192352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec0ff1e0 msgr2=0x7f01ec195180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01ebfff640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec0ff1e0 0x7f01ec195180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01ebfff640 1 -- 172.21.15.63:0/224192352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8009560 con 0x7f01ec1956c0 2024-05-09T06:49:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01f08ea640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec0ff1e0 0x7f01ec195180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.162+0000 7f01ebfff640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec1956c0 0x7f01ec199a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01d8006010 tx=0x7f01d802cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.163+0000 7f01e9ffb640 1 -- 172.21.15.63:0/224192352 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d80044a0 con 0x7f01ec1956c0 2024-05-09T06:49:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.163+0000 7f01e9ffb640 1 -- 172.21.15.63:0/224192352 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01d8003ab0 con 0x7f01ec1956c0 2024-05-09T06:49:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.163+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec19a000 con 0x7f01ec1956c0 2024-05-09T06:49:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.163+0000 7f01e9ffb640 1 -- 172.21.15.63:0/224192352 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d8005590 con 0x7f01ec1956c0 2024-05-09T06:49:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.163+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec19a560 con 0x7f01ec1956c0 2024-05-09T06:49:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.164+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec1006b0 con 0x7f01ec1956c0 2024-05-09T06:49:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.164+0000 7f01e9ffb640 1 -- 172.21.15.63:0/224192352 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f01d8004dd0 con 0x7f01ec1956c0 2024-05-09T06:49:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.165+0000 7f01e9ffb640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f01dc06e170 0x7f01dc070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.165+0000 7f01e9ffb640 1 -- 172.21.15.63:0/224192352 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f01d80b05e0 con 0x7f01ec1956c0 2024-05-09T06:49:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.165+0000 7f01f08ea640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f01dc06e170 0x7f01dc070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.165+0000 7f01f08ea640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f01dc06e170 0x7f01dc070630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009b40 tx=0x7f01e0005e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.169+0000 7f01e9ffb640 1 -- 172.21.15.63:0/224192352 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f01d8079740 con 0x7f01ec1956c0 2024-05-09T06:49:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.392+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f01ec19a7d0 con 0x7f01ec1956c0 2024-05-09T06:49:40.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.393+0000 7f01e9ffb640 1 -- 172.21.15.63:0/224192352 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v34) v1 ==== 107+0+4224 (secure 0 0 0) 0x7f01d807ce00 con 0x7f01ec1956c0 2024-05-09T06:49:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:40.372 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:45:37.706625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":0,"incarnation":23,"state":"up:active","state_seq":157,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24285,"qdb_cluster":[24285]},"id":1}]} 2024-05-09T06:49:40.372 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-05-09T06:49:40.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f01dc06e170 msgr2=0x7f01dc070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f01dc06e170 0x7f01dc070630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009b40 tx=0x7f01e0005e50 comp rx=0 tx=0).stop 2024-05-09T06:49:40.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec1956c0 msgr2=0x7f01ec199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec1956c0 0x7f01ec199a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01d8006010 tx=0x7f01d802cea0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 shutdown_connections 2024-05-09T06:49:40.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f01dc06e170 0x7f01dc070630 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ec1956c0 0x7f01ec199a60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 --2- 172.21.15.63:0/224192352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01ec0ff1e0 0x7f01ec195180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 >> 172.21.15.63:0/224192352 conn(0x7f01ec0faa60 msgr2=0x7f01ec0fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:40.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 shutdown_connections 2024-05-09T06:49:40.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.395+0000 7f01f2b4e640 1 -- 172.21.15.63:0/224192352 wait complete. 2024-05-09T06:49:40.385 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-05-09T06:49:40.386 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-05-09T06:49:40.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.512+0000 7fa586b47640 1 -- 172.21.15.63:0/2675725924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800727b0 msgr2=0x7fa580072b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.512+0000 7fa586b47640 1 --2- 172.21.15.63:0/2675725924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800727b0 0x7fa580072b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa570009880 tx=0x7fa57002bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:40.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 -- 172.21.15.63:0/2675725924 shutdown_connections 2024-05-09T06:49:40.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 --2- 172.21.15.63:0/2675725924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5800730d0 0x7fa58010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 --2- 172.21.15.63:0/2675725924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800727b0 0x7fa580072b90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 -- 172.21.15.63:0/2675725924 >> 172.21.15.63:0/2675725924 conn(0x7fa5800fbb70 msgr2=0x7fa5800fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:40.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 -- 172.21.15.63:0/2675725924 shutdown_connections 2024-05-09T06:49:40.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 -- 172.21.15.63:0/2675725924 wait complete. 2024-05-09T06:49:40.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 Processor -- start 2024-05-09T06:49:40.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 -- start start 2024-05-09T06:49:40.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5800727b0 0x7fa580198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:40.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800730d0 0x7fa580198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:40.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580198e40 con 0x7fa5800730d0 2024-05-09T06:49:40.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.513+0000 7fa586b47640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580198fb0 con 0x7fa5800727b0 2024-05-09T06:49:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800730d0 0x7fa580198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800730d0 0x7fa580198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40356/0 (socket says 172.21.15.63:40356) 2024-05-09T06:49:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57ffff640 1 -- 172.21.15.63:0/3383453172 learned_addr learned my addr 172.21.15.63:0/3383453172 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa5848e3640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5800727b0 0x7fa580198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57ffff640 1 -- 172.21.15.63:0/3383453172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5800727b0 msgr2=0x7fa580198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57ffff640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5800727b0 0x7fa580198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57ffff640 1 -- 172.21.15.63:0/3383453172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa570009560 con 0x7fa5800730d0 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa5848e3640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5800727b0 0x7fa580198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57ffff640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800730d0 0x7fa580198840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa57400c8a0 tx=0x7fa57400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57dffb640 1 -- 172.21.15.63:0/3383453172 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa574007b80 con 0x7fa5800730d0 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57dffb640 1 -- 172.21.15.63:0/3383453172 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa574007ce0 con 0x7fa5800730d0 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa57dffb640 1 -- 172.21.15.63:0/3383453172 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa574018810 con 0x7fa5800730d0 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58019cd80 con 0x7fa5800730d0 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.514+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58019d2e0 con 0x7fa5800730d0 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.515+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa580073e60 con 0x7fa5800730d0 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.516+0000 7fa57dffb640 1 -- 172.21.15.63:0/3383453172 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa574018a80 con 0x7fa5800730d0 2024-05-09T06:49:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.516+0000 7fa57dffb640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa56806e170 0x7fa568070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:40.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.516+0000 7fa57dffb640 1 -- 172.21.15.63:0/3383453172 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fa57408e520 con 0x7fa5800730d0 2024-05-09T06:49:40.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.516+0000 7fa5848e3640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa56806e170 0x7fa568070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:40.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.517+0000 7fa5848e3640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa56806e170 0x7fa568070630 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa570000c00 tx=0x7fa570005d90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:40.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.521+0000 7fa57dffb640 1 -- 172.21.15.63:0/3383453172 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa574057680 con 0x7fa5800730d0 2024-05-09T06:49:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.746+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fa580106d80 con 0x7fa5800730d0 2024-05-09T06:49:40.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.747+0000 7fa57dffb640 1 -- 172.21.15.63:0/3383453172 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v34) v1 ==== 107+0+5074 (secure 0 0 0) 0x7fa57405ad40 con 0x7fa5800730d0 2024-05-09T06:49:40.726 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:40.726 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44323,"name":"cephfs.smithi193.ggtmyb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/2007543899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2007543899},{"type":"v1","addr":"172.21.15.193:6825","nonce":2007543899}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:45:37.706625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi193.sygxnf","rank":0,"incarnation":23,"state":"up:active","state_seq":157,"addr":"172.21.15.193:6827/354024957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":354024957},{"type":"v1","addr":"172.21.15.193:6827","nonce":354024957}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24285,"qdb_cluster":[24285]},"id":1}]} 2024-05-09T06:49:40.727 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 27 2024-05-09T06:49:40.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa56806e170 msgr2=0x7fa568070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa56806e170 0x7fa568070630 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa570000c00 tx=0x7fa570005d90 comp rx=0 tx=0).stop 2024-05-09T06:49:40.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800730d0 msgr2=0x7fa580198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800730d0 0x7fa580198840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa57400c8a0 tx=0x7fa57400cd60 comp rx=0 tx=0).stop 2024-05-09T06:49:40.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 shutdown_connections 2024-05-09T06:49:40.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa56806e170 0x7fa568070630 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5800730d0 0x7fa580198840 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 --2- 172.21.15.63:0/3383453172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5800727b0 0x7fa580198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 >> 172.21.15.63:0/3383453172 conn(0x7fa5800fbb70 msgr2=0x7fa5801048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:40.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 shutdown_connections 2024-05-09T06:49:40.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.749+0000 7fa586b47640 1 -- 172.21.15.63:0/3383453172 wait complete. 2024-05-09T06:49:40.739 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-05-09T06:49:40.740 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-05-09T06:49:40.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.867+0000 7faac4b5c640 1 -- 172.21.15.63:0/2832146871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 msgr2=0x7faac00ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.867+0000 7faac4b5c640 1 --2- 172.21.15.63:0/2832146871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 0x7faac00ff680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faab8009c20 tx=0x7faab802c1b0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.867+0000 7faac4b5c640 1 -- 172.21.15.63:0/2832146871 shutdown_connections 2024-05-09T06:49:40.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.867+0000 7faac4b5c640 1 --2- 172.21.15.63:0/2832146871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac00ffc50 0x7faac0103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.867+0000 7faac4b5c640 1 --2- 172.21.15.63:0/2832146871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 0x7faac00ff680 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.867+0000 7faac4b5c640 1 -- 172.21.15.63:0/2832146871 >> 172.21.15.63:0/2832146871 conn(0x7faac0074aa0 msgr2=0x7faac0074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:40.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.867+0000 7faac4b5c640 1 -- 172.21.15.63:0/2832146871 shutdown_connections 2024-05-09T06:49:40.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.867+0000 7faac4b5c640 1 -- 172.21.15.63:0/2832146871 wait complete. 2024-05-09T06:49:40.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faac4b5c640 1 Processor -- start 2024-05-09T06:49:40.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faac4b5c640 1 -- start start 2024-05-09T06:49:40.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faac4b5c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 0x7faac0196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:40.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faac4b5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac0197500 0x7faac0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:40.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faac4b5c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac0068370 con 0x7faac00ff2a0 2024-05-09T06:49:40.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faac4b5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01a0030 con 0x7faac0197500 2024-05-09T06:49:40.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faabe59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 0x7faac0196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:40.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faabe59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 0x7faac0196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40366/0 (socket says 172.21.15.63:40366) 2024-05-09T06:49:40.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faabe59c640 1 -- 172.21.15.63:0/2978851233 learned_addr learned my addr 172.21.15.63:0/2978851233 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:40.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faabdd9b640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac0197500 0x7faac0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:40.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faabe59c640 1 -- 172.21.15.63:0/2978851233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac0197500 msgr2=0x7faac0197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:40.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faabe59c640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac0197500 0x7faac0197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:40.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faabe59c640 1 -- 172.21.15.63:0/2978851233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab8009900 con 0x7faac00ff2a0 2024-05-09T06:49:40.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faabdd9b640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac0197500 0x7faac0197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:40.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.868+0000 7faabe59c640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 0x7faac0196fc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faab8000f80 tx=0x7faab8032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:40.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.869+0000 7faab77fe640 1 -- 172.21.15.63:0/2978851233 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab8034610 con 0x7faac00ff2a0 2024-05-09T06:49:40.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.869+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac01a0230 con 0x7faac00ff2a0 2024-05-09T06:49:40.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.869+0000 7faab77fe640 1 -- 172.21.15.63:0/2978851233 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faab8003e30 con 0x7faac00ff2a0 2024-05-09T06:49:40.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.869+0000 7faab77fe640 1 -- 172.21.15.63:0/2978851233 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab8004830 con 0x7faac00ff2a0 2024-05-09T06:49:40.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.869+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac01a0730 con 0x7faac00ff2a0 2024-05-09T06:49:40.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.870+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac01006c0 con 0x7faac00ff2a0 2024-05-09T06:49:40.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.870+0000 7faab77fe640 1 -- 172.21.15.63:0/2978851233 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faab803b040 con 0x7faac00ff2a0 2024-05-09T06:49:40.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.871+0000 7faab77fe640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faaac06e120 0x7faaac0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:40.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.871+0000 7faab77fe640 1 -- 172.21.15.63:0/2978851233 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7faab80b0fe0 con 0x7faac00ff2a0 2024-05-09T06:49:40.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.871+0000 7faabdd9b640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faaac06e120 0x7faaac0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:40.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.871+0000 7faabdd9b640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faaac06e120 0x7faaac0705e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faac0072e40 tx=0x7faaa8005e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:40.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:40.875+0000 7faab77fe640 1 -- 172.21.15.63:0/2978851233 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faab807a140 con 0x7faac00ff2a0 2024-05-09T06:49:41.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.096+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7faac00ffc50 con 0x7faac00ff2a0 2024-05-09T06:49:41.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.097+0000 7faab77fe640 1 -- 172.21.15.63:0/2978851233 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v34) v1 ==== 107+0+4273 (secure 0 0 0) 0x7faab807d800 con 0x7faac00ff2a0 2024-05-09T06:49:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:41.076 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44323,"name":"cephfs.smithi193.ggtmyb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/2007543899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2007543899},{"type":"v1","addr":"172.21.15.193:6825","nonce":2007543899}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:45:51.754032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T06:49:41.076 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-05-09T06:49:41.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faaac06e120 msgr2=0x7faaac0705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faaac06e120 0x7faaac0705e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faac0072e40 tx=0x7faaa8005e50 comp rx=0 tx=0).stop 2024-05-09T06:49:41.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 msgr2=0x7faac0196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 0x7faac0196fc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faab8000f80 tx=0x7faab8032570 comp rx=0 tx=0).stop 2024-05-09T06:49:41.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 shutdown_connections 2024-05-09T06:49:41.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7faaac06e120 0x7faaac0705e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac0197500 0x7faac0197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 --2- 172.21.15.63:0/2978851233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faac00ff2a0 0x7faac0196fc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 >> 172.21.15.63:0/2978851233 conn(0x7faac0074aa0 msgr2=0x7faac0102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:41.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 shutdown_connections 2024-05-09T06:49:41.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.099+0000 7faac4b5c640 1 -- 172.21.15.63:0/2978851233 wait complete. 2024-05-09T06:49:41.088 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:40 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2786159496' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-09T06:49:41.088 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:40 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/224192352' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-09T06:49:41.089 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-05-09T06:49:41.090 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-05-09T06:49:41.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:40 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2786159496' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-09T06:49:41.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:40 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/224192352' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-09T06:49:41.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.215+0000 7f5f3c42d640 1 -- 172.21.15.63:0/908995969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f340ffc50 msgr2=0x7f5f34103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.215+0000 7f5f3c42d640 1 --2- 172.21.15.63:0/908995969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f340ffc50 0x7f5f34103be0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009880 tx=0x7f5f2402bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 -- 172.21.15.63:0/908995969 shutdown_connections 2024-05-09T06:49:41.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 --2- 172.21.15.63:0/908995969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f340ffc50 0x7f5f34103be0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 --2- 172.21.15.63:0/908995969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f340ff2a0 0x7f5f340ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 -- 172.21.15.63:0/908995969 >> 172.21.15.63:0/908995969 conn(0x7f5f34074aa0 msgr2=0x7f5f34074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 -- 172.21.15.63:0/908995969 shutdown_connections 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 -- 172.21.15.63:0/908995969 wait complete. 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 Processor -- start 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 -- start start 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f340ff2a0 0x7f5f34195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f34195590 0x7f5f341998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f34195b90 con 0x7f5f34195590 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.216+0000 7f5f3c42d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f34195d00 con 0x7f5f340ff2a0 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f399c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f34195590 0x7f5f341998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f399c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f34195590 0x7f5f341998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40392/0 (socket says 172.21.15.63:40392) 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f399c8640 1 -- 172.21.15.63:0/1114588663 learned_addr learned my addr 172.21.15.63:0/1114588663 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f3a1c9640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f340ff2a0 0x7f5f34195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f399c8640 1 -- 172.21.15.63:0/1114588663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f340ff2a0 msgr2=0x7f5f34195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f399c8640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f340ff2a0 0x7f5f34195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f399c8640 1 -- 172.21.15.63:0/1114588663 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f24009560 con 0x7f5f34195590 2024-05-09T06:49:41.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f399c8640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f34195590 0x7f5f341998e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f2402cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:41.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f2b7fe640 1 -- 172.21.15.63:0/1114588663 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f240044a0 con 0x7f5f34195590 2024-05-09T06:49:41.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f34199e80 con 0x7f5f34195590 2024-05-09T06:49:41.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f2b7fe640 1 -- 172.21.15.63:0/1114588663 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f24003ab0 con 0x7f5f34195590 2024-05-09T06:49:41.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f2b7fe640 1 -- 172.21.15.63:0/1114588663 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f24005590 con 0x7f5f34195590 2024-05-09T06:49:41.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.217+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3419a3e0 con 0x7f5f34195590 2024-05-09T06:49:41.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.218+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f341006c0 con 0x7f5f34195590 2024-05-09T06:49:41.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.219+0000 7f5f2b7fe640 1 -- 172.21.15.63:0/1114588663 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5f24004dd0 con 0x7f5f34195590 2024-05-09T06:49:41.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.219+0000 7f5f2b7fe640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5f2006e170 0x7f5f20070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:41.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.219+0000 7f5f3a1c9640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5f2006e170 0x7f5f20070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:41.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.219+0000 7f5f2b7fe640 1 -- 172.21.15.63:0/1114588663 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f5f240b0650 con 0x7f5f34195590 2024-05-09T06:49:41.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.220+0000 7f5f3a1c9640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5f2006e170 0x7f5f20070630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000bbe0 tx=0x7f5f3000b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:41.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.224+0000 7f5f2b7fe640 1 -- 172.21.15.63:0/1114588663 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5f24079860 con 0x7f5f34195590 2024-05-09T06:49:41.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.448+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f5f3419a6c0 con 0x7f5f34195590 2024-05-09T06:49:41.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.449+0000 7f5f2b7fe640 1 -- 172.21.15.63:0/1114588663 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v34) v1 ==== 107+0+4352 (secure 0 0 0) 0x7f5f2407cf20 con 0x7f5f34195590 2024-05-09T06:49:41.428 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:41.428 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44323,"name":"cephfs.smithi193.ggtmyb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/2007543899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2007543899},{"type":"v1","addr":"172.21.15.193:6825","nonce":2007543899}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:45:51.758882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T06:49:41.428 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 29 2024-05-09T06:49:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5f2006e170 msgr2=0x7f5f20070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5f2006e170 0x7f5f20070630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000bbe0 tx=0x7f5f3000b390 comp rx=0 tx=0).stop 2024-05-09T06:49:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f34195590 msgr2=0x7f5f341998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f34195590 0x7f5f341998e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f2402cea0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 shutdown_connections 2024-05-09T06:49:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f5f2006e170 0x7f5f20070630 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f34195590 0x7f5f341998e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 --2- 172.21.15.63:0/1114588663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f340ff2a0 0x7f5f34195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 >> 172.21.15.63:0/1114588663 conn(0x7f5f34074aa0 msgr2=0x7f5f34102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:41.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 shutdown_connections 2024-05-09T06:49:41.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.451+0000 7f5f3c42d640 1 -- 172.21.15.63:0/1114588663 wait complete. 2024-05-09T06:49:41.441 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-05-09T06:49:41.441 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-05-09T06:49:41.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.568+0000 7f90fd870640 1 -- 172.21.15.63:0/324483639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f80ffbc0 msgr2=0x7f90f8103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.568+0000 7f90fd870640 1 --2- 172.21.15.63:0/324483639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f80ffbc0 0x7f90f8103c00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009880 tx=0x7f90e002bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.568+0000 7f90fd870640 1 -- 172.21.15.63:0/324483639 shutdown_connections 2024-05-09T06:49:41.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.568+0000 7f90fd870640 1 --2- 172.21.15.63:0/324483639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f80ffbc0 0x7f90f8103c00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.568+0000 7f90fd870640 1 --2- 172.21.15.63:0/324483639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f80ff210 0x7f90f80ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.568+0000 7f90fd870640 1 -- 172.21.15.63:0/324483639 >> 172.21.15.63:0/324483639 conn(0x7f90f80faa90 msgr2=0x7f90f80fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.568+0000 7f90fd870640 1 -- 172.21.15.63:0/324483639 shutdown_connections 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.568+0000 7f90fd870640 1 -- 172.21.15.63:0/324483639 wait complete. 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.568+0000 7f90fd870640 1 Processor -- start 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90fd870640 1 -- start start 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90fd870640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f80ff210 0x7f90f8196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90fd870640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f8197510 0x7f90f8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90fd870640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f81a0040 con 0x7f90f8197510 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90fd870640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f81a0180 con 0x7f90f80ff210 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90f67fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f8197510 0x7f90f8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90f67fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f8197510 0x7f90f8197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40418/0 (socket says 172.21.15.63:40418) 2024-05-09T06:49:41.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90f67fc640 1 -- 172.21.15.63:0/3947088968 learned_addr learned my addr 172.21.15.63:0/3947088968 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90f6ffd640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f80ff210 0x7f90f8196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90f67fc640 1 -- 172.21.15.63:0/3947088968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f80ff210 msgr2=0x7f90f8196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90f67fc640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f80ff210 0x7f90f8196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90f67fc640 1 -- 172.21.15.63:0/3947088968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0009560 con 0x7f90f8197510 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90f6ffd640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f80ff210 0x7f90f8196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.569+0000 7f90f67fc640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f8197510 0x7f90f8197970 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f90e0036040 tx=0x7f90e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.570+0000 7f90fc86e640 1 -- 172.21.15.63:0/3947088968 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90e00035f0 con 0x7f90f8197510 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.570+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f81a02c0 con 0x7f90f8197510 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.570+0000 7f90fc86e640 1 -- 172.21.15.63:0/3947088968 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90e0003c10 con 0x7f90f8197510 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.570+0000 7f90fc86e640 1 -- 172.21.15.63:0/3947088968 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90e0034440 con 0x7f90f8197510 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.570+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f81a0820 con 0x7f90f8197510 2024-05-09T06:49:41.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.571+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f81006e0 con 0x7f90f8197510 2024-05-09T06:49:41.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.571+0000 7f90fc86e640 1 -- 172.21.15.63:0/3947088968 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f90e0004f60 con 0x7f90f8197510 2024-05-09T06:49:41.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.572+0000 7f90fc86e640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f90e406e170 0x7f90e4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:41.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.572+0000 7f90fc86e640 1 -- 172.21.15.63:0/3947088968 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f90e00b06e0 con 0x7f90f8197510 2024-05-09T06:49:41.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.572+0000 7f90f6ffd640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f90e406e170 0x7f90e4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:41.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.572+0000 7f90f6ffd640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f90e406e170 0x7f90e4070630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005fd0 tx=0x7f90e8005ea0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:41.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.576+0000 7f90fc86e640 1 -- 172.21.15.63:0/3947088968 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f90e0079840 con 0x7f90f8197510 2024-05-09T06:49:41.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.798+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f90f8107510 con 0x7f90f8197510 2024-05-09T06:49:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.799+0000 7f90fc86e640 1 -- 172.21.15.63:0/3947088968 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v34) v1 ==== 107+0+5199 (secure 0 0 0) 0x7f90e007cf00 con 0x7f90f8197510 2024-05-09T06:49:41.778 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:41.778 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44323,"name":"cephfs.smithi193.ggtmyb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/2007543899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2007543899},{"type":"v1","addr":"172.21.15.193:6825","nonce":2007543899}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44347,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/790240017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":790240017},{"type":"v1","addr":"172.21.15.193:6827","nonce":790240017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:45:51.758882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T06:49:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 30 2024-05-09T06:49:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.801+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f90e406e170 msgr2=0x7f90e4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.801+0000 7f90fd870640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f90e406e170 0x7f90e4070630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005fd0 tx=0x7f90e8005ea0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.801+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f8197510 msgr2=0x7f90f8197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.801+0000 7f90fd870640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f8197510 0x7f90f8197970 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f90e0036040 tx=0x7f90e0004040 comp rx=0 tx=0).stop 2024-05-09T06:49:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.801+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 shutdown_connections 2024-05-09T06:49:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.801+0000 7f90fd870640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f90e406e170 0x7f90e4070630 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.801+0000 7f90fd870640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90f8197510 0x7f90f8197970 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.801+0000 7f90fd870640 1 --2- 172.21.15.63:0/3947088968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f80ff210 0x7f90f8196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.801+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 >> 172.21.15.63:0/3947088968 conn(0x7f90f80faa90 msgr2=0x7f90f80fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.802+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 shutdown_connections 2024-05-09T06:49:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.802+0000 7f90fd870640 1 -- 172.21.15.63:0/3947088968 wait complete. 2024-05-09T06:49:41.790 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:41 smithi063 ceph-mon[92765]: pgmap v333: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:41.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:41 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3383453172' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-09T06:49:41.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:41 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2978851233' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-09T06:49:41.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:41 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1114588663' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-09T06:49:41.792 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-05-09T06:49:41.792 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-05-09T06:49:41.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.919+0000 7f6a74ba9640 1 -- 172.21.15.63:0/2966540247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70100cb0 msgr2=0x7f6a70104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.919+0000 7f6a74ba9640 1 --2- 172.21.15.63:0/2966540247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70100cb0 0x7f6a70104cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009880 tx=0x7f6a6402bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.919+0000 7f6a74ba9640 1 -- 172.21.15.63:0/2966540247 shutdown_connections 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.919+0000 7f6a74ba9640 1 --2- 172.21.15.63:0/2966540247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70100cb0 0x7f6a70104cf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.919+0000 7f6a74ba9640 1 --2- 172.21.15.63:0/2966540247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a70100300 0x7f6a701006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.919+0000 7f6a74ba9640 1 -- 172.21.15.63:0/2966540247 >> 172.21.15.63:0/2966540247 conn(0x7f6a700fbb60 msgr2=0x7f6a700fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.919+0000 7f6a74ba9640 1 -- 172.21.15.63:0/2966540247 shutdown_connections 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.919+0000 7f6a74ba9640 1 -- 172.21.15.63:0/2966540247 wait complete. 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.919+0000 7f6a74ba9640 1 Processor -- start 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a74ba9640 1 -- start start 2024-05-09T06:49:41.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a74ba9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a70100300 0x7f6a70072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a74ba9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70072c40 0x7f6a700730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a74ba9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a70074660 con 0x7f6a70072c40 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a74ba9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a701a22e0 con 0x7f6a70100300 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a6dd9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70072c40 0x7f6a700730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a6dd9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70072c40 0x7f6a700730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40426/0 (socket says 172.21.15.63:40426) 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a6dd9b640 1 -- 172.21.15.63:0/3972267512 learned_addr learned my addr 172.21.15.63:0/3972267512 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a6e59c640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a70100300 0x7f6a70072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a6dd9b640 1 -- 172.21.15.63:0/3972267512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a70100300 msgr2=0x7f6a70072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a6dd9b640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a70100300 0x7f6a70072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a6dd9b640 1 -- 172.21.15.63:0/3972267512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64009560 con 0x7f6a70072c40 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.920+0000 7f6a6dd9b640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70072c40 0x7f6a700730a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a6402c220 tx=0x7f6a6402cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:41.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.921+0000 7f6a5f7fe640 1 -- 172.21.15.63:0/3972267512 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a640044a0 con 0x7f6a70072c40 2024-05-09T06:49:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.921+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a701a2480 con 0x7f6a70072c40 2024-05-09T06:49:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.921+0000 7f6a5f7fe640 1 -- 172.21.15.63:0/3972267512 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a64003ab0 con 0x7f6a70072c40 2024-05-09T06:49:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.921+0000 7f6a5f7fe640 1 -- 172.21.15.63:0/3972267512 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a64005560 con 0x7f6a70072c40 2024-05-09T06:49:41.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.921+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a701a29b0 con 0x7f6a70072c40 2024-05-09T06:49:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.922+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a701017d0 con 0x7f6a70072c40 2024-05-09T06:49:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.922+0000 7f6a5f7fe640 1 -- 172.21.15.63:0/3972267512 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6a64004dd0 con 0x7f6a70072c40 2024-05-09T06:49:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.923+0000 7f6a5f7fe640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6a5806e170 0x7f6a58070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.923+0000 7f6a5f7fe640 1 -- 172.21.15.63:0/3972267512 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f6a640b0660 con 0x7f6a70072c40 2024-05-09T06:49:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.923+0000 7f6a6e59c640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6a5806e170 0x7f6a58070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:41.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.923+0000 7f6a6e59c640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6a5806e170 0x7f6a58070630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a60005fd0 tx=0x7f6a6000b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:41.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:41.927+0000 7f6a5f7fe640 1 -- 172.21.15.63:0/3972267512 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6a64079870 con 0x7f6a70072c40 2024-05-09T06:49:42.033 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:41 smithi193 ceph-mon[81665]: pgmap v333: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:42.033 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:41 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3383453172' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-09T06:49:42.033 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:41 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2978851233' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-09T06:49:42.033 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:41 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1114588663' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-09T06:49:42.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.155+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f6a7004e160 con 0x7f6a70072c40 2024-05-09T06:49:42.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.156+0000 7f6a5f7fe640 1 -- 172.21.15.63:0/3972267512 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v34) v1 ==== 107+0+5203 (secure 0 0 0) 0x7f6a6407cf30 con 0x7f6a70072c40 2024-05-09T06:49:42.135 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:42.135 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44323,"name":"cephfs.smithi193.ggtmyb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/2007543899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2007543899},{"type":"v1","addr":"172.21.15.193:6825","nonce":2007543899}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44347,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/790240017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":790240017},{"type":"v1","addr":"172.21.15.193:6827","nonce":790240017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:46:00.885514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":0,"incarnation":29,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T06:49:42.135 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 31 2024-05-09T06:49:42.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6a5806e170 msgr2=0x7f6a58070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6a5806e170 0x7f6a58070630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a60005fd0 tx=0x7f6a6000b390 comp rx=0 tx=0).stop 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70072c40 msgr2=0x7f6a700730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70072c40 0x7f6a700730a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a6402c220 tx=0x7f6a6402cea0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 shutdown_connections 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f6a5806e170 0x7f6a58070630 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a70072c40 0x7f6a700730a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 --2- 172.21.15.63:0/3972267512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a70100300 0x7f6a70072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 >> 172.21.15.63:0/3972267512 conn(0x7f6a700fbb60 msgr2=0x7f6a700fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 shutdown_connections 2024-05-09T06:49:42.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.158+0000 7f6a74ba9640 1 -- 172.21.15.63:0/3972267512 wait complete. 2024-05-09T06:49:42.149 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-05-09T06:49:42.149 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-05-09T06:49:42.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.276+0000 7f3262b4e640 1 -- 172.21.15.63:0/2511184362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 msgr2=0x7f325c0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.276+0000 7f3262b4e640 1 --2- 172.21.15.63:0/2511184362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 0x7f325c0ff5f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3250009880 tx=0x7f325002bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.276+0000 7f3262b4e640 1 -- 172.21.15.63:0/2511184362 shutdown_connections 2024-05-09T06:49:42.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.276+0000 7f3262b4e640 1 --2- 172.21.15.63:0/2511184362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f325c0ffbc0 0x7f325c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.276+0000 7f3262b4e640 1 --2- 172.21.15.63:0/2511184362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 0x7f325c0ff5f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.276+0000 7f3262b4e640 1 -- 172.21.15.63:0/2511184362 >> 172.21.15.63:0/2511184362 conn(0x7f325c0faa90 msgr2=0x7f325c0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.276+0000 7f3262b4e640 1 -- 172.21.15.63:0/2511184362 shutdown_connections 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.276+0000 7f3262b4e640 1 -- 172.21.15.63:0/2511184362 wait complete. 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f3262b4e640 1 Processor -- start 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f3262b4e640 1 -- start start 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f3262b4e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 0x7f325c1961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f3262b4e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f325c1966e0 0x7f325c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f3262b4e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c196ce0 con 0x7f325c0ff210 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f3262b4e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c196e50 con 0x7f325c1966e0 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f32608ea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 0x7f325c1961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f32608ea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 0x7f325c1961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40448/0 (socket says 172.21.15.63:40448) 2024-05-09T06:49:42.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f32608ea640 1 -- 172.21.15.63:0/3020450829 learned_addr learned my addr 172.21.15.63:0/3020450829 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.277+0000 7f325bfff640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f325c1966e0 0x7f325c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f32608ea640 1 -- 172.21.15.63:0/3020450829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f325c1966e0 msgr2=0x7f325c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f32608ea640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f325c1966e0 0x7f325c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f32608ea640 1 -- 172.21.15.63:0/3020450829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3250009560 con 0x7f325c0ff210 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f325bfff640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f325c1966e0 0x7f325c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f32608ea640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 0x7f325c1961a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3250006010 tx=0x7f325002cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f3259ffb640 1 -- 172.21.15.63:0/3020450829 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32500044a0 con 0x7f325c0ff210 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f3259ffb640 1 -- 172.21.15.63:0/3020450829 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3250003ab0 con 0x7f325c0ff210 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325c19afd0 con 0x7f325c0ff210 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f3259ffb640 1 -- 172.21.15.63:0/3020450829 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3250005590 con 0x7f325c0ff210 2024-05-09T06:49:42.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.278+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c19b530 con 0x7f325c0ff210 2024-05-09T06:49:42.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.279+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325c1006e0 con 0x7f325c0ff210 2024-05-09T06:49:42.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.280+0000 7f3259ffb640 1 -- 172.21.15.63:0/3020450829 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3250004dd0 con 0x7f325c0ff210 2024-05-09T06:49:42.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.280+0000 7f3259ffb640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f324c06e170 0x7f324c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:42.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.281+0000 7f325bfff640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f324c06e170 0x7f324c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:42.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.281+0000 7f3259ffb640 1 -- 172.21.15.63:0/3020450829 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f32500b0550 con 0x7f325c0ff210 2024-05-09T06:49:42.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.281+0000 7f325bfff640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f324c06e170 0x7f324c070630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f325c066a80 tx=0x7f324800b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:42.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.285+0000 7f3259ffb640 1 -- 172.21.15.63:0/3020450829 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3250079760 con 0x7f325c0ff210 2024-05-09T06:49:42.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.509+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f325c19b7a0 con 0x7f325c0ff210 2024-05-09T06:49:42.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.509+0000 7f3259ffb640 1 -- 172.21.15.63:0/3020450829 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v34) v1 ==== 107+0+5200 (secure 0 0 0) 0x7f325007ce20 con 0x7f325c0ff210 2024-05-09T06:49:42.488 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:42.488 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44323,"name":"cephfs.smithi193.ggtmyb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/2007543899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2007543899},{"type":"v1","addr":"172.21.15.193:6825","nonce":2007543899}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44347,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/790240017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":790240017},{"type":"v1","addr":"172.21.15.193:6827","nonce":790240017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:46:01.687635+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":0,"incarnation":29,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T06:49:42.488 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 32 2024-05-09T06:49:42.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f324c06e170 msgr2=0x7f324c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f324c06e170 0x7f324c070630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f325c066a80 tx=0x7f324800b390 comp rx=0 tx=0).stop 2024-05-09T06:49:42.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 msgr2=0x7f325c1961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 0x7f325c1961a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3250006010 tx=0x7f325002cea0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 shutdown_connections 2024-05-09T06:49:42.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f324c06e170 0x7f324c070630 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f325c1966e0 0x7f325c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 --2- 172.21.15.63:0/3020450829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f325c0ff210 0x7f325c1961a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 >> 172.21.15.63:0/3020450829 conn(0x7f325c0faa90 msgr2=0x7f325c068960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:42.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 shutdown_connections 2024-05-09T06:49:42.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.511+0000 7f3262b4e640 1 -- 172.21.15.63:0/3020450829 wait complete. 2024-05-09T06:49:42.501 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-05-09T06:49:42.502 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-05-09T06:49:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.629+0000 7fd0a3269640 1 -- 172.21.15.63:0/4078066440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0730c0 msgr2=0x7fd09c110b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.629+0000 7fd0a3269640 1 --2- 172.21.15.63:0/4078066440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0730c0 0x7fd09c110b60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd098009880 tx=0x7fd09802bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.629+0000 7fd0a3269640 1 -- 172.21.15.63:0/4078066440 shutdown_connections 2024-05-09T06:49:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.629+0000 7fd0a3269640 1 --2- 172.21.15.63:0/4078066440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0730c0 0x7fd09c110b60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.629+0000 7fd0a3269640 1 --2- 172.21.15.63:0/4078066440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0727a0 0x7fd09c072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.629+0000 7fd0a3269640 1 -- 172.21.15.63:0/4078066440 >> 172.21.15.63:0/4078066440 conn(0x7fd09c0fbb60 msgr2=0x7fd09c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.629+0000 7fd0a3269640 1 -- 172.21.15.63:0/4078066440 shutdown_connections 2024-05-09T06:49:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.629+0000 7fd0a3269640 1 -- 172.21.15.63:0/4078066440 wait complete. 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.630+0000 7fd0a3269640 1 Processor -- start 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.630+0000 7fd0a3269640 1 -- start start 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.630+0000 7fd0a3269640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0727a0 0x7fd09c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.630+0000 7fd0a3269640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0730c0 0x7fd09c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.630+0000 7fd0a3269640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c19d260 con 0x7fd09c0727a0 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.630+0000 7fd0a3269640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c1a0f70 con 0x7fd09c0730c0 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.630+0000 7fd0a1005640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0727a0 0x7fd09c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.630+0000 7fd0a1005640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0727a0 0x7fd09c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40470/0 (socket says 172.21.15.63:40470) 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.630+0000 7fd0a1005640 1 -- 172.21.15.63:0/513821948 learned_addr learned my addr 172.21.15.63:0/513821948 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.631+0000 7fd0a1005640 1 -- 172.21.15.63:0/513821948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0730c0 msgr2=0x7fd09c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.631+0000 7fd0a1005640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0730c0 0x7fd09c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.631+0000 7fd0a1005640 1 -- 172.21.15.63:0/513821948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd098009560 con 0x7fd09c0727a0 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.631+0000 7fd0a1005640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0727a0 0x7fd09c19c690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e970 tx=0x7fd09000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.631+0000 7fd08e7fc640 1 -- 172.21.15.63:0/513821948 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd09000cc90 con 0x7fd09c0727a0 2024-05-09T06:49:42.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.631+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09c1a1250 con 0x7fd09c0727a0 2024-05-09T06:49:42.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.631+0000 7fd08e7fc640 1 -- 172.21.15.63:0/513821948 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd09000cdf0 con 0x7fd09c0727a0 2024-05-09T06:49:42.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.631+0000 7fd08e7fc640 1 -- 172.21.15.63:0/513821948 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd090010430 con 0x7fd09c0727a0 2024-05-09T06:49:42.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.631+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c1a17b0 con 0x7fd09c0727a0 2024-05-09T06:49:42.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.632+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09c073ee0 con 0x7fd09c0727a0 2024-05-09T06:49:42.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.632+0000 7fd08e7fc640 1 -- 172.21.15.63:0/513821948 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd0900046c0 con 0x7fd09c0727a0 2024-05-09T06:49:42.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.633+0000 7fd08e7fc640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd08806e170 0x7fd088070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:42.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.633+0000 7fd08e7fc640 1 -- 172.21.15.63:0/513821948 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fd090014070 con 0x7fd09c0727a0 2024-05-09T06:49:42.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.633+0000 7fd0a0804640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd08806e170 0x7fd088070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:42.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.633+0000 7fd0a0804640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd08806e170 0x7fd088070630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd098036040 tx=0x7fd098000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:42.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.637+0000 7fd08e7fc640 1 -- 172.21.15.63:0/513821948 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd090057a50 con 0x7fd09c0727a0 2024-05-09T06:49:42.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:42 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3947088968' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-09T06:49:42.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:42 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3972267512' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-09T06:49:42.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:49:42.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:42 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3020450829' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-09T06:49:42.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.865+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fd09c04e160 con 0x7fd09c0727a0 2024-05-09T06:49:42.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.866+0000 7fd08e7fc640 1 -- 172.21.15.63:0/513821948 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v34) v1 ==== 107+0+5209 (secure 0 0 0) 0x7fd09005b110 con 0x7fd09c0727a0 2024-05-09T06:49:42.845 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:42.845 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi063.fbzxbf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/2182150991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2182150991},{"type":"v1","addr":"172.21.15.63:6829","nonce":2182150991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44323,"name":"cephfs.smithi193.ggtmyb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/2007543899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":2007543899},{"type":"v1","addr":"172.21.15.193:6825","nonce":2007543899}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44347,"name":"cephfs.smithi193.sygxnf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/790240017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":790240017},{"type":"v1","addr":"172.21.15.193:6827","nonce":790240017}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T06:35:03.455858+0000","modified":"2024-05-09T06:46:03.694426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi063.aoiibw","rank":0,"incarnation":29,"state":"up:active","state_seq":20,"addr":"172.21.15.63:6827/566601990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":566601990},{"type":"v1","addr":"172.21.15.63:6827","nonce":566601990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-05-09T06:49:42.845 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd08806e170 msgr2=0x7fd088070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd08806e170 0x7fd088070630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd098036040 tx=0x7fd098000bc0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0727a0 msgr2=0x7fd09c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0727a0 0x7fd09c19c690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e970 tx=0x7fd09000ed70 comp rx=0 tx=0).stop 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 shutdown_connections 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fd08806e170 0x7fd088070630 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0730c0 0x7fd09c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 --2- 172.21.15.63:0/513821948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd09c0727a0 0x7fd09c19c690 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 >> 172.21.15.63:0/513821948 conn(0x7fd09c0fbb60 msgr2=0x7fd09c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 shutdown_connections 2024-05-09T06:49:42.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:42.868+0000 7fd0a3269640 1 -- 172.21.15.63:0/513821948 wait complete. 2024-05-09T06:49:42.858 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-05-09T06:49:42.858 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-09T06:49:42.869 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-09T06:49:42.870 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:49:42.870 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:49:42.888 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:49:42.888 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:49:42.948 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:49:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.113+0000 7fe4a4a58640 1 -- 172.21.15.63:0/3690477304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a0101c00 msgr2=0x7fe4a010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.113+0000 7fe4a4a58640 1 --2- 172.21.15.63:0/3690477304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a0101c00 0x7fe4a010a0e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe494009880 tx=0x7fe49402bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.113+0000 7fe4a4a58640 1 -- 172.21.15.63:0/3690477304 shutdown_connections 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.113+0000 7fe4a4a58640 1 --2- 172.21.15.63:0/3690477304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a0101c00 0x7fe4a010a0e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.113+0000 7fe4a4a58640 1 --2- 172.21.15.63:0/3690477304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4a01012e0 0x7fe4a01016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.113+0000 7fe4a4a58640 1 -- 172.21.15.63:0/3690477304 >> 172.21.15.63:0/3690477304 conn(0x7fe4a00faac0 msgr2=0x7fe4a00fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.113+0000 7fe4a4a58640 1 -- 172.21.15.63:0/3690477304 shutdown_connections 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.113+0000 7fe4a4a58640 1 -- 172.21.15.63:0/3690477304 wait complete. 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe4a4a58640 1 Processor -- start 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe4a4a58640 1 -- start start 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe4a4a58640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a01012e0 0x7fe4a0196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:43.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe4a4a58640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4a0101c00 0x7fe4a0196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe4a4a58640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0196d20 con 0x7fe4a01012e0 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe4a4a58640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a019a9e0 con 0x7fe4a0101c00 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe49e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a01012e0 0x7fe4a0196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe49e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a01012e0 0x7fe4a0196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40484/0 (socket says 172.21.15.63:40484) 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe49e59c640 1 -- 172.21.15.63:0/2165302951 learned_addr learned my addr 172.21.15.63:0/2165302951 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe49e59c640 1 -- 172.21.15.63:0/2165302951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4a0101c00 msgr2=0x7fe4a0196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe49e59c640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4a0101c00 0x7fe4a0196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.114+0000 7fe49e59c640 1 -- 172.21.15.63:0/2165302951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494009560 con 0x7fe4a01012e0 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.115+0000 7fe49e59c640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a01012e0 0x7fe4a0196150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d6f0 tx=0x7fe49000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.115+0000 7fe48f7fe640 1 -- 172.21.15.63:0/2165302951 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe490009a30 con 0x7fe4a01012e0 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.115+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a019acc0 con 0x7fe4a01012e0 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.115+0000 7fe48f7fe640 1 -- 172.21.15.63:0/2165302951 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4900054e0 con 0x7fe4a01012e0 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.115+0000 7fe48f7fe640 1 -- 172.21.15.63:0/2165302951 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe490010460 con 0x7fe4a01012e0 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.115+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a019b220 con 0x7fe4a01012e0 2024-05-09T06:49:43.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.116+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a0068dc0 con 0x7fe4a01012e0 2024-05-09T06:49:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.116+0000 7fe48f7fe640 1 -- 172.21.15.63:0/2165302951 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe490009b90 con 0x7fe4a01012e0 2024-05-09T06:49:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.117+0000 7fe48f7fe640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe48806e170 0x7fe488070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.117+0000 7fe49dd9b640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe48806e170 0x7fe488070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.117+0000 7fe48f7fe640 1 -- 172.21.15.63:0/2165302951 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fe49008deb0 con 0x7fe4a01012e0 2024-05-09T06:49:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.117+0000 7fe49dd9b640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe48806e170 0x7fe488070630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe494036040 tx=0x7fe494000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:43.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.122+0000 7fe48f7fe640 1 -- 172.21.15.63:0/2165302951 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe490057010 con 0x7fe4a01012e0 2024-05-09T06:49:43.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:42 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3947088968' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-09T06:49:43.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:42 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3972267512' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-09T06:49:43.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:49:43.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:42 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3020450829' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-09T06:49:43.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.310+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4a004e160 con 0x7fe4a01012e0 2024-05-09T06:49:43.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.311+0000 7fe48f7fe640 1 -- 172.21.15.63:0/2165302951 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17332 (secure 0 0 0) 0x7fe490017090 con 0x7fe4a01012e0 2024-05-09T06:49:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:43.291 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":96,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","created":"2024-05-09T06:27:53.398729+0000","modified":"2024-05-09T06:45:51.753800+0000","last_up_change":"2024-05-09T06:44:43.058485+0000","last_in_change":"2024-05-09T06:33:25.853191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:32:32.042216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-09T06:35:02.184376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-09T06:35:02.871327+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eea045d9-973d-4960-a34e-689f9403d55f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":89,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6803","nonce":1522382249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6805","nonce":1522382249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6809","nonce":1522382249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6807","nonce":1522382249}]},"public_addr":"172.21.15.63:6803/1522382249","cluster_addr":"172.21.15.63:6805/1522382249","heartbeat_back_addr":"172.21.15.63:6809/1522382249","heartbeat_front_addr":"172.21.15.63:6807/1522382249","state":["exists","up"]},{"osd":1,"uuid":"da187f97-762d-40d3-94d0-fef449ce1dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":66,"up_thru":89,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6811","nonce":2256761098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6813","nonce":2256761098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6817","nonce":2256761098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6815","nonce":2256761098}]},"public_addr":"172.21.15.63:6811/2256761098","cluster_addr":"172.21.15.63:6813/2256761098","heartbeat_back_addr":"172.21.15.63:6817/2256761098","heartbeat_front_addr":"172.21.15.63:6815/2256761098","state":["exists","up"]},{"osd":2,"uuid":"6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":70,"up_from":74,"up_thru":89,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":616478},{"type":"v1","addr":"172.21.15.63:6819","nonce":616478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":616478},{"type":"v1","addr":"172.21.15.63:6821","nonce":616478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":616478},{"type":"v1","addr":"172.21.15.63:6825","nonce":616478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":616478},{"type":"v1","addr":"172.21.15.63:6823","nonce":616478}]},"public_addr":"172.21.15.63:6819/616478","cluster_addr":"172.21.15.63:6821/616478","heartbeat_back_addr":"172.21.15.63:6825/616478","heartbeat_front_addr":"172.21.15.63:6823/616478","state":["exists","up"]},{"osd":3,"uuid":"ff1a9d2a-4481-4e61-aaa5-e5046f3c0055","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":75,"up_from":79,"up_thru":89,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6801","nonce":2657331774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6803","nonce":2657331774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6807","nonce":2657331774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6805","nonce":2657331774}]},"public_addr":"172.21.15.193:6801/2657331774","cluster_addr":"172.21.15.193:6803/2657331774","heartbeat_back_addr":"172.21.15.193:6807/2657331774","heartbeat_front_addr":"172.21.15.193:6805/2657331774","state":["exists","up"]},{"osd":4,"uuid":"23f0fabc-4f38-438c-a691-307209bb0f31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":80,"up_from":84,"up_thru":89,"down_at":81,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6809","nonce":909880686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6811","nonce":909880686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6815","nonce":909880686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6813","nonce":909880686}]},"public_addr":"172.21.15.193:6809/909880686","cluster_addr":"172.21.15.193:6811/909880686","heartbeat_back_addr":"172.21.15.193:6815/909880686","heartbeat_front_addr":"172.21.15.193:6813/909880686","state":["exists","up"]},{"osd":5,"uuid":"eb6226de-63c7-4200-8978-4ee9491c1427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":85,"up_from":89,"up_thru":89,"down_at":86,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6817","nonce":2078959605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6819","nonce":2078959605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6823","nonce":2078959605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6821","nonce":2078959605}]},"public_addr":"172.21.15.193:6817/2078959605","cluster_addr":"172.21.15.193:6819/2078959605","heartbeat_back_addr":"172.21.15.193:6823/2078959605","heartbeat_front_addr":"172.21.15.193:6821/2078959605","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:32.202943+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:59.919211+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:28.132162+0000","dead_epoch":70},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:52.038182+0000","dead_epoch":75},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:17.334991+0000","dead_epoch":80},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:43.406303+0000","dead_epoch":85}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":0,"to":2}]},{"pgid":"2.9","mappings":[{"from":0,"to":2}]},{"pgid":"2.18","mappings":[{"from":3,"to":1}]},{"pgid":"2.19","mappings":[{"from":0,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"2.1d","mappings":[{"from":3,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":0}]},{"pgid":"3.4","mappings":[{"from":3,"to":0}]},{"pgid":"3.7","mappings":[{"from":4,"to":5}]},{"pgid":"3.b","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6827/354024957":"2024-05-10T06:45:51.753775+0000","172.21.15.193:6826/354024957":"2024-05-10T06:45:51.753775+0000","172.21.15.193:6824/296432349":"2024-05-10T06:45:26.766385+0000","172.21.15.63:0/2992018924":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/1538574058":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/2580859343":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/4061231873":"2024-05-10T06:29:03.468046+0000","172.21.15.193:0/3682498688":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6801/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/2209960091":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/576235286":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/750127211":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/1465547870":"2024-05-10T06:30:11.518272+0000","172.21.15.193:6825/296432349":"2024-05-10T06:45:26.766385+0000","172.21.15.63:0/1507850411":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/4049710905":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/3349953626":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6826/1318147125":"2024-05-10T06:35:21.124789+0000","172.21.15.63:0/1971919126":"2024-05-10T06:28:29.800161+0000","172.21.15.63:6800/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.193:0/3815434278":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6800/1655504434":"2024-05-10T06:40:12.081321+0000","172.21.15.193:0/3760737819":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6801/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6800/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/452783656":"2024-05-10T06:37:59.346754+0000","172.21.15.63:6827/1318147125":"2024-05-10T06:35:21.124789+0000","172.21.15.63:6801/4130975635":"2024-05-10T06:37:59.346754+0000","172.21.15.193:6828/1432929717":"2024-05-10T06:39:11.862568+0000","172.21.15.193:0/389588125":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6801/1655504434":"2024-05-10T06:40:12.081321+0000","172.21.15.63:6801/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/3728039632":"2024-05-10T06:28:29.800161+0000","172.21.15.193:6829/1432929717":"2024-05-10T06:39:11.862568+0000","172.21.15.63:0/4215281381":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/494791920":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.193:0/398074734":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6800/4130975635":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/2254623220":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/2340959335":"2024-05-10T06:40:12.081321+0000","172.21.15.193:0/3446861232":"2024-05-10T06:39:11.862568+0000","172.21.15.63:0/2926131890":"2024-05-10T06:40:12.081321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:49:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe48806e170 msgr2=0x7fe488070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe48806e170 0x7fe488070630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe494036040 tx=0x7fe494000bc0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a01012e0 msgr2=0x7fe4a0196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a01012e0 0x7fe4a0196150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d6f0 tx=0x7fe49000dbb0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 shutdown_connections 2024-05-09T06:49:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe48806e170 0x7fe488070630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4a0101c00 0x7fe4a0196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 --2- 172.21.15.63:0/2165302951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a01012e0 0x7fe4a0196150 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 >> 172.21.15.63:0/2165302951 conn(0x7fe4a00faac0 msgr2=0x7fe4a0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 shutdown_connections 2024-05-09T06:49:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.313+0000 7fe4a4a58640 1 -- 172.21.15.63:0/2165302951 wait complete. 2024-05-09T06:49:43.304 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:32:32.042216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-09T06:35:02.184376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-09T06:35:02.871327+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:49:43.304 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:49:43.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.432+0000 7ff82e2fc640 1 -- 172.21.15.63:0/994402011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8280ff220 msgr2=0x7ff8280ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.432+0000 7ff82e2fc640 1 --2- 172.21.15.63:0/994402011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8280ff220 0x7ff8280ff600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff818009880 tx=0x7ff81802bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.432+0000 7ff82e2fc640 1 -- 172.21.15.63:0/994402011 shutdown_connections 2024-05-09T06:49:43.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.432+0000 7ff82e2fc640 1 --2- 172.21.15.63:0/994402011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8280ffbd0 0x7ff828103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.432+0000 7ff82e2fc640 1 --2- 172.21.15.63:0/994402011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8280ff220 0x7ff8280ff600 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.432+0000 7ff82e2fc640 1 -- 172.21.15.63:0/994402011 >> 172.21.15.63:0/994402011 conn(0x7ff8280faac0 msgr2=0x7ff8280fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.432+0000 7ff82e2fc640 1 -- 172.21.15.63:0/994402011 shutdown_connections 2024-05-09T06:49:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.432+0000 7ff82e2fc640 1 -- 172.21.15.63:0/994402011 wait complete. 2024-05-09T06:49:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff82e2fc640 1 Processor -- start 2024-05-09T06:49:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff82e2fc640 1 -- start start 2024-05-09T06:49:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff82e2fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8280ff220 0x7ff828196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff82e2fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8281974f0 0x7ff828197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff82e2fc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281a0020 con 0x7ff8281974f0 2024-05-09T06:49:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff82e2fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281a0160 con 0x7ff8280ff220 2024-05-09T06:49:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8281974f0 0x7ff828197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8281974f0 0x7ff828197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40506/0 (socket says 172.21.15.63:40506) 2024-05-09T06:49:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff8277fe640 1 -- 172.21.15.63:0/718482456 learned_addr learned my addr 172.21.15.63:0/718482456 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff827fff640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8280ff220 0x7ff828196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff8277fe640 1 -- 172.21.15.63:0/718482456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8280ff220 msgr2=0x7ff828196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff8277fe640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8280ff220 0x7ff828196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.433+0000 7ff8277fe640 1 -- 172.21.15.63:0/718482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818009560 con 0x7ff8281974f0 2024-05-09T06:49:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.434+0000 7ff8277fe640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8281974f0 0x7ff828197950 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00d6f0 tx=0x7ff81c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.434+0000 7ff8257fa640 1 -- 172.21.15.63:0/718482456 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81c0051f0 con 0x7ff8281974f0 2024-05-09T06:49:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.434+0000 7ff8257fa640 1 -- 172.21.15.63:0/718482456 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff81c009ce0 con 0x7ff8281974f0 2024-05-09T06:49:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.434+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8281a0300 con 0x7ff8281974f0 2024-05-09T06:49:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.434+0000 7ff8257fa640 1 -- 172.21.15.63:0/718482456 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81c004e80 con 0x7ff8281974f0 2024-05-09T06:49:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.434+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8281a0830 con 0x7ff8281974f0 2024-05-09T06:49:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.435+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8281006f0 con 0x7ff8281974f0 2024-05-09T06:49:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.435+0000 7ff8257fa640 1 -- 172.21.15.63:0/718482456 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff81c0046c0 con 0x7ff8281974f0 2024-05-09T06:49:43.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.436+0000 7ff8257fa640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff81006e180 0x7ff810070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:43.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.436+0000 7ff8257fa640 1 -- 172.21.15.63:0/718482456 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7ff81c08dca0 con 0x7ff8281974f0 2024-05-09T06:49:43.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.436+0000 7ff827fff640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff81006e180 0x7ff810070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:43.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.436+0000 7ff827fff640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff81006e180 0x7ff810070640 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff818000c00 tx=0x7ff81802c820 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.441+0000 7ff8257fa640 1 -- 172.21.15.63:0/718482456 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff81c092070 con 0x7ff8281974f0 2024-05-09T06:49:43.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.624+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff828107550 con 0x7ff8281974f0 2024-05-09T06:49:43.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.624+0000 7ff8257fa640 1 -- 172.21.15.63:0/718482456 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff81c0200a0 con 0x7ff8281974f0 2024-05-09T06:49:43.603 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-09T06:49:43.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.626+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff81006e180 msgr2=0x7ff810070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff81006e180 0x7ff810070640 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff818000c00 tx=0x7ff81802c820 comp rx=0 tx=0).stop 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8281974f0 msgr2=0x7ff828197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8281974f0 0x7ff828197950 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00d6f0 tx=0x7ff81c00dbb0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 shutdown_connections 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7ff81006e180 0x7ff810070640 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8281974f0 0x7ff828197950 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 --2- 172.21.15.63:0/718482456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8280ff220 0x7ff828196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 >> 172.21.15.63:0/718482456 conn(0x7ff8280faac0 msgr2=0x7ff8280fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 shutdown_connections 2024-05-09T06:49:43.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.627+0000 7ff82e2fc640 1 -- 172.21.15.63:0/718482456 wait complete. 2024-05-09T06:49:43.616 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-09T06:49:43.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.745+0000 7f953a6d5640 1 -- 172.21.15.63:0/2695404655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101270 msgr2=0x7f9534101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.745+0000 7f953a6d5640 1 --2- 172.21.15.63:0/2695404655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101270 0x7f9534101650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9524009880 tx=0x7f952402bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:43.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.745+0000 7f953a6d5640 1 -- 172.21.15.63:0/2695404655 shutdown_connections 2024-05-09T06:49:43.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.745+0000 7f953a6d5640 1 --2- 172.21.15.63:0/2695404655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9534101b90 0x7f953410a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.745+0000 7f953a6d5640 1 --2- 172.21.15.63:0/2695404655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101270 0x7f9534101650 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.745+0000 7f953a6d5640 1 -- 172.21.15.63:0/2695404655 >> 172.21.15.63:0/2695404655 conn(0x7f95340faa90 msgr2=0x7f95340fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.745+0000 7f953a6d5640 1 -- 172.21.15.63:0/2695404655 shutdown_connections 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.745+0000 7f953a6d5640 1 -- 172.21.15.63:0/2695404655 wait complete. 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f953a6d5640 1 Processor -- start 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f953a6d5640 1 -- start start 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f953a6d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9534101270 0x7f9534196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f953a6d5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101b90 0x7f95341966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f953a6d5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9534196d40 con 0x7f9534101b90 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f953a6d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953419aa00 con 0x7f9534101270 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f952bfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101b90 0x7f95341966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f952bfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101b90 0x7f95341966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40538/0 (socket says 172.21.15.63:40538) 2024-05-09T06:49:43.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f952bfff640 1 -- 172.21.15.63:0/1069511558 learned_addr learned my addr 172.21.15.63:0/1069511558 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f9533fff640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9534101270 0x7f9534196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f952bfff640 1 -- 172.21.15.63:0/1069511558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9534101270 msgr2=0x7f9534196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f952bfff640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9534101270 0x7f9534196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.746+0000 7f952bfff640 1 -- 172.21.15.63:0/1069511558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9524009560 con 0x7f9534101b90 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.747+0000 7f952bfff640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101b90 0x7f95341966b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f951c00e970 tx=0x7f951c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.747+0000 7f9531ffb640 1 -- 172.21.15.63:0/1069511558 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f951c00cc90 con 0x7f9534101b90 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.747+0000 7f9531ffb640 1 -- 172.21.15.63:0/1069511558 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f951c00cdf0 con 0x7f9534101b90 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.747+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953419ace0 con 0x7f9534101b90 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.747+0000 7f9531ffb640 1 -- 172.21.15.63:0/1069511558 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f951c010430 con 0x7f9534101b90 2024-05-09T06:49:43.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.747+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953419b240 con 0x7f9534101b90 2024-05-09T06:49:43.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.748+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9534068e60 con 0x7f9534101b90 2024-05-09T06:49:43.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.748+0000 7f9531ffb640 1 -- 172.21.15.63:0/1069511558 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f951c004f20 con 0x7f9534101b90 2024-05-09T06:49:43.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.749+0000 7f9531ffb640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f952006e170 0x7f9520070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:43.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.749+0000 7f9531ffb640 1 -- 172.21.15.63:0/1069511558 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f951c014070 con 0x7f9534101b90 2024-05-09T06:49:43.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.749+0000 7f9533fff640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f952006e170 0x7f9520070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:43.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.749+0000 7f9533fff640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f952006e170 0x7f9520070630 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9524000c00 tx=0x7f9524005b90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:43.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.753+0000 7f9531ffb640 1 -- 172.21.15.63:0/1069511558 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f951c0579e0 con 0x7f9534101b90 2024-05-09T06:49:43.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.935+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f953404e160 con 0x7f9534101b90 2024-05-09T06:49:43.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.936+0000 7f9531ffb640 1 -- 172.21.15.63:0/1069511558 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f951c05b0a0 con 0x7f9534101b90 2024-05-09T06:49:43.915 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f952006e170 msgr2=0x7f9520070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f952006e170 0x7f9520070630 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9524000c00 tx=0x7f9524005b90 comp rx=0 tx=0).stop 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101b90 msgr2=0x7f95341966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101b90 0x7f95341966b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f951c00e970 tx=0x7f951c00ee30 comp rx=0 tx=0).stop 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 shutdown_connections 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f952006e170 0x7f9520070630 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9534101b90 0x7f95341966b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 --2- 172.21.15.63:0/1069511558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9534101270 0x7f9534196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 >> 172.21.15.63:0/1069511558 conn(0x7f95340faa90 msgr2=0x7f9534102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 shutdown_connections 2024-05-09T06:49:43.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:43.938+0000 7f953a6d5640 1 -- 172.21.15.63:0/1069511558 wait complete. 2024-05-09T06:49:43.927 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:43 smithi063 ceph-mon[92765]: pgmap v334: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:43.927 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:43 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/513821948' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-09T06:49:43.927 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:43 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2165302951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:49:43.927 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:43 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/718482456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:49:43.928 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-09T06:49:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 -- 172.21.15.63:0/3818783199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 msgr2=0x7f428c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 --2- 172.21.15.63:0/3818783199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 0x7f428c1006d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f427c009880 tx=0x7f427c02bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 -- 172.21.15.63:0/3818783199 shutdown_connections 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 --2- 172.21.15.63:0/3818783199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f428c100ca0 0x7f428c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 --2- 172.21.15.63:0/3818783199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 0x7f428c1006d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 -- 172.21.15.63:0/3818783199 >> 172.21.15.63:0/3818783199 conn(0x7f428c0fbb70 msgr2=0x7f428c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 -- 172.21.15.63:0/3818783199 shutdown_connections 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 -- 172.21.15.63:0/3818783199 wait complete. 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 Processor -- start 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.057+0000 7f4290bf2640 1 -- start start 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f4290bf2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 0x7f428c196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f4290bf2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f428c196680 0x7f428c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f4290bf2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c196bf0 con 0x7f428c1002f0 2024-05-09T06:49:44.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f4290bf2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c196d60 con 0x7f428c196680 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f428a59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 0x7f428c196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f428a59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 0x7f428c196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40554/0 (socket says 172.21.15.63:40554) 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f428a59c640 1 -- 172.21.15.63:0/299133440 learned_addr learned my addr 172.21.15.63:0/299133440 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f4289d9b640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f428c196680 0x7f428c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f428a59c640 1 -- 172.21.15.63:0/299133440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f428c196680 msgr2=0x7f428c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f428a59c640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f428c196680 0x7f428c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f428a59c640 1 -- 172.21.15.63:0/299133440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c009560 con 0x7f428c1002f0 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f428a59c640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 0x7f428c196140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f427c02c220 tx=0x7f427c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f427b7fe640 1 -- 172.21.15.63:0/299133440 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f427c0035f0 con 0x7f428c1002f0 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f427b7fe640 1 -- 172.21.15.63:0/299133440 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f427c003c10 con 0x7f428c1002f0 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f427b7fe640 1 -- 172.21.15.63:0/299133440 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f427c034440 con 0x7f428c1002f0 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.058+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c19afc0 con 0x7f428c1002f0 2024-05-09T06:49:44.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.059+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c19b4d0 con 0x7f428c1002f0 2024-05-09T06:49:44.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.060+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c1017c0 con 0x7f428c1002f0 2024-05-09T06:49:44.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.060+0000 7f427b7fe640 1 -- 172.21.15.63:0/299133440 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f427c004f60 con 0x7f428c1002f0 2024-05-09T06:49:44.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.061+0000 7f427b7fe640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f427406e170 0x7f4274070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:44.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.061+0000 7f427b7fe640 1 -- 172.21.15.63:0/299133440 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f427c0b07d0 con 0x7f428c1002f0 2024-05-09T06:49:44.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.061+0000 7f4289d9b640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f427406e170 0x7f4274070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:44.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.061+0000 7f4289d9b640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f427406e170 0x7f4274070630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4280005e10 tx=0x7f4280005da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:44.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.065+0000 7f427b7fe640 1 -- 172.21.15.63:0/299133440 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f427c079890 con 0x7f428c1002f0 2024-05-09T06:49:44.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:43 smithi193 ceph-mon[81665]: pgmap v334: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:44.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:43 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/513821948' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-09T06:49:44.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:43 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2165302951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:49:44.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:43 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/718482456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:49:44.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.251+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f428c19b7b0 con 0x7f428c1002f0 2024-05-09T06:49:44.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.252+0000 7f427b7fe640 1 -- 172.21.15.63:0/299133440 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f427c07cf50 con 0x7f428c1002f0 2024-05-09T06:49:44.231 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f427406e170 msgr2=0x7f4274070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f427406e170 0x7f4274070630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4280005e10 tx=0x7f4280005da0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 msgr2=0x7f428c196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 0x7f428c196140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f427c02c220 tx=0x7f427c004040 comp rx=0 tx=0).stop 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 shutdown_connections 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f427406e170 0x7f4274070630 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f428c196680 0x7f428c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 --2- 172.21.15.63:0/299133440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f428c1002f0 0x7f428c196140 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 >> 172.21.15.63:0/299133440 conn(0x7f428c0fbb70 msgr2=0x7f428c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:44.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 shutdown_connections 2024-05-09T06:49:44.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.254+0000 7f4290bf2640 1 -- 172.21.15.63:0/299133440 wait complete. 2024-05-09T06:49:44.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 -- 172.21.15.63:0/1873217294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101b80 msgr2=0x7f56a410a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 --2- 172.21.15.63:0/1873217294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101b80 0x7f56a410a060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5698009880 tx=0x7f569802bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 -- 172.21.15.63:0/1873217294 shutdown_connections 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 --2- 172.21.15.63:0/1873217294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101b80 0x7f56a410a060 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 --2- 172.21.15.63:0/1873217294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56a4101260 0x7f56a4101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 -- 172.21.15.63:0/1873217294 >> 172.21.15.63:0/1873217294 conn(0x7f56a4075c40 msgr2=0x7f56a40fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 -- 172.21.15.63:0/1873217294 shutdown_connections 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 -- 172.21.15.63:0/1873217294 wait complete. 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 Processor -- start 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.371+0000 7f56aa3eb640 1 -- start start 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56aa3eb640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101260 0x7f56a41960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:44.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56aa3eb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56a4101b80 0x7f56a4196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56aa3eb640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a4196c90 con 0x7f56a4101260 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56aa3eb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a419a9a0 con 0x7f56a4101b80 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a3fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101260 0x7f56a41960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a3fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101260 0x7f56a41960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40568/0 (socket says 172.21.15.63:40568) 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a3fff640 1 -- 172.21.15.63:0/3570233953 learned_addr learned my addr 172.21.15.63:0/3570233953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a37fe640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56a4101b80 0x7f56a4196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a3fff640 1 -- 172.21.15.63:0/3570233953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56a4101b80 msgr2=0x7f56a4196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a3fff640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56a4101b80 0x7f56a4196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a3fff640 1 -- 172.21.15.63:0/3570233953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5698009560 con 0x7f56a4101260 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a37fe640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56a4101b80 0x7f56a4196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a3fff640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101260 0x7f56a41960c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f569400d6f0 tx=0x7f569400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a17fa640 1 -- 172.21.15.63:0/3570233953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5694009a30 con 0x7f56a4101260 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a419ac80 con 0x7f56a4101260 2024-05-09T06:49:44.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.372+0000 7f56a17fa640 1 -- 172.21.15.63:0/3570233953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56940054e0 con 0x7f56a4101260 2024-05-09T06:49:44.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.373+0000 7f56a17fa640 1 -- 172.21.15.63:0/3570233953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5694010460 con 0x7f56a4101260 2024-05-09T06:49:44.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.373+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a419b1e0 con 0x7f56a4101260 2024-05-09T06:49:44.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.374+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a4067c80 con 0x7f56a4101260 2024-05-09T06:49:44.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.374+0000 7f56a17fa640 1 -- 172.21.15.63:0/3570233953 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5694004ec0 con 0x7f56a4101260 2024-05-09T06:49:44.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.374+0000 7f56a17fa640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f568c06e170 0x7f568c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:44.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.375+0000 7f56a17fa640 1 -- 172.21.15.63:0/3570233953 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f569408df30 con 0x7f56a4101260 2024-05-09T06:49:44.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.375+0000 7f56a37fe640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f568c06e170 0x7f568c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:44.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.375+0000 7f56a37fe640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f568c06e170 0x7f568c070630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5698005ec0 tx=0x7f569802caf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:44.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.379+0000 7f56a17fa640 1 -- 172.21.15.63:0/3570233953 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f56940c2740 con 0x7f56a4101260 2024-05-09T06:49:44.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.577+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f56a404e160 con 0x7f56a4101260 2024-05-09T06:49:44.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.578+0000 7f56a17fa640 1 -- 172.21.15.63:0/3570233953 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v96) v1 ==== 81+0+2550 (secure 0 0 0) 0x7f5694020a10 con 0x7f56a4101260 2024-05-09T06:49:44.557 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6827/354024957 2024-05-10T06:45:51.753775+0000 2024-05-09T06:49:44.557 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6826/354024957 2024-05-10T06:45:51.753775+0000 2024-05-09T06:49:44.557 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6824/296432349 2024-05-10T06:45:26.766385+0000 2024-05-09T06:49:44.558 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2992018924 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:44.558 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1538574058 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:44.558 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2580859343 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:44.558 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4061231873 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:44.558 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/3682498688 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:44.558 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3629307253 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:44.558 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2209960091 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:44.558 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/576235286 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:44.558 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/750127211 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:44.559 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1465547870 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:44.559 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6825/296432349 2024-05-10T06:45:26.766385+0000 2024-05-09T06:49:44.559 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1507850411 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:44.559 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4049710905 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:44.559 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3349953626 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:44.559 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6826/1318147125 2024-05-10T06:35:21.124789+0000 2024-05-09T06:49:44.559 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1971919126 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:44.559 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1739893919 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/3815434278 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1655504434 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/3760737819 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3098785268 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3629307253 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/452783656 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6827/1318147125 2024-05-10T06:35:21.124789+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/4130975635 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6828/1432929717 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:44.560 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/389588125 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1655504434 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1739893919 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3728039632 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6829/1432929717 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4215281381 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/494791920 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3098785268 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/398074734 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/4130975635 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2254623220 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2340959335 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/3446861232 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:44.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2926131890 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:44.562 INFO:teuthology.orchestra.run.smithi063.stderr:listed 43 entries 2024-05-09T06:49:44.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f568c06e170 msgr2=0x7f568c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f568c06e170 0x7f568c070630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5698005ec0 tx=0x7f569802caf0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101260 msgr2=0x7f56a41960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101260 0x7f56a41960c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f569400d6f0 tx=0x7f569400dbb0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 shutdown_connections 2024-05-09T06:49:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f568c06e170 0x7f568c070630 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56a4101b80 0x7f56a4196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 --2- 172.21.15.63:0/3570233953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56a4101260 0x7f56a41960c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 >> 172.21.15.63:0/3570233953 conn(0x7f56a4075c40 msgr2=0x7f56a4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 shutdown_connections 2024-05-09T06:49:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.580+0000 7f56aa3eb640 1 -- 172.21.15.63:0/3570233953 wait complete. 2024-05-09T06:49:44.569 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T06:49:44.570 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:49:44.588 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:49:44.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.753+0000 7fe10f6c7640 1 -- 172.21.15.63:0/4010232054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 msgr2=0x7fe108101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.753+0000 7fe10f6c7640 1 --2- 172.21.15.63:0/4010232054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 0x7fe108101630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009880 tx=0x7fe0fc02bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:44.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.753+0000 7fe10f6c7640 1 -- 172.21.15.63:0/4010232054 shutdown_connections 2024-05-09T06:49:44.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.753+0000 7fe10f6c7640 1 --2- 172.21.15.63:0/4010232054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe108101b70 0x7fe10810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.753+0000 7fe10f6c7640 1 --2- 172.21.15.63:0/4010232054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 0x7fe108101630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.753+0000 7fe10f6c7640 1 -- 172.21.15.63:0/4010232054 >> 172.21.15.63:0/4010232054 conn(0x7fe1080faab0 msgr2=0x7fe1080fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:44.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.753+0000 7fe10f6c7640 1 -- 172.21.15.63:0/4010232054 shutdown_connections 2024-05-09T06:49:44.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.753+0000 7fe10f6c7640 1 -- 172.21.15.63:0/4010232054 wait complete. 2024-05-09T06:49:44.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10f6c7640 1 Processor -- start 2024-05-09T06:49:44.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10f6c7640 1 -- start start 2024-05-09T06:49:44.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10f6c7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 0x7fe1081960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:44.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10f6c7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe108101b70 0x7fe108196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:44.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10f6c7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe108196c90 con 0x7fe108101250 2024-05-09T06:49:44.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10f6c7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10819a9a0 con 0x7fe108101b70 2024-05-09T06:49:44.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10d463640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 0x7fe1081960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10d463640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 0x7fe1081960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39390/0 (socket says 172.21.15.63:39390) 2024-05-09T06:49:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10d463640 1 -- 172.21.15.63:0/1417854764 learned_addr learned my addr 172.21.15.63:0/1417854764 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10d463640 1 -- 172.21.15.63:0/1417854764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe108101b70 msgr2=0x7fe108196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10cc62640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe108101b70 0x7fe108196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10d463640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe108101b70 0x7fe108196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10d463640 1 -- 172.21.15.63:0/1417854764 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc009560 con 0x7fe108101250 2024-05-09T06:49:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10cc62640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe108101b70 0x7fe108196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.754+0000 7fe10d463640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 0x7fe1081960c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc02c2b0 tx=0x7fe0fc02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.755+0000 7fe0fa7fc640 1 -- 172.21.15.63:0/1417854764 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0fc0035f0 con 0x7fe108101250 2024-05-09T06:49:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.755+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10819ac20 con 0x7fe108101250 2024-05-09T06:49:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.755+0000 7fe0fa7fc640 1 -- 172.21.15.63:0/1417854764 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0fc003c10 con 0x7fe108101250 2024-05-09T06:49:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.755+0000 7fe0fa7fc640 1 -- 172.21.15.63:0/1417854764 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0fc005690 con 0x7fe108101250 2024-05-09T06:49:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.755+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10819b180 con 0x7fe108101250 2024-05-09T06:49:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.756+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe108068e20 con 0x7fe108101250 2024-05-09T06:49:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.756+0000 7fe0fa7fc640 1 -- 172.21.15.63:0/1417854764 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe0fc004ed0 con 0x7fe108101250 2024-05-09T06:49:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.757+0000 7fe0fa7fc640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe0f406e110 0x7fe0f40705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.757+0000 7fe0fa7fc640 1 -- 172.21.15.63:0/1417854764 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fe0fc0b0ea0 con 0x7fe108101250 2024-05-09T06:49:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.757+0000 7fe10cc62640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe0f406e110 0x7fe0f40705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.757+0000 7fe10cc62640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe0f406e110 0x7fe0f40705d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe108197610 tx=0x7fe104005d30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.761+0000 7fe0fa7fc640 1 -- 172.21.15.63:0/1417854764 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe0fc07a000 con 0x7fe108101250 2024-05-09T06:49:44.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:44 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1069511558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-09T06:49:44.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:44 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/299133440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-09T06:49:44.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:44 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3570233953' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:49:44.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.951+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe108104860 con 0x7fe108101250 2024-05-09T06:49:44.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.952+0000 7fe0fa7fc640 1 -- 172.21.15.63:0/1417854764 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17332 (secure 0 0 0) 0x7fe0fc07d6c0 con 0x7fe108101250 2024-05-09T06:49:44.931 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T06:49:44.931 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":96,"fsid":"2386be1c-0dcd-11ef-bc97-c7b262605968","created":"2024-05-09T06:27:53.398729+0000","modified":"2024-05-09T06:45:51.753800+0000","last_up_change":"2024-05-09T06:44:43.058485+0000","last_in_change":"2024-05-09T06:33:25.853191+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:32:32.042216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-09T06:35:02.184376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-09T06:35:02.871327+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eea045d9-973d-4960-a34e-689f9403d55f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":89,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6803","nonce":1522382249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6805","nonce":1522382249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6809","nonce":1522382249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1522382249},{"type":"v1","addr":"172.21.15.63:6807","nonce":1522382249}]},"public_addr":"172.21.15.63:6803/1522382249","cluster_addr":"172.21.15.63:6805/1522382249","heartbeat_back_addr":"172.21.15.63:6809/1522382249","heartbeat_front_addr":"172.21.15.63:6807/1522382249","state":["exists","up"]},{"osd":1,"uuid":"da187f97-762d-40d3-94d0-fef449ce1dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":66,"up_thru":89,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6811","nonce":2256761098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6813","nonce":2256761098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6817","nonce":2256761098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2256761098},{"type":"v1","addr":"172.21.15.63:6815","nonce":2256761098}]},"public_addr":"172.21.15.63:6811/2256761098","cluster_addr":"172.21.15.63:6813/2256761098","heartbeat_back_addr":"172.21.15.63:6817/2256761098","heartbeat_front_addr":"172.21.15.63:6815/2256761098","state":["exists","up"]},{"osd":2,"uuid":"6d92243d-024a-4eb0-b8cb-fcfaa6dd8fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":70,"up_from":74,"up_thru":89,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":616478},{"type":"v1","addr":"172.21.15.63:6819","nonce":616478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":616478},{"type":"v1","addr":"172.21.15.63:6821","nonce":616478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":616478},{"type":"v1","addr":"172.21.15.63:6825","nonce":616478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":616478},{"type":"v1","addr":"172.21.15.63:6823","nonce":616478}]},"public_addr":"172.21.15.63:6819/616478","cluster_addr":"172.21.15.63:6821/616478","heartbeat_back_addr":"172.21.15.63:6825/616478","heartbeat_front_addr":"172.21.15.63:6823/616478","state":["exists","up"]},{"osd":3,"uuid":"ff1a9d2a-4481-4e61-aaa5-e5046f3c0055","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":75,"up_from":79,"up_thru":89,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6801","nonce":2657331774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6803","nonce":2657331774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6807","nonce":2657331774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2657331774},{"type":"v1","addr":"172.21.15.193:6805","nonce":2657331774}]},"public_addr":"172.21.15.193:6801/2657331774","cluster_addr":"172.21.15.193:6803/2657331774","heartbeat_back_addr":"172.21.15.193:6807/2657331774","heartbeat_front_addr":"172.21.15.193:6805/2657331774","state":["exists","up"]},{"osd":4,"uuid":"23f0fabc-4f38-438c-a691-307209bb0f31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":80,"up_from":84,"up_thru":89,"down_at":81,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6809","nonce":909880686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6811","nonce":909880686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6815","nonce":909880686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":909880686},{"type":"v1","addr":"172.21.15.193:6813","nonce":909880686}]},"public_addr":"172.21.15.193:6809/909880686","cluster_addr":"172.21.15.193:6811/909880686","heartbeat_back_addr":"172.21.15.193:6815/909880686","heartbeat_front_addr":"172.21.15.193:6813/909880686","state":["exists","up"]},{"osd":5,"uuid":"eb6226de-63c7-4200-8978-4ee9491c1427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":85,"up_from":89,"up_thru":89,"down_at":86,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6817","nonce":2078959605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6819","nonce":2078959605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6823","nonce":2078959605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2078959605},{"type":"v1","addr":"172.21.15.193:6821","nonce":2078959605}]},"public_addr":"172.21.15.193:6817/2078959605","cluster_addr":"172.21.15.193:6819/2078959605","heartbeat_back_addr":"172.21.15.193:6823/2078959605","heartbeat_front_addr":"172.21.15.193:6821/2078959605","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:32.202943+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:31:59.919211+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:28.132162+0000","dead_epoch":70},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:32:52.038182+0000","dead_epoch":75},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:17.334991+0000","dead_epoch":80},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:33:43.406303+0000","dead_epoch":85}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":0,"to":2}]},{"pgid":"2.9","mappings":[{"from":0,"to":2}]},{"pgid":"2.18","mappings":[{"from":3,"to":1}]},{"pgid":"2.19","mappings":[{"from":0,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"2.1d","mappings":[{"from":3,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":0}]},{"pgid":"3.4","mappings":[{"from":3,"to":0}]},{"pgid":"3.7","mappings":[{"from":4,"to":5}]},{"pgid":"3.b","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6827/354024957":"2024-05-10T06:45:51.753775+0000","172.21.15.193:6826/354024957":"2024-05-10T06:45:51.753775+0000","172.21.15.193:6824/296432349":"2024-05-10T06:45:26.766385+0000","172.21.15.63:0/2992018924":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/1538574058":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/2580859343":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/4061231873":"2024-05-10T06:29:03.468046+0000","172.21.15.193:0/3682498688":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6801/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/2209960091":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/576235286":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/750127211":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/1465547870":"2024-05-10T06:30:11.518272+0000","172.21.15.193:6825/296432349":"2024-05-10T06:45:26.766385+0000","172.21.15.63:0/1507850411":"2024-05-10T06:30:11.518272+0000","172.21.15.63:0/4049710905":"2024-05-10T06:40:12.081321+0000","172.21.15.63:0/3349953626":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6826/1318147125":"2024-05-10T06:35:21.124789+0000","172.21.15.63:0/1971919126":"2024-05-10T06:28:29.800161+0000","172.21.15.63:6800/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.193:0/3815434278":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6800/1655504434":"2024-05-10T06:40:12.081321+0000","172.21.15.193:0/3760737819":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6801/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.63:6800/3629307253":"2024-05-10T06:29:03.468046+0000","172.21.15.63:0/452783656":"2024-05-10T06:37:59.346754+0000","172.21.15.63:6827/1318147125":"2024-05-10T06:35:21.124789+0000","172.21.15.63:6801/4130975635":"2024-05-10T06:37:59.346754+0000","172.21.15.193:6828/1432929717":"2024-05-10T06:39:11.862568+0000","172.21.15.193:0/389588125":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6801/1655504434":"2024-05-10T06:40:12.081321+0000","172.21.15.63:6801/1739893919":"2024-05-10T06:28:29.800161+0000","172.21.15.63:0/3728039632":"2024-05-10T06:28:29.800161+0000","172.21.15.193:6829/1432929717":"2024-05-10T06:39:11.862568+0000","172.21.15.63:0/4215281381":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/494791920":"2024-05-10T06:29:03.468046+0000","172.21.15.63:6800/3098785268":"2024-05-10T06:30:11.518272+0000","172.21.15.193:0/398074734":"2024-05-10T06:39:11.862568+0000","172.21.15.63:6800/4130975635":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/2254623220":"2024-05-10T06:37:59.346754+0000","172.21.15.63:0/2340959335":"2024-05-10T06:40:12.081321+0000","172.21.15.193:0/3446861232":"2024-05-10T06:39:11.862568+0000","172.21.15.63:0/2926131890":"2024-05-10T06:40:12.081321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:49:44.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe0f406e110 msgr2=0x7fe0f40705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe0f406e110 0x7fe0f40705d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe108197610 tx=0x7fe104005d30 comp rx=0 tx=0).stop 2024-05-09T06:49:44.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 msgr2=0x7fe1081960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:44.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 0x7fe1081960c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc02c2b0 tx=0x7fe0fc02c8e0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 shutdown_connections 2024-05-09T06:49:44.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fe0f406e110 0x7fe0f40705d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe108101b70 0x7fe108196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 --2- 172.21.15.63:0/1417854764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe108101250 0x7fe1081960c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:44.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 >> 172.21.15.63:0/1417854764 conn(0x7fe1080faab0 msgr2=0x7fe108102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:44.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 shutdown_connections 2024-05-09T06:49:44.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:44.954+0000 7fe10f6c7640 1 -- 172.21.15.63:0/1417854764 wait complete. 2024-05-09T06:49:44.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:32:32.042216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-09T06:35:02.184376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-09T06:35:02.871327+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:49:44.945 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.073+0000 7fa4264dc640 1 -- 172.21.15.63:0/1491117354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa420101bd0 msgr2=0x7fa42010a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.073+0000 7fa4264dc640 1 --2- 172.21.15.63:0/1491117354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa420101bd0 0x7fa42010a0b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa414009880 tx=0x7fa41402bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.073+0000 7fa4264dc640 1 -- 172.21.15.63:0/1491117354 shutdown_connections 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.073+0000 7fa4264dc640 1 --2- 172.21.15.63:0/1491117354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa420101bd0 0x7fa42010a0b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.073+0000 7fa4264dc640 1 --2- 172.21.15.63:0/1491117354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4201012b0 0x7fa420101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.073+0000 7fa4264dc640 1 -- 172.21.15.63:0/1491117354 >> 172.21.15.63:0/1491117354 conn(0x7fa4200faab0 msgr2=0x7fa4200fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.073+0000 7fa4264dc640 1 -- 172.21.15.63:0/1491117354 shutdown_connections 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.073+0000 7fa4264dc640 1 -- 172.21.15.63:0/1491117354 wait complete. 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.074+0000 7fa4264dc640 1 Processor -- start 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.074+0000 7fa4264dc640 1 -- start start 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.074+0000 7fa4264dc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4201012b0 0x7fa4200fef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.074+0000 7fa4264dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420101bd0 0x7fa4200ff440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.074+0000 7fa4264dc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200ff980 con 0x7fa4201012b0 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.074+0000 7fa4264dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200ffaf0 con 0x7fa420101bd0 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.074+0000 7fa41ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4201012b0 0x7fa4200fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4201012b0 0x7fa4200fef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39412/0 (socket says 172.21.15.63:39412) 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41ffff640 1 -- 172.21.15.63:0/3029107641 learned_addr learned my addr 172.21.15.63:0/3029107641 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41f7fe640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420101bd0 0x7fa4200ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41ffff640 1 -- 172.21.15.63:0/3029107641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420101bd0 msgr2=0x7fa4200ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41ffff640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420101bd0 0x7fa4200ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41ffff640 1 -- 172.21.15.63:0/3029107641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa414009560 con 0x7fa4201012b0 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41f7fe640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420101bd0 0x7fa4200ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41ffff640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4201012b0 0x7fa4200fef00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa41000d6f0 tx=0x7fa41000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41d7fa640 1 -- 172.21.15.63:0/3029107641 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa410009a30 con 0x7fa4201012b0 2024-05-09T06:49:45.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41d7fa640 1 -- 172.21.15.63:0/3029107641 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4100054e0 con 0x7fa4201012b0 2024-05-09T06:49:45.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa41d7fa640 1 -- 172.21.15.63:0/3029107641 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa410010460 con 0x7fa4201012b0 2024-05-09T06:49:45.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4201a2340 con 0x7fa4201012b0 2024-05-09T06:49:45.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.075+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4201a26b0 con 0x7fa4201012b0 2024-05-09T06:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.076+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa420068dc0 con 0x7fa4201012b0 2024-05-09T06:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.077+0000 7fa41d7fa640 1 -- 172.21.15.63:0/3029107641 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa410009b90 con 0x7fa4201012b0 2024-05-09T06:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.077+0000 7fa41d7fa640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa40806e180 0x7fa408070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.077+0000 7fa41d7fa640 1 -- 172.21.15.63:0/3029107641 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fa41008deb0 con 0x7fa4201012b0 2024-05-09T06:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.077+0000 7fa41f7fe640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa40806e180 0x7fa408070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.078+0000 7fa41f7fe640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa40806e180 0x7fa408070640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa41402ca80 tx=0x7fa41402c8b0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:45.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.082+0000 7fa41d7fa640 1 -- 172.21.15.63:0/3029107641 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa410057010 con 0x7fa4201012b0 2024-05-09T06:49:45.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:44 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1069511558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-09T06:49:45.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:44 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/299133440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-09T06:49:45.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:44 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3570233953' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:49:45.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.273+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa42004e160 con 0x7fa4201012b0 2024-05-09T06:49:45.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.273+0000 7fa41d7fa640 1 -- 172.21.15.63:0/3029107641 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa4100170a0 con 0x7fa4201012b0 2024-05-09T06:49:45.253 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-09T06:49:45.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.276+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa40806e180 msgr2=0x7fa408070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.276+0000 7fa4264dc640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa40806e180 0x7fa408070640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa41402ca80 tx=0x7fa41402c8b0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.276+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4201012b0 msgr2=0x7fa4200fef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.276+0000 7fa4264dc640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4201012b0 0x7fa4200fef00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa41000d6f0 tx=0x7fa41000dbb0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.276+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 shutdown_connections 2024-05-09T06:49:45.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.276+0000 7fa4264dc640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fa40806e180 0x7fa408070640 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.276+0000 7fa4264dc640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420101bd0 0x7fa4200ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.276+0000 7fa4264dc640 1 --2- 172.21.15.63:0/3029107641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4201012b0 0x7fa4200fef00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.276+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 >> 172.21.15.63:0/3029107641 conn(0x7fa4200faab0 msgr2=0x7fa420102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:45.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.277+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 shutdown_connections 2024-05-09T06:49:45.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.277+0000 7fa4264dc640 1 -- 172.21.15.63:0/3029107641 wait complete. 2024-05-09T06:49:45.268 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-09T06:49:45.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.397+0000 7f74f6623640 1 -- 172.21.15.63:0/2231597448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101b90 msgr2=0x7f74f010a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.397+0000 7f74f6623640 1 --2- 172.21.15.63:0/2231597448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101b90 0x7f74f010a070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009880 tx=0x7f74e402bd70 comp rx=0 tx=0).stop 2024-05-09T06:49:45.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.397+0000 7f74f6623640 1 -- 172.21.15.63:0/2231597448 shutdown_connections 2024-05-09T06:49:45.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.397+0000 7f74f6623640 1 --2- 172.21.15.63:0/2231597448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101b90 0x7f74f010a070 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.397+0000 7f74f6623640 1 --2- 172.21.15.63:0/2231597448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74f0101270 0x7f74f0101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.397+0000 7f74f6623640 1 -- 172.21.15.63:0/2231597448 >> 172.21.15.63:0/2231597448 conn(0x7f74f00faa90 msgr2=0x7f74f00fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:45.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.397+0000 7f74f6623640 1 -- 172.21.15.63:0/2231597448 shutdown_connections 2024-05-09T06:49:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.397+0000 7f74f6623640 1 -- 172.21.15.63:0/2231597448 wait complete. 2024-05-09T06:49:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.398+0000 7f74f6623640 1 Processor -- start 2024-05-09T06:49:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.398+0000 7f74f6623640 1 -- start start 2024-05-09T06:49:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.398+0000 7f74f6623640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101270 0x7f74f00feec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.398+0000 7f74f6623640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74f0101b90 0x7f74f00ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.398+0000 7f74f6623640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00ff940 con 0x7f74f0101270 2024-05-09T06:49:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.398+0000 7f74f6623640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f00ffab0 con 0x7f74f0101b90 2024-05-09T06:49:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.398+0000 7f74effff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101270 0x7f74f00feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.398+0000 7f74effff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101270 0x7f74f00feec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39424/0 (socket says 172.21.15.63:39424) 2024-05-09T06:49:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.398+0000 7f74effff640 1 -- 172.21.15.63:0/2420599338 learned_addr learned my addr 172.21.15.63:0/2420599338 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74ef7fe640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74f0101b90 0x7f74f00ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74effff640 1 -- 172.21.15.63:0/2420599338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74f0101b90 msgr2=0x7f74f00ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74effff640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74f0101b90 0x7f74f00ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74effff640 1 -- 172.21.15.63:0/2420599338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e4009560 con 0x7f74f0101270 2024-05-09T06:49:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74ef7fe640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74f0101b90 0x7f74f00ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:49:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74effff640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101270 0x7f74f00feec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f74e000d6f0 tx=0x7f74e000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74ed7fa640 1 -- 172.21.15.63:0/2420599338 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74e0009a30 con 0x7f74f0101270 2024-05-09T06:49:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74ed7fa640 1 -- 172.21.15.63:0/2420599338 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74e00054e0 con 0x7f74f0101270 2024-05-09T06:49:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f01a22d0 con 0x7f74f0101270 2024-05-09T06:49:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74ed7fa640 1 -- 172.21.15.63:0/2420599338 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74e0010460 con 0x7f74f0101270 2024-05-09T06:49:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.399+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f01a2640 con 0x7f74f0101270 2024-05-09T06:49:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.400+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f0068e60 con 0x7f74f0101270 2024-05-09T06:49:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.401+0000 7f74ed7fa640 1 -- 172.21.15.63:0/2420599338 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f74e0004ec0 con 0x7f74f0101270 2024-05-09T06:49:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.401+0000 7f74ed7fa640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f74d806e170 0x7f74d8070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.401+0000 7f74ef7fe640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f74d806e170 0x7f74d8070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.401+0000 7f74ed7fa640 1 -- 172.21.15.63:0/2420599338 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f74e008de20 con 0x7f74f0101270 2024-05-09T06:49:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.402+0000 7f74ef7fe640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f74d806e170 0x7f74d8070630 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74f01005f0 tx=0x7f74e4005a20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:45.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.406+0000 7f74ed7fa640 1 -- 172.21.15.63:0/2420599338 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f74e0057000 con 0x7f74f0101270 2024-05-09T06:49:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.588+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f74f004e160 con 0x7f74f0101270 2024-05-09T06:49:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.588+0000 7f74ed7fa640 1 -- 172.21.15.63:0/2420599338 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f74e00200b0 con 0x7f74f0101270 2024-05-09T06:49:45.568 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-09T06:49:45.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f74d806e170 msgr2=0x7f74d8070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f74d806e170 0x7f74d8070630 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74f01005f0 tx=0x7f74e4005a20 comp rx=0 tx=0).stop 2024-05-09T06:49:45.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101270 msgr2=0x7f74f00feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101270 0x7f74f00feec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f74e000d6f0 tx=0x7f74e000dbb0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 shutdown_connections 2024-05-09T06:49:45.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7f74d806e170 0x7f74d8070630 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74f0101b90 0x7f74f00ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 --2- 172.21.15.63:0/2420599338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74f0101270 0x7f74f00feec0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 >> 172.21.15.63:0/2420599338 conn(0x7f74f00faa90 msgr2=0x7f74f0102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 shutdown_connections 2024-05-09T06:49:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.590+0000 7f74f6623640 1 -- 172.21.15.63:0/2420599338 wait complete. 2024-05-09T06:49:45.580 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-09T06:49:45.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 -- 172.21.15.63:0/3679317636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 msgr2=0x7fdd5c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 --2- 172.21.15.63:0/3679317636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 0x7fdd5c1006d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009880 tx=0x7fdd5802bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 -- 172.21.15.63:0/3679317636 shutdown_connections 2024-05-09T06:49:45.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 --2- 172.21.15.63:0/3679317636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c100ca0 0x7fdd5c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 --2- 172.21.15.63:0/3679317636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 0x7fdd5c1006d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 -- 172.21.15.63:0/3679317636 >> 172.21.15.63:0/3679317636 conn(0x7fdd5c0fbb70 msgr2=0x7fdd5c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:45.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 -- 172.21.15.63:0/3679317636 shutdown_connections 2024-05-09T06:49:45.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 -- 172.21.15.63:0/3679317636 wait complete. 2024-05-09T06:49:45.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 Processor -- start 2024-05-09T06:49:45.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.712+0000 7fdd63e5f640 1 -- start start 2024-05-09T06:49:45.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd63e5f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 0x7fdd5c072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:45.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd63e5f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c072c40 0x7fdd5c0730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:45.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd63e5f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0745d0 con 0x7fdd5c1002f0 2024-05-09T06:49:45.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd63e5f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1a2300 con 0x7fdd5c072c40 2024-05-09T06:49:45.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd61bfb640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 0x7fdd5c072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:45.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd61bfb640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 0x7fdd5c072700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39444/0 (socket says 172.21.15.63:39444) 2024-05-09T06:49:45.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd61bfb640 1 -- 172.21.15.63:0/492146300 learned_addr learned my addr 172.21.15.63:0/492146300 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:45.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd613fa640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c072c40 0x7fdd5c0730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:45.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd61bfb640 1 -- 172.21.15.63:0/492146300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c072c40 msgr2=0x7fdd5c0730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd61bfb640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c072c40 0x7fdd5c0730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd61bfb640 1 -- 172.21.15.63:0/492146300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58009560 con 0x7fdd5c1002f0 2024-05-09T06:49:45.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd61bfb640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 0x7fdd5c072700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdd58036040 tx=0x7fdd58004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:45.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd52ffd640 1 -- 172.21.15.63:0/492146300 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd580035f0 con 0x7fdd5c1002f0 2024-05-09T06:49:45.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd52ffd640 1 -- 172.21.15.63:0/492146300 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd58003c10 con 0x7fdd5c1002f0 2024-05-09T06:49:45.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c1a2440 con 0x7fdd5c1002f0 2024-05-09T06:49:45.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.713+0000 7fdd52ffd640 1 -- 172.21.15.63:0/492146300 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd58005660 con 0x7fdd5c1002f0 2024-05-09T06:49:45.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.714+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c1a2910 con 0x7fdd5c1002f0 2024-05-09T06:49:45.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.715+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c1017c0 con 0x7fdd5c1002f0 2024-05-09T06:49:45.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.715+0000 7fdd52ffd640 1 -- 172.21.15.63:0/492146300 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdd58004ed0 con 0x7fdd5c1002f0 2024-05-09T06:49:45.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.715+0000 7fdd52ffd640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd4806e170 0x7fdd48070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:45.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.716+0000 7fdd613fa640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd4806e170 0x7fdd48070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:45.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.716+0000 7fdd52ffd640 1 -- 172.21.15.63:0/492146300 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fdd580b0690 con 0x7fdd5c1002f0 2024-05-09T06:49:45.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.716+0000 7fdd613fa640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd4806e170 0x7fdd48070630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c069640 tx=0x7fdd4c00a2d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:45.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.720+0000 7fdd52ffd640 1 -- 172.21.15.63:0/492146300 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdd580797f0 con 0x7fdd5c1002f0 2024-05-09T06:49:45.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:45 smithi063 ceph-mon[92765]: pgmap v335: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:45.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:45 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1417854764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:49:45.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:45 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/3029107641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:49:45.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:45 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/2420599338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-09T06:49:45.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.904+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fdd5c04e160 con 0x7fdd5c1002f0 2024-05-09T06:49:45.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.905+0000 7fdd52ffd640 1 -- 172.21.15.63:0/492146300 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdd5807ceb0 con 0x7fdd5c1002f0 2024-05-09T06:49:45.884 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-09T06:49:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd4806e170 msgr2=0x7fdd48070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd4806e170 0x7fdd48070630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c069640 tx=0x7fdd4c00a2d0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 msgr2=0x7fdd5c072700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 0x7fdd5c072700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdd58036040 tx=0x7fdd58004040 comp rx=0 tx=0).stop 2024-05-09T06:49:45.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 shutdown_connections 2024-05-09T06:49:45.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fdd4806e170 0x7fdd48070630 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c072c40 0x7fdd5c0730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 --2- 172.21.15.63:0/492146300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd5c1002f0 0x7fdd5c072700 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:45.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 >> 172.21.15.63:0/492146300 conn(0x7fdd5c0fbb70 msgr2=0x7fdd5c103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:45.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 shutdown_connections 2024-05-09T06:49:45.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:45.907+0000 7fdd63e5f640 1 -- 172.21.15.63:0/492146300 wait complete. 2024-05-09T06:49:45.901 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:49:46.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.027+0000 7fc584973640 1 -- 172.21.15.63:0/2706374999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100cf0 msgr2=0x7fc580104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.027+0000 7fc584973640 1 --2- 172.21.15.63:0/2706374999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100cf0 0x7fc580104d30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc574009880 tx=0x7fc57402bce0 comp rx=0 tx=0).stop 2024-05-09T06:49:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.027+0000 7fc584973640 1 -- 172.21.15.63:0/2706374999 shutdown_connections 2024-05-09T06:49:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.027+0000 7fc584973640 1 --2- 172.21.15.63:0/2706374999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100cf0 0x7fc580104d30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.027+0000 7fc584973640 1 --2- 172.21.15.63:0/2706374999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc580100340 0x7fc580100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.027+0000 7fc584973640 1 -- 172.21.15.63:0/2706374999 >> 172.21.15.63:0/2706374999 conn(0x7fc5800fbbc0 msgr2=0x7fc5800fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.027+0000 7fc584973640 1 -- 172.21.15.63:0/2706374999 shutdown_connections 2024-05-09T06:49:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.027+0000 7fc584973640 1 -- 172.21.15.63:0/2706374999 wait complete. 2024-05-09T06:49:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.028+0000 7fc584973640 1 Processor -- start 2024-05-09T06:49:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.028+0000 7fc584973640 1 -- start start 2024-05-09T06:49:46.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.028+0000 7fc584973640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100340 0x7fc580196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:46.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.028+0000 7fc584973640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5801966c0 0x7fc58019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:46.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.028+0000 7fc584973640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc580196cc0 con 0x7fc580100340 2024-05-09T06:49:46.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.028+0000 7fc584973640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc580196e30 con 0x7fc5801966c0 2024-05-09T06:49:46.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.028+0000 7fc57e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100340 0x7fc580196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:46.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.028+0000 7fc57e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100340 0x7fc580196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39464/0 (socket says 172.21.15.63:39464) 2024-05-09T06:49:46.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.028+0000 7fc57e59c640 1 -- 172.21.15.63:0/1628120379 learned_addr learned my addr 172.21.15.63:0/1628120379 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-09T06:49:46.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc57e59c640 1 -- 172.21.15.63:0/1628120379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5801966c0 msgr2=0x7fc58019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc57dd9b640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5801966c0 0x7fc58019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc57e59c640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5801966c0 0x7fc58019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc57e59c640 1 -- 172.21.15.63:0/1628120379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc574009560 con 0x7fc580100340 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc57dd9b640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5801966c0 0x7fc58019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc57e59c640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100340 0x7fc580196180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c920 tx=0x7fc57000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc56f7fe640 1 -- 172.21.15.63:0/1628120379 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc570007b80 con 0x7fc580100340 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58019b010 con 0x7fc580100340 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc56f7fe640 1 -- 172.21.15.63:0/1628120379 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5700054e0 con 0x7fc580100340 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc56f7fe640 1 -- 172.21.15.63:0/1628120379 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc57000f450 con 0x7fc580100340 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.029+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58019b570 con 0x7fc580100340 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.030+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58019b850 con 0x7fc580100340 2024-05-09T06:49:46.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.031+0000 7fc56f7fe640 1 -- 172.21.15.63:0/1628120379 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc570004f60 con 0x7fc580100340 2024-05-09T06:49:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.031+0000 7fc56f7fe640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc56806e170 0x7fc568070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:49:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.031+0000 7fc56f7fe640 1 -- 172.21.15.63:0/1628120379 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fc57008e1b0 con 0x7fc580100340 2024-05-09T06:49:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.031+0000 7fc57dd9b640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc56806e170 0x7fc568070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:49:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.032+0000 7fc57dd9b640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc56806e170 0x7fc568070630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc574036040 tx=0x7fc574000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:49:46.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.036+0000 7fc56f7fe640 1 -- 172.21.15.63:0/1628120379 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc570057310 con 0x7fc580100340 2024-05-09T06:49:46.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:45 smithi193 ceph-mon[81665]: pgmap v335: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:46.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:45 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1417854764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:49:46.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:45 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/3029107641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:49:46.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:45 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/2420599338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-09T06:49:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.233+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc580197430 con 0x7fc580100340 2024-05-09T06:49:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.234+0000 7fc56f7fe640 1 -- 172.21.15.63:0/1628120379 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v96) v1 ==== 81+0+2550 (secure 0 0 0) 0x7fc57005a9d0 con 0x7fc580100340 2024-05-09T06:49:46.212 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6827/354024957 2024-05-10T06:45:51.753775+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6826/354024957 2024-05-10T06:45:51.753775+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6824/296432349 2024-05-10T06:45:26.766385+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2992018924 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1538574058 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2580859343 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4061231873 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/3682498688 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3629307253 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2209960091 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/576235286 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/750127211 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1465547870 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:46.213 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6825/296432349 2024-05-10T06:45:26.766385+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1507850411 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4049710905 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3349953626 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6826/1318147125 2024-05-10T06:35:21.124789+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1971919126 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1739893919 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/3815434278 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1655504434 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/3760737819 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3098785268 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3629307253 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/452783656 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6827/1318147125 2024-05-10T06:35:21.124789+0000 2024-05-09T06:49:46.214 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/4130975635 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6828/1432929717 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/389588125 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1655504434 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1739893919 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3728039632 2024-05-10T06:28:29.800161+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:6829/1432929717 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4215281381 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/494791920 2024-05-10T06:29:03.468046+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3098785268 2024-05-10T06:30:11.518272+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/398074734 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/4130975635 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2254623220 2024-05-10T06:37:59.346754+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2340959335 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.193:0/3446861232 2024-05-10T06:39:11.862568+0000 2024-05-09T06:49:46.215 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2926131890 2024-05-10T06:40:12.081321+0000 2024-05-09T06:49:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:listed 43 entries 2024-05-09T06:49:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.235+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc56806e170 msgr2=0x7fc568070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.235+0000 7fc584973640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc56806e170 0x7fc568070630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc574036040 tx=0x7fc574000bc0 comp rx=0 tx=0).stop 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.235+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100340 msgr2=0x7fc580196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.235+0000 7fc584973640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100340 0x7fc580196180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c920 tx=0x7fc57000cde0 comp rx=0 tx=0).stop 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.235+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 shutdown_connections 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.235+0000 7fc584973640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:6800/2497028716,v1:172.21.15.63:6801/2497028716] conn(0x7fc56806e170 0x7fc568070630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.235+0000 7fc584973640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5801966c0 0x7fc58019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.235+0000 7fc584973640 1 --2- 172.21.15.63:0/1628120379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc580100340 0x7fc580196180 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.236+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 >> 172.21.15.63:0/1628120379 conn(0x7fc5800fbbc0 msgr2=0x7fc5800fbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.236+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 shutdown_connections 2024-05-09T06:49:46.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T06:49:46.236+0000 7fc584973640 1 -- 172.21.15.63:0/1628120379 wait complete. 2024-05-09T06:49:46.225 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi063.front.sepia.ceph.com... 2024-05-09T06:49:46.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:49:46.226 DEBUG:teuthology.orchestra.run.smithi063:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:49:46.251 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:49:47.061 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:46 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/492146300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-09T06:49:47.061 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:46 smithi063 ceph-mon[92765]: from='client.? 172.21.15.63:0/1628120379' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:49:47.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:46 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/492146300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-09T06:49:47.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:46 smithi193 ceph-mon[81665]: from='client.? 172.21.15.63:0/1628120379' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:49:48.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:47 smithi063 ceph-mon[92765]: pgmap v336: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:48.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:47 smithi193 ceph-mon[81665]: pgmap v336: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:50.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:49 smithi063 ceph-mon[92765]: pgmap v337: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:50.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:49 smithi193 ceph-mon[81665]: pgmap v337: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:49:52.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:51 smithi063 ceph-mon[92765]: pgmap v338: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:52.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:51 smithi193 ceph-mon[81665]: pgmap v338: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:54.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:53 smithi063 ceph-mon[92765]: pgmap v339: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:54.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:53 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:49:54.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:53 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:49:54.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:53 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:49:54.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:53 smithi193 ceph-mon[81665]: pgmap v339: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:54.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:53 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:49:54.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:53 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:49:54.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:53 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:49:55.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:49:55.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:49:55.105 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:49:55.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:49:55.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:49:55.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:49:56.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:55 smithi063 ceph-mon[92765]: pgmap v340: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:56.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:55 smithi193 ceph-mon[81665]: pgmap v340: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:49:58.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:57 smithi063 ceph-mon[92765]: pgmap v341: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:58.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:49:58.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:57 smithi193 ceph-mon[81665]: pgmap v341: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:49:58.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:50:00.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:49:59 smithi063 ceph-mon[92765]: pgmap v342: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:00.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:49:59 smithi193 ceph-mon[81665]: pgmap v342: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:01.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:00 smithi063 ceph-mon[92765]: overall HEALTH_OK 2024-05-09T06:50:01.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:00 smithi193 ceph-mon[81665]: overall HEALTH_OK 2024-05-09T06:50:02.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:01 smithi063 ceph-mon[92765]: pgmap v343: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:02.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:01 smithi193 ceph-mon[81665]: pgmap v343: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:03.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:02 smithi063 ceph-mon[92765]: pgmap v344: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:03.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:02 smithi193 ceph-mon[81665]: pgmap v344: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:05.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:05 smithi063 ceph-mon[92765]: pgmap v345: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:05.621 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:05 smithi193 ceph-mon[81665]: pgmap v345: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:07.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:07 smithi063 ceph-mon[92765]: pgmap v346: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:07.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:07 smithi193 ceph-mon[81665]: pgmap v346: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:09.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:09 smithi063 ceph-mon[92765]: pgmap v347: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:09.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:09 smithi193 ceph-mon[81665]: pgmap v347: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:11.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:11 smithi063 ceph-mon[92765]: pgmap v348: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:11.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:11 smithi193 ceph-mon[81665]: pgmap v348: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:12.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:50:12.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:50:13.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:13 smithi063 ceph-mon[92765]: pgmap v349: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:13.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:13 smithi193 ceph-mon[81665]: pgmap v349: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:15.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:15 smithi063 ceph-mon[92765]: pgmap v350: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:15.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:15 smithi193 ceph-mon[81665]: pgmap v350: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:17.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:17 smithi063 ceph-mon[92765]: pgmap v351: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:17.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:17 smithi193 ceph-mon[81665]: pgmap v351: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:19.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:19 smithi063 ceph-mon[92765]: pgmap v352: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:19.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:19 smithi193 ceph-mon[81665]: pgmap v352: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:21.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:21 smithi063 ceph-mon[92765]: pgmap v353: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:21.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:21 smithi193 ceph-mon[81665]: pgmap v353: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:23.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:23 smithi063 ceph-mon[92765]: pgmap v354: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:23.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:23 smithi193 ceph-mon[81665]: pgmap v354: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:25.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:25 smithi063 ceph-mon[92765]: pgmap v355: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:25.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:25 smithi193 ceph-mon[81665]: pgmap v355: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:27.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:27 smithi063 ceph-mon[92765]: pgmap v356: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:27.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:50:27.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:27 smithi193 ceph-mon[81665]: pgmap v356: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:27.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:50:29.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:29 smithi063 ceph-mon[92765]: pgmap v357: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:29.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:29 smithi193 ceph-mon[81665]: pgmap v357: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:31.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:31 smithi063 ceph-mon[92765]: pgmap v358: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:31.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:31 smithi193 ceph-mon[81665]: pgmap v358: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:33.609 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:33 smithi063 ceph-mon[92765]: pgmap v359: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:33.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:33 smithi193 ceph-mon[81665]: pgmap v359: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:35.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:35 smithi063 ceph-mon[92765]: pgmap v360: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:35.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:35 smithi193 ceph-mon[81665]: pgmap v360: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:37.574 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:37 smithi193 ceph-mon[81665]: pgmap v361: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:37.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:37 smithi063 ceph-mon[92765]: pgmap v361: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:39.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:39 smithi063 ceph-mon[92765]: pgmap v362: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:39.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:39 smithi193 ceph-mon[81665]: pgmap v362: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:41.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:41 smithi063 ceph-mon[92765]: pgmap v363: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:41.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:41 smithi193 ceph-mon[81665]: pgmap v363: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:42.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:50:42.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:50:43.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:43 smithi063 ceph-mon[92765]: pgmap v364: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:43.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:43 smithi193 ceph-mon[81665]: pgmap v364: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:45.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:45 smithi063 ceph-mon[92765]: pgmap v365: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:45.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:45 smithi193 ceph-mon[81665]: pgmap v365: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:47.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:47 smithi063 ceph-mon[92765]: pgmap v366: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:47.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:47 smithi193 ceph-mon[81665]: pgmap v366: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:49.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:49 smithi063 ceph-mon[92765]: pgmap v367: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:49.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:49 smithi193 ceph-mon[81665]: pgmap v367: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:50:51.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:51 smithi063 ceph-mon[92765]: pgmap v368: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:51.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:51 smithi193 ceph-mon[81665]: pgmap v368: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:53.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:53 smithi063 ceph-mon[92765]: pgmap v369: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:53.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:53 smithi193 ceph-mon[81665]: pgmap v369: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:54.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:54 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:50:54.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:54 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:50:56.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:55 smithi063 ceph-mon[92765]: pgmap v370: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:56.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:50:56.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:50:56.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:50:56.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:50:56.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:50:56.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:55 smithi193 ceph-mon[81665]: pgmap v370: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:50:56.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:50:56.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:50:56.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:50:56.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:50:56.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:50:58.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:57 smithi063 ceph-mon[92765]: pgmap v371: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:58.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:50:58.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:57 smithi193 ceph-mon[81665]: pgmap v371: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:50:58.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:51:00.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:50:59 smithi063 ceph-mon[92765]: pgmap v372: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:00.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:50:59 smithi193 ceph-mon[81665]: pgmap v372: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:02.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:01 smithi063 ceph-mon[92765]: pgmap v373: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:02.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:01 smithi193 ceph-mon[81665]: pgmap v373: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:04.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:03 smithi063 ceph-mon[92765]: pgmap v374: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:04.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:03 smithi193 ceph-mon[81665]: pgmap v374: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:06.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:05 smithi063 ceph-mon[92765]: pgmap v375: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:06.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:05 smithi193 ceph-mon[81665]: pgmap v375: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:07.981 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:07 smithi063 ceph-mon[92765]: pgmap v376: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:08.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:07 smithi193 ceph-mon[81665]: pgmap v376: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:10.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:09 smithi063 ceph-mon[92765]: pgmap v377: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:10.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:09 smithi193 ceph-mon[81665]: pgmap v377: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:12.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:11 smithi063 ceph-mon[92765]: pgmap v378: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:12.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:11 smithi193 ceph-mon[81665]: pgmap v378: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:12.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:51:13.021 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:51:14.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:13 smithi063 ceph-mon[92765]: pgmap v379: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:14.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:13 smithi193 ceph-mon[81665]: pgmap v379: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:16.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:15 smithi063 ceph-mon[92765]: pgmap v380: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:16.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:15 smithi193 ceph-mon[81665]: pgmap v380: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:17.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:17 smithi193 ceph-mon[81665]: pgmap v381: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:18.096 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:17 smithi063 ceph-mon[92765]: pgmap v381: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:20.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:19 smithi063 ceph-mon[92765]: pgmap v382: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:20.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:19 smithi193 ceph-mon[81665]: pgmap v382: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:22.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:21 smithi063 ceph-mon[92765]: pgmap v383: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:22.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:21 smithi193 ceph-mon[81665]: pgmap v383: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:24.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:23 smithi063 ceph-mon[92765]: pgmap v384: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:24.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:23 smithi193 ceph-mon[81665]: pgmap v384: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:26.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:25 smithi063 ceph-mon[92765]: pgmap v385: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:26.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:25 smithi193 ceph-mon[81665]: pgmap v385: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:28.064 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:27 smithi193 ceph-mon[81665]: pgmap v386: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:28.064 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:51:28.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:27 smithi063 ceph-mon[92765]: pgmap v386: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:28.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:51:30.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:29 smithi063 ceph-mon[92765]: pgmap v387: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:30.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:29 smithi193 ceph-mon[81665]: pgmap v387: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:32.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:31 smithi063 ceph-mon[92765]: pgmap v388: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:32.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:31 smithi193 ceph-mon[81665]: pgmap v388: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:34.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:33 smithi063 ceph-mon[92765]: pgmap v389: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:34.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:33 smithi193 ceph-mon[81665]: pgmap v389: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:36.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:35 smithi063 ceph-mon[92765]: pgmap v390: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:36.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:35 smithi193 ceph-mon[81665]: pgmap v390: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:38.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:37 smithi063 ceph-mon[92765]: pgmap v391: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:38.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:37 smithi193 ceph-mon[81665]: pgmap v391: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:40.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:39 smithi063 ceph-mon[92765]: pgmap v392: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:40.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:39 smithi193 ceph-mon[81665]: pgmap v392: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:42.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:41 smithi063 ceph-mon[92765]: pgmap v393: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:42.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:41 smithi193 ceph-mon[81665]: pgmap v393: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:43.021 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:51:43.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:51:44.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:43 smithi063 ceph-mon[92765]: pgmap v394: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:44.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:43 smithi193 ceph-mon[81665]: pgmap v394: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:46.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:45 smithi063 ceph-mon[92765]: pgmap v395: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:46.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:45 smithi193 ceph-mon[81665]: pgmap v395: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:48.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:47 smithi063 ceph-mon[92765]: pgmap v396: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:48.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:47 smithi193 ceph-mon[81665]: pgmap v396: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:50.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:49 smithi063 ceph-mon[92765]: pgmap v397: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:50.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:49 smithi193 ceph-mon[81665]: pgmap v397: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:52.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:51 smithi063 ceph-mon[92765]: pgmap v398: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:52.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:51 smithi193 ceph-mon[81665]: pgmap v398: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:53.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:53 smithi063 ceph-mon[92765]: pgmap v399: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:54.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:53 smithi193 ceph-mon[81665]: pgmap v399: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:55.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:54 smithi063 ceph-mon[92765]: pgmap v400: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:55.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:54 smithi193 ceph-mon[81665]: pgmap v400: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:51:56.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:55 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:51:56.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:55 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:51:57.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:56 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:51:57.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:56 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:51:57.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:56 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:51:57.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:56 smithi063 ceph-mon[92765]: pgmap v401: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:57.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:56 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:51:57.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:56 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:51:57.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:56 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:51:57.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:56 smithi193 ceph-mon[81665]: pgmap v401: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:51:58.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:58 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:51:58.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:58 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:51:59.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:51:59 smithi063 ceph-mon[92765]: pgmap v402: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:51:59.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:51:59 smithi193 ceph-mon[81665]: pgmap v402: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:01.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:01 smithi063 ceph-mon[92765]: pgmap v403: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:01.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:01 smithi193 ceph-mon[81665]: pgmap v403: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:03.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:03 smithi063 ceph-mon[92765]: pgmap v404: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:03.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:03 smithi193 ceph-mon[81665]: pgmap v404: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:05.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:05 smithi063 ceph-mon[92765]: pgmap v405: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:05.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:05 smithi193 ceph-mon[81665]: pgmap v405: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:07.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:07 smithi063 ceph-mon[92765]: pgmap v406: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:07.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:07 smithi193 ceph-mon[81665]: pgmap v406: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:09.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:09 smithi063 ceph-mon[92765]: pgmap v407: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:09.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:09 smithi193 ceph-mon[81665]: pgmap v407: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:11.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:11 smithi063 ceph-mon[92765]: pgmap v408: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:11.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:11 smithi193 ceph-mon[81665]: pgmap v408: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:12.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:52:12.619 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:52:13.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:13 smithi063 ceph-mon[92765]: pgmap v409: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:13.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:13 smithi193 ceph-mon[81665]: pgmap v409: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:15.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:15 smithi063 ceph-mon[92765]: pgmap v410: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:15.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:15 smithi193 ceph-mon[81665]: pgmap v410: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:17.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:17 smithi063 ceph-mon[92765]: pgmap v411: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:17.619 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:17 smithi193 ceph-mon[81665]: pgmap v411: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:19.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:19 smithi063 ceph-mon[92765]: pgmap v412: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:19.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:19 smithi193 ceph-mon[81665]: pgmap v412: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:21.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:21 smithi063 ceph-mon[92765]: pgmap v413: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:21.619 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:21 smithi193 ceph-mon[81665]: pgmap v413: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:23.602 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:23 smithi193 ceph-mon[81665]: pgmap v414: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:23.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:23 smithi063 ceph-mon[92765]: pgmap v414: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:25.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:25 smithi063 ceph-mon[92765]: pgmap v415: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:25.619 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:25 smithi193 ceph-mon[81665]: pgmap v415: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:27.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:27 smithi063 ceph-mon[92765]: pgmap v416: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:27.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:52:27.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:27 smithi193 ceph-mon[81665]: pgmap v416: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:27.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:52:29.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:29 smithi063 ceph-mon[92765]: pgmap v417: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:29.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:29 smithi193 ceph-mon[81665]: pgmap v417: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:31.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:31 smithi063 ceph-mon[92765]: pgmap v418: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:31.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:31 smithi193 ceph-mon[81665]: pgmap v418: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:33.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:33 smithi063 ceph-mon[92765]: pgmap v419: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:33.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:33 smithi193 ceph-mon[81665]: pgmap v419: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:35.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:35 smithi063 ceph-mon[92765]: pgmap v420: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:35.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:35 smithi193 ceph-mon[81665]: pgmap v420: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:37.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:37 smithi063 ceph-mon[92765]: pgmap v421: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:37.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:37 smithi193 ceph-mon[81665]: pgmap v421: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:39.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:39 smithi063 ceph-mon[92765]: pgmap v422: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:39.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:39 smithi193 ceph-mon[81665]: pgmap v422: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:41.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:41 smithi063 ceph-mon[92765]: pgmap v423: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:41.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:41 smithi193 ceph-mon[81665]: pgmap v423: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:42.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:52:42.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:52:43.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:43 smithi063 ceph-mon[92765]: pgmap v424: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:43.619 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:43 smithi193 ceph-mon[81665]: pgmap v424: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:45.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:45 smithi193 ceph-mon[81665]: pgmap v425: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:45.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:45 smithi063 ceph-mon[92765]: pgmap v425: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:47.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:47 smithi193 ceph-mon[81665]: pgmap v426: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:47.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:47 smithi063 ceph-mon[92765]: pgmap v426: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:49.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:49 smithi063 ceph-mon[92765]: pgmap v427: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:49.619 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:49 smithi193 ceph-mon[81665]: pgmap v427: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:51.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:51 smithi193 ceph-mon[81665]: pgmap v428: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:51.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:51 smithi063 ceph-mon[92765]: pgmap v428: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:53.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:53 smithi193 ceph-mon[81665]: pgmap v429: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:53.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:53 smithi063 ceph-mon[92765]: pgmap v429: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:55.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:55 smithi063 ceph-mon[92765]: pgmap v430: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:55.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:55 smithi193 ceph-mon[81665]: pgmap v430: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:52:56.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:56 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:52:56.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:56 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:52:57.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:57 smithi063 ceph-mon[92765]: pgmap v431: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:52:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:52:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:52:57.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:52:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:57 smithi193 ceph-mon[81665]: pgmap v431: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:52:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:52:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:52:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:52:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:52:59.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:52:59 smithi063 ceph-mon[92765]: pgmap v432: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:52:59.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:52:59 smithi193 ceph-mon[81665]: pgmap v432: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:01.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:01 smithi063 ceph-mon[92765]: pgmap v433: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:01.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:01 smithi193 ceph-mon[81665]: pgmap v433: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:03.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:03 smithi063 ceph-mon[92765]: pgmap v434: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:03.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:03 smithi193 ceph-mon[81665]: pgmap v434: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:05.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:05 smithi063 ceph-mon[92765]: pgmap v435: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:05.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:05 smithi193 ceph-mon[81665]: pgmap v435: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:07.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:07 smithi063 ceph-mon[92765]: pgmap v436: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:07.872 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:07 smithi193 ceph-mon[81665]: pgmap v436: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:09.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:09 smithi063 ceph-mon[92765]: pgmap v437: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:09.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:09 smithi193 ceph-mon[81665]: pgmap v437: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:11.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:11 smithi063 ceph-mon[92765]: pgmap v438: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:11.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:11 smithi193 ceph-mon[81665]: pgmap v438: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:12.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:53:12.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:53:13.770 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:13 smithi193 ceph-mon[81665]: pgmap v439: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:13.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:13 smithi063 ceph-mon[92765]: pgmap v439: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:15.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:15 smithi063 ceph-mon[92765]: pgmap v440: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:15.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:15 smithi193 ceph-mon[81665]: pgmap v440: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:17.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:17 smithi063 ceph-mon[92765]: pgmap v441: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:17.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:17 smithi193 ceph-mon[81665]: pgmap v441: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:19.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:19 smithi063 ceph-mon[92765]: pgmap v442: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:19.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:19 smithi193 ceph-mon[81665]: pgmap v442: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:21.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:21 smithi063 ceph-mon[92765]: pgmap v443: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:21.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:21 smithi193 ceph-mon[81665]: pgmap v443: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:23.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:23 smithi063 ceph-mon[92765]: pgmap v444: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:23.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:23 smithi193 ceph-mon[81665]: pgmap v444: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:25.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:25 smithi063 ceph-mon[92765]: pgmap v445: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:25.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:25 smithi193 ceph-mon[81665]: pgmap v445: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:27.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:27 smithi063 ceph-mon[92765]: pgmap v446: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:27.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:53:27.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:27 smithi193 ceph-mon[81665]: pgmap v446: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:27.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:53:29.619 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:29 smithi193 ceph-mon[81665]: pgmap v447: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:29.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:29 smithi063 ceph-mon[92765]: pgmap v447: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:31.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:31 smithi063 ceph-mon[92765]: pgmap v448: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:31.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:31 smithi193 ceph-mon[81665]: pgmap v448: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:33.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:33 smithi063 ceph-mon[92765]: pgmap v449: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:33.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:33 smithi193 ceph-mon[81665]: pgmap v449: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:35.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:35 smithi063 ceph-mon[92765]: pgmap v450: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:35.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:35 smithi193 ceph-mon[81665]: pgmap v450: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:37.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:37 smithi063 ceph-mon[92765]: pgmap v451: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:37.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:37 smithi193 ceph-mon[81665]: pgmap v451: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:39.620 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:39 smithi193 ceph-mon[81665]: pgmap v452: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:39.702 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:39 smithi063 ceph-mon[92765]: pgmap v452: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:41.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:41 smithi063 ceph-mon[92765]: pgmap v453: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:41.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:41 smithi193 ceph-mon[81665]: pgmap v453: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:42.751 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:53:42.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:53:43.771 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:43 smithi193 ceph-mon[81665]: pgmap v454: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:43.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:43 smithi063 ceph-mon[92765]: pgmap v454: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:45.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:45 smithi063 ceph-mon[92765]: pgmap v455: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:45.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:45 smithi193 ceph-mon[81665]: pgmap v455: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:47.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:47 smithi063 ceph-mon[92765]: pgmap v456: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:47.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:47 smithi193 ceph-mon[81665]: pgmap v456: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:49.817 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:49 smithi063 ceph-mon[92765]: pgmap v457: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:49.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:49 smithi193 ceph-mon[81665]: pgmap v457: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:51.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:51 smithi063 ceph-mon[92765]: pgmap v458: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:51.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:51 smithi193 ceph-mon[81665]: pgmap v458: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:53.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:53 smithi063 ceph-mon[92765]: pgmap v459: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:53.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:53 smithi193 ceph-mon[81665]: pgmap v459: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:55.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:55 smithi063 ceph-mon[92765]: pgmap v460: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:55.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:55 smithi193 ceph-mon[81665]: pgmap v460: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:53:57.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:57 smithi063 ceph-mon[92765]: pgmap v461: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:57.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:53:57.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:57 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:53:57.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:57 smithi193 ceph-mon[81665]: pgmap v461: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:53:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:53:57.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:57 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:53:58.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:58 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:53:58.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:58 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:53:58.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:58 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:53:58.869 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:58 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:53:58.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:58 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:53:58.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:58 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' 2024-05-09T06:53:59.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:53:59 smithi193 ceph-mon[81665]: pgmap v462: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:53:59.931 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:53:59 smithi063 ceph-mon[92765]: pgmap v462: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:54:02.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:01 smithi063 ceph-mon[92765]: pgmap v463: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:02.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:01 smithi193 ceph-mon[81665]: pgmap v463: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:04.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:03 smithi063 ceph-mon[92765]: pgmap v464: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:04.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:03 smithi193 ceph-mon[81665]: pgmap v464: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:06.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:05 smithi063 ceph-mon[92765]: pgmap v465: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:06.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:05 smithi193 ceph-mon[81665]: pgmap v465: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:08.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:07 smithi063 ceph-mon[92765]: pgmap v466: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:08.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:07 smithi193 ceph-mon[81665]: pgmap v466: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:10.046 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:09 smithi063 ceph-mon[92765]: pgmap v467: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:54:10.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:09 smithi193 ceph-mon[81665]: pgmap v467: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:54:12.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:11 smithi063 ceph-mon[92765]: pgmap v468: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:12.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:11 smithi193 ceph-mon[81665]: pgmap v468: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:13.021 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:12 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:54:13.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:12 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:54:14.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:13 smithi063 ceph-mon[92765]: pgmap v469: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:14.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:13 smithi193 ceph-mon[81665]: pgmap v469: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:16.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:15 smithi063 ceph-mon[92765]: pgmap v470: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:16.122 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:15 smithi193 ceph-mon[81665]: pgmap v470: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:18.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:17 smithi063 ceph-mon[92765]: pgmap v471: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-09T06:54:18.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:17 smithi193 ceph-mon[81665]: pgmap v471: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-09T06:54:20.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:19 smithi063 ceph-mon[92765]: pgmap v472: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:20.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:19 smithi193 ceph-mon[81665]: pgmap v472: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:22.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:21 smithi063 ceph-mon[92765]: pgmap v473: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:22.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:21 smithi193 ceph-mon[81665]: pgmap v473: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:24.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:23 smithi063 ceph-mon[92765]: pgmap v474: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:24.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:23 smithi193 ceph-mon[81665]: pgmap v474: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:26.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:25 smithi063 ceph-mon[92765]: pgmap v475: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:26.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:25 smithi193 ceph-mon[81665]: pgmap v475: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:28.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:27 smithi063 ceph-mon[92765]: pgmap v476: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:28.104 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:27 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:54:28.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:27 smithi193 ceph-mon[81665]: pgmap v476: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:28.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:27 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:54:30.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:29 smithi063 ceph-mon[92765]: pgmap v477: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:54:30.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:29 smithi193 ceph-mon[81665]: pgmap v477: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:54:32.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:31 smithi063 ceph-mon[92765]: pgmap v478: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:32.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:31 smithi193 ceph-mon[81665]: pgmap v478: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:34.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:33 smithi063 ceph-mon[92765]: pgmap v479: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:34.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:33 smithi193 ceph-mon[81665]: pgmap v479: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:36.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:35 smithi063 ceph-mon[92765]: pgmap v480: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:36.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:35 smithi193 ceph-mon[81665]: pgmap v480: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:38.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:37 smithi063 ceph-mon[92765]: pgmap v481: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:38.120 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:37 smithi193 ceph-mon[81665]: pgmap v481: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:40.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:39 smithi063 ceph-mon[92765]: pgmap v482: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:54:40.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:39 smithi193 ceph-mon[81665]: pgmap v482: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-09T06:54:42.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:41 smithi063 ceph-mon[92765]: pgmap v483: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:42.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:41 smithi193 ceph-mon[81665]: pgmap v483: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:43.021 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:42 smithi193 ceph-mon[81665]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:54:43.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:42 smithi063 ceph-mon[92765]: from='mgr.34102 172.21.15.63:0/2617022916' entity='mgr.smithi063.egfguy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-09T06:54:44.103 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:43 smithi063 ceph-mon[92765]: pgmap v484: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:44.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:43 smithi193 ceph-mon[81665]: pgmap v484: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-09T06:54:45.330 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-05-09T06:54:45.330 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-09T06:54:45.332 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T06:54:45.345 INFO:tasks.cephadm:Teardown begin 2024-05-09T06:54:45.345 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-09T06:54:45.347 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:54:45.382 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:54:45.416 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T06:54:45.416 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T06:54:45.440 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T06:54:45.474 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T06:54:45.474 INFO:tasks.cephadm.mon.smithi063:Stopping mon.smithi063... 2024-05-09T06:54:45.474 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063 2024-05-09T06:54:45.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:45 smithi063 systemd[1]: Stopping Ceph mon.smithi063 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:54:46.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:45 smithi063 ceph-mon[92765]: pgmap v485: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:46.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:45 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063[92761]: 2024-05-09T06:54:45.950+0000 7f21bd0cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi063 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:54:46.052 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:45 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063[92761]: 2024-05-09T06:54:45.951+0000 7f21bd0cc700 -1 mon.smithi063@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T06:54:46.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:45 smithi193 ceph-mon[81665]: pgmap v485: 65 pgs: 65 active+clean; 205 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-09T06:54:46.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:46 smithi063 podman[152330]: 2024-05-09 06:54:46.135119077 +0000 UTC m=+0.434316068 container died 8b7491de89ee74a84df6f094de48da563508e1e520281437387c946370db3770 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T06:54:46.762 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:46 smithi063 podman[152330]: 2024-05-09 06:54:46.616927775 +0000 UTC m=+0.916124783 container cleanup 8b7491de89ee74a84df6f094de48da563508e1e520281437387c946370db3770 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-09T06:54:46.763 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:46 smithi063 bash[152330]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063 2024-05-09T06:54:47.019 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:May 09 06:54:46 smithi063 podman[152355]: 2024-05-09 06:54:46.785250611 +0000 UTC m=+0.645358780 container remove 8b7491de89ee74a84df6f094de48da563508e1e520281437387c946370db3770 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi063, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T06:54:47.024 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi063.service' 2024-05-09T06:54:47.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:54:47.073 INFO:tasks.cephadm.mon.smithi063:Stopped mon.smithi063 2024-05-09T06:54:47.073 INFO:tasks.cephadm.mon.smithi193:Stopping mon.smithi193... 2024-05-09T06:54:47.074 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi193 2024-05-09T06:54:47.370 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:47 smithi193 systemd[1]: Stopping Ceph mon.smithi193 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:54:47.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:47 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193[81661]: 2024-05-09T06:54:47.394+0000 7fcc3856a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi193 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:54:47.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:47 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193[81661]: 2024-05-09T06:54:47.394+0000 7fcc3856a700 -1 mon.smithi193@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T06:54:47.870 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:47 smithi193 podman[122551]: 2024-05-09 06:54:47.619754993 +0000 UTC m=+0.325005647 container died dafb35e76458fefc325a774ca0e6ae888b1c9c92d7d4706055a45b0b7ecb3ba0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.license=GPLv2) 2024-05-09T06:54:48.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:48 smithi193 podman[122551]: 2024-05-09 06:54:48.086850634 +0000 UTC m=+0.792101281 container cleanup dafb35e76458fefc325a774ca0e6ae888b1c9c92d7d4706055a45b0b7ecb3ba0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T06:54:48.358 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:May 09 06:54:48 smithi193 bash[122551]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-mon-smithi193 2024-05-09T06:54:48.486 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@mon.smithi193.service' 2024-05-09T06:54:48.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:54:48.534 INFO:tasks.cephadm.mon.smithi193:Stopped mon.smithi193 2024-05-09T06:54:48.535 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T06:54:48.535 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.0 2024-05-09T06:54:48.833 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:48 smithi063 systemd[1]: Stopping Ceph osd.0 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:54:49.103 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:48 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[101832]: 2024-05-09T06:54:48.854+0000 7f1c21528700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:54:49.103 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:48 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[101832]: 2024-05-09T06:54:48.854+0000 7f1c21528700 -1 osd.0 96 *** Got signal Terminated *** 2024-05-09T06:54:49.103 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:48 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0[101832]: 2024-05-09T06:54:48.854+0000 7f1c21528700 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:54:54.400 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:54 smithi063 podman[152464]: 2024-05-09 06:54:54.344232057 +0000 UTC m=+5.588301775 container died 6e387840e5b75ed5723b3b36033b5a0b25ac299c0ac54fb82a010a1e3fe5a5f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-09T06:54:54.959 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:54 smithi063 podman[152464]: 2024-05-09 06:54:54.721654941 +0000 UTC m=+5.965724652 container cleanup 6e387840e5b75ed5723b3b36033b5a0b25ac299c0ac54fb82a010a1e3fe5a5f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T06:54:54.959 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:54 smithi063 bash[152464]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0 2024-05-09T06:54:55.340 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:54 smithi063 podman[152482]: 2024-05-09 06:54:54.981466344 +0000 UTC m=+0.632526031 container remove 6e387840e5b75ed5723b3b36033b5a0b25ac299c0ac54fb82a010a1e3fe5a5f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:54:55.603 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:55 smithi063 podman[152584]: 2024-05-09 06:54:55.266075558 +0000 UTC m=+0.019986212 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:54:55.604 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:55 smithi063 podman[152584]: 2024-05-09 06:54:55.365699714 +0000 UTC m=+0.119610368 container create be86671fe9dbaff0bfd83d6a67beb7345e3c9ba751dd99c9f0b52a678a7196eb (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:54:55.604 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:55 smithi063 podman[152584]: 2024-05-09 06:54:55.602430243 +0000 UTC m=+0.356340915 container init be86671fe9dbaff0bfd83d6a67beb7345e3c9ba751dd99c9f0b52a678a7196eb (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:54:55.604 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:55 smithi063 podman[152584]: 2024-05-09 06:54:55.607350129 +0000 UTC m=+0.361260784 container start be86671fe9dbaff0bfd83d6a67beb7345e3c9ba751dd99c9f0b52a678a7196eb (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-09T06:54:55.894 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:55 smithi063 podman[152584]: 2024-05-09 06:54:55.719048773 +0000 UTC m=+0.472959437 container attach be86671fe9dbaff0bfd83d6a67beb7345e3c9ba751dd99c9f0b52a678a7196eb (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True) 2024-05-09T06:54:56.208 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 09 06:54:55 smithi063 podman[152584]: 2024-05-09 06:54:55.916928782 +0000 UTC m=+0.670839438 container died be86671fe9dbaff0bfd83d6a67beb7345e3c9ba751dd99c9f0b52a678a7196eb (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:54:56.537 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.0.service' 2024-05-09T06:54:56.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:54:56.583 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T06:54:56.583 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T06:54:56.584 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.1 2024-05-09T06:54:56.853 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:54:56 smithi063 systemd[1]: Stopping Ceph osd.1 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:54:57.353 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:54:56 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[107519]: 2024-05-09T06:54:56.934+0000 7f70c24e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:54:57.353 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:54:56 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[107519]: 2024-05-09T06:54:56.934+0000 7f70c24e3700 -1 osd.1 96 *** Got signal Terminated *** 2024-05-09T06:54:57.353 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:54:56 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1[107519]: 2024-05-09T06:54:56.934+0000 7f70c24e3700 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:55:02.555 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:02 smithi063 podman[152689]: 2024-05-09 06:55:02.299385139 +0000 UTC m=+5.504817909 container died f1a348429588844ad28718d0138f8222f8f127b34d3fa073a0d5aecc45b6f341 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T06:55:02.853 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:02 smithi063 podman[152689]: 2024-05-09 06:55:02.667553942 +0000 UTC m=+5.872986711 container cleanup f1a348429588844ad28718d0138f8222f8f127b34d3fa073a0d5aecc45b6f341 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-96805dc) 2024-05-09T06:55:02.853 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:02 smithi063 bash[152689]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1 2024-05-09T06:55:03.272 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:02 smithi063 podman[152706]: 2024-05-09 06:55:02.960673431 +0000 UTC m=+0.656363846 container remove f1a348429588844ad28718d0138f8222f8f127b34d3fa073a0d5aecc45b6f341 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:55:03.542 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:03 smithi063 podman[152797]: 2024-05-09 06:55:03.295060606 +0000 UTC m=+0.113355725 container create 6a76cd726708284cdbd62fc06bbe5b41eba84aef3ac7e3360a598348fac5c2ea (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-09T06:55:03.542 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:03 smithi063 podman[152797]: 2024-05-09 06:55:03.200907467 +0000 UTC m=+0.019202615 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:55:03.542 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:03 smithi063 podman[152797]: 2024-05-09 06:55:03.489314703 +0000 UTC m=+0.307609833 container init 6a76cd726708284cdbd62fc06bbe5b41eba84aef3ac7e3360a598348fac5c2ea (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc) 2024-05-09T06:55:03.542 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:03 smithi063 podman[152797]: 2024-05-09 06:55:03.492818527 +0000 UTC m=+0.311113660 container start 6a76cd726708284cdbd62fc06bbe5b41eba84aef3ac7e3360a598348fac5c2ea (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:55:03.854 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:03 smithi063 podman[152797]: 2024-05-09 06:55:03.564005914 +0000 UTC m=+0.382301051 container attach 6a76cd726708284cdbd62fc06bbe5b41eba84aef3ac7e3360a598348fac5c2ea (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-09T06:55:03.854 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:03 smithi063 podman[152797]: 2024-05-09 06:55:03.813471749 +0000 UTC m=+0.631766881 container died 6a76cd726708284cdbd62fc06bbe5b41eba84aef3ac7e3360a598348fac5c2ea (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-05-09T06:55:04.345 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.1.service' 2024-05-09T06:55:04.353 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:04 smithi063 podman[152823]: 2024-05-09 06:55:04.342927965 +0000 UTC m=+0.523858331 container remove 6a76cd726708284cdbd62fc06bbe5b41eba84aef3ac7e3360a598348fac5c2ea (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T06:55:04.353 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:04 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.1.service: Deactivated successfully. 2024-05-09T06:55:04.353 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:04 smithi063 systemd[1]: Stopped Ceph osd.1 for 2386be1c-0dcd-11ef-bc97-c7b262605968. 2024-05-09T06:55:04.353 INFO:journalctl@ceph.osd.1.smithi063.stdout:May 09 06:55:04 smithi063 systemd[1]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.1.service: Consumed 13.063s CPU time. 2024-05-09T06:55:04.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:55:04.388 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T06:55:04.388 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T06:55:04.388 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.2 2024-05-09T06:55:04.716 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:04 smithi063 systemd[1]: Stopping Ceph osd.2 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:55:05.103 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:04 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[113377]: 2024-05-09T06:55:04.737+0000 7f91920fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:55:05.103 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:04 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[113377]: 2024-05-09T06:55:04.737+0000 7f91920fe700 -1 osd.2 96 *** Got signal Terminated *** 2024-05-09T06:55:05.103 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:04 smithi063 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2[113377]: 2024-05-09T06:55:04.737+0000 7f91920fe700 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:55:10.349 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:10 smithi063 podman[152902]: 2024-05-09 06:55:10.095907296 +0000 UTC m=+5.451143454 container died 313cdd3a98673ebd1a2d175ff13db803878435517149edf48b8c5cfd3032aca4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:55:10.605 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:10 smithi063 podman[152902]: 2024-05-09 06:55:10.470730868 +0000 UTC m=+5.825967032 container cleanup 313cdd3a98673ebd1a2d175ff13db803878435517149edf48b8c5cfd3032aca4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T06:55:10.606 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:10 smithi063 bash[152902]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2 2024-05-09T06:55:11.087 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:10 smithi063 podman[152921]: 2024-05-09 06:55:10.746916017 +0000 UTC m=+0.649729120 container remove 313cdd3a98673ebd1a2d175ff13db803878435517149edf48b8c5cfd3032aca4 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T06:55:11.353 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:11 smithi063 podman[153018]: 2024-05-09 06:55:11.013300965 +0000 UTC m=+0.019022560 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:55:11.353 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:11 smithi063 podman[153018]: 2024-05-09 06:55:11.131185101 +0000 UTC m=+0.136906684 container create 539735886bbebaeb326523f3f2eb3a4cd0d05fc349f584c65c04e95f04f9ab34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS) 2024-05-09T06:55:11.353 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:11 smithi063 podman[153018]: 2024-05-09 06:55:11.325948238 +0000 UTC m=+0.331669836 container init 539735886bbebaeb326523f3f2eb3a4cd0d05fc349f584c65c04e95f04f9ab34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:55:11.354 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:11 smithi063 podman[153018]: 2024-05-09 06:55:11.329551614 +0000 UTC m=+0.335273198 container start 539735886bbebaeb326523f3f2eb3a4cd0d05fc349f584c65c04e95f04f9ab34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-09T06:55:11.853 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:11 smithi063 podman[153018]: 2024-05-09 06:55:11.45075626 +0000 UTC m=+0.456477844 container attach 539735886bbebaeb326523f3f2eb3a4cd0d05fc349f584c65c04e95f04f9ab34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-09T06:55:11.854 INFO:journalctl@ceph.osd.2.smithi063.stdout:May 09 06:55:11 smithi063 podman[153018]: 2024-05-09 06:55:11.637343576 +0000 UTC m=+0.643065155 container died 539735886bbebaeb326523f3f2eb3a4cd0d05fc349f584c65c04e95f04f9ab34 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T06:55:12.185 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.2.service' 2024-05-09T06:55:12.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:55:12.228 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T06:55:12.228 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T06:55:12.228 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.3 2024-05-09T06:55:12.533 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:12 smithi193 systemd[1]: Stopping Ceph osd.3 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:55:12.870 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:12 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[91448]: 2024-05-09T06:55:12.555+0000 7f42acb97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:55:12.870 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:12 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[91448]: 2024-05-09T06:55:12.555+0000 7f42acb97700 -1 osd.3 96 *** Got signal Terminated *** 2024-05-09T06:55:12.870 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:12 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3[91448]: 2024-05-09T06:55:12.555+0000 7f42acb97700 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:55:14.869 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:14 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:14.519+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:15.869 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:15 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:15.541+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:16.868 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:16 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:16.494+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:17.482 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:17 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:17.212+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:49.896176+0000 front 2024-05-09T06:54:49.896164+0000 (oldest deadline 2024-05-09T06:55:16.296370+0000) 2024-05-09T06:55:17.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:17 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:17.504+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:18.168 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:17 smithi193 podman[122673]: 2024-05-09 06:55:17.937790326 +0000 UTC m=+5.488947054 container died 96afdd638dd5254f20b49cfbe743bbd54eb68f9a6d9d0e21b476f31d2cacf69b (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:55:18.501 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:18 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:18.249+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:49.896176+0000 front 2024-05-09T06:54:49.896164+0000 (oldest deadline 2024-05-09T06:55:16.296370+0000) 2024-05-09T06:55:18.502 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:18 smithi193 podman[122673]: 2024-05-09 06:55:18.273048005 +0000 UTC m=+5.824204725 container cleanup 96afdd638dd5254f20b49cfbe743bbd54eb68f9a6d9d0e21b476f31d2cacf69b (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, ceph=True) 2024-05-09T06:55:18.502 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:18 smithi193 bash[122673]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3 2024-05-09T06:55:18.824 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:18 smithi193 podman[122704]: 2024-05-09 06:55:18.524140473 +0000 UTC m=+0.578035919 container remove 96afdd638dd5254f20b49cfbe743bbd54eb68f9a6d9d0e21b476f31d2cacf69b (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T06:55:18.825 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:18 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:18.523+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:19.120 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:18 smithi193 podman[122796]: 2024-05-09 06:55:18.749798417 +0000 UTC m=+0.017811317 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:55:19.120 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:18 smithi193 podman[122796]: 2024-05-09 06:55:18.858284411 +0000 UTC m=+0.126297300 container create 0f1813319d501bc6d680cec94bf9b8c3206dc83ab2b165b33cf7e58080d14605 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T06:55:19.120 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:19 smithi193 podman[122796]: 2024-05-09 06:55:19.077820982 +0000 UTC m=+0.345833877 container init 0f1813319d501bc6d680cec94bf9b8c3206dc83ab2b165b33cf7e58080d14605 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-96805dc) 2024-05-09T06:55:19.120 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:19 smithi193 podman[122796]: 2024-05-09 06:55:19.081599404 +0000 UTC m=+0.349612292 container start 0f1813319d501bc6d680cec94bf9b8c3206dc83ab2b165b33cf7e58080d14605 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:55:19.495 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:19 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:19.278+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:49.896176+0000 front 2024-05-09T06:54:49.896164+0000 (oldest deadline 2024-05-09T06:55:16.296370+0000) 2024-05-09T06:55:19.495 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:19 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:19.516+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:19.496 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:19 smithi193 podman[122796]: 2024-05-09 06:55:19.210981343 +0000 UTC m=+0.478994244 container attach 0f1813319d501bc6d680cec94bf9b8c3206dc83ab2b165b33cf7e58080d14605 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-09T06:55:19.496 INFO:journalctl@ceph.osd.3.smithi193.stdout:May 09 06:55:19 smithi193 podman[122796]: 2024-05-09 06:55:19.377005293 +0000 UTC m=+0.645018171 container died 0f1813319d501bc6d680cec94bf9b8c3206dc83ab2b165b33cf7e58080d14605 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T06:55:19.944 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.3.service' 2024-05-09T06:55:19.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:55:19.984 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T06:55:19.984 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T06:55:19.984 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.4 2024-05-09T06:55:20.213 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:20 smithi193 systemd[1]: Stopping Ceph osd.4 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:55:20.214 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:20 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:20.235+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:49.896176+0000 front 2024-05-09T06:54:49.896164+0000 (oldest deadline 2024-05-09T06:55:16.296370+0000) 2024-05-09T06:55:20.509 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:20 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:20.326+0000 7f0a5cbf1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:55:20.509 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:20 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:20.326+0000 7f0a5cbf1700 -1 osd.4 96 *** Got signal Terminated *** 2024-05-09T06:55:20.509 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:20 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:20.326+0000 7f0a5cbf1700 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:55:20.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:20 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:20.530+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:21.496 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:21 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:21.241+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:49.896176+0000 front 2024-05-09T06:54:49.896164+0000 (oldest deadline 2024-05-09T06:55:16.296370+0000) 2024-05-09T06:55:21.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:21 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:21.519+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:22.539 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:22 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:22.207+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:49.896176+0000 front 2024-05-09T06:54:49.896164+0000 (oldest deadline 2024-05-09T06:55:16.296370+0000) 2024-05-09T06:55:22.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:22 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:22.561+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:23.494 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:23 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:23.187+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:49.896176+0000 front 2024-05-09T06:54:49.896164+0000 (oldest deadline 2024-05-09T06:55:16.296370+0000) 2024-05-09T06:55:23.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:23 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:23.516+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:23.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:23 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:23.516+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:24.528 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:24 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:24.232+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:49.896176+0000 front 2024-05-09T06:54:49.896164+0000 (oldest deadline 2024-05-09T06:55:16.296370+0000) 2024-05-09T06:55:24.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:24 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:24.550+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:24.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:24 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:24.550+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:25.481 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:25 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:25.228+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:49.896176+0000 front 2024-05-09T06:54:49.896164+0000 (oldest deadline 2024-05-09T06:55:16.296370+0000) 2024-05-09T06:55:25.481 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:25 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4[96357]: 2024-05-09T06:55:25.228+0000 7f0a559de700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:54:59.097514+0000 front 2024-05-09T06:54:59.097432+0000 (oldest deadline 2024-05-09T06:55:24.397301+0000) 2024-05-09T06:55:25.744 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:25 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:25.503+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:25.744 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:25 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:25.503+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:26.006 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:25 smithi193 podman[122902]: 2024-05-09 06:55:25.766723652 +0000 UTC m=+5.532514731 container died 14c9b4920f5a3effa6e414b14a412559f4d5072be4ad4006cbc95c05d2a3c723 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.build-date=20240507) 2024-05-09T06:55:26.272 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:26 smithi193 podman[122902]: 2024-05-09 06:55:26.109480207 +0000 UTC m=+5.875271283 container cleanup 14c9b4920f5a3effa6e414b14a412559f4d5072be4ad4006cbc95c05d2a3c723 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-96805dc) 2024-05-09T06:55:26.272 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:26 smithi193 bash[122902]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4 2024-05-09T06:55:26.620 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:26 smithi193 podman[122919]: 2024-05-09 06:55:26.294287313 +0000 UTC m=+0.525114159 container remove 14c9b4920f5a3effa6e414b14a412559f4d5072be4ad4006cbc95c05d2a3c723 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0) 2024-05-09T06:55:26.620 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:26 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:26.454+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:26.620 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:26 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:26.454+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:27.006 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:26 smithi193 podman[123012]: 2024-05-09 06:55:26.610157184 +0000 UTC m=+0.018359514 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:55:27.007 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:26 smithi193 podman[123012]: 2024-05-09 06:55:26.745597913 +0000 UTC m=+0.153800277 container create 938d7f7dcd6ed7c4683632e39cabaa99922a63d14db96ed6c8f85bb7f8bd78e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-96805dc) 2024-05-09T06:55:27.370 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:27 smithi193 podman[123012]: 2024-05-09 06:55:27.156421072 +0000 UTC m=+0.564623390 container init 938d7f7dcd6ed7c4683632e39cabaa99922a63d14db96ed6c8f85bb7f8bd78e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True) 2024-05-09T06:55:27.370 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:27 smithi193 podman[123012]: 2024-05-09 06:55:27.160509489 +0000 UTC m=+0.568711808 container start 938d7f7dcd6ed7c4683632e39cabaa99922a63d14db96ed6c8f85bb7f8bd78e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T06:55:27.370 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:27 smithi193 podman[123012]: 2024-05-09 06:55:27.256259074 +0000 UTC m=+0.664461404 container attach 938d7f7dcd6ed7c4683632e39cabaa99922a63d14db96ed6c8f85bb7f8bd78e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-09T06:55:27.779 INFO:journalctl@ceph.osd.4.smithi193.stdout:May 09 06:55:27 smithi193 podman[123012]: 2024-05-09 06:55:27.454511793 +0000 UTC m=+0.862714105 container died 938d7f7dcd6ed7c4683632e39cabaa99922a63d14db96ed6c8f85bb7f8bd78e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:55:27.779 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:27 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:27.424+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:27.779 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:27 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:27.424+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:28.044 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.4.service' 2024-05-09T06:55:28.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:55:28.083 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T06:55:28.083 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T06:55:28.083 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.5 2024-05-09T06:55:28.370 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:28 smithi193 systemd[1]: Stopping Ceph osd.5 for 2386be1c-0dcd-11ef-bc97-c7b262605968... 2024-05-09T06:55:28.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:28.437+0000 7f2ebe329700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:55:28.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:28.437+0000 7f2ebe329700 -1 osd.5 96 *** Got signal Terminated *** 2024-05-09T06:55:28.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:28.437+0000 7f2ebe329700 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:55:28.871 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:28.473+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:28.871 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:28 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:28.473+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:29.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:29 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:29.509+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:29.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:29 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:29.509+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:30.620 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:30 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:30.556+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:30.620 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:30 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:30.556+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:30.620 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:30 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:30.556+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6822 osd.2 since back 2024-05-09T06:55:04.305440+0000 front 2024-05-09T06:55:04.305427+0000 (oldest deadline 2024-05-09T06:55:30.205352+0000) 2024-05-09T06:55:31.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:31 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:31.530+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:31.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:31 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:31.530+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:31.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:31 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:31.530+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6822 osd.2 since back 2024-05-09T06:55:04.305440+0000 front 2024-05-09T06:55:04.305427+0000 (oldest deadline 2024-05-09T06:55:30.205352+0000) 2024-05-09T06:55:32.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:32 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:32.526+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-09T06:54:51.603468+0000 front 2024-05-09T06:54:51.603437+0000 (oldest deadline 2024-05-09T06:55:14.503348+0000) 2024-05-09T06:55:32.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:32 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:32.526+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-05-09T06:55:00.904645+0000 front 2024-05-09T06:55:00.904704+0000 (oldest deadline 2024-05-09T06:55:22.604482+0000) 2024-05-09T06:55:32.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:32 smithi193 ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5[101271]: 2024-05-09T06:55:32.526+0000 7f2ebb11e700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.63:6822 osd.2 since back 2024-05-09T06:55:04.305440+0000 front 2024-05-09T06:55:04.305427+0000 (oldest deadline 2024-05-09T06:55:30.205352+0000) 2024-05-09T06:55:34.159 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:33 smithi193 podman[123119]: 2024-05-09 06:55:33.903185247 +0000 UTC m=+5.577629734 container died d219b4bb599f002375395018a5a108c1c485b68d9d4ac3d63ba42ebdbd8668f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-96805dc, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T06:55:34.525 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:34 smithi193 podman[123119]: 2024-05-09 06:55:34.287751989 +0000 UTC m=+5.962196472 container cleanup d219b4bb599f002375395018a5a108c1c485b68d9d4ac3d63ba42ebdbd8668f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:55:34.525 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:34 smithi193 bash[123119]: ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5 2024-05-09T06:55:34.838 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:34 smithi193 podman[123139]: 2024-05-09 06:55:34.547440855 +0000 UTC m=+0.635660985 container remove d219b4bb599f002375395018a5a108c1c485b68d9d4ac3d63ba42ebdbd8668f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T06:55:35.120 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:34 smithi193 podman[123226]: 2024-05-09 06:55:34.762598713 +0000 UTC m=+0.018384999 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T06:55:35.120 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:34 smithi193 podman[123226]: 2024-05-09 06:55:34.873170351 +0000 UTC m=+0.128956667 container create 0d61f508bf6fc3ea5e1f4d4a4c5c7db06cbc03b5e0405962bce37c08b2ae0204 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-09T06:55:35.120 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:35 smithi193 podman[123226]: 2024-05-09 06:55:35.100549276 +0000 UTC m=+0.356335539 container init 0d61f508bf6fc3ea5e1f4d4a4c5c7db06cbc03b5e0405962bce37c08b2ae0204 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:55:35.121 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:35 smithi193 podman[123226]: 2024-05-09 06:55:35.10397542 +0000 UTC m=+0.359761688 container start 0d61f508bf6fc3ea5e1f4d4a4c5c7db06cbc03b5e0405962bce37c08b2ae0204 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T06:55:35.620 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:35 smithi193 podman[123226]: 2024-05-09 06:55:35.184017819 +0000 UTC m=+0.439804098 container attach 0d61f508bf6fc3ea5e1f4d4a4c5c7db06cbc03b5e0405962bce37c08b2ae0204 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-05-09T06:55:35.620 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 09 06:55:35 smithi193 podman[123226]: 2024-05-09 06:55:35.407083038 +0000 UTC m=+0.662869307 container died 0d61f508bf6fc3ea5e1f4d4a4c5c7db06cbc03b5e0405962bce37c08b2ae0204 (image=quay.ceph.io/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-2386be1c-0dcd-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:55:35.945 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2386be1c-0dcd-11ef-bc97-c7b262605968@osd.5.service' 2024-05-09T06:55:35.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:55:35.985 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T06:55:35.986 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T06:55:36.153 INFO:teuthology.orchestra.run.smithi063.stdout:Deleting cluster with fsid: 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:55:40.583 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[75595]: fuse finished with error 0 and tester_r 0 2024-05-09T06:55:58.469 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T06:55:58.627 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:56:12.930 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:56:12.965 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:56:12.993 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T06:56:12.994 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535/remote/smithi063/crash 2024-05-09T06:56:12.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/crash -- . 2024-05-09T06:56:13.034 INFO:teuthology.orchestra.run.smithi063.stderr:tar: /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T06:56:13.034 INFO:teuthology.orchestra.run.smithi063.stderr:tar: Error is not recoverable: exiting now 2024-05-09T06:56:13.036 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535/remote/smithi193/crash 2024-05-09T06:56:13.037 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/crash -- . 2024-05-09T06:56:13.063 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T06:56:13.063 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-05-09T06:56:13.063 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T06:56:13.063 DEBUG:teuthology.orchestra.run.smithi063:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-05-09T06:56:13.117 INFO:tasks.cephadm:Compressing logs... 2024-05-09T06:56:13.117 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T06:56:13.160 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T06:56:13.184 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-05-09T06:56:13.185 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-05-09T06:56:13.185 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.log 2024-05-09T06:56:13.185 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T06:56:13.185 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-09T06:56:13.185 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-09T06:56:13.192 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-09T06:56:13.192 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.5.log: /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-client.ceph-exporter.smithi193.log 2024-05-09T06:56:13.192 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mgr.smithi193.mmmqzh.log 2024-05-09T06:56:13.192 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-client.ceph-exporter.smithi193.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T06:56:13.192 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mgr.smithi193.mmmqzh.log: 88.5% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T06:56:13.192 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi193.sygxnf.log 2024-05-09T06:56:13.192 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.cephadm.log: 94.2% 85.6% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T06:56:13.192 INFO:teuthology.orchestra.run.smithi193.stderr: -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-client.ceph-exporter.smithi193.log.gz 2024-05-09T06:56:13.193 INFO:teuthology.orchestra.run.smithi193.stderr: 91.6% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T06:56:13.195 INFO:teuthology.orchestra.run.smithi063.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-09T06:56:13.195 INFO:teuthology.orchestra.run.smithi063.stderr:: No such file or directory 2024-05-09T06:56:13.195 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.log 2024-05-09T06:56:13.195 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mgr.smithi063.egfguy.log 2024-05-09T06:56:13.195 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mgr.smithi063.egfguy.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mon.smithi063.log 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.2.log: /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-client.ceph-exporter.smithi063.log 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mon.smithi063.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-client.ceph-exporter.smithi063.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.1.log: 88.5% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr: 94.2% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-client.ceph-exporter.smithi063.log.gz 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi063.aoiibw.log 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi063.fbzxbf.log 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi063.aoiibw.log: 91.5% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T06:56:13.196 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T06:56:13.197 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mon.smithi193.log 2024-05-09T06:56:13.199 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi193.sygxnf.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T06:56:13.199 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mon.smithi193.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi193.ggtmyb.log 2024-05-09T06:56:13.206 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi063.fbzxbf.log: gzip -5 --verbose -- /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T06:56:13.207 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.cephadm.log: 85.7% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T06:56:13.213 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T06:56:13.213 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T06:56:13.235 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi193.ggtmyb.log: /var/log/ceph/cephadm.log: 89.1% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mgr.smithi193.mmmqzh.log.gz 2024-05-09T06:56:13.235 INFO:teuthology.orchestra.run.smithi193.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T06:56:13.235 INFO:teuthology.orchestra.run.smithi193.stderr: 91.1% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T06:56:13.264 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 91.2% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T06:56:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T06:56:13.490 INFO:teuthology.orchestra.run.smithi193.stderr: 92.6% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mon.smithi193.log.gz 2024-05-09T06:56:13.624 INFO:teuthology.orchestra.run.smithi063.stderr: 88.6% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mgr.smithi063.egfguy.log.gz 2024-05-09T06:56:14.449 INFO:teuthology.orchestra.run.smithi063.stderr: 90.5% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mon.smithi063.log.gz 2024-05-09T06:56:18.931 INFO:teuthology.orchestra.run.smithi193.stderr: 94.7% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi193.sygxnf.log.gz 2024-05-09T06:56:18.946 INFO:teuthology.orchestra.run.smithi063.stderr: 93.5% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-09T06:56:19.614 INFO:teuthology.orchestra.run.smithi193.stderr: 93.5% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-09T06:56:20.490 INFO:teuthology.orchestra.run.smithi063.stderr: 93.1% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-09T06:56:21.001 INFO:teuthology.orchestra.run.smithi063.stderr: 94.7% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi063.aoiibw.log.gz 2024-05-09T06:56:21.070 INFO:teuthology.orchestra.run.smithi193.stderr: 93.4% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-09T06:56:21.219 INFO:teuthology.orchestra.run.smithi063.stderr: 93.3% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-09T06:56:21.980 INFO:teuthology.orchestra.run.smithi193.stderr: 93.6% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-09T06:56:26.484 INFO:teuthology.orchestra.run.smithi063.stderr: 94.8% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi063.fbzxbf.log.gz 2024-05-09T06:56:29.275 INFO:teuthology.orchestra.run.smithi193.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-05-09T06:56:29.275 INFO:teuthology.orchestra.run.smithi193.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-05-09T06:56:30.711 INFO:teuthology.orchestra.run.smithi063.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-09T06:56:30.714 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-05-09T06:56:30.714 INFO:teuthology.orchestra.run.smithi063.stderr:real 0m17.539s 2024-05-09T06:56:30.714 INFO:teuthology.orchestra.run.smithi063.stderr:user 0m59.505s 2024-05-09T06:56:30.714 INFO:teuthology.orchestra.run.smithi063.stderr:sys 0m1.817s 2024-05-09T06:57:37.105 INFO:teuthology.orchestra.run.smithi193.stderr: 92.6% -- replaced with /var/log/ceph/2386be1c-0dcd-11ef-bc97-c7b262605968/ceph-mds.cephfs.smithi193.ggtmyb.log.gz 2024-05-09T06:57:37.106 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-05-09T06:57:37.106 INFO:teuthology.orchestra.run.smithi193.stderr:real 1m23.928s 2024-05-09T06:57:37.106 INFO:teuthology.orchestra.run.smithi193.stderr:user 2m5.088s 2024-05-09T06:57:37.107 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m3.860s 2024-05-09T06:57:37.107 INFO:tasks.cephadm:Archiving logs... 2024-05-09T06:57:37.107 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535/remote/smithi063/log 2024-05-09T06:57:37.108 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T06:57:44.357 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535/remote/smithi193/log 2024-05-09T06:57:44.358 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T06:58:03.662 INFO:tasks.cephadm:Removing cluster... 2024-05-09T06:58:03.663 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 --force 2024-05-09T06:58:03.826 INFO:teuthology.orchestra.run.smithi063.stdout:Deleting cluster with fsid: 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:58:04.152 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2386be1c-0dcd-11ef-bc97-c7b262605968 --force 2024-05-09T06:58:04.305 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 2386be1c-0dcd-11ef-bc97-c7b262605968 2024-05-09T06:58:04.699 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T06:58:04.699 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T06:58:04.717 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T06:58:04.732 INFO:tasks.cephadm:Teardown complete 2024-05-09T06:58:04.733 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T06:58:04.751 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-09T06:58:04.752 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T06:58:04.752 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T06:58:04.759 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T06:58:04.848 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T06:58:04.858 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T06:58:04.859 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:58:04.861 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:58:04.874 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:58:04.875 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:58:04.878 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:58:04.879 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-09T06:58:04.879 INFO:teuthology.orchestra.run.smithi063.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +1068us[+1073us] +/- 49ms 2024-05-09T06:58:04.879 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -990us[ -985us] +/- 73ms 2024-05-09T06:58:04.879 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -831us[ -827us] +/- 91ms 2024-05-09T06:58:04.879 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:58:04.880 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:58:04.880 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-05-09T06:58:04.880 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +1090us[+1089us] +/- 49ms 2024-05-09T06:58:04.880 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -973us[ -973us] +/- 73ms 2024-05-09T06:58:04.880 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -823us[ -824us] +/- 91ms 2024-05-09T06:58:04.880 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:58:04.880 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T06:58:04.894 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T06:58:04.894 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T06:58:04.911 DEBUG:teuthology.orchestra.run.smithi063:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T06:58:05.002 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T06:58:05.076 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:58:05.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:58:05.106 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:58:05.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:58:05.135 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T06:58:05.147 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T06:58:05.158 INFO:teuthology.task.internal:Duration was 2274.990441 seconds 2024-05-09T06:58:05.159 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T06:58:05.176 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T06:58:05.176 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T06:58:05.179 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T06:58:05.217 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:58:05.217 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:58:05.671 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T06:58:05.671 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T06:58:05.671 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T06:58:05.692 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-05-09T06:58:05.693 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T06:58:05.734 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T06:58:05.734 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T06:58:05.737 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T06:58:05.801 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T06:58:05.802 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T06:58:05.805 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T06:58:06.474 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T06:58:06.486 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T06:58:06.487 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T06:58:06.512 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T06:58:06.539 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T06:58:06.552 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:58:06.555 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:58:06.576 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-05-09T06:58:06.604 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-05-09T06:58:06.647 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:58:06.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:58:06.667 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:58:06.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:58:06.703 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T06:58:06.717 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T06:58:06.718 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535/remote/smithi063 2024-05-09T06:58:06.719 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T06:58:06.771 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535/remote/smithi193 2024-05-09T06:58:06.771 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T06:58:06.809 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T06:58:06.809 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T06:58:06.812 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T06:58:06.858 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T06:58:06.872 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T06:58:06.872 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T06:58:06.886 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T06:58:06.886 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T06:58:06.888 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T06:58:06.901 INFO:teuthology.orchestra.run.smithi063.stdout: 266028 4 drwxr-xr-x 3 ubuntu ubuntu 4096 May 9 06:58 /home/ubuntu/cephtest 2024-05-09T06:58:06.901 INFO:teuthology.orchestra.run.smithi063.stdout: 655385 4 d--------- 2 ubuntu ubuntu 4096 May 9 06:35 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:58:06.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:58:06.903 INFO:teuthology.orchestra.run.smithi063.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-05-09T06:58:06.903 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-05-09T06:58:06.903 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi063 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-05-09T06:58:06.904 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T06:58:06.918 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T06:58:06.932 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-09T06:58:06.932 INFO:teuthology.nuke:Checking targets against current locks 2024-05-09T06:58:06.956 DEBUG:teuthology.nuke:shortname: smithi063 2024-05-09T06:58:06.956 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T06:58:06.981 DEBUG:teuthology.nuke:shortname: smithi193 2024-05-09T06:58:06.981 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T06:58:06.984 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:11:21.541231', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOYFkKgAT/AIVxNH9mZa3ADyR3pC0nOjqV5uXAx2u+voylf2wL8cziVKm7ogjmYdUemwhflKsYmPWj6X8+SOM0U='} 2024-05-09T06:58:07.006 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698535', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:11:21.540236', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0dYtLFJMb1Cd3PqN/S8GwRyA6YC7HvN4IeszCSa/JDbc1gqrxc5QN+KNkbCHxxP5IfNYzSjDWGkFwmjsrjkrI='} 2024-05-09T06:58:07.036 INFO:teuthology.orchestra.console.smithi063:Power off 2024-05-09T06:58:07.036 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T06:58:07.062 DEBUG:teuthology.orchestra.console.smithi063:power off output: Chassis Power Control: Down/Off 2024-05-09T06:58:07.062 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:58:07.078 INFO:teuthology.orchestra.console.smithi193:Power off 2024-05-09T06:58:07.078 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T06:58:07.087 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-05-09T06:58:07.100 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off 2024-05-09T06:58:07.100 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:58:07.122 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-05-09T06:58:11.088 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:58:11.123 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:58:11.199 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-05-09T06:58:11.233 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-05-09T06:58:15.202 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:58:15.234 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:58:15.312 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off 2024-05-09T06:58:15.345 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off 2024-05-09T06:58:15.412 INFO:teuthology.orchestra.console.smithi063:Power off completed 2024-05-09T06:58:15.446 INFO:teuthology.orchestra.console.smithi193:Power off completed 2024-05-09T06:58:15.551 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2274.9904408454895 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-05-09T06:58:15.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T06:58:15.624 INFO:teuthology.run:FAIL